junior-ml-dev

A junior ML engineer that shows its work and catches its own mistakes.

It profiles a tabular dataset, checks for leakage before modelling, trains and compares models against baselines, has a deterministic reviewer sign off, and writes a report. Deployment is gated by a single-use approval.

This is a static site, not a live demo. The example runs below were computed once with the project's scripted planner (no LLM) on a developer laptop (Windows 11), CPU only, and are shown as recorded. A live, interactive version runs separately on a free Render instance: https://junior-ml-dev-demo.onrender.com/ (it sleeps when idle, so the first visit can take about a minute; it is slower and lighter than the runs here: leakage detectors skipped, no FLAML). Hugging Face Docker and Gradio Spaces need a paid account. The source repository is currently private, so file references below (result JSONs, code) are names only, not links.

Headline: Agent benchmark v5 (headline; unseen partition, each arm run once)

SCRIPTED planner (no LLM)
0.987 [0.949, 1.011]
pass vs linear: 7/10 (Wilson 0.40-0.89)
REAL-LLM planner v2
0.989 [0.950, 1.012]
pass vs linear: 8/10 (Wilson 0.49-0.94)

Read from docs/METRICS.md when this site was built (file hash 51e907642319, built 2026-11-09 18:20 UTC). Full tables, intervals and caveats: benchmark results.

Example runs

Three small public OpenML tables, each run once with the same code and limits the repository's demo mode uses (scripted planner, FLAML 15 s). A demo run has two limits: a 55 s soft wall after which the agent stops searching and wraps up (review, report), and a 90 s hard limit at which the whole process is killed (the AutoML ceiling is 20 s). Wrap-up and process start come on top of the soft wall, so a finished run can take longer than it. Wall times are from one machine and will differ elsewhere. These are demo-scale tasks, not the benchmark.

Pima diabetes (OpenML 37)
768 rows, binary, metric roc_auc
chosen model linear, validation roc_auc 0.8337, reviewer APPROVE
42 s wall, 13 steps
German credit (OpenML 31)
1000 rows, binary, metric roc_auc
chosen model linear, validation roc_auc 0.7913, reviewer APPROVE
46 s wall, 13 steps
Red wine quality (OpenML 40691)
1599 rows, regression, metric rmse
chosen model rf, validation rmse 0.5802, reviewer APPROVE
35 s wall, 13 steps

How it works

The architecture page shows the planner, the guarded runner, the tool servers, the reviewer and the approval gate, and which properties are enforced where.