A junior ML engineer that shows its work and catches its own mistakes.
It profiles a tabular dataset, checks for leakage before modelling, trains and compares models against baselines, has a deterministic reviewer sign off, and writes a report. Deployment is gated by a single-use approval.
Headline: Agent benchmark v5 (headline; unseen partition, each arm run once)
Read from docs/METRICS.md when this site was built (file hash
51e907642319, built 2026-11-09 18:20 UTC). Full tables, intervals and caveats:
benchmark results.
Example runs
Three small public OpenML tables, each run once with the same code and limits the repository's demo mode uses (scripted planner, FLAML 15 s). A demo run has two limits: a 55 s soft wall after which the agent stops searching and wraps up (review, report), and a 90 s hard limit at which the whole process is killed (the AutoML ceiling is 20 s). Wrap-up and process start come on top of the soft wall, so a finished run can take longer than it. Wall times are from one machine and will differ elsewhere. These are demo-scale tasks, not the benchmark.
linear, validation roc_auc 0.8337, reviewer APPROVElinear, validation roc_auc 0.7913, reviewer APPROVErf, validation rmse 0.5802, reviewer APPROVEHow it works
The architecture page shows the planner, the guarded runner, the tool servers, the reviewer and the approval gate, and which properties are enforced where.