hillclimb gives AI agents 48 hours to improve real machine-learning tasks
The new benchmark scores agents on 15 research-derived problems, using private verifiers to judge whether their changes hold up on data they cannot see.
By Ryan Merket · Published
Primary source: X
Why it matters
DeepRSI shifts evaluation toward sustained machine-learning work with private checks, while its results remain bounded by Hillclimb's task design, fixed agent setup and two runs per model. Strong scores show task improvement, not recursive self-improvement on their own.
Jun Park (@jparkjmc) introduced DeepRSI on October 12th as a benchmark for testing whether AI agents can improve machine-learning systems over a 48-hour run. Built by Park's hillclimb with three co-authors, it runs agents on 15 open-ended tasks drawn from research environments and scores their gains against task-specific baselines. The DeepRSI benchmark and its methodology are published online.