Robocurve unveils Inspect Robots, free framework to benchmark AI-driven robots

Why it matters

An open, independent benchmark gives investors, developers, and regulators a reliable way to compare AI‑driven robots beyond staged demos, reducing hype and accelerating safe deployment.

On July 23rd 2026, Jay Chooi (@chooi_jeq) announced that his Public Benefit Corporation, Robocurve, had released Inspect Robots, an open‑source framework for measuring the real‑world capabilities of AI‑powered robots.

The announcement came as a 15‑post thread on X, where Chooi highlighted recent viral robot demos – a robot that can bake bread, cut carrots, and even perform karate kicks – and asked how well any of those feats reflect genuine progress. He positioned Robocurve as an independent evaluator, akin to METR for language models, but focused on the robotics domain.

"We are building @robocurve, a Public Benefit Corporation, to inform the public of the state of robotics progress. We are scaling the field of robotics evals as fast as possible," Chooi wrote in the second tweet of the thread.

Inspect Robots is released under an MIT license and is hosted at inspect-robots.org. The framework promises free, open‑source evaluation across any robot hardware and any AI model. It supports a growing catalog of benchmark tasks, from making a sandwich to assembling data‑center components, and the team invites the community to propose new benchmarks and receive a gift for the winning idea.

Technical details disclosed in the thread include:

• Compatibility with 40 VLAs (Vision‑Language Agents), among them models such as π0.5, GR00T N1.7, and MolmoAct2.
• Out‑of‑the‑box support for 260+ frontier LLMs on the first day of release.
• An architecture that can evaluate "any model, on any robot, on any benchmark, real and sim."

The framework also integrates with the Claude Fable 5 system, which can control YAM robotic arms to complete simple tasks, demonstrating a concrete example of the evaluation pipeline.

Robocurve's launch is backed by a Y Combinator batch, as referenced by a link to the YC launch page (Y Combinator launch). The YC affiliation underscores the early‑stage backing and the founders' access to a network of investors and mentors familiar with AI infrastructure.

The timing of the release is notable. In the same thread, Chooi noted that "Frontier labs are targeting general‑purpose robots by 2028, but nobody independent can tell you how close they are, yet." By providing an open, standardized yardstick, Robocurve aims to fill a gap that has hampered both investors and developers: the lack of transparent, reproducible performance data for physical AI systems. Without such metrics, claims about robot capabilities often rely on curated demo videos that obscure real‑world reliability, safety, and scalability.

Industry observers have highlighted the strategic value of an open evaluation suite. As AI models become more adept at perception and manipulation, the bottleneck shifts from algorithmic breakthroughs to hardware integration and safety verification. A public benchmark can accelerate iteration by surfacing failure modes early, allowing developers to prioritize robustness over flashy demos.

Robocurve also emphasized community engagement. The thread invited the public to suggest benchmark ideas, promising to build the best one and send a gift to the proposer. This crowdsourced approach mirrors trends in open‑source AI tooling, where community contributions drive rapid adoption and validation.

While the framework is free, the long‑term sustainability model for Robocurve remains unclear. As a Public Benefit Corporation, its charter likely focuses on mission over profit, but the company will eventually need revenue streams—potentially through consulting, custom evaluation services, or premium tooling.

Overall, the launch of Inspect Robots marks the first major step toward a systematic, transparent methodology for assessing AI‑controlled robots in the physical world. Whether the framework gains traction will depend on its ability to attract a critical mass of researchers, robot manufacturers, and end‑users who need reliable performance data beyond staged demonstrations.

Robocurve's open‑source evaluation suite arrives as the robotics community seeks independent, reproducible metrics to gauge the next generation of AI‑enabled machines.