Thinking Machines' ReViSQL-K2.6 scores 91.37% on cleaned SQL benchmark

Why it matters

ReViSQL supports Murati's argument that specialized training can beat bigger models and intricate agent workflows, but it also exposes the bill: expert data cleanup and carefully designed rewards remain essential.

Mira Murati (@miramurati)'s Thinking Machines Lab and four academic researchers trained a single model to outperform elaborate text-to-SQL agent pipelines, after finding that errors riddled the benchmark data used to teach and judge those systems.

The researchers, Yuxuan Zhu, Tengjun Jin, Yoojin Choi and Daniel Kang, published the results on August 27. Their model, ReViSQL-K2.6, reached 88.55% accuracy with greedy decoding and 91.37% when selecting among 16 generated answers on Arcwise-Plat-SQL, an expert-verified variant of BIRD Mini-Dev.

The paper identifies 92.96% as BIRD's reported human execution accuracy on BIRD Test, a different evaluation set. The two figures should not be treated as a same-test comparison.

The more consequential result is how the researchers got there: they replaced a chain of separately prompted agents with reinforcement learning on data reviewed by SQL experts.

For Murati, the project supplies a concrete example of the thesis behind Thinking Machines. The former OpenAI chief technology officer founded Thinking Machines to let researchers and organizations shape models around their own knowledge instead of relying on fixed general-purpose systems. Chief scientist John Schulman, an OpenAI co-founder who led reinforcement-learning work behind ChatGPT, gives that bet a direct lineage to modern model post-training.

The benchmark needed work before the model did

Zhu is a third-year computer science Ph.D. student at the University of Illinois Urbana-Champaign, advised by Kang. He previously studied computer science at the University of Michigan, where his research included SQL query optimization and testing. Kang is a UIUC computer science professor whose work centers on making machine learning and data analytics usable by scientists and analysts. He is also affiliated with Bridgewater AIA Labs, which studies AI systems for investment research.

Their starting point was less glamorous than designing another agent: checking whether the training examples were correct.

The researchers audited approximately 2,500 examples sampled from BIRD Train, a widely used text-to-SQL dataset. They reported that 52.1% contained an incorrect reference SQL query, 26.2% had a flawed natural-language question and 18.2% supplied incorrect external knowledge. The categories overlapped, and 61.1% of the audited examples contained at least one identified problem.

An OpenAI o3 model used in the initial review achieved 90.6% precision when it flagged an annotation error, according to the researchers. It found only 24.5% of the errors identified by human reviewers. The group then used multiple rounds of expert review and conflict resolution to produce BIRD-Platinum, the corrected training set.

That audit changes how the headline benchmark numbers should be read. A model can be punished for producing a sound query when the reference answer is wrong. It can also receive credit for matching a flawed reference. The researchers separately reported that 52.8% of BIRD Mini-Dev examples had annotation errors after two cleanup passes.

The result is a demonstration on curated academic test sets, rather than evidence that ReViSQL-K2.6 can safely query a production data warehouse. Enterprise databases bring access controls, shifting schemas, organization-specific terminology and far larger column counts. The paper does not report a customer deployment or evaluation on a live corporate database.

Training replaces the prompt assembly line

Text-to-SQL systems commonly divide the job among several model calls. One stage identifies relevant tables and columns. Another writes a query. Later calls execute, repair and rank candidates. Each addition can raise accuracy, while also adding latency, cost and another piece of benchmark-specific orchestration.

The UIUC researchers took the opposite route. They fine-tuned Moonshot AI's Kimi-K2.6 with reinforcement learning through Tinker, Thinking Machines' training API. Tinker handles the underlying compute while exposing functions for sampling, gradient calculation, weight updates and checkpointing.

The training method used verifiable rewards, which assign a score based on whether generated SQL produces the expected result. The researchers found that this familiar test could reward a query for reaching the right answer through faulty logic. ReViSQL therefore added reward-shaping techniques aimed at the domain's common failure modes, including whether a query was logically equivalent to the reference and whether it followed supplied background constraints.

ReViSQL-K2.6 reached 88.55% with greedy decoding, defined as one sample at temperature 0, at a reported cost of $0.035 per task. Generating 16 candidates at temperature 1 and choosing the majority execution result raised accuracy to 91.37% under 16-sample self-consistency and cost to approximately $0.56 per task. Thinking Machines says that configuration cost 12% to 15% as much as the frontier-model systems in its comparison.

Majority voting still spends additional inference compute, even if it avoids a multi-stage agent pipeline. The distinction matters: ReViSQL generates parallel answers from the same prompt and model, rather than sending the problem through specialized prompting, correction and judging agents.

The researchers released the code, corrected data and training recipe. The repository provides commands for reproducing the training and inference process, although users must train their own checkpoint from the Kimi-K2.6 base model. The supplied materials do not establish whether a downloadable ReViSQL-K2.6 checkpoint or software license is available.

Murati gets a proof point for Tinker

Thinking Machines has ample capital to pursue large models after closing a $2 billion seed round led by Andreessen Horowitz in July 2025. Nvidia, Accel, ServiceNow, Cisco, AMD and Jane Street also participated in the financing, which valued Thinking Machines at $12 billion, according to a Thinking Machines spokesperson cited by TechCrunch.

ReViSQL points toward a nearer-term business for Murati: selling the infrastructure that lets specialists train focused models without operating their own GPU clusters. A hospital, bank or software vendor does not need to build a general-purpose frontier model to capture proprietary expertise. It needs reliable training tools, clean examples and a reward signal that measures the behavior it actually wants.

Thinking Machines has been turning that argument into a series of public demonstrations. RuntimeWire reported on August 26 that Thinking Machines would offer up to $50,000 in Tinker credits for outside safety research involving its Inkling model. The SQL work puts the same infrastructure behind a commercial data task with measurable accuracy and cost.

The harder test comes outside the benchmark. ReViSQL benefited from SQL experts who manually repaired thousands of examples and helped encode domain-specific failure modes into the reward. That labor is part of the product thesis rather than an incidental expense. Murati is betting that organizations will invest in teaching models what their own experts know, and that Tinker can become the machinery underneath that transfer.