GPT-6 Astra reaches 53% in Rails coding benchmark as Gemini regresses
GPT-6 Astra reached 53% accuracy, Gemini regressed, and a previously scoreless OpenAI model suddenly completed 16 runs.
By Ryan Merket · Published
Primary source: Ruby on Rails on X
Why it matters
Reasoning settings can change coding-agent economics as much as model choice. Rails found gains ranging from zero to 27 points while aggregate costs nearly doubled.
Ruby on Rails pushed every eligible model in its feature-development benchmark to the highest reasoning setting available, producing large gains for OpenAI's models, little movement elsewhere, and an overall bill that nearly doubled.
…
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.