Specific Labs launches Real-SWE, where 38.8% is enough for first place
Specific Labs co-founders Janak Sunil and Siddhant Paliwal built the benchmark from licensed production code, turning a recurring lab question into a data product.
By RuntimeWire Staff · Published
Primary source: Specific Labs
Why it matters
Public coding scores can reward familiarity with visible repositories. Real-SWE tests unfamiliar production systems while giving Specific a route from evaluation failures to training-data customers.
Specific Labs co-founders Janak Sunil (@janaksunil) and Siddhant Paliwal (@siddhantpa1iwal) have launched Real-SWE, a benchmark that puts coding agents to work on licensed, private production codebases. The assignments cover billing, taxes, customer migrations and infrastructure changes rather than neatly isolated bugs from public repositories.…
When you purchase through links in our articles, we may earn a small commission. This doesn’t affect our editorial independence.