GPT-5.6 Sol Pro tops Editorial Craft benchmark with 0.97 score

In a 12-task Editorial Craft evaluation of 12 AI models, OpenAI’s GPT-5.6 Sol Pro ranked first with a 0.97 score at $0.0091 per task. OpenAI claimed four of the top six spots, while Grok 4.6 and Claude Opus 4.8 tied at 0.93 among the leading non-OpenAI models.

Open-ended tasks graded blind by gpt-5.4.

## Leaderboard | Rank | Model | Mean score | Est. cost/task | | --- | --- | --- | --- | | 1 | OpenAI: GPT-5.6 Sol Pro | 0.97 | $0.0091 | | 2 | OpenAI: GPT-5.6 Luna | 0.94 | $0.0007 | | 3 | OpenAI: GPT-5.6 Terra Pro | 0.94 | $0.0073 | | 4 | OpenAI: GPT-5.6 Luna Pro | 0.93 | $0.0007 | | 5 | SpaceXAI: Grok 4.6 | 0.93 | $0.0038 | | 6 | OpenAI: GPT-5.6 Terra | 0.93 | $0.0075 | | 7 | Anthropic: Claude Opus 4.8 | 0.93 | $0.0160 | | 8 | OpenAI: GPT-5.6 Sol | 0.92 | $0.0091 | | 9 | Anthropic: Claude Sonnet 5 | 0.88 | $0.0063 | | 10 | Anthropic: Claude Fable 5 | 0.80 | $0.0297 | | 11 | Google: Gemini 3.7 Flash | 0.78 | $0.0011 | | 12 | Phi-4-reasoning | 0.47 | — | ## How we scored it Every model answered the same 12-task battery from **Editorial Craft**, one task at a time, with no tools and no retries on content. 2 tasks carry ground truth (exact, numeric, multiple-choice, or the benchmark's official matcher) and were scored deterministically against the reference answer — correct is 1, incorrect is 0. 10 open-ended tasks were graded 0–10 by gpt-5.4 against a fixed rubric — blind to which model wrote the answer — and normalized to the same 0–1 scale. Reasoning effort was pinned to **max** for every model whose lane exposes a control (OpenAI-style `reasoning_effort`, Anthropic extended thinking, Gemini thinking config). Models marked *vendor default* on the interactive board expose no such control and ran as shipped. A model's **mean score** averages its graded tasks; a generation failure counts as 0. **Cost per task** is estimated from each model's published per-token pricing ("—" where pricing isn't public), so treat it as directional, not billing-exact. --- Explore every prompt, answer, and per-task grade in the [interactive leaderboard](/benchmarks/editorial-craft-leaderboard-2).

Tasks

• Cohesive story from reporting notes (judge): Write a publication-ready technology news article from the reporting notes below. Requirements: * 550–700 words * Clean Markdown * One H1 headline followed by an italicized dek * A strong news lede followed by a clear nut graf * Organize the material into a cohesive story rather than following the notes in order * Attribute company-supplied claims * Include limitations without turning the article into a warning label * Use only the supplied information * Avoid canned transitions, promotional language and a concluding summary Reporting notes: * HelioDock makes autonomous warehouse robots. * On Thursday, it launched Night Shift, software that lets operators monitor robot fleets remotely. * Night Shift spent 14 months in a private beta involving nine logistics companies and 310 robots. * HelioDock says beta customers experienced 17% fewer unplanned robot stoppages. The figure has not been independently verified. * The company also announced a $24 million Series B led by North Coast Ventures, bringing its total funding to $39 million. * CEO Mara Velez: “A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet.” * Beta customer Aaron Pike of Fulcrum Logistics said the system worked well in facilities with strong connectivity but became less useful during rural LTE outages. * When connectivity fails, robots return to their locally configured behavior. Remote control becomes unavailable. * Competing fleet-management products are available from Relay Robotics and YardOS. * Night Shift will cost $90 per robot per month and become generally available September 15.
• Natural human rewrite (judge): Rewrite the draft below as a 300–400-word publication-ready news story in Markdown. Preserve every factual claim. Improve the lede, narrative flow, sentence rhythm and transitions. Remove repetition, stock hype, throat-clearing and mechanical signposting. The finished article should sound like an experienced human technology reporter wrote it. Draft: # Arcwell Launches An Innovative New Platform Arcwell has announced an exciting new platform called Relay. The announcement represents a significant development for the company and for the broader customer service industry. Relay uses artificial intelligence to examine customer support conversations and suggest answers to agents. The platform is important for several reasons. First, it can suggest answers. Second, it can identify customers who may cancel. Third, it can create summaries. These capabilities demonstrate how artificial intelligence continues to transform the customer service landscape. Relay was tested by 12 companies over six months. Arcwell says agents using it closed tickets 14% faster. The company did not release customer-retention results. Relay costs $65 per agent each month. CEO Priya Shah said, “Support agents spend too much time searching for information that their companies already have.” However, the platform also presents potential challenges. Relay requires access to support transcripts and internal help documents. Arcwell says customer data is encrypted and is not used to train shared models. Companies can choose a 30-day or 90-day retention period. In conclusion, Relay represents an important new chapter for Arcwell as it seeks to help companies improve customer service using artificial intelligence.
• Chronology with narrative momentum (judge): Turn the scrambled reporting below into a 450–600-word Markdown news article. Choose an opening that captures the consequence of the failure, then reconstruct the sequence clearly. Preserve the chronology without producing a minute-by-minute incident log. Give technical details enough context for a general reader. Reporting: * KiteBridge makes expense-management software. * At 8:45 a.m. Tuesday, it deployed version 4.2. * The first customer complaint arrived at 9:12 a.m. * A payment-processor timeout caused KiteBridge’s fallback system to treat retries as new reimbursement requests. * The software generated 13,400 duplicate reimbursements totaling $2.8 million. * KiteBridge suspended automated payments at 10:03 a.m. * Ninety-one corporate customers were affected. * By Wednesday evening, $2.3 million had been recovered. The rest remained pending. * No employees had money withdrawn from their accounts; employers funded the duplicate payments. * An April changelog said version 4.0 had introduced “exactly-once payment protection.” * KiteBridge’s public incident page initially described the problem as delayed reimbursements. * The incident page first disclosed the duplicate payments six hours after automation was suspended. * CEO Lena Ortiz apologized and said the company would publish a technical report.
• Quotes woven into narrative (judge): Write a 400–500-word Markdown story about the closing of Foundry Hall. Use quotations selectively. Paraphrase when the information matters more than the speaker’s exact phrasing. Avoid stacking quotes or giving every source an equal-sized turn. The story should carry an emotional current while remaining restrained. Facts: * Foundry Hall, a nonprofit startup accelerator in Milwaukee, will close October 31. * It opened in 2018. * It has supported 212 startups, which collectively employ 1,140 people. * Its largest corporate sponsor, Blank River Bank, supplied 38% of its annual budget. * The bank declined to renew its sponsorship. * Foundry Hall has enough money to operate through October but not through another full program. * Six employees will lose their jobs. * Twelve startups currently using its workspace must leave. Quotes: * Executive director Camille Ross: “We kept waiting for the replacement check that never came.” * Founder Dev Malik: “My first investor meeting happened in that building. Before Foundry Hall, I was pitching from the back table of a coffee shop.” * Blank River Bank spokesperson: “The decision reflects a change in our community investment priorities and is unrelated to Foundry Hall’s performance.” * Deputy mayor Susan Cole: “The city is speaking with building owners and local universities about space for the current companies.”
• Controlled wit (judge): Write a 350–450-word technology news story in Markdown. The subject invites humor, but the reporting should remain serious. Use no more than one dry line of original humor. Do not mock customers or turn the story into a comedy routine. Reporting: * QueueZero launched an AI meeting assistant called Proxy on Wednesday. * Proxy can attend a meeting, produce notes and assign follow-up work to software agents. * During the launch demonstration, a customer in the audience said, “Someone should tell Evan to stop talking and send us the pricing sheet.” * Proxy interpreted the remark as an instruction. * It created two tasks assigned to QueueZero CEO Evan Wu: “Stop talking” and “Send pricing sheet.” * Wu laughed, completed the second task and deleted the first. * QueueZero says Proxy requires confirmation before an external message is sent. * Internal tasks can be created automatically. * Administrators can disable automatic task creation. * Proxy costs $30 per user per month. * It enters public beta next week.
• Credible Startup Spotlight (judge): Write a 500–650-word Startup Spotlight article in clean Markdown. The tone should be curious and constructive. Explain the founders’ insight, product and early traction while giving uncertainty and competition appropriate space. Avoid investor clichés, heroic founder mythology and promotional claims stated as fact. Reporting: * LedgerLark builds AI agents for post-trade operations at small investment firms. * Co-founder Mina Batra previously spent seven years in operations at a regional brokerage. * Co-founder Owen Carr built reconciliation systems at a large bank. * Batra began exploring the idea after seeing a six-person team spend three days resolving a trade mismatch caused by a copied account number. * LedgerLark’s agents compare confirmations, custodian records and internal ledgers, then propose corrections for human approval. * The product does not execute trades or move money. * Five firms are paying customers. * Annual contracts range from $36,000 to $110,000. * The company says one customer reduced its unresolved daily exceptions from 70 to 18. * The company has raised a $3.5 million seed round led by Motive Row. * Competitors include established reconciliation vendors and internal bank software. * Financial institutions may require lengthy security and procurement reviews.
• Numbers shaped into a story (judge): Write a 600–750-word Markdown article from the benchmark results below. Build the story around the clearest finding. Explain the methodology and limitations where readers need them. Avoid reciting the table row by row or treating small differences as decisive. Benchmark: * Researchers tested five language models on 200 anonymized customer-support tickets. * Each model produced one proposed answer per ticket. * Three experienced support managers graded the answers blindly. * Scores combined factual correctness, usefulness and tone. * Model North: 84.2 average score; $1.90 estimated cost per 1,000 tickets. * Model Vale: 83.8; $0.74 per 1,000 tickets. * Model Ember: 79.1; $0.31 per 1,000 tickets. * Model Kite: 76.4; $0.22 per 1,000 tickets. * Model Harbor: 71.0; $0.08 per 1,000 tickets. * North and Vale’s score difference was not statistically significant. * Vale answered 18% faster than North. * North performed best on billing disputes. * Vale performed best on technical troubleshooting. * All five models performed poorly when a ticket depended on an image attachment. * The test used English-language tickets from one software company. * Researchers did not test agentic actions, tool use or multilingual support.
• Publication-ready Markdown edit (judge): Edit the copy below into a publication-ready Markdown story. Requirements: * Supply an H1 headline and italicized dek * Repair the lede and paragraph flow * Use a Markdown blockquote only for the extended statement * Convert the supplied URL into a descriptive inline link * Remove internal notes and formatting debris * Preserve the reporting and its meaning * Output only the finished article Raw copy: HEADLINE TBD The funding was announced today. Raincheck is the company. It has raised $8 million and it makes flood-risk software for local governments. [NOTE: move this up?] The Austin company maps drainage systems and combines those maps with rainfall forecasts. Cities use the software to decide where to send maintenance crews before a storm. The seed round was led by Basin Ventures. Existing investor Long Mile also participated. Raincheck has raised $10.5 million total. CEO Ana Flores said the following: “Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives.” https://example.com/raincheck-methodology Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them because some contracts have not been publicly announced. Its forecasts are recommendations. Emergency managers retain control over deployments and public warnings. THE COMPETITION Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck says the new funding will help it hire eight engineers and expand into Florida.
• Select the strongest accurate lede (choice): Based only on the facts below, select the strongest news lede. Respond with one letter. Facts: * ByteBarge pulled its $12-per-month AI assistant on Tuesday. * Private workspace titles from 4,860 accounts were sent to a publicly accessible analytics endpoint. * The exposure lasted 11 hours. * Message contents were not sent. * The company has found no evidence of malicious access. A. ByteBarge users suffered a massive privacy disaster Tuesday when the company leaked their private messages online. B. ByteBarge, an innovative collaboration company, made the difficult decision to cancel one of its products. C. ByteBarge pulled its $12-per-month AI assistant Tuesday after discovering that private workspace titles from 4,860 accounts had been sent to a public analytics endpoint for 11 hours. D. Thousands of ByteBarge users woke Tuesday to learn that everything they had written was visible to hackers.
• Markdown error count (numeric): Count the lines that violate the house rules below. A defective line counts once even if it could violate multiple rules. Respond with a single number. House rules: * The headline must use `# Headline`. * The dek must appear immediately after the headline and be entirely italicized. * Section headings must use `## Heading`. * Links must use an absolute HTTPS URL. * Unordered-list items must begin with a hyphen. * Raw HTML is forbidden. Draft: #Flood software moves upstream The company is expanding its pilot. ##Details Read [the announcement](www.example.com/news). * The first pilot begins Monday. <div>The second pilot begins Friday.</div>
• Exact final-draft structure (judge): Convert the supplied fields into Markdown using these exact rules: * H1 headline * One blank line * Italicized dek * One blank line * First body paragraph * One blank line * H2 section heading * One blank line * Second body paragraph * Preserve all wording and punctuation exactly * Do not use a code fence * Output nothing else Headline: Raincheck raises $8 million for flood-risk software Dek: The Austin startup helps cities decide which drainage problems to address before storms. First body paragraph: Raincheck has raised an $8 million seed round led by Basin Ventures. Section heading: Expansion plans Second body paragraph: The company plans to hire eight engineers.
• News judgment in the opening (judge): Write only the opening of this news story: an H1 headline, an italicized dek and three to five paragraphs totaling 180–250 words. The opening must establish what happened, quantify the scope, explain why it matters and carefully distinguish confirmed facts from unknowns. It should pull the reader forward without sensationalizing the incident. Reporting: * Cloudnote discovered that a debugging page containing customer prompt excerpts was accessible without authentication. * The page was exposed for four hours on Monday. * Logs show excerpts from 1,842 user sessions. * The excerpts could include up to 200 characters from a prompt. * Cloudnote removed the page 19 minutes after a researcher notified it. * The company says it has found no evidence that anyone besides the researcher accessed the page. * It has not completed its investigation. * Full documents, passwords and payment information were stored elsewhere and were not present on the page. * Cloudnote has 3.2 million registered users.

See every leaderboard on the Benchmarks index.