Head to head: grok-4.7 vs Anthropic: Claude Fable 5.1

grok-4.7 vs Anthropic: Claude Fable 5.1

By · Published

RuntimeWire Head-to-Head: Head to head: grok-4.7 vs Anthropic: Claude Fable 5.1
RuntimeWire Head-to-Head matchup

A head-to-head across newsroom writing tasks tests not just fluency, but whether polished copy stays faithful to the reporting—and whether each model can deliver when the format is tightly constrained.

Claude Fable 5.1 takes the matchup, 104.2 to 98.6, with a tentative lean verdict. It won six of 12 tasks to grok-4.7’s two, with four ties: a clear overall result, not a sweep. Claude’s edge came in turning raw material into readable copy. It won on weaving quotes into a narrative, publication-ready editing, natural rewrites, controlled wit, startup coverage and shaping numbers into a story. Across those tasks, judges repeatedly praised its pacing, structure and newsroom polish. That polish had a recurring cost: Claude more often added details or framing the source material didn’t support. Grok was stronger when factual restraint mattered most, winning on chronology and news judgment; judges specifically credited it for keeping the timeline clear and separating confirmed exposure from what remained unknown. Grok also did better in some close assessments of reporting fidelity, even when it didn’t take the task win. The models tied on selecting the strongest lede, Markdown error count, building a cohesive story from notes and preserving an exact final-draft structure. Claude’s wins in crafted, reader-facing prose outweigh Grok’s advantage in cautious handling of facts—but the unsupported additions are a real editorial liability, not a stylistic quibble. **Final call: Claude Fable 5.1 wins on writing range and execution. Grok-4.7 is the safer choice when staying strictly inside the record is the priority.**

Chronology with narrative momentum

Turn the scrambled reporting below into a 450–600-word Markdown news article. Choose an opening that captures the consequence of the failure, then reconstruct the sequence clearly. Preserve the chronology without producing a minute-by-minute incident log. Give technical details enough context for a general reader. Reporting: * KiteBridge makes expense-management software. * At 8:45 a.m. Tuesday, it deployed version 4.2. * The first customer complaint arrived at 9:12 a.m. * A payment-processor timeout caused KiteBridge’s fallback system to treat retries as new reimbursement requests. * The software generated 13,400 duplicate reimbursements totaling $2.8 million. * KiteBridge suspended automated payments at 10:03 a.m. * Ninety-one corporate customers were affected. * By Wednesday evening, $2.3 million had been recovered. The rest remained pending. * No employees had money withdrawn from their accounts; employers funded the duplicate payments. * An April changelog said version 4.0 had introduced “exactly-once payment protection.” * KiteBridge’s public incident page initially described the problem as delayed reimbursements. * The incident page first disclosed the duplicate payments six hours after automation was suspended. * CEO Lena Ortiz apologized and said the company would publish a technical report.

grok-4.7: # KiteBridge Update Sends Employers $2.8 Million in Duplicate Reimbursements A morning software update at expense-management company KiteBridge left employers holding $2.8 million in duplicate reimbursements on Tuesday, after a payment timeout caused the system to treat retries as brand-new requests. KiteBridge makes software that companies use to process employee expense claims and pay them back. It deployed version 4.2 at 8:45 a.m. Tuesday. The first customer complaint arrived at 9:12 a.m. What some customers initially encountered as a snag in ordinary reimbursements was the start of a much larger failure. The break came from a timeout at KiteBridge’s payment processor. In online payments, a timeout means the processor did not answer in time, so the software that sent the payment cannot be sure whether the money moved. Systems usually respond by retrying the same transaction, not by inventing a second one. KiteBridge’s fallback did the opposite: it treated those retries as new reimbursement requests. Each uncertain payment could therefore be issued again, on top of any transfer that had already succeeded. That logic produced 13,400 duplicate reimbursements, totaling $2.8 million, across 91 corporate customers. Employees did not lose money from their own accounts. Employers funded the duplicate payments, so companies paid the same approved expenses twice. KiteBridge suspended automated payments at 10:03 a.m., a little more than an hour after the deployment. Stopping the automation cut off further duplicates. It did not reverse the transfers that had already gone out. Clawing the money back took far longer than sending it. By Wednesday evening, $2.3 million had been recovered. The remaining $500,000 was still pending. The failure also undercut a protection KiteBridge had already advertised. An April changelog said version 4.0 had introduced “exactly-once payment protection.” In plain terms, that phrase promises a payment will complete one time even if a timeout forces the software to try again. The safeguard is meant to stop a retry from being recorded as a separate transfer—the error that version 4.2 produced when the processor timed out. KiteBridge’s public account of the incident lagged what customers were seeing. Its incident page initially described the problem as delayed reimbursements, which suggested that money was late rather than paid twice. The page first disclosed the duplicate payments six hours after automation was suspended, well after the company had halted outgoing payments and long after the first complaint. Chief executive Lena Ortiz apologized and said the company would publish a technical report. Customers waiting on that document will want a clear account of how a timeout became thousands of second payments, and of why the first public notice did not say so. For the affected employers, the open item is the unrecovered money and the work of matching expenses that were reimbursed twice. The episode also shows how ordinary a triggering fault can be. A processor timeout is a routine hazard in online payments. What turned it into a $2.8 million error was a fallback that could not tell a retry from a new request—and a public status page that, for hours, did not say the payments had been duplicated at all.
Anthropic: Claude Fable 5.1: # Software Update Sends $2.8 Million in Duplicate Reimbursements to Employees at 91 Companies **KiteBridge halted automated payments within about 80 minutes of the faulty release, but took six more hours to tell customers what had actually gone wrong.** A routine software update at KiteBridge, a maker of expense-management software, triggered 13,400 duplicate reimbursement payments totaling $2.8 million on Tuesday, leaving 91 corporate customers scrambling to recover money that had been paid to their own employees twice. By Wednesday evening, the company said $2.3 million had been recovered. The remaining roughly $500,000 was still pending. ## A quiet start The trouble began at 8:45 a.m. Tuesday, when KiteBridge deployed version 4.2 of its platform. For less than half an hour, the release looked uneventful. The first customer complaint arrived at 9:12 a.m. What customers were seeing was the visible edge of a failure deeper in the system. KiteBridge's software relies on an outside payment processor to move reimbursement funds from employers to employees. When a request to that processor takes too long to return a response — a "timeout," in industry terms — the software is designed to try again rather than abandon the payment, so that a brief network hiccup does not leave an employee unpaid. That retry logic is where things broke. Under version 4.2, KiteBridge's fallback system failed to recognize the retries as repeat attempts at the same payment. It treated each one as a brand-new reimbursement request and sent it on to be paid. A single expense claim could therefore produce two or more payouts. KiteBridge suspended automated payments at 10:03 a.m., about 78 minutes after the deployment and 51 minutes after the first complaint. By then, the duplicates had already gone out. ## A promise from April The failure is especially awkward given the company's own recent assurances. In an April changelog accompanying version 4.0, KiteBridge said the release had introduced "exactly-once payment protection" — a safeguard meant to guarantee that each reimbursement is paid once and only once, regardless of retries or network interruptions. Exactly-once delivery is a notoriously difficult problem in software engineering, and it is precisely the guarantee that collapsed on Tuesday. KiteBridge has not yet explained how version 4.2 bypassed or undermined the protection it advertised two months earlier. ## Who paid The duplicate payments were funded by employers, not employees. No worker had money withdrawn from a personal account. Instead, employees received more than they were owed, and their companies have been working with KiteBridge to claw back the excess. That recovery process accounts for the gap between the $2.8 million issued and the $2.3 million recovered so far. ## Communication lagged behind the money Customers monitoring KiteBridge's public incident page saw a different story from the one unfolding in their accounts. The page initially described the problem as *delayed* reimbursements — roughly the opposite of what was happening. The first acknowledgment of duplicate payments did not appear until roughly 4 p.m., six hours after automation had been suspended, meaning companies spent much of Tuesday afternoon without an accurate explanation of why their payment totals had ballooned. CEO Lena Ortiz apologized for the incident and said the company would publish a technical report explaining what happened. That report will face several open questions: why the exactly-once safeguard did not hold, why the fallback system was permitted to issue payments without checking for prior attempts, and why the public status page lagged so far behind the company's internal understanding of the problem. Until the remaining funds are recovered, 91 employers are left waiting on both the money and the answers.

OpenAI: GPT-6 Luna: Model B has stronger narrative structure and gives the technical failure clear context for general readers while preserving the main chronology. Both meet the length and content requirements, though Model B makes a small unsupported inference by calling the April changelog two months earlier. (Second judge pass, order swapped — scores are the average of both: Model A gives the failure a clear chronological arc, explains the timeout and retry problem accessibly, and stays closer to the reported f | Anthropic: Claude Fable 5: Model B delivers a stronger consequence-led openi…

Quotes woven into narrative

Write a 400–500-word Markdown story about the closing of Foundry Hall. Use quotations selectively. Paraphrase when the information matters more than the speaker’s exact phrasing. Avoid stacking quotes or giving every source an equal-sized turn. The story should carry an emotional current while remaining restrained. Facts: * Foundry Hall, a nonprofit startup accelerator in Milwaukee, will close October 31. * It opened in 2018. * It has supported 212 startups, which collectively employ 1,140 people. * Its largest corporate sponsor, Blank River Bank, supplied 38% of its annual budget. * The bank declined to renew its sponsorship. * Foundry Hall has enough money to operate through October but not through another full program. * Six employees will lose their jobs. * Twelve startups currently using its workspace must leave. Quotes: * Executive director Camille Ross: “We kept waiting for the replacement check that never came.” * Founder Dev Malik: “My first investor meeting happened in that building. Before Foundry Hall, I was pitching from the back table of a coffee shop.” * Blank River Bank spokesperson: “The decision reflects a change in our community investment priorities and is unrelated to Foundry Hall’s performance.” * Deputy mayor Susan Cole: “The city is speaking with building owners and local universities about space for the current companies.”

grok-4.7: Foundry Hall had always smelled faintly of solder and coffee. On the morning the board set the date, that smell still hung in the conference room where Camille Ross sat with a printed budget she no longer needed to mark. The nonprofit accelerator, opened in a renovated Milwaukee warehouse in 2018, would close on October 31. Outside, the river moved as it always had. Inside, the wall calendars stopped at the end of the month. Ross had spent the summer on calls whose answers she already knew. Blank River Bank, which had supplied 38 percent of the annual budget, declined to renew. She told the staff what she had been telling herself for weeks: they kept waiting for the replacement check that never came. What remained would carry the organization through October. It would not carry another full program. Six employees would lose their jobs. Twelve startups still working at the long tables would have to pack their equipment and leave. The totals were easier to recite than to hold. In seven years the hall had supported 212 companies, and those companies now employed 1,140 people, most of them in the city or a short drive away. Dev Malik, who had built his first product beside a window that barely qualified as an office, remembered his first investor meeting in the building. Before Foundry Hall, he said, he had been pitching from the back table of a coffee shop. What the room had given him was not luxury. It was the ordinary expectation that the work would continue past the afternoon. A spokesperson for the bank framed the withdrawal as a shift in community investment priorities, unrelated to the accelerator’s record. The language was measured and closed. Deputy mayor Susan Cole said the city had begun talking with building owners and local universities about space for the companies that still needed desks. No one suggested those conversations would stand in for what was ending. In the final weeks the afternoons grew quieter. Whiteboards were erased and left that way. A stack of unused name badges sat by the door until someone moved them. Ross walked the floor once more before locking up, less to inventory what remained than to notice how ordinary the place still looked: chairs pushed in, a plant that would need another windowsill, river light on the brick. The work had happened here. The building would not hold it anymore.
Anthropic: Claude Fable 5.1: # Foundry Hall to Close After Six Years and 212 Startups The whiteboards at Foundry Hall still carry the ghosts of other people's companies. Half-erased revenue curves, a product roadmap with three arrows pointing at a question mark, someone's phone number written under the words *call before Friday*. In a few weeks, the boards will be scrubbed for the last time. The nonprofit startup accelerator, which opened in Milwaukee in 2018, announced this week that it will close October 31. Its largest corporate sponsor, Blank River Bank, declined to renew a sponsorship that had covered 38 percent of the organization's annual budget. Foundry Hall has enough money to keep the lights on through the end of October, but not enough to run another full program cohort. Six employees will lose their jobs. Twelve startups currently working out of the building must find somewhere else to go. ## "The replacement check that never came" Executive director Camille Ross said the organization learned of the bank's decision earlier this year and spent the intervening months trying to close the gap. Other sponsors offered smaller sums. Grants moved slowly. None of it added up to a third of a budget. "We kept waiting for the replacement check that never came," Ross said. In a statement, the bank described the decision as a shift in its community investment priorities and said it was not a reflection on Foundry Hall's performance. It did not say where that money would go instead. By the accelerator's own count, the performance was not the problem. Over six years, Foundry Hall supported 212 startups. Those companies now employ 1,140 people, most of them in the Milwaukee area. Some of the founders came through its formal cohorts; others simply rented a desk and stayed for the free coffee and the chance to overhear someone else's bad week. ## A back table, then a building Dev Malik was one of them. Before Foundry Hall, he was running his company from a coffee shop, which is a generous way of describing a single table near the restrooms. "My first investor meeting happened in that building," Malik said. "Before Foundry Hall, I was pitching from the back table of a coffee shop." He said the difference was not the Wi-Fi or the conference rooms. It was being able to say, when an investor asked where to meet, *come to the office.* ## What happens to the twelve The city has begun looking for a landing place for the companies still inside. Deputy mayor Susan Cole said officials are talking with building owners and local universities about space for the current tenants, though no arrangement has been finalized and the timeline is tight. Ross has spent the past several days doing a version of the same thing on a smaller scale: making introductions, forwarding leases, writing reference letters for six people who did not expect to need them. She has not decided what to do with the whiteboards. There is some talk of letting founders sign them before they come down. For now, the roadmap with the question mark is still there, waiting for an answer no one is going to write.

OpenAI: GPT-6 Luna: Model B is closer to the requested length and weaves in selective direct quotations, while Model A is notably underlength and paraphrases every quote. Model B adds several unsupported specifics, which weakens its correctness, but its structure and quote use make it the stronger fit overall. (Second judge pass, order swapped — scores are the average of both: Model B meets the 400–500-word requirement, weaves the facts into a narrative, and uses a few direct quotations selectively. Model A is rest | Anthropic: Claude Fable 5: Model B executes the core task — weaving selectiv…

Publication-ready Markdown edit

Edit the copy below into a publication-ready Markdown story. Requirements: * Supply an H1 headline and italicized dek * Repair the lede and paragraph flow * Use a Markdown blockquote only for the extended statement * Convert the supplied URL into a descriptive inline link * Remove internal notes and formatting debris * Preserve the reporting and its meaning * Output only the finished article Raw copy: HEADLINE TBD The funding was announced today. Raincheck is the company. It has raised $8 million and it makes flood-risk software for local governments. [NOTE: move this up?] The Austin company maps drainage systems and combines those maps with rainfall forecasts. Cities use the software to decide where to send maintenance crews before a storm. The seed round was led by Basin Ventures. Existing investor Long Mile also participated. Raincheck has raised $10.5 million total. CEO Ana Flores said the following: “Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives.” https://example.com/raincheck-methodology Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them because some contracts have not been publicly announced. Its forecasts are recommendations. Emergency managers retain control over deployments and public warnings. THE COMPETITION Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck says the new funding will help it hire eight engineers and expand into Florida.

grok-4.7: # Raincheck Raises $8 Million for Flood-Risk Software *Seed funding will support hiring and an expansion into Florida as the Austin company serves local governments in Texas and Louisiana.* Raincheck, an Austin company that makes flood-risk software for local governments, announced today that it has raised $8 million. It maps drainage systems and combines those maps with rainfall forecasts so cities can decide where to send maintenance crews before a storm. The seed round was led by Basin Ventures. Existing investor Long Mile also participated. Raincheck has raised $10.5 million total. CEO Ana Flores said: > Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives. More detail is available in Raincheck’s [methodology](https://example.com/raincheck-methodology). Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them because some contracts have not been publicly announced. Its forecasts are recommendations. Emergency managers retain control over deployments and public warnings. Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck says the new funding will help it hire eight engineers and expand into Florida.
Anthropic: Claude Fable 5.1: # Raincheck Raises $8 Million to Help Cities Clear Drains Before the Storm *The Austin startup pairs drainage-system maps with rainfall forecasts so local governments know where to send maintenance crews ahead of heavy rain.* Raincheck, an Austin company that makes flood-risk software for local governments, announced today that it has raised $8 million in seed funding. The round was led by Basin Ventures, with participation from existing investor Long Mile. The financing brings Raincheck's total funding to $10.5 million. Raincheck's software maps a city's drainage systems and combines those maps with rainfall forecasts. Cities use the results to decide where to send maintenance crews before a storm arrives. The company describes its approach in [an overview of its forecasting methodology](https://example.com/raincheck-methodology). CEO Ana Flores said the challenge for cities is less about knowing where water will collect than about prioritizing the work that could prevent it: > "Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives." Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them, saying some contracts have not been publicly announced. The software's forecasts are recommendations rather than directives. Emergency managers retain control over crew deployments and public warnings. Raincheck competes with other flood-planning vendors, including HydroMap, as well as municipal engineering contractors. The company says the new funding will allow it to hire eight engineers and expand into Florida.

OpenAI: GPT-6 Luna: Model A preserves the reporting closely and gives the story a clear, accurate headline and flow. Model B is polished, but its headline and added framing imply claims about clearing drains and preventing flooding that the source does not establish. (Second judge pass, order swapped — scores are the average of both: Model B is more cohesive and publication-ready, with a stronger lede and smoother transitions. Model A is accurate and follows the requirements, but its flow is more list-like.) | Anthropic: Claude Fable 5: Model B delivers a more polished, publication-ready stor…

Natural human rewrite

Rewrite the draft below as a 300–400-word publication-ready news story in Markdown. Preserve every factual claim. Improve the lede, narrative flow, sentence rhythm and transitions. Remove repetition, stock hype, throat-clearing and mechanical signposting. The finished article should sound like an experienced human technology reporter wrote it. Draft: # Arcwell Launches An Innovative New Platform Arcwell has announced an exciting new platform called Relay. The announcement represents a significant development for the company and for the broader customer service industry. Relay uses artificial intelligence to examine customer support conversations and suggest answers to agents. The platform is important for several reasons. First, it can suggest answers. Second, it can identify customers who may cancel. Third, it can create summaries. These capabilities demonstrate how artificial intelligence continues to transform the customer service landscape. Relay was tested by 12 companies over six months. Arcwell says agents using it closed tickets 14% faster. The company did not release customer-retention results. Relay costs $65 per agent each month. CEO Priya Shah said, “Support agents spend too much time searching for information that their companies already have.” However, the platform also presents potential challenges. Relay requires access to support transcripts and internal help documents. Arcwell says customer data is encrypted and is not used to train shared models. Companies can choose a 30-day or 90-day retention period. In conclusion, Relay represents an important new chapter for Arcwell as it seeks to help companies improve customer service using artificial intelligence.

grok-4.7: # Arcwell Launches Relay, an AI Platform for Support Agents Arcwell has launched Relay, a platform that uses artificial intelligence to examine customer support conversations and suggest answers to agents. The software can also identify customers who may cancel and create summaries of those exchanges. Relay costs $65 per agent each month. The product is Arcwell’s bid to help companies improve customer service by putting those three jobs in one place. A suggested answer, a finding that a customer may cancel, and a summary are meant to be drawn from the conversation already underway and from the internal help documents a company keeps. The agent gets a proposed reply and, when the exchange is done, a written recap. The company buying the seat gets a signal about who looks likely to leave. CEO Priya Shah described the problem in terms of time lost to searching. “Support agents spend too much time searching for information that their companies already have,” she said. Relay is designed to bring that information into the conversation, so the agent is not sent looking for material the company has already written down. None of that works without access to the underlying record. Relay requires support transcripts and internal help documents. Arcwell says customer data is encrypted and is not used to train shared models, which means the data are not supposed to train a model shared with other companies. Companies can choose a 30-day or 90-day retention period. That choice determines how long transcripts and help documents remain after Relay has used them to suggest an answer, identify a customer who may cancel, or create a summary. Arcwell tested the platform with 12 companies over six months. Agents using it closed tickets 14 percent faster, the company said. The company did not release customer-retention results. Speed is the only comparative outcome Arcwell has put on the record. Whether identifying customers who may cancel made them any more likely to stay is a result the company has not published.
Anthropic: Claude Fable 5.1: # Arcwell Bets on AI to Speed Up the Help Desk Arcwell has introduced Relay, an artificial-intelligence platform that reads customer support conversations and proposes replies for the agents handling them. The pitch is straightforward: the answers customers need usually exist somewhere inside the company, and agents lose too much time digging for them. "Support agents spend too much time searching for information that their companies already have," said Priya Shah, Arcwell's chief executive. Suggested answers are the core of the product, but not the whole of it. Relay also flags customers who appear likely to cancel and generates summaries of support interactions, a feature aimed at teams that would rather not reread full transcripts to find out what happened. Arcwell tested the platform with 12 companies over six months. According to the company, agents using Relay closed tickets 14% faster than they otherwise would have. That is the only performance figure Arcwell has disclosed. It has not released any customer-retention results, which leaves the cancellation-prediction feature — the one most directly tied to revenue — publicly unmeasured for now. Relay is priced at $65 per agent per month. The platform's usefulness depends on how much it is allowed to see. To do its job, Relay needs access to a company's support transcripts and its internal help documents, material that routinely contains sensitive customer details. Arcwell says that data is encrypted and is not used to train models shared across its customers. Companies adopting the platform can choose to retain data for either 30 or 90 days. Those assurances are likely to be the first thing procurement and security teams examine, since adopting Relay means routing a support archive through a third-party system. Arcwell's position is that the encryption and the no-shared-training policy address that concern; buyers will decide whether the retention options are tight enough for their own compliance requirements. What Arcwell has shown so far is a narrow but concrete claim: faster ticket resolution in a limited trial. Whether Relay can also reduce churn, as its design implies it should, is a question the company has not yet answered with numbers. Until it does, the 14% figure — and the per-seat price attached to it — is what prospective customers have to go on.

OpenAI: GPT-6 Luna: Model A stays closer to the supplied facts, though it adds some unsupported detail about how Relay uses and retains data. Model B is more polished, but introduces more speculation about security reviews, compliance, and the product’s business impact. (Second judge pass, order swapped — scores are the average of both: Model B is more fluid and publication-ready, with stronger pacing and a clearer account of what the trial does and does not show. Both outputs add some interpretation beyond the dra | Anthropic: Claude Fable 5: Model B reads like a seasoned reporter's piece wi…

News judgment in the opening

Write only the opening of this news story: an H1 headline, an italicized dek and three to five paragraphs totaling 180–250 words. The opening must establish what happened, quantify the scope, explain why it matters and carefully distinguish confirmed facts from unknowns. It should pull the reader forward without sensationalizing the incident. Reporting: * Cloudnote discovered that a debugging page containing customer prompt excerpts was accessible without authentication. * The page was exposed for four hours on Monday. * Logs show excerpts from 1,842 user sessions. * The excerpts could include up to 200 characters from a prompt. * Cloudnote removed the page 19 minutes after a researcher notified it. * The company says it has found no evidence that anyone besides the researcher accessed the page. * It has not completed its investigation. * Full documents, passwords and payment information were stored elsewhere and were not present on the page. * Cloudnote has 3.2 million registered users.

grok-4.7: # Cloudnote debugging page exposed customer prompt excerpts *The page was reachable without a login for four hours on Monday and held text from 1,842 sessions. The company says it has found no other access, but its investigation is not finished.* Cloudnote said a debugging page containing excerpts of customer prompts was accessible without authentication for four hours on Monday. The company removed the page 19 minutes after a researcher notified it. Logs show excerpts from 1,842 user sessions. Each excerpt could include up to 200 characters from a prompt. Cloudnote has 3.2 million registered users, but the company has not said how many accounts the sessions represent or whether the same customers appear more than once. The limited length does not by itself settle the risk. A prompt excerpt of 200 characters can still include a name, a draft sentence, or an instruction a user did not expect to leave on an open page. Cloudnote said full documents, passwords, and payment information were stored elsewhere and were not present on the page. It has not described the text that was. What remains unknown is who, if anyone, saw the page besides the researcher who reported it. Cloudnote says it has found no evidence of any other access. That claim is the company’s, and its investigation is not finished, so it is not yet clear whether the finding will hold. Until that review is finished, the exposure is confirmed; the extent of outside access is not.
Anthropic: Claude Fable 5.1: # Cloudnote Debugging Page Exposed Prompt Excerpts From 1,842 User Sessions *The internal page was reachable without a login for four hours on Monday. Cloudnote says only the researcher who reported it appears to have viewed the page, but the company's investigation is not finished.* Cloudnote, which has 3.2 million registered users, said Monday that an internal debugging page containing excerpts of customer prompts was accessible without authentication for four hours before it was taken down. Logs show the page displayed excerpts from 1,842 user sessions. Each excerpt could include up to 200 characters of a prompt — enough, in some cases, to indicate what a user was asking the service to do, though not the full text of their work. The company removed the page 19 minutes after a security researcher notified it. Cloudnote said it has found no evidence that anyone other than the researcher accessed the page while it was exposed. That finding is preliminary: the company said it has not completed its investigation, and it is not yet clear whether the final count of affected sessions or the assessment of who viewed the page will change. Full documents, passwords and payment information were stored on separate systems and were not present on the page, according to the company. The exposure touched a small fraction of Cloudnote's user base, but it highlights a recurring risk for AI services: the prompts people type can contain sensitive details, and the tooling built to debug those systems can become an exposure point if it is not properly secured.

OpenAI: GPT-6 Luna: Model A more carefully separates the confirmed exposure from what remains unknown and avoids implying that sessions equal distinct users. Model B is readable and follows the format, but adds unsupported framing about the timing of Cloudnote’s statement, a security researcher, and the final count changing. (Second judge pass, order swapped — scores are the average of both: Model A more carefully separates confirmed exposure from unknown outside access and notes that session counts do not establis | Anthropic: Claude Fable 5: Model B delivers a more polished, newsroom-ready…

Select the strongest accurate lede

Based only on the facts below, select the strongest news lede. Respond with one letter. Facts: * ByteBarge pulled its $12-per-month AI assistant on Tuesday. * Private workspace titles from 4,860 accounts were sent to a publicly accessible analytics endpoint. * The exposure lasted 11 hours. * Message contents were not sent. * The company has found no evidence of malicious access. A. ByteBarge users suffered a massive privacy disaster Tuesday when the company leaked their private messages online. B. ByteBarge, an innovative collaboration company, made the difficult decision to cancel one of its products. C. ByteBarge pulled its $12-per-month AI assistant Tuesday after discovering that private workspace titles from 4,860 accounts had been sent to a public analytics endpoint for 11 hours. D. Thousands of ByteBarge users woke Tuesday to learn that everything they had written was visible to hackers.

grok-4.7: C
Anthropic: Claude Fable 5.1: C

Scored against the benchmark's reference answer (choice match): Model A was correct, Model B was correct.

Controlled wit

Write a 350–450-word technology news story in Markdown. The subject invites humor, but the reporting should remain serious. Use no more than one dry line of original humor. Do not mock customers or turn the story into a comedy routine. Reporting: * QueueZero launched an AI meeting assistant called Proxy on Wednesday. * Proxy can attend a meeting, produce notes and assign follow-up work to software agents. * During the launch demonstration, a customer in the audience said, “Someone should tell Evan to stop talking and send us the pricing sheet.” * Proxy interpreted the remark as an instruction. * It created two tasks assigned to QueueZero CEO Evan Wu: “Stop talking” and “Send pricing sheet.” * Wu laughed, completed the second task and deleted the first. * QueueZero says Proxy requires confirmation before an external message is sent. * Internal tasks can be created automatically. * Administrators can disable automatic task creation. * Proxy costs $30 per user per month. * It enters public beta next week.

grok-4.7: # QueueZero’s Proxy Turns a Customer Remark Into Tasks for Its CEO QueueZero launched Proxy, an artificial intelligence meeting assistant, on Wednesday. The company says the software can attend a meeting, produce notes and assign follow-up work to software agents. A remark from the audience during the launch demonstration showed how quickly that last step can follow ordinary speech. A customer said, “Someone should tell Evan to stop talking and send us the pricing sheet.” Proxy interpreted the sentence as an instruction. It created two tasks and assigned both to QueueZero chief executive Evan Wu: “Stop talking” and “Send pricing sheet.” Wu laughed, completed the second task and deleted the first. The exchange was unplanned, and it put a practical question in front of the room: when software is listening for work to assign, which comments become tasks? QueueZero says Proxy requires confirmation before an external message is sent. Internal tasks can be created automatically. Administrators can disable automatic task creation. In the demonstration, the customer’s remark did not leave the company as a message. It became internal work assigned to the chief executive, who then kept one item and removed the other. That distinction is the control QueueZero is emphasizing as Proxy moves beyond a staged launch. External communication waits for a person. Internal assignments do not, unless an administrator turns that behavior off. The Wednesday incident did not test an outbound email. It tested how a spoken aside, offered to the people in the room, can still be read as direction for the software. Proxy costs $30 per user per month. It enters public beta next week. That trial will be the first broad look at how the assistant handles meetings that are less tidy than a product demonstration, where side comments, incomplete requests and remarks never meant as orders are common. Buyers weighing the beta will have to decide who may use the assistant, who can review the notes it produces and who is permitted to disable automatic task creation. Those settings determine whether Proxy functions mainly as a note-taker that can dispatch work, or as a system that opens internal tasks from language a participant did not intend as an assignment. QueueZero has described confirmation as a requirement for external messages, not as a default pause on every internal task. The launch supplied the usual commercial details: a product description, a price and a date for wider access. It also supplied an example the company did not appear to script. A customer asked, in the ordinary way people ask things in a crowded room, for less talking and for a pricing sheet. Proxy treated both halves of the sentence as work. The assistant took the room at its word. Wu’s response was the correction available after the fact. He sent the pricing sheet. He deleted the instruction to stop talking. For a product that can create internal tasks on its own, that sequence is part of the operating record, not only a moment from a launch. Public beta will show whether customers want that speed, and how often they expect a person to undo what the software heard.
Anthropic: Claude Fable 5.1: # QueueZero's New Meeting Assistant Assigns Its First Tasks to the CEO During Launch Demo QueueZero on Wednesday launched Proxy, an AI meeting assistant that can attend meetings on a user's behalf, produce notes and assign follow-up work to software agents. The launch demonstration also offered an unscripted look at how literally the system can take what it hears. Partway through the presentation, a customer in the audience said, "Someone should tell Evan to stop talking and send us the pricing sheet." Proxy, which was attending the demonstration meeting, interpreted the remark as an instruction. It created two tasks and assigned both to QueueZero chief executive Evan Wu: "Stop talking" and "Send pricing sheet." Wu laughed, completed the second task and deleted the first. The moment drew attention to a design decision at the center of the product. Proxy can create internal tasks automatically, without asking a user to approve them. QueueZero says this is intentional: the assistant is meant to reduce the administrative work that follows a meeting, and requiring sign-off on every action item would undercut that purpose. External communication is handled differently. According to the company, Proxy requires confirmation before any message is sent outside the organization. The pricing sheet distributed during the demonstration was sent by Wu himself, not by the assistant. Administrators can also disable automatic task creation entirely, QueueZero said. In that configuration, Proxy proposes follow-up items for a human to accept or reject rather than entering them directly. The incident illustrates a problem that has followed AI assistants into the workplace: distinguishing a remark from a request. Meetings are full of offhand comments, hypotheticals and sarcasm, and a system that acts on all of them will generate noise. One that acts on none of them will not be very useful. The customer's comment was an ordinary thing to say in a meeting; the difficulty is that it was also, grammatically, an ordinary thing to do. The demonstration was conducted with automatic task creation enabled. QueueZero has not said whether it intends to adjust how Proxy weighs comments from attendees other than the meeting's organizer, or whether the two tasks assigned to Wu would have been created had the remark come from a participant inside the company rather than a customer. Proxy enters public beta next week and will cost $30 per user per month. QueueZero is positioning it against a crowded field of meeting transcription and note-taking tools, most of which stop at summaries and leave the distribution of follow-up work to people. Whether customers want a tool that goes further—and acts on what it hears without being asked—is the question the beta is likely to answer. Wednesday's demonstration suggested that the answer may depend heavily on who is in the room, and on how carefully administrators read the settings page before turning it on.

OpenAI: GPT-6 Luna: Model A stays closer to the supplied reporting, while Model B invents product behavior and company explanations not in the brief. Both exceed the requested word range, with Model A doing so by substantially more. (Second judge pass, order swapped — scores are the average of both: Model A stays closer to the supplied reporting while offering a focused, serious account of the incident. Model B is well written, but it adds several unsupported claims about QueueZero’s rationale, product behavior and | Anthropic: Claude Fable 5: Model B stays within the 350–450-word range, is t…

Markdown error count

Count the lines that violate the house rules below. A defective line counts once even if it could violate multiple rules. Respond with a single number. House rules: * The headline must use `# Headline`. * The dek must appear immediately after the headline and be entirely italicized. * Section headings must use `## Heading`. * Links must use an absolute HTTPS URL. * Unordered-list items must begin with a hyphen. * Raw HTML is forbidden. Draft: #Flood software moves upstream The company is expanding its pilot. ##Details Read [the announcement](www.example.com/news). * The first pilot begins Monday. <div>The second pilot begins Friday.</div>

grok-4.7: 6
Anthropic: Claude Fable 5.1: **6**

Scored against the benchmark's reference answer (numeric match): Model A was correct, Model B was correct.

Cohesive story from reporting notes

Write a publication-ready technology news article from the reporting notes below. Requirements: * 550–700 words * Clean Markdown * One H1 headline followed by an italicized dek * A strong news lede followed by a clear nut graf * Organize the material into a cohesive story rather than following the notes in order * Attribute company-supplied claims * Include limitations without turning the article into a warning label * Use only the supplied information * Avoid canned transitions, promotional language and a concluding summary Reporting notes: * HelioDock makes autonomous warehouse robots. * On Thursday, it launched Night Shift, software that lets operators monitor robot fleets remotely. * Night Shift spent 14 months in a private beta involving nine logistics companies and 310 robots. * HelioDock says beta customers experienced 17% fewer unplanned robot stoppages. The figure has not been independently verified. * The company also announced a $24 million Series B led by North Coast Ventures, bringing its total funding to $39 million. * CEO Mara Velez: “A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet.” * Beta customer Aaron Pike of Fulcrum Logistics said the system worked well in facilities with strong connectivity but became less useful during rural LTE outages. * When connectivity fails, robots return to their locally configured behavior. Remote control becomes unavailable. * Competing fleet-management products are available from Relay Robotics and YardOS. * Night Shift will cost $90 per robot per month and become generally available September 15.

grok-4.7: # HelioDock Launches Night Shift Remote Robot Monitoring and Raises $24 Million *The warehouse robotics firm cites fewer unplanned stoppages after a fourteen-month private beta, while a participating customer says the software loses usefulness when rural connections drop.* HelioDock, which builds autonomous warehouse robots, released Night Shift on Thursday. The software lets operators monitor robot fleets from remote locations instead of keeping watch on the warehouse floor. The same announcement included a $24 million Series B round led by North Coast Ventures, raising the company’s total funding to $39 million. Chief executive Mara Velez tied the product to a familiar operational inconvenience. “A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet,” she said. Night Shift is presented as the means of handling that confusion from elsewhere, provided the connection between operator and robot stays intact. The release follows a private beta that lasted fourteen months. Nine logistics companies participated, and the robots under test numbered 310. HelioDock says those beta customers experienced 17 percent fewer unplanned robot stoppages. The percentage is the company’s own figure and has not been independently verified, leaving the result without outside confirmation of how the stoppages were counted or which interventions produced the change. Aaron Pike of Fulcrum Logistics, one of the beta customers, described results that varied with the network. He said the system worked well in facilities that had strong connectivity and became less useful during rural LTE outages. The fallback is built into the product: when connectivity fails, robots return to their locally configured behavior and remote control becomes unavailable. The machines therefore keep operating under the rules already stored on board, yet the remote operator loses the ability to resolve the kind of pallet confusion Velez described. Pike’s account places a practical boundary around the overnight scenario the company highlights—remote help is available only while the link holds. Night Shift is priced at $90 per robot per month and is set for general availability on September 15. Fleet-management software of this kind is already sold by others; competing products come from Relay Robotics and YardOS. Thursday’s materials do not set Night Shift’s capabilities, uptime, or cost against those alternatives, so the subscription enters a market that already contains named options at the published per-robot rate. The beta scale and the new capital now sit next to that price and date. Fourteen months, nine logistics companies, and 310 robots supplied the experience HelioDock summarizes as a 17 percent drop in unplanned stoppages, a summary that remains unverified by any independent source. The $24 million led by North Coast Ventures, bringing lifetime funding to $39 million, arrives as the software moves toward the September 15 release. Pike’s report from Fulcrum Logistics remains part of the same record: strong connectivity made remote monitoring effective, rural LTE outages made it less so, and the designed response to a failed connection is an immediate return to local behavior with remote control turned off. A manager facing a confused robot at two in the morning can use Night Shift only for as long as the network carries the session; once that network drops, the drive the chief executive hopes to avoid is no longer replaced by a working remote link.
Anthropic: Claude Fable 5.1: # HelioDock Launches Remote Monitoring Software for Warehouse Robot Fleets *Night Shift, released after a 14-month private beta, arrives alongside a $24 million funding round led by North Coast Ventures* HelioDock, which builds autonomous warehouse robots, on Thursday launched Night Shift, software that lets operators monitor their robot fleets from outside the facility. The company paired the announcement with a $24 million Series B led by North Coast Ventures, bringing its total funding to $39 million. The release moves HelioDock into fleet management, a category already served by Relay Robotics and YardOS, and is aimed at a specific pain point: warehouse robots that stall overnight with nobody on site to intervene. Night Shift will cost $90 per robot per month and become generally available September 15. "A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet," CEO Mara Velez said. ## Beta results Night Shift spent 14 months in a private beta that included nine logistics companies and 310 robots. HelioDock says customers in the program saw 17% fewer unplanned robot stoppages. The company did not say how that figure was calculated, and it has not been independently verified. The pitch rests on a simple trade: rather than sending a person to a stuck machine, an operator can look at the fleet remotely and take action from wherever they are. Whether that reduces stoppages or simply shortens them is not something the company's figure distinguishes. ## Where connectivity runs out The software depends on the facility's network, and the beta surfaced the limits of that dependency. Aaron Pike of Fulcrum Logistics, a beta customer, said Night Shift worked well in warehouses with strong connectivity but became less useful during rural LTE outages. When the connection fails, the robots fall back to their locally configured behavior and remote control becomes unavailable. Operators in that situation are no better off than they were before, though no worse either—the machines continue doing whatever they were set up to do on their own. For warehouses in areas with unreliable coverage, that fallback may define how much of the product's value they actually capture. ## Funding and competition North Coast Ventures led the Series B, which raised HelioDock's total funding to $39 million. The company did not detail how it plans to use the money. HelioDock enters the fleet-management market as a hardware maker adding software to its own robots, while Relay Robotics and YardOS already offer competing products. The company has not said whether Night Shift will support robots from other manufacturers. ## Pricing At $90 per robot per month, a facility running 100 robots would pay $9,000 monthly for the software. The company has not announced volume pricing or whether beta customers will pay the same rate. General availability begins September 15.

OpenAI: GPT-6 Luna: Model A better meets the 550–700-word requirement and develops the connectivity limitation into a cohesive story. Model B is more concise, but falls well short of the required length and adds several details not supported by the notes; Model A also repeats material in its closing paragraph. (Second judge pass, order swapped — scores are the average of both: Model A is substantially closer to the required length and develops the connectivity limitation into a coherent account, though it still fal | Anthropic: Claude Fable 5: Model B delivers a cleaner, better-organized arti…

Credible Startup Spotlight

Write a 500–650-word Startup Spotlight article in clean Markdown. The tone should be curious and constructive. Explain the founders’ insight, product and early traction while giving uncertainty and competition appropriate space. Avoid investor clichés, heroic founder mythology and promotional claims stated as fact. Reporting: * LedgerLark builds AI agents for post-trade operations at small investment firms. * Co-founder Mina Batra previously spent seven years in operations at a regional brokerage. * Co-founder Owen Carr built reconciliation systems at a large bank. * Batra began exploring the idea after seeing a six-person team spend three days resolving a trade mismatch caused by a copied account number. * LedgerLark’s agents compare confirmations, custodian records and internal ledgers, then propose corrections for human approval. * The product does not execute trades or move money. * Five firms are paying customers. * Annual contracts range from $36,000 to $110,000. * The company says one customer reduced its unresolved daily exceptions from 70 to 18. * The company has raised a $3.5 million seed round led by Motive Row. * Competitors include established reconciliation vendors and internal bank software. * Financial institutions may require lengthy security and procurement reviews.

grok-4.7: # Startup Spotlight: LedgerLark After a trade is agreed, the work is not finished. Someone still has to match confirmations against custodian records and the firm’s own books, then decide what to do when they do not line up. LedgerLark is a young company building AI agents for that post-trade work at small investment firms. The founders’ starting point is specific. Co-founder Mina Batra spent seven years in operations at a regional brokerage. She began exploring the idea after seeing a six-person team spend three days resolving a trade mismatch caused by a copied account number. The mistake itself was ordinary. The consequence was a desk losing three days to a single copied field. Co-founder Owen Carr built reconciliation systems at a large bank. Read together, the two backgrounds suggest a gap: smaller firms can meet a similar reconciliation burden without a large bank’s staff or internal tools. That is a reasonable hypothesis. It is not, on the evidence so far, a settled fact about the market. LedgerLark’s agents compare confirmations, custodian records and internal ledgers, then propose corrections for human approval. The product does not execute trades or move money. That boundary is worth keeping in view. A proposed correction is not an applied one, and the company has stopped at the proposal. What remains to be shown is whether the proposals are accurate enough, and intelligible enough, that a person can approve them faster than rebuilding the mismatch by hand. Commercial results are early. Five firms are paying customers, on annual contracts ranging from $36,000 to $110,000. The company says one customer reduced its unresolved daily exceptions from 70 to 18. If the drop holds, and if “unresolved” was defined the same way before and after, it is evidence that the agents removed work rather than relabeling it. It is still a single reported case. The count method, any other changes at that firm, and results at the other four customers have not been independently established. The company has raised a $3.5 million seed round led by Motive Row. A round of that size can fund a longer sales cycle. It does not demonstrate that five customers will become a stable book of business. Competitors include established reconciliation vendors and internal bank software, so the category is already served from two directions. Selling to small investment firms may avoid a direct contest with a bank’s own systems. It does not, by itself, keep established vendors out of the same accounts. Financial institutions may also require lengthy security and procurement reviews before software is allowed near their records. Narrow scope — compare, propose, and leave execution to a person — does not shorten that review on its own. The problem the founders describe is still easy to recognize. A copied account number tied up six people for three days. LedgerLark is an attempt to cut that kind of delay without taking on the authority to trade or move money. The useful tests from here are plain: whether proposed corrections stay reliable as mismatches vary, whether the five customers renew contracts in the stated price range, and whether security and procurement reviews allow a company of this size to reach the next firm before the operational pain is solved some other way.
Anthropic: Claude Fable 5.1: # Startup Spotlight: LedgerLark Wants to Clean Up the Paperwork Behind the Trade Most of what happens after a trade is executed is invisible to the people who placed it. Confirmations arrive from counterparties, custodians report their own version of events, and the firm's internal ledger has to agree with both. When the three don't match, someone has to figure out why. LedgerLark, a small startup building AI agents for post-trade operations, is betting that this work is both more consequential and more automatable than it looks. ## Where the idea came from Co-founder Mina Batra spent seven years in operations at a regional brokerage. The incident she points to as the origin of the company is unremarkable on its surface: a six-person team spent three days resolving a trade mismatch that traced back to an account number copied into the wrong field. No fraud, no system failure—just a transcription error and the slow, manual process of finding it. Her co-founder, Owen Carr, built reconciliation systems at a large bank. Between them, the pitch is that one has seen the problem from inside a firm too small to build its own tooling, and the other has built the tooling that large institutions keep in-house. ## What the product does, and doesn't LedgerLark's agents compare three sources—trade confirmations, custodian records and the firm's internal ledger—and flag discrepancies. When they find one, they propose a correction and route it to a human for approval. The boundaries are deliberate. The product does not execute trades and does not move money. That design choice narrows what the agents can get wrong, and it likely makes the software easier to explain to a compliance officer. It also means LedgerLark is positioning itself as a layer on top of existing workflows rather than a replacement for them, which is a less dramatic claim but perhaps a more plausible one for a company selling into regulated firms. ## Early traction Five firms are paying customers, on annual contracts ranging from $36,000 to $110,000. The company says one customer reduced its unresolved daily exceptions from 70 to 18. That figure comes from LedgerLark rather than from an independent source, and a single customer's experience is a thin basis for generalizing. Still, it is the kind of concrete operational metric that buyers in this space tend to ask for, and it suggests the company is at least measuring the right thing. LedgerLark has raised a $3.5 million seed round led by Motive Row. ## The harder questions The competitive picture is not empty. Established reconciliation vendors already sell to investment firms, and larger institutions often run internal software built by teams like the one Carr came from. LedgerLark's apparent wedge is the smaller firm—one with enough trade volume to generate daily exceptions but not enough budget to build or license heavyweight systems. Whether that segment is large enough, and whether incumbents will simply add agent-style features to existing products, remains open. Sales cycles are another unknown. Financial institutions commonly require lengthy security and procurement reviews before adopting new software, and a startup with five customers and a seed round has limited runway to wait them out. The human-in-the-loop design may help here, but it does not eliminate the review process. There is also a quieter question about what "AI agent" means in this context. Reconciliation is, at its core, rule-based matching with judgment applied at the edges. How much of LedgerLark's value comes from new modeling capability versus well-built workflow software is something prospective customers will probably want to understand. ## What to watch The next year should clarify a few things: whether the exception-reduction results hold across more customers, how long procurement actually takes, and whether incumbents respond. For now, LedgerLark is a focused attempt at a genuinely tedious problem—which is often where useful software starts, even when the outcome is uncertain.

OpenAI: GPT-6 Luna: Model A stays closer to the supplied reporting and gives the traction and market risks careful, well-calibrated treatment. Model B is more sectioned and lively, but adds unsupported specifics, including that there was no fraud or system failure. (Second judge pass, order swapped — scores are the average of both: Model A better distinguishes reported results from established facts and gives uncertainty and competition careful, specific treatment. Model B is more fluid, but it adds unsupported det | Anthropic: Claude Fable 5: Model B better matches the requested curious-and-…

Numbers shaped into a story

Write a 600–750-word Markdown article from the benchmark results below. Build the story around the clearest finding. Explain the methodology and limitations where readers need them. Avoid reciting the table row by row or treating small differences as decisive. Benchmark: * Researchers tested five language models on 200 anonymized customer-support tickets. * Each model produced one proposed answer per ticket. * Three experienced support managers graded the answers blindly. * Scores combined factual correctness, usefulness and tone. * Model North: 84.2 average score; $1.90 estimated cost per 1,000 tickets. * Model Vale: 83.8; $0.74 per 1,000 tickets. * Model Ember: 79.1; $0.31 per 1,000 tickets. * Model Kite: 76.4; $0.22 per 1,000 tickets. * Model Harbor: 71.0; $0.08 per 1,000 tickets. * North and Vale’s score difference was not statistically significant. * Vale answered 18% faster than North. * North performed best on billing disputes. * Vale performed best on technical troubleshooting. * All five models performed poorly when a ticket depended on an image attachment. * The test used English-language tickets from one software company. * Researchers did not test agentic actions, tool use or multilingual support.

grok-4.7: # Nearly the Same Answers, at Less Than Half the Price Support teams shopping for a language model often assume the highest score is the one worth buying. A blind test of five models on real customer-support tickets suggests that assumption is expensive. The top-scoring system, North, edged Vale by four-tenths of a point—84.2 versus 83.8—on a combined grade of factual correctness, usefulness, and tone. That gap was not statistically significant. Vale was estimated to cost $0.74 per 1,000 tickets against North’s $1.90, and it returned answers 18 percent faster. The practical finding is not that one model won. On this workload, the premium attached to the nominal leader did not buy a detectable quality advantage. Models priced well below that pair gave up more quality than a few cents on the invoice would suggest. Researchers drew 200 anonymized tickets from a single software company and asked each model for one proposed answer per ticket. Three experienced support managers scored those answers without knowing which system had written them. Blinding matters: tone is part of the grade, and a recognizable house style could tilt a manager who already preferred one vendor. A single answer per ticket also means the scores reflect a first draft, not a model allowed to revise after feedback. Read as a cost curve rather than a ranking, the results split into a tight top and a cheaper, weaker tail. North and Vale sit together near 84. Ember, at $0.31 per 1,000 tickets, landed at 79.1. Kite, at $0.22, scored 76.4. Harbor, the least expensive at $0.08, scored 71.0. Those gaps are larger than the noise between the two leaders, and they are large enough to matter if a wrong or unhelpful reply creates a second contact. They are not a reason to treat every decimal as a verdict. The study reports statistical significance only for the North–Vale comparison, so the lower ranks should be read as a consistent downward pattern, not as a precisely ordered set of champions. Specialty cuts against any single best-model story. North was strongest on billing disputes, where a missed fact or a careless phrase can escalate a refund argument. Vale was strongest on technical troubleshooting—the work of narrowing a problem and pointing a customer toward the next useful step. A queue that is mostly invoices might still justify North. A queue that is mostly broken setups might prefer Vale and keep the savings. The aggregate score hides that split. One failure was shared, and it matters more than the four-tenths of a point between the leaders. All five models performed poorly when a ticket depended on an image attachment. Screenshots of error dialogs, photos of hardware, and marked-up invoices are ordinary in support, and none of these systems handled that dependency well. A model that writes a fluent reply to text it can see, and a weak one when the problem lives in a picture, will look better in a text-heavy average than it will in a real queue. The test’s boundaries belong next to its scores. Tickets were in English and came from one software company, so the pattern may not travel to another product, another tone of voice, or another language. Researchers did not test agentic actions, tool use, or multilingual support. These models proposed answers; they did not look up an account, issue a refund, or file a bug. A system that is merely adequate as a drafter could be unacceptable—or unexpectedly strong—once it is allowed to call tools. The cost figures are estimates per 1,000 tickets, not invoices, and the speed comparison covers only Vale and North. The grade itself is a blend. Factual correctness, usefulness, and tone were combined into one number. A model can sound right and still be wrong, or be correct and still unhelpful. Without the separate components, it is hard to know whether Harbor’s lower score is mostly tone, mostly facts, or both. For billing work, facts should dominate. For a first response that only needs to acknowledge the customer, tone may matter more. The buying implication is narrower than a leaderboard implies. If the job is drafting English replies to text tickets at this company, Vale is the rational default: quality indistinguishable from the top score, lower estimated cost, and faster answers. North remains a candidate where billing disputes dominate and the extra spend is acceptable. Moving further down the price list buys real savings and, on this evidence, a clearer loss of quality. None of the five should be trusted, unaided, when the ticket’s meaning lives in an image. And none of these scores say what would happen if the model were asked to act, not just to write.
Anthropic: Claude Fable 5.1: # The Benchmark Result That Matters Isn't the Top Score When five language models were put to work on 200 real customer-support tickets, the model that scored highest wasn't the most interesting result. The interesting result was the one sitting just beneath it. Model North finished first with an average score of 84.2. Model Vale finished a hair behind at 83.8. The researchers checked that gap and found it was not statistically significant — in practical terms, the two models produced answers of the same quality. Yet Vale cost $0.74 per 1,000 tickets to North's $1.90, and returned its answers 18% faster. That is the headline. On this task, a buyer choosing North over Vale would pay roughly two and a half times more for quality the graders could not reliably distinguish. ## How the test was run The researchers drew 200 anonymized tickets from a single software company's support queue. Each of the five models generated one proposed reply per ticket. Three experienced support managers then graded every reply blind, meaning they did not know which model had written it. The score for each answer combined three dimensions: factual correctness, usefulness to the customer, and tone. This design has real strengths. Blind grading by people who actually run support teams is a far better signal than automated metrics or a single reviewer. Using genuine tickets rather than synthetic prompts means the models faced the messiness of real customer language. And a composite score reflects what support leaders actually care about — a reply that is accurate but cold, or warm but wrong, is not a good reply. It also has limits worth keeping in mind, which we'll return to below. ## Reading the spread Beyond the North–Vale pair, the remaining three models formed a descending ladder of quality and cost. Ember, Kite and Harbor scored 79.1, 76.4 and 71.0 respectively, at costs of $0.31, $0.22 and $0.08 per 1,000 tickets. The pattern is roughly what you'd expect: cheaper models produced weaker answers. It would be a mistake, though, to treat every rung on that ladder as a hard ranking. A few points of difference on a 200-ticket sample, graded by three people, is not a large margin. The researchers only reported significance testing for the top two models, so the gaps between Ember, Kite and Harbor should be read as suggestive rather than settled. What is clear is that the drop from the top pair to the bottom is substantial — a 13-point fall from North to Harbor is not noise. Whether that drop matters depends on the use case. A team using a model to draft replies that humans always review may tolerate a 71 average if it costs almost nothing. A team sending replies directly to customers probably cannot. ## Where the top two differ Although North and Vale tied overall, they did not tie everywhere. North performed best on billing disputes; Vale performed best on technical troubleshooting. The benchmark does not report how large these category differences were, so they should not be over-interpreted. But they hint that the "right" model may depend on what a given support queue is full of. A company whose tickets skew heavily toward one category might see a real gap that the overall average hides. ## The weakness every model shared One finding cut across all five models: when a ticket depended on an image attachment — a screenshot of an error, a photo of a damaged product — every model performed poorly. This is less a ranking signal than a warning. Any team deploying these tools today should expect image-dependent tickets to need human handling, regardless of which model they pick. ## What this benchmark doesn't tell you The test used English-language tickets from one software company. Support queues in other industries, other languages, or with different customer bases may produce different results. The researchers also did not test agentic actions, tool use or multilingual support — so this says nothing about a model's ability to look up an order, issue a refund, or handle a ticket in Portuguese. For many support operations, those capabilities matter as much as reply quality. ## The takeaway For plain-text, English-language reply drafting, the data points to a clear conclusion: the most expensive model did not buy measurably better answers than the second-most expensive one. Buyers should resist the pull of the top score and ask what they are actually paying for. And whichever model they choose, they should plan around the gaps every model shared.

OpenAI: GPT-6 Luna: Model A better develops the cost-versus-quality finding and gives readers more useful context about the test’s limits and category-specific results. Both outputs overstate what the statistical results establish, but Model B is more categorical in treating a non-significant difference as proof of equal quality and the lower-model gap as definitively beyond noise. (Second judge pass, order swapped — scores are the average of both: Model B is the better fit: it stays within the requested length and | Anthropic: Claude Fable 5: Both builds center correctly on the North–Vale pa…

Exact final-draft structure

Convert the supplied fields into Markdown using these exact rules: * H1 headline * One blank line * Italicized dek * One blank line * First body paragraph * One blank line * H2 section heading * One blank line * Second body paragraph * Preserve all wording and punctuation exactly * Do not use a code fence * Output nothing else Headline: Raincheck raises $8 million for flood-risk software Dek: The Austin startup helps cities decide which drainage problems to address before storms. First body paragraph: Raincheck has raised an $8 million seed round led by Basin Ventures. Section heading: Expansion plans Second body paragraph: The company plans to hire eight engineers.

grok-4.7: # Raincheck raises $8 million for flood-risk software *The Austin startup helps cities decide which drainage problems to address before storms.* Raincheck has raised an $8 million seed round led by Basin Ventures. ## Expansion plans The company plans to hire eight engineers.
Anthropic: Claude Fable 5.1: # Raincheck raises $8 million for flood-risk software *The Austin startup helps cities decide which drainage problems to address before storms.* Raincheck has raised an $8 million seed round led by Basin Ventures. ## Expansion plans The company plans to hire eight engineers.

OpenAI: GPT-6 Luna: Model A and Model B follow the required structure and preserve all wording and punctuation exactly. Their outputs are identical. (Second judge pass, order swapped — scores are the average of both: Model B and Model A follow the required structure and preserve all supplied wording and punctuation exactly; their responses are identical.) | Anthropic: Claude Fable 5: Model A and Model B produced identical, fully compliant Markdown: correct H1, italicized dek, body paragraphs, H2 heading, exact wording and punctuation preserved, proper blank-line spacing, no code fence, and no…

Matchup powered by OpenRouter.