Head to head: Google: Gemini 4 Carbon vs Anthropic: Claude Sonnet 5.5

Google: Gemini 4 Carbon vs Anthropic: Claude Sonnet 5.5

By · Published · Updated

RuntimeWire Head-to-Head: Head to head: Google: Gemini 4 Carbon vs Anthropic: Claude Sonnet 5.5
RuntimeWire Head-to-Head matchup

A head-to-head on turning supplied reporting into readable, accurate stories: one model more often built a stronger narrative, while the other showed sharper restraint around what the evidence could support.

Gemini 4 Carbon leads the saved score, **102.0 to 96.4**, with five task wins to Claude Sonnet 5.5’s three and four ties. The gap reflects a modest editorial advantage, not a clean sweep: the models tied on the Markdown edit, strongest-lede selection, Markdown error count and exact final-draft structure. Gemini’s wins came largely in story craft and execution. It did better on chronology, weaving quotes into a narrative, controlled wit, building a cohesive story from reporting notes and meeting the requested length in a natural rewrite. But several of those evaluations also flagged unsupported detail or overreach, tempering the polish. Claude’s strongest work came where evidentiary care mattered most: shaping benchmark numbers, covering a startup credibly and choosing the opening angle for a news story. It was more careful about what the figures established, procurement and competition risks, and the limits of comparing session counts with registered users. That restraint sometimes came at a cost: in the natural rewrite, Claude fell well short of the requested word count. **Call: Gemini 4 Carbon, slight edge (not statistical).** It more consistently delivered the requested narrative shape, while Claude was the safer hand on claims and qualifications. This assessment applies to the saved task sample.

Publication-ready Markdown edit

Edit the copy below into a publication-ready Markdown story. Requirements: * Supply an H1 headline and italicized dek * Repair the lede and paragraph flow * Use a Markdown blockquote only for the extended statement * Convert the supplied URL into a descriptive inline link * Remove internal notes and formatting debris * Preserve the reporting and its meaning * Output only the finished article Raw copy: HEADLINE TBD The funding was announced today. Raincheck is the company. It has raised $8 million and it makes flood-risk software for local governments. [NOTE: move this up?] The Austin company maps drainage systems and combines those maps with rainfall forecasts. Cities use the software to decide where to send maintenance crews before a storm. The seed round was led by Basin Ventures. Existing investor Long Mile also participated. Raincheck has raised $10.5 million total. CEO Ana Flores said the following: “Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives.” https://example.com/raincheck-methodology Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them because some contracts have not been publicly announced. Its forecasts are recommendations. Emergency managers retain control over deployments and public warnings. THE COMPETITION Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck says the new funding will help it hire eight engineers and expand into Florida.

Google: Gemini 4 Carbon: # Raincheck Raises $8 Million to Help Cities Tackle Pre-Storm Flood Risks *The Austin startup pairs drainage maps with rainfall forecasts so local governments can prioritize maintenance before severe weather arrives.* Raincheck, an Austin-based maker of flood-risk software for local governments, announced today that it has raised an $8 million seed round. The company [maps drainage systems and combines them with rainfall forecasts](https://example.com/raincheck-methodology) to help cities decide where to send maintenance crews before a storm. Basin Ventures led the round, with participation from existing investor Long Mile. The new investment brings Raincheck’s total funding to $10.5 million. According to CEO Ana Flores, the software is designed to help local officials prioritize urgent infrastructure work ahead of severe weather: > “Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives.” The platform's forecasts serve only as recommendations, leaving emergency managers in control of deployments and public warnings. Raincheck currently works with seven cities across Texas and Louisiana, though the company declined to identify them because some contracts have not yet been publicly announced. Competing alongside municipal engineering contractors and other flood-planning vendors such as HydroMap, Raincheck says the fresh funding will help it hire eight engineers and expand into Florida.
Anthropic: Claude Sonnet 5.5: # Raincheck Raises $8 Million to Help Cities Prepare for Floods Before the Rain Arrives *The Austin company's software maps drainage systems and pairs them with rainfall forecasts to guide maintenance crews ahead of storms.* Raincheck, an Austin company that makes flood-risk software for local governments, has raised $8 million in a seed round, the company announced today. The software maps drainage systems and combines those maps with rainfall forecasts. Cities use it to decide where to send maintenance crews before a storm. Basin Ventures led the round, and existing investor Long Mile also participated. Raincheck has raised $10.5 million in total. CEO Ana Flores said the challenge is not identifying where flooding happens, but deciding what to fix first: > "Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives." More detail on the company's approach is available in its [methodology documentation](https://example.com/raincheck-methodology). Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them because some contracts have not been publicly announced. Its forecasts are recommendations, and emergency managers retain control over deployments and public warnings. Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck says the new funding will help it hire eight engineers and expand into Florida.

OpenAI: GPT-6 Luna: Model A has smoother paragraph flow and integrates the funding, product, and competition details into a more cohesive story. Model B is also accurate and follows the requirements, but its more segmented presentation is slightly less polished. (Second judge pass, order swapped — scores are the average of both: Model B preserves the supplied reporting with minimal interpretation and uses the URL as a clearly labeled methodology link. Model A is also polished, but its linked sentence implies the me | Anthropic: Claude Fable 5.1: Model A delivers a smoother, more publication-r…

Chronology with narrative momentum

Turn the scrambled reporting below into a 450–600-word Markdown news article. Choose an opening that captures the consequence of the failure, then reconstruct the sequence clearly. Preserve the chronology without producing a minute-by-minute incident log. Give technical details enough context for a general reader. Reporting: * KiteBridge makes expense-management software. * At 8:45 a.m. Tuesday, it deployed version 4.2. * The first customer complaint arrived at 9:12 a.m. * A payment-processor timeout caused KiteBridge’s fallback system to treat retries as new reimbursement requests. * The software generated 13,400 duplicate reimbursements totaling $2.8 million. * KiteBridge suspended automated payments at 10:03 a.m. * Ninety-one corporate customers were affected. * By Wednesday evening, $2.3 million had been recovered. The rest remained pending. * No employees had money withdrawn from their accounts; employers funded the duplicate payments. * An April changelog said version 4.0 had introduced “exactly-once payment protection.” * KiteBridge’s public incident page initially described the problem as delayed reimbursements. * The incident page first disclosed the duplicate payments six hours after automation was suspended. * CEO Lena Ortiz apologized and said the company would publish a technical report.

Google: Gemini 4 Carbon: # Software Glitch Triggers $2.8 Million in Duplicate Payouts for KiteBridge Clients Ninety-one corporate customers unexpectedly funded $2.8 million in erroneous payouts this week after a software update from expense-management provider KiteBridge went awry, generating 13,400 duplicate employee reimbursements in little more than an hour. KiteBridge builds software that businesses rely on to automate and pay out worker expense claims. Because employers funded the duplicate payments directly, corporate clients absorbed the immediate financial strain of the Tuesday morning malfunction. While millions of dollars in extra payouts were issued, no employees had money withdrawn from their accounts as a result of the error or the subsequent recovery process. The breakdown began at 8:45 a.m. Tuesday with the deployment of KiteBridge version 4.2. Soon after the update went live, the platform encountered a payment-processor timeout, a common hiccup in digital finance in which an external processing network takes too long to confirm whether a transfer succeeded. When timeouts occur, payment software typically retries the connection solely to check the status of the original transaction. During Tuesday’s disruption, however, KiteBridge’s fallback system, a backup mechanism meant to handle processing snags, treated those automated retries as brand-new reimbursement requests and triggered fresh payouts each time. Signs of trouble surfaced less than half an hour into the rollout, when the first customer complaint arrived at 9:12 a.m. Even so, erroneous transfers continued to mount across client accounts before engineers intervened. KiteBridge ultimately suspended automated payments at 10:03 a.m., halting the runaway payouts more than an hour after version 4.2 was deployed. Although the company stopped its automated payment engine mid-morning, its public transparency lagged well behind the crisis. KiteBridge’s public incident page initially described the problem as delayed reimbursements, an explanation that suggested funds were merely running late rather than going out multiple times. Customers monitoring the page were left without an accurate picture of the outage until late afternoon: the incident page first disclosed the duplicate payments six hours after automation was suspended. The malfunction also undercut a key reliability promise in the software maker’s recent history. According to an April changelog, KiteBridge’s version 4.0 release had introduced “exactly-once payment protection.” In financial technology, that type of safeguard is specifically built to ensure a single approved expense cannot trigger multiple payouts, even when network timeouts force a system to resend a request. Tuesday’s cascade of duplicate transfers showed that when KiteBridge’s fallback system stepped in under the new update, that protection failed to prevent repeat disbursements. Following Tuesday’s emergency shutdown, efforts shifted to recouping the misdirected corporate funds. By Wednesday evening, $2.3 million of the $2.8 million total had been recovered, leaving the remaining $500,000 in duplicate reimbursements still pending resolution for affected employers. KiteBridge Chief Executive Lena Ortiz apologized for the failure and the disruption inflicted on the businesses that use the platform. Facing scrutiny over how the flaw slipped through and bypassed the platform’s stated safeguards, Ortiz said the company would publish a technical report examining the incident.
Anthropic: Claude Sonnet 5.5: # KiteBridge Software Error Sent $2.8 Million in Duplicate Reimbursements to 91 Companies Expense-management company KiteBridge mistakenly issued 13,400 duplicate reimbursements worth $2.8 million on Tuesday. The error affected 91 corporate customers, and about $500,000 was still unrecovered Wednesday evening. The money came from employers, not workers. No employees had funds withdrawn from their accounts, but the companies that fund reimbursements paid out the extra money. ## What happened KiteBridge deployed version 4.2 of its software at 8:45 a.m. Tuesday. The first customer complaint arrived at 9:12 a.m. According to the reporting, the trouble began when a payment processor, the outside service that moves money between accounts, timed out. A timeout means a system did not receive a response in the expected window. In such cases, software typically retries the request, and the retry should be recognized as a repeat of the original payment. KiteBridge's fallback system, which takes over when the main payment path fails, did not do that. It treated the retries as new reimbursement requests, and each one generated another payment. The result was the 13,400 duplicates. KiteBridge suspended automated payments at 10:03 a.m., about 51 minutes after the first complaint. ## A safeguard in question An April changelog, the public record of changes to KiteBridge's software, said version 4.0 had introduced "exactly-once payment protection." The phrase describes a design goal in payment systems: each payment should be processed one time, even if a request is sent repeatedly. The reporting does not establish why that protection did not stop the duplicates, or how version 4.2 relates to the failure. Those questions are likely to be central to the technical report the company has promised. ## Communication under scrutiny KiteBridge's public incident page initially described the problem as delayed reimbursements. That description did not match what was happening: some payments were being sent more than once, not held up. The page first disclosed the duplicate payments about six hours after automation was suspended. During that period, customers relying on the page would not have known that extra money was going out. ## Recovery and response By Wednesday evening, $2.3 million of the $2.8 million had been recovered, which is about 82 percent. The remainder was still pending. CEO Lena Ortiz apologized and said the company would publish a technical report. KiteBridge has not yet released it, and it is unclear when it will appear. ## What to watch The report may answer several open questions: - How the fallback system came to treat retries as new requests - Whether the version 4.2 deployment played a role - Why the safeguard described in the April changelog did not prevent the duplicates - Why the incident page described the problem as delayed reimbursements for hours For the 91 affected companies, the immediate concern is getting back the remaining money. The broader question is whether KiteBridge can explain the failure well enough to restore confidence in its automated payments.

OpenAI: GPT-6 Luna: Model A gives the failure a stronger narrative arc, explains the technical cause clearly, and reads like a cohesive news article. Model B is organized and informative, but its “What to watch” section and bullet list make it feel more like an analysis brief than the requested article. (Second judge pass, order swapped — scores are the average of both: Model B is more careful about what the reporting establishes, especially regarding the relationship between version 4.2 and the “exactly-once” safe | Anthropic: Claude Fable 5.1: Model A delivers a cohesive narrative article w…

Quotes woven into narrative

Write a 400–500-word Markdown story about the closing of Foundry Hall. Use quotations selectively. Paraphrase when the information matters more than the speaker’s exact phrasing. Avoid stacking quotes or giving every source an equal-sized turn. The story should carry an emotional current while remaining restrained. Facts: * Foundry Hall, a nonprofit startup accelerator in Milwaukee, will close October 31. * It opened in 2018. * It has supported 212 startups, which collectively employ 1,140 people. * Its largest corporate sponsor, Blank River Bank, supplied 38% of its annual budget. * The bank declined to renew its sponsorship. * Foundry Hall has enough money to operate through October but not through another full program. * Six employees will lose their jobs. * Twelve startups currently using its workspace must leave. Quotes: * Executive director Camille Ross: “We kept waiting for the replacement check that never came.” * Founder Dev Malik: “My first investor meeting happened in that building. Before Foundry Hall, I was pitching from the back table of a coffee shop.” * Blank River Bank spokesperson: “The decision reflects a change in our community investment priorities and is unrelated to Foundry Hall’s performance.” * Deputy mayor Susan Cole: “The city is speaking with building owners and local universities about space for the current companies.”

Google: Gemini 4 Carbon: # Milwaukee’s Foundry Hall Prepares to Lock Its Doors for Good For the twelve fledgling companies currently working out of Foundry Hall, the autumn months have turned into an unexpected scramble for cardboard boxes and new short-term leases. The Milwaukee nonprofit startup accelerator, a fixture of the local business ecosystem since 2018, will permanently close its doors on October 31. The shutdown follows a sudden chasm in the organization’s ledger. Blank River Bank, Foundry Hall’s largest corporate sponsor and the source of 38 percent of its annual budget, declined to renew its backing. A bank spokesperson indicated that the withdrawal stemmed from shifting community investment priorities rather than any issue with the accelerator’s performance. Still, the departure left a financial gap too wide to bridge in time. While Foundry Hall holds enough cash reserves to operate through October, leadership determined there was not enough money left to responsibly launch another full program, leaving an orderly wind-down as the only viable path. Executive director Camille Ross spent months courting new donors to salvage the organization, hoping another major backer might step forward before the financial runway disappeared. “We kept waiting for the replacement check that never came,” Ross said. The closure will cost six employees their jobs at the end of the month. Meanwhile, municipal officials are stepping in to soften the disruption for the displaced entrepreneurs. Deputy Mayor Susan Cole noted that the city is talking with local universities and building owners to secure workspace for the twelve current companies that must now vacate the premises. Since opening, Foundry Hall quietly became a vital engine for Milwaukee’s economy. The organization guided 212 startups through their earliest, most fragile stages; collectively, those alumni companies employ 1,140 people. Beyond structured programming, the accelerator offered immediate legitimacy to founders who otherwise lacked a professional foothold in the city. For many entrepreneurs, that legitimacy was tangible. Founder Dev Malik recalled how having access to a dedicated conference room, rather than noisy public venues, altered the early trajectory of his business. “My first investor meeting happened in that building. Before Foundry Hall, I was pitching from the back table of a coffee shop,” Malik said. In these final weeks, the accelerator’s workspace remains active, though the usual hum of ambition is tempered by the quiet logistics of packing up. Whiteboards once reserved for product roadmaps and pitch revisions now track moving deadlines. Come November, the building will empty out, leaving behind a network of more than a thousand workers whose livelihoods trace back to its rooms, and an unanswered question about where Milwaukee’s next fledgling founders will go.
Anthropic: Claude Sonnet 5.5: # Foundry Hall, Milwaukee Startup Accelerator, Will Close October 31 Foundry Hall, the nonprofit startup accelerator that has operated in Milwaukee since 2018, will close its doors on October 31. The closure follows the loss of its largest corporate sponsor. Blank River Bank, which supplied 38% of Foundry Hall's annual budget, declined to renew its sponsorship. The organization has enough money to operate through October but not to run another full program. Executive director Camille Ross said the staff spent months hoping the gap would be filled. "We kept waiting for the replacement check that never came," she said. Six employees will lose their jobs. Twelve startups that currently use the workspace will have to leave. A Blank River Bank spokesperson said the decision reflected a change in the bank's community investment priorities and had nothing to do with how Foundry Hall performed. The accelerator's record would seem to support that. Over its lifetime, Foundry Hall has supported 212 startups, which together employ 1,140 people. For some of those founders, the building was where their companies first became real. Dev Malik remembers it that way. He said his first investor meeting took place at Foundry Hall, in a space that gave the company a legitimacy it hadn't yet earned on its own. "My first investor meeting happened in that building," Malik said. "Before Foundry Hall, I was pitching from the back table of a coffee shop." He did not describe the closure as an abstraction. For him, he said, it means a place that took early-stage ideas seriously will no longer be there for the next person with a laptop and a half-finished deck. The immediate question is where the twelve current companies will go. Deputy Mayor Susan Cole said the city is talking with building owners and local universities about finding space for them. She did not say how soon an arrangement might be reached, and the companies have until the end of the month to move. Ross has not said what will happen to the programs, advisers, and relationships Foundry Hall built, many of which depended on the building as much as on the people in it. Those pieces will not move as easily as desks and monitors. In the weeks ahead, the staff will do what an accelerator does: help companies prepare for what comes next. This time the work includes their own departure. Six people who spent their days helping others find footing will be looking for their own. Foundry Hall's final weeks will look much like its others. Founders will come in to work, and meetings will still be held at the same tables. The difference is that everyone will know the date. The doors close October 31.

OpenAI: GPT-6 Luna: Model A builds a more cohesive narrative and weaves the quotes into it effectively, while Model B relies more on speculative claims about what Ross and Malik thought or said beyond the supplied material. Both meet the length requirement, though both add some unsupported details. (Second judge pass, order swapped — scores are the average of both: Model B is more restrained and stays closer to the supplied facts, while Model A adds more unsupported details, including donor-courting efforts and a c | Anthropic: Claude Fable 5.1: Both pick the same two quotes to run verbatim a…

Numbers shaped into a story

Write a 600–750-word Markdown article from the benchmark results below. Build the story around the clearest finding. Explain the methodology and limitations where readers need them. Avoid reciting the table row by row or treating small differences as decisive. Benchmark: * Researchers tested five language models on 200 anonymized customer-support tickets. * Each model produced one proposed answer per ticket. * Three experienced support managers graded the answers blindly. * Scores combined factual correctness, usefulness and tone. * Model North: 84.2 average score; $1.90 estimated cost per 1,000 tickets. * Model Vale: 83.8; $0.74 per 1,000 tickets. * Model Ember: 79.1; $0.31 per 1,000 tickets. * Model Kite: 76.4; $0.22 per 1,000 tickets. * Model Harbor: 71.0; $0.08 per 1,000 tickets. * North and Vale’s score difference was not statistically significant. * Vale answered 18% faster than North. * North performed best on billing disputes. * Vale performed best on technical troubleshooting. * All five models performed poorly when a ticket depended on an image attachment. * The test used English-language tickets from one software company. * Researchers did not test agentic actions, tool use or multilingual support.

Google: Gemini 4 Carbon: # Top Customer-Support AI Doesn’t Require Top Dollar, Benchmark Finds Companies shopping for artificial intelligence to assist their customer-support teams often assume they must pay flagship prices to get flagship quality. A new benchmark evaluating five language models challenges that assumption. In blind evaluations of real-world support inquiries, a mid-priced contender matched the highest-scoring model in overall quality while answering 18% faster and cutting costs by more than 60%. That standout performer, Model Vale, earned an average score of 83.8 at an estimated cost of $0.74 per 1,000 tickets. Model North posted a nominally higher 84.2, but at an estimated $1.90 per 1,000 tickets—more than two and a half times Vale’s price. Crucially, researchers found that the narrow 0.4-point difference between the two leaders was not statistically significant. For support organizations operating at high volume, the clearest takeaway is that top-tier drafting quality does not require paying top-of-market rates. ## Inside the Blind Evaluation To measure how these models perform under realistic conditions, researchers tested all five systems on 200 anonymized customer-support tickets from a single software company. Each model was given one shot per ticket to produce a proposed answer, testing its ability to draft a reliable response on the first try. Rather than relying on automated benchmarks or having another AI grade the homework, the study put evaluation in the hands of three experienced support managers. Working blindly so model identities could not bias their scores, the managers graded every proposed reply on a combination of factual correctness, usefulness, and tone. That three-part rubric reflects the daily reality of customer service, where a polite answer is worthless if its facts are wrong, and an accurate answer still fails if it alienates a frustrated user. ## Where Specialization Beats Picking One Winner Although North and Vale finished in a statistical dead heat overall, closer inspection reveals distinct domain strengths that point toward smarter ticket routing. Vale performed best on technical troubleshooting, where its 18% speed advantage over North also helps teams resolve urgent product hiccups more briskly. North, by contrast, performed best on billing disputes—high-stakes interactions where nuance and precision around charges matter most. Instead of locking an entire help desk into a single model, support leaders could take a hybrid approach: routing general queues and technical troubleshooting to the faster, far cheaper Vale while reserving the pricier North specifically for billing disputes. ## The Price-Performance Cliff Below the Leaders Below the two frontrunners, the benchmark illustrates a real trade-off between ultra-low pricing and output quality. Teams looking to trim their AI spend to pennies per thousand tickets have options, but the step down in manager grades is unmistakable. Model Ember (averaging 79.1 at $0.31 per 1,000 tickets) and Model Kite (76.4 at $0.22) occupy a middle tier, cutting costs well below Vale’s already modest rate while sacrificing several points in overall quality. At the bottom of the price scale, Model Harbor costs an estimated $0.08 per 1,000 tickets—nearly 24 times cheaper than North—but saw its average score drop to 71.0. While dropping from $1.90 to $0.74 cost nothing measurable in quality, pushing below fifty cents per thousand tickets brought clear declines across correctness, usefulness, or tone. ## Shared Blind Spots and Unanswered Questions Even the strongest models hit a wall when text alone could not tell the whole story. All five models performed poorly whenever a ticket depended on an image attachment. For software support teams—where customers routinely attach visuals rather than typing out what they see—any ticket carrying an image remains a strong candidate for direct human handling rather than AI drafting. Buyers should also weigh the study’s boundaries before extrapolating these scores to every workflow. Because the benchmark drew exclusively on English-language tickets from one software company, results may not translate cleanly to multilingual queues or industries like retail and logistics. Just as important, researchers tested pure answer drafting, not tool use or agentic actions. Modern AI roadmaps increasingly promise systems that can look up account records, process refunds, or modify subscriptions autonomously—capabilities this test did not measure. Within the scope of drafting frontline English replies, however, the evidence is decisive: rigorous human grading shows that smart model selection can slash AI support costs by more than half without shortchanging the customer.
Anthropic: Claude Sonnet 5.5: # The Most Expensive AI Support Model Didn't Win in a Meaningful Way **In a blind test of five language models on real support tickets, the top two finished in a statistical tie, even though one costs more than twice as much as the other.** Teams choosing a language model to draft customer-support replies face a familiar trade-off: pay more for better answers, or pay less and accept worse ones. A new benchmark of five models suggests that at the top of the market, the trade-off may disappear. The pricier option bought no measurable quality advantage. ## The headline result Model North posted the highest average score, 84.2. Model Vale scored 83.8. The researchers report that this gap was not statistically significant, so the data gives no good reason to think North writes better answers than Vale in general. The costs differ sharply. North's estimated price is $1.90 per 1,000 tickets, while Vale's is $0.74, about 61% less. Vale also answered 18% faster. For a team handling a million tickets, that is roughly $1,900 against $740 in model costs, with no demonstrated quality gain for the higher figure. In absolute terms, these sums are small next to support staffing costs. But the pattern matters for anyone scaling up, and it shows that the highest price tag didn't buy a clearly better product. ## How the test worked The researchers collected 200 anonymized customer-support tickets. Each of the five models wrote one proposed answer per ticket. Three experienced support managers then graded the answers blindly, without knowing which model produced which. Scores combined three things: factual correctness, usefulness and tone. Blind grading by people who do this work for a living is a strength. Reviewers judged the answers as a manager would judge a colleague's draft, rather than relying on an automated metric. ## Where the rest of the field landed The lower-priced models scored progressively lower. Ember reached 79.1 at $0.31 per 1,000 tickets, Kite 76.4 at $0.22, and Harbor 71.0 at $0.08. Harbor is by far the cheapest, but its score trails the leaders by more than 13 points. Whether that gap matters depends on the job. A drafting tool where a human agent reviews every reply might tolerate a weaker model. A workflow that sends answers straight to customers probably can't. The benchmark doesn't settle that question, and the right answer depends on what a bad reply costs a given business. ## Specialties, with caution The breakdown by ticket type adds some nuance. North performed best on billing disputes, and Vale performed best on technical troubleshooting. That could point toward routing different ticket types to different models, but it should be treated as a hypothesis. With 200 tickets spread across several categories, each category holds only a fraction of the sample, and the summary doesn't say whether these category-level differences were statistically significant. Given that the overall North–Vale gap was not, small differences within categories deserve skepticism until someone retests them with more data. ## The shared weakness: images One finding applied to every model. All five performed poorly when a ticket depended on an image attachment, such as a screenshot of an error or a photo of a billing statement. Price and overall rank made no difference here. Any team that receives many tickets with screenshots shouldn't assume the headline scores reflect what it will see. Those tickets likely need human handling or a different pipeline. ## What the benchmark can't tell us The limits are significant: - **One company, one language.** The tickets were in English and came from a single software company. Results could differ for other industries, products or customer bases. - **No multilingual testing.** Many support operations serve customers in several languages, and this benchmark says nothing about that. - **No agentic work.** The researchers did not test agentic actions or tool use, such as issuing a refund, looking up an account or changing a setting. Drafting a good reply is a narrower task than resolving a case. - **One answer per ticket.** Each model produced a single response per ticket, so the test doesn't show how consistent the models are across repeated attempts. - **Estimated costs.** The prices are estimates. Real spending will depend on ticket length, prompt design and vendor pricing, which can change. ## The takeaway The clearest lesson is about value, not a winner. On this test, Vale delivered quality statistically indistinguishable from North's at well under half the price and with faster responses. Unless a team's workload is dominated by billing disputes, where North looked strongest, the cheaper of the two looks like the more sensible default. That conclusion comes with the caveats above. Teams should rerun a comparison like this on their own tickets, in their own languages, including the image-heavy ones, before committing. The benchmark's most useful result may be the reminder that a higher price doesn't reliably mean better answers.

OpenAI: GPT-6 Luna: Model B is more careful about what the benchmark establishes, especially by qualifying the category-level findings and distinguishing drafting from broader support work. Model A is engaging but overstates some conclusions, including declines in individual scoring dimensions that the provided results do not report. (Second judge pass, order swapped — scores are the average of both: Model B gives a more measured account of the statistically insignificant top-score gap and flags uncertainty around | Anthropic: Claude Fable 5.1: Model A is fluent and on-target but overreaches…

Credible Startup Spotlight

Write a 500–650-word Startup Spotlight article in clean Markdown. The tone should be curious and constructive. Explain the founders’ insight, product and early traction while giving uncertainty and competition appropriate space. Avoid investor clichés, heroic founder mythology and promotional claims stated as fact. Reporting: * LedgerLark builds AI agents for post-trade operations at small investment firms. * Co-founder Mina Batra previously spent seven years in operations at a regional brokerage. * Co-founder Owen Carr built reconciliation systems at a large bank. * Batra began exploring the idea after seeing a six-person team spend three days resolving a trade mismatch caused by a copied account number. * LedgerLark’s agents compare confirmations, custodian records and internal ledgers, then propose corrections for human approval. * The product does not execute trades or move money. * Five firms are paying customers. * Annual contracts range from $36,000 to $110,000. * The company says one customer reduced its unresolved daily exceptions from 70 to 18. * The company has raised a $3.5 million seed round led by Motive Row. * Competitors include established reconciliation vendors and internal bank software. * Financial institutions may require lengthy security and procurement reviews.

Google: Gemini 4 Carbon: # Startup Spotlight: LedgerLark Brings AI Agents to the Post-Trade Back Office In financial markets, executing a trade takes milliseconds, while confirming that every record agrees on the details can take days. For Mina Batra, the everyday friction of post-trade work became hard to ignore during her seven years in operations at a regional brokerage. She began exploring ideas for better tooling after watching a six-person team spend three days resolving a single trade mismatch caused by a copied account number. That operational bottleneck is the focus of LedgerLark, a startup Batra co-founded with Owen Carr, who previously built reconciliation systems at a large bank. Together, they are developing AI agents tailored for post-trade operations at small investment firms. While major banks employ large engineering teams to maintain custom reconciliation engines, smaller firms often rely on spreadsheets and manual checks to meet the same regulatory standards for accuracy. ### Proposing Fixes, Keeping Humans in Control Post-trade operations revolve around ensuring every party to a transaction shares identical records. When a trade confirmation, a custodian statement, and an internal portfolio ledger fail to match, an "exception" occurs, forcing operations staff to hunt across documents and portals to find the root cause. LedgerLark’s agents automate that investigative legwork. The software compares trade confirmations, custodian records, and internal ledgers, identifies where discrepancies appear, and proposes specific corrections for human approval. Crucially for a risk-conscious sector, the founders have placed clear guardrails around what the software is permitted to do. Every proposed adjustment requires human sign-off, and the product does not execute trades or move money. By restricting its agents to read-only analysis and workflow suggestions, LedgerLark aims to limit operational risk and ease concerns among compliance officers evaluating AI tools. ### Early Traction and Seed Funding This defined scope has helped LedgerLark win its first cohort of users. Five firms are currently paying customers, signing annual contracts that range from $36,000 to $110,000. While it is still too early to measure broad industry impact, the company says one customer has reduced its unresolved daily exceptions from 70 to 18. To support product development and expand its customer base, LedgerLark has raised a $3.5 million seed round led by Motive Row. ### Navigating Procurement and Incumbents Turning early traction into durable growth in financial software presents distinct hurdles. Even small financial institutions often require lengthy security and procurement reviews before allowing an external vendor to touch sensitive trading and custodian records. Navigating these drawn-out sales cycles requires patience and capital discipline for a seed-stage team. Moreover, the margin for error in back-office accounting is slim; if the agents surface inaccurate proposals, operations teams could spend just as much time auditing the software as they previously spent checking spreadsheets. LedgerLark also faces capable competition. Its rivals include established reconciliation vendors with deep industry ties and growing automation features of their own, alongside internal bank software—the very type of infrastructure Carr used to build—which larger institutions sometimes bundle or extend to smaller client firms. Batra and Carr are wagering that traditional enterprise tools remain too rigid and expensive to implement for smaller investment managers, leaving room for lighter-weight agents that can interpret varied document formats quickly. Over the next year, LedgerLark’s test will be whether it can replicate its early exception-reduction results across a wider variety of firms—and move smoothly through the security gates required to win their business.
Anthropic: Claude Sonnet 5.5: # Startup Spotlight: LedgerLark Wants AI to Handle the Tedious Part of Post-Trade Work ## The problem After a trade is made, someone has to confirm that everyone involved agrees on what happened. At small investment firms, that job often falls to a handful of operations staff comparing trade confirmations, custodian records and internal ledgers by hand. Most days the records match. When they don't, finding out why can take far longer than the error seems to deserve. Mina Batra saw this during seven years in operations at a regional brokerage. In one case, a six-person team spent three days resolving a trade mismatch that traced back to a copied account number. The fix was trivial. Finding it was not. ## The founders Batra started exploring the idea of automating that detective work and teamed up with Owen Carr, who had built reconciliation systems at a large bank. Their backgrounds are complementary: she knows how the work feels on a small team, and he knows how reconciliation software is built and where it breaks. ## What LedgerLark does LedgerLark builds AI agents for post-trade operations. The agents compare a firm's confirmations, custodian records and internal ledgers, look for discrepancies, and propose corrections. A person reviews and approves each proposal before anything changes. The product does not execute trades or move money. That limit keeps the software away from the highest-risk actions, and it also narrows what LedgerLark can promise. The agents are only as useful as their proposals are accurate. If reviewers must check every suggestion from scratch, the time savings shrink. ## Early traction Five firms are paying customers, with annual contracts ranging from $36,000 to $110,000. That is a meaningful signal for a young company, since these buyers are paying real money for software that touches sensitive records. It is still a small sample, and the range suggests the firms differ considerably in size or scope of use. The company says one customer cut its unresolved daily exceptions from 70 to 18. That figure comes from LedgerLark, and the details matter: how exceptions are counted, how long the customer had used the product, and whether trade volume changed over that period. A drop of that size is encouraging, but one customer's result doesn't show what the typical firm will see. LedgerLark has raised a $3.5 million seed round led by Motive Row. ## Competition and open questions LedgerLark is entering a field that is far from empty. Established reconciliation vendors already serve financial institutions and have long-standing relationships and proven integrations. Banks and larger firms also frequently rely on internal software built around their own workflows. A small firm might reasonably ask whether an incumbent's product, perhaps with new AI features, would be enough. LedgerLark's bet appears to be that small firms are underserved: too small to build their own tooling and too lean to want a heavy implementation. That is plausible, but it is a hypothesis, and the answer will show up in renewals more than in first-year signups. Sales may be another challenge. Financial institutions often require lengthy security and procurement reviews, and a startup handling trade and custody data will face close scrutiny. Those reviews can stretch sales cycles and strain a small team, even when the customer is eager. ## What to watch Several questions will say more than the early numbers: - **Accuracy over time.** How often are the agents' proposed corrections right, and how do customers handle the ones that aren't? - **Renewals.** Do the first five customers renew, and do they expand their use? - **Reach.** Can the company win firms beyond those that found it early? - **Incumbent response.** How quickly do established vendors offer something comparable to small firms? The founding story is a modest one: a tedious problem, seen up close, that may be well suited to software that proposes and people who approve. Whether that is enough to build a durable business is what LedgerLark's next year or two should reveal.

OpenAI: GPT-6 Luna: Model B better balances the reported traction with explicit uncertainty, competition and procurement risks. Model A is polished but adds unsupported claims about firms’ workflows and the product’s read-only design, and leans more promotional in places. (Second judge pass, order swapped — scores are the average of both: Model B better balances the reported traction with uncertainty and makes fewer unsupported claims. Model A is polished, but presents several unreported details and causal explanat | Anthropic: Claude Fable 5.1: Model B stays closer to the supplied reporting,…

Controlled wit

Write a 350–450-word technology news story in Markdown. The subject invites humor, but the reporting should remain serious. Use no more than one dry line of original humor. Do not mock customers or turn the story into a comedy routine. Reporting: * QueueZero launched an AI meeting assistant called Proxy on Wednesday. * Proxy can attend a meeting, produce notes and assign follow-up work to software agents. * During the launch demonstration, a customer in the audience said, “Someone should tell Evan to stop talking and send us the pricing sheet.” * Proxy interpreted the remark as an instruction. * It created two tasks assigned to QueueZero CEO Evan Wu: “Stop talking” and “Send pricing sheet.” * Wu laughed, completed the second task and deleted the first. * QueueZero says Proxy requires confirmation before an external message is sent. * Internal tasks can be created automatically. * Administrators can disable automatic task creation. * Proxy costs $30 per user per month. * It enters public beta next week.

Google: Gemini 4 Carbon: # QueueZero’s New AI Meeting Assistant Takes Launch Demo Remark Literally Enterprise software company QueueZero launched an artificial intelligence meeting assistant called Proxy on Wednesday, introducing a workplace tool designed to move team discussions directly into automated task execution. Unlike conventional transcription services that stop at summarizing conversations, Proxy can attend a meeting, produce structured notes, and assign follow-up work to software agents as well as human team members. During Wednesday’s launch demonstration, however, the software’s literal approach to capturing action items produced an unscripted test of its boundaries. As QueueZero Chief Executive Evan Wu presented the product, a customer in the audience said, “Someone should tell Evan to stop talking and send us the pricing sheet.” Proxy, which was attending the demonstration, interpreted the remark as an instruction. Within moments, the system created two tasks assigned to Wu: “Stop talking” and “Send pricing sheet.” Wu laughed, completed the second task, and deleted the first. Artificial intelligence, it seems, has yet to learn executive deference. The exchange underscored a persistent technical challenge in the rapidly growing market for enterprise AI tools. As software vendors connect large language models directly to workplace task managers and autonomous agents, systems must reliably distinguish between genuine operational directives and the offhand remarks common in live meetings. QueueZero said Proxy includes built-in controls to manage how spoken comments translate into actions. While internal tasks can be created automatically to reduce manual data entry after a meeting, the company said Proxy requires human confirmation before any external message is sent. That safeguard is meant to ensure misread remarks remain internal adjustments rather than unauthorized communications with clients or partners. For organizations that prefer stricter oversight over their internal project boards and workflows, company administrators can disable automatic task creation altogether, requiring human users to review proposed action items before they are formally logged and routed. The launch positions QueueZero in an increasingly crowded segment of the business software industry, where startups and major collaboration platforms alike are racing to pair meeting notes with agent-driven workflows capable of handling routine administrative follow-ups. Proxy will enter public beta next week, opening the platform to broader testing across corporate teams looking to automate post-meeting coordination. QueueZero said the service costs $30 per user per month, a price point comparable to other commercial AI workplace assistants.
Anthropic: Claude Sonnet 5.5: # QueueZero Launches Proxy, an AI Meeting Assistant That Assigns Work to Software Agents QueueZero on Wednesday launched Proxy, an AI meeting assistant that can attend meetings, produce notes and assign follow-up work to software agents. The company said the product enters public beta next week. The launch demonstration included an unscripted test of how the system handles casual speech. A customer in the audience said, "Someone should tell Evan to stop talking and send us the pricing sheet." Proxy treated the remark as an instruction. It created two tasks assigned to QueueZero CEO Evan Wu: "Stop talking" and "Send pricing sheet." Wu laughed, completed the second task and deleted the first. It was a quick bit of prioritization. ## How Proxy works Proxy joins meetings, generates notes and converts discussion into tasks. Those tasks can be assigned to people or to software agents that carry out follow-up work. The incident points to a basic challenge for tools that act on conversation: telling the difference between an offhand comment, a suggestion and a directive. People often phrase requests indirectly, and a system that treats every such remark as a command can produce unwanted work. ## Safeguards QueueZero said Proxy requires confirmation before an external message is sent. That limits the risk that a misread comment results in an email or other communication reaching a customer, partner or other outside party without a person approving it. Internal tasks are different. The company said Proxy can create them automatically, which is how the two tasks appeared for Wu during the demonstration. Administrators can disable automatic task creation, giving organizations the option to require review before any task is generated. QueueZero did not say in the information provided how Proxy decides whether a remark is actionable, or whether customers can adjust that threshold beyond turning automatic creation on or off. ## Pricing and availability Proxy costs $30 per user per month. The public beta begins next week. The pricing places Proxy among a growing group of workplace AI products sold per seat. Its distinguishing feature, as presented Wednesday, is that it goes beyond summarizing a meeting. It also starts the work that follows. That raises a practical question for administrators weighing the beta: how much a meeting tool should do without being asked. Automatic task creation saves time when it correctly captures a commitment. It adds clutter when it captures a complaint, a joke or a suggestion that was never meant as an assignment. In Wednesday's demonstration, the cost of a misread comment was a deleted task. QueueZero's design, which allows internal tasks to be created automatically but requires confirmation before external messages go out, reflects the same distinction. Mistakes inside the company are cheap to fix, while mistakes sent outside it are harder to undo. Customers who try the beta will test how well that line holds in everyday meetings.

OpenAI: GPT-6 Luna: Model A is the more polished, focused news story and uses a single restrained joke. Model B adds repetitive analysis and an awkward meta-reference to “the information provided”; both also include some claims beyond the supplied reporting. (Second judge pass, order swapped — scores are the average of both: Model A stays within the requested length and is more concise, while Model B appears to exceed 450 words and repeats its analysis. Model B is more factually restrained; Model A adds unsupported | Anthropic: Claude Fable 5.1: Model A delivers a tight, within-range news sto…

Cohesive story from reporting notes

Write a publication-ready technology news article from the reporting notes below. Requirements: * 550–700 words * Clean Markdown * One H1 headline followed by an italicized dek * A strong news lede followed by a clear nut graf * Organize the material into a cohesive story rather than following the notes in order * Attribute company-supplied claims * Include limitations without turning the article into a warning label * Use only the supplied information * Avoid canned transitions, promotional language and a concluding summary Reporting notes: * HelioDock makes autonomous warehouse robots. * On Thursday, it launched Night Shift, software that lets operators monitor robot fleets remotely. * Night Shift spent 14 months in a private beta involving nine logistics companies and 310 robots. * HelioDock says beta customers experienced 17% fewer unplanned robot stoppages. The figure has not been independently verified. * The company also announced a $24 million Series B led by North Coast Ventures, bringing its total funding to $39 million. * CEO Mara Velez: “A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet.” * Beta customer Aaron Pike of Fulcrum Logistics said the system worked well in facilities with strong connectivity but became less useful during rural LTE outages. * When connectivity fails, robots return to their locally configured behavior. Remote control becomes unavailable. * Competing fleet-management products are available from Relay Robotics and YardOS. * Night Shift will cost $90 per robot per month and become generally available September 15.

Google: Gemini 4 Carbon: # HelioDock Launches Remote Fleet Software Alongside $24 Million Series B *The autonomous warehouse robot maker says its Night Shift platform cuts down on unplanned stoppages, though rural cellular outages can leave operators without remote control.* HelioDock, a maker of autonomous warehouse robots, launched a remote fleet-monitoring software platform called Night Shift on Thursday, pairing the product debut with $24 million in new Series B funding. The dual announcement combines fresh venture capital with a push into subscription software designed to let logistics operators oversee automated warehouse fleets from afar. By giving off-site personnel a way to monitor machines and step in digitally when equipment halts, HelioDock aims to cut down on disruptive floor stoppages while competing for customers in an active fleet-management software market. Autonomous warehouse robots are built to handle floor tasks without constant human supervision, yet unexpected snags inside a facility can still bring individual machines to a standstill. Without remote tools, resolving even a routine holdup during overnight shifts can require physical intervention from on-site staff or managers called in after hours. Night Shift gives operators a way to monitor robot fleets remotely and take remote control when a unit stalls. “A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet,” HelioDock CEO Mara Velez said. Thursday’s commercial launch follows a 14-month private beta in which nine logistics companies tested Night Shift across a combined 310 robots. HelioDock says beta customers experienced 17% fewer unplanned robot stoppages while running the platform across those deployments. That performance figure was supplied by the company and has not been independently verified, leaving prospective buyers to weigh HelioDock’s internal metrics alongside feedback from early testers. Feedback from the beta trial indicates that the software’s day-to-day usefulness depends heavily on reliable network coverage at each warehouse location. Aaron Pike of Fulcrum Logistics, one of the beta customers that evaluated Night Shift during the 14-month test, saw differing results depending on facility connectivity. Pike said the system worked well in facilities with strong connectivity, though its advantages faded in regions with spotty cellular service, noting that the software became less useful during rural LTE outages. Network drops directly curtail what managers can accomplish through Night Shift when a machine runs into trouble. When connectivity fails, robots return to their locally configured behavior, and remote control becomes unavailable. Falling back to local configuration ensures that machines follow their pre-set on-site rules during a communication outage rather than losing operational direction, yet it strips away the off-site intervention tools central to the platform. If a robot encounters an unplanned stoppage during a rural LTE outage, operators have no way to take remote control until connectivity returns. HelioDock’s software debut arrives alongside new venture capital to back the robotics maker’s operations. North Coast Ventures led the $24 million Series B round, bringing HelioDock’s total funding to $39 million and accounting for more than half of the capital the company has raised to date across its financing rounds. As it brings Night Shift out of private testing and into the broader market, HelioDock faces competition from other software providers serving warehouse operators. Competing fleet-management products are already available from both Relay Robotics and YardOS, giving logistics firms existing alternatives for overseeing automated equipment fleets. Logistics companies outside the initial nine-firm beta group will gain access to the platform in mid-September, when HelioDock opens commercial subscriptions to the wider logistics industry. Night Shift will become generally available on September 15 and will cost $90 per robot per month, tying software expenses directly to the size of an operator’s fleet.
Anthropic: Claude Sonnet 5.5: # HelioDock Launches Remote Monitoring for Warehouse Robot Fleets and Raises $24 Million *The company's Night Shift software costs $90 per robot per month. A beta customer says it was less useful during rural LTE outages.* HelioDock, which makes autonomous warehouse robots, on Thursday launched Night Shift, software that lets operators monitor their robot fleets remotely. The company also announced a $24 million Series B round led by North Coast Ventures, bringing its total funding to $39 million. Night Shift is meant to let warehouse staff watch over robots without being on the floor, including overnight. HelioDock says customers in its beta program had 17% fewer unplanned robot stoppages, though the figure has not been independently verified. One beta customer said the software performed well in facilities with strong connectivity but lost value when the connection did not hold. ## What the software does The product is aimed at the kind of problem CEO Mara Velez described in a statement about the launch. "A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet," she said. Night Shift will be generally available September 15 and will cost $90 per robot per month. Before the launch, the software spent 14 months in a private beta involving nine logistics companies and 310 robots. ## Where connectivity matters Because Night Shift depends on a network link between the operator and the robots, its usefulness tracks the quality of that link. Aaron Pike of Fulcrum Logistics, a beta customer, said the system worked well in facilities with strong connectivity. It became less useful during rural LTE outages, he said. HelioDock's design determines what happens when a connection drops. In that case, robots return to their locally configured behavior, and remote control becomes unavailable. The robots do not stop working because the software cannot reach them, but an operator who is off-site cannot step in until the connection returns. That makes the 17% reduction in stoppages harder to generalize. The figure comes from the company and covers beta participants, who ran the software at sites of their own choosing. The notes do not say how many of those sites had weak connectivity. ## A crowded category HelioDock is not the only company selling tools to manage robot fleets. Relay Robotics and YardOS both offer competing fleet-management products. Night Shift's price is set per robot, so a customer's monthly bill will grow with the size of its fleet. The new funding adds to what HelioDock has raised so far. The Series B is the largest single figure in the company's disclosed total: $24 million of the $39 million. HelioDock did not say in the information provided how it plans to divide the money among product development, sales or manufacturing. ## What comes next General availability on September 15 will move Night Shift beyond the nine companies that tested it. Those customers ran the software across 310 robots, a fleet size that is small next to the number of facilities that could take on the product once it is publicly available. For now, the evidence for Night Shift rests on the company's own beta results and the experience of customers like Pike. The company's claim that the software reduces unplanned stoppages is a measure of how often robots halt unexpectedly, and the 17% figure has not been checked by anyone outside HelioDock. Pike's account suggests that where networks are reliable, remote monitoring does what it promises, and that in places where LTE service fails, operators are back to relying on the robots' local settings. Velez's two-in-the-morning scenario describes a specific kind of call: a single robot confused by a single pallet. Night Shift is built to let a manager deal with that from a screen rather than from a car. Whether the screen is reachable depends on the facility's connection.

OpenAI: GPT-6 Luna: Model A is the more cohesive, publication-ready article and ends with concrete launch details rather than a recap. Model B adds unsupported claims about beta-site selection and robot behavior during outages, and closes with a summary despite the instruction to avoid one. (Second judge pass, order swapped — scores are the average of both: Model A is more cohesive and publication-ready, and it handles the connectivity limitation and company-supplied performance claim clearly. Model B adds more uns | Anthropic: Claude Fable 5.1: Model A delivers a cleanly organized news artic…

Select the strongest accurate lede

Based only on the facts below, select the strongest news lede. Respond with one letter. Facts: * ByteBarge pulled its $12-per-month AI assistant on Tuesday. * Private workspace titles from 4,860 accounts were sent to a publicly accessible analytics endpoint. * The exposure lasted 11 hours. * Message contents were not sent. * The company has found no evidence of malicious access. A. ByteBarge users suffered a massive privacy disaster Tuesday when the company leaked their private messages online. B. ByteBarge, an innovative collaboration company, made the difficult decision to cancel one of its products. C. ByteBarge pulled its $12-per-month AI assistant Tuesday after discovering that private workspace titles from 4,860 accounts had been sent to a public analytics endpoint for 11 hours. D. Thousands of ByteBarge users woke Tuesday to learn that everything they had written was visible to hackers.

Google: Gemini 4 Carbon: C
Anthropic: Claude Sonnet 5.5: C

Scored against the benchmark's reference answer (choice match): Model A was correct, Model B was correct.

Natural human rewrite

Rewrite the draft below as a 300–400-word publication-ready news story in Markdown. Preserve every factual claim. Improve the lede, narrative flow, sentence rhythm and transitions. Remove repetition, stock hype, throat-clearing and mechanical signposting. The finished article should sound like an experienced human technology reporter wrote it. Draft: # Arcwell Launches An Innovative New Platform Arcwell has announced an exciting new platform called Relay. The announcement represents a significant development for the company and for the broader customer service industry. Relay uses artificial intelligence to examine customer support conversations and suggest answers to agents. The platform is important for several reasons. First, it can suggest answers. Second, it can identify customers who may cancel. Third, it can create summaries. These capabilities demonstrate how artificial intelligence continues to transform the customer service landscape. Relay was tested by 12 companies over six months. Arcwell says agents using it closed tickets 14% faster. The company did not release customer-retention results. Relay costs $65 per agent each month. CEO Priya Shah said, “Support agents spend too much time searching for information that their companies already have.” However, the platform also presents potential challenges. Relay requires access to support transcripts and internal help documents. Arcwell says customer data is encrypted and is not used to train shared models. Companies can choose a 30-day or 90-day retention period. In conclusion, Relay represents an important new chapter for Arcwell as it seeks to help companies improve customer service using artificial intelligence.

Google: Gemini 4 Carbon: # Arcwell Debuts Relay AI Platform to Speed Up Customer Support Arcwell has unveiled Relay, a new artificial intelligence platform built to help customer service teams resolve inquiries faster by surfacing information their organizations already hold. Priced at $65 per agent each month, the software examines customer support conversations to assist human representatives in three ways: suggesting answers for agents to send, creating summaries of interactions, and identifying customers who may cancel. The pitch centers on cutting down the legwork behind everyday support tickets rather than taking human staff out of the loop. “Support agents spend too much time searching for information that their companies already have,” Arcwell CEO Priya Shah said. Prior to the launch, 12 companies tested Relay over a six-month period. Arcwell says agents using the platform closed tickets 14% faster. Even so, a notable piece of the software’s value proposition remains publicly unmeasured: while Arcwell highlights Relay’s ability to flag customers at risk of canceling, the company did not release any customer-retention results from the trial. Adopting the tool also presents potential hurdles around corporate data access. To suggest responses, summarize threads, and spot cancellation risks, Relay requires access to a company’s support transcripts as well as its internal help documents. Opening up those repositories can be a sticking point for businesses cautious about feeding proprietary documentation and customer exchanges into outside AI systems. Arcwell is seeking to ease those concerns with explicit boundaries on how client information is stored and processed. According to the company, customer data handled by Relay is encrypted and is not used to train shared AI models. Participating companies can also govern how long their records remain on the system by choosing either a 30-day or 90-day retention period. For Arcwell, the rollout marks a concerted push to embed AI assistance into everyday customer service workflows. Relay’s traction with business buyers will likely hinge on whether a 14% gain in ticket-closing speed, paired with the promise of early cancellation warnings, is enough to justify both the monthly per-agent fee and the deep document access the platform demands.
Anthropic: Claude Sonnet 5.5: # Arcwell's Relay Promises Faster Support, but Retention Gains Remain Unproven Arcwell says support agents closed tickets 14% faster with Relay, its new AI platform, during a six-month trial. The company has not said whether the tool helps keep customers from leaving. Relay analyzes customer support conversations and suggests answers to agents as they work. It can also flag customers who may cancel and write summaries. Twelve companies tested the platform. The 14% figure is Arcwell's own, and the company did not release customer-retention results. That leaves the cancellation-spotting feature without public evidence of its effect. Arcwell's pitch rests on a simple observation about how agents spend their time. "Support agents spend too much time searching for information that their companies already have," said CEO Priya Shah. Relay costs $65 per agent each month. The product's usefulness depends on access. To work, Relay needs to read a company's support transcripts and its internal help documents. Arcwell says customer data is encrypted and is not used to train shared models. Companies can choose a retention period of either 30 or 90 days. Those assurances address the most obvious concern for any business weighing the tool, which is what happens to customer conversations once an outside system has read them. They do not answer the question the trial leaves open: whether faster ticket closure translates into customers who stay. For now, Relay's case rests on speed, which Arcwell has measured, rather than retention, which it has not shared.

OpenAI: GPT-6 Luna: Model A delivers a publication-ready story within the requested word range and includes the draft’s key facts. Model B is tighter and more restrained, but falls well short of the 300-word minimum. (Second judge pass, order swapped — scores are the average of both: Model A is the stronger choice because it meets the requested 300–400-word length while delivering a coherent news story. Model B is more restrained, but falls well short of the length requirement; Model A’s main weakness is some specu | Anthropic: Claude Fable 5.1: Model A delivers a complete ~340-word story tha…

News judgment in the opening

Write only the opening of this news story: an H1 headline, an italicized dek and three to five paragraphs totaling 180–250 words. The opening must establish what happened, quantify the scope, explain why it matters and carefully distinguish confirmed facts from unknowns. It should pull the reader forward without sensationalizing the incident. Reporting: * Cloudnote discovered that a debugging page containing customer prompt excerpts was accessible without authentication. * The page was exposed for four hours on Monday. * Logs show excerpts from 1,842 user sessions. * The excerpts could include up to 200 characters from a prompt. * Cloudnote removed the page 19 minutes after a researcher notified it. * The company says it has found no evidence that anyone besides the researcher accessed the page. * It has not completed its investigation. * Full documents, passwords and payment information were stored elsewhere and were not present on the page. * Cloudnote has 3.2 million registered users.

Google: Gemini 4 Carbon: # Cloudnote Exposes User Prompt Excerpts on Unsecured Debugging Page *A four-hour lapse left text snippets from 1,842 sessions accessible without authentication, though full documents and passwords remained protected.* Cloudnote discovered that a debugging page containing customer prompt excerpts was accessible without authentication for four hours on Monday, exposing partial inputs logged across 1,842 user sessions. The exposed excerpts could include up to 200 characters from a single prompt. While the affected sessions represent only a tiny fraction of Cloudnote’s 3.2 million registered users, even brief text snippets can contain confidential workplace context, sensitive queries or personal details. The lapse underscores a persistent vulnerability for online platforms: diagnostic tools designed to troubleshoot software can quietly undermine privacy safeguards when left unsecured. More sensitive customer records were unaffected by the exposure. Full documents, passwords and payment information were stored elsewhere and were not present on the debugging page. Once a researcher notified Cloudnote of the vulnerability, the company removed the page 19 minutes later. Still, the full impact of the incident remains an open question. Cloudnote says it has found no evidence so far that anyone besides the researcher accessed the page during Monday’s four-hour window. Because the company has not yet completed its investigation, however, it remains unknown whether any unauthorized parties viewed or copied the prompt excerpts before access was cut off.
Anthropic: Claude Sonnet 5.5: # Cloudnote debugging page exposed excerpts of user prompts for four hours *The company says it has found no sign that anyone but a security researcher viewed the page, but its investigation is not finished.* Cloudnote left an internal debugging page containing excerpts of customers' prompts open to anyone without a login for four hours on Monday, the company said. A security researcher flagged the problem. Logs show the page displayed excerpts from 1,842 user sessions, each of which could include up to 200 characters of a prompt. Cloudnote took the page down 19 minutes after the researcher notified it. The company said full documents, passwords and payment information are stored elsewhere and were not on the page. Cloudnote said it has found no evidence that anyone other than the researcher accessed the page. It has not completed its investigation, however, and that assessment could change. The incident matters because prompts hold whatever users choose to type, from casual questions to personal or work details, and even a short snippet can carry sensitive context. The 1,842 sessions are a small fraction of Cloudnote's 3.2 million registered users, but the figures are not directly comparable, because a session is not the same as a user. It is still unclear what the excerpts contained and how many separate users were affected.

OpenAI: GPT-6 Luna: Model B is more careful about the limits of the numbers, noting that sessions are not directly comparable to registered users and that the investigation is incomplete. Model A is polished, but its claim that the affected sessions are a tiny fraction of users risks implying a comparison the reporting does not support. (Second judge pass, order swapped — scores are the average of both: Model B is more precise: it explains that session counts cannot be directly compared with registered users, while | Anthropic: Claude Fable 5.1: Model B shows sharper news judgment: it attribu…

Markdown error count

Count the lines that violate the house rules below. A defective line counts once even if it could violate multiple rules. Respond with a single number. House rules: * The headline must use `# Headline`. * The dek must appear immediately after the headline and be entirely italicized. * Section headings must use `## Heading`. * Links must use an absolute HTTPS URL. * Unordered-list items must begin with a hyphen. * Raw HTML is forbidden. Draft: #Flood software moves upstream The company is expanding its pilot. ##Details Read [the announcement](www.example.com/news). * The first pilot begins Monday. <div>The second pilot begins Friday.</div>

Google: Gemini 4 Carbon: 6
Anthropic: Claude Sonnet 5.5: Going through the draft line by line: 1. `#Flood software moves upstream` has no space after the #, so it is not `# Headline`. Violation. 2. `The company is expanding its pilot.` is the dek, which must be entirely italicized, and it is not. Violation. 3. `##Details` has no space after the ##, so it is not `## Heading`. Violation. 4. `Read [the announcement](www.example.com/news).` has a link that is not an absolute HTTPS URL. Violation. 5. `* The first pilot begins Monday.` is a list item that begins with an asterisk instead of a hyphen. Violation. 6. `<div>The second pilot begins Friday.</div>` is raw HTML. Violation. Total: 6 lines. **6**

Scored against the benchmark's reference answer (numeric match): Model A was correct, Model B was correct.

Exact final-draft structure

Convert the supplied fields into Markdown using these exact rules: * H1 headline * One blank line * Italicized dek * One blank line * First body paragraph * One blank line * H2 section heading * One blank line * Second body paragraph * Preserve all wording and punctuation exactly * Do not use a code fence * Output nothing else Headline: Raincheck raises $8 million for flood-risk software Dek: The Austin startup helps cities decide which drainage problems to address before storms. First body paragraph: Raincheck has raised an $8 million seed round led by Basin Ventures. Section heading: Expansion plans Second body paragraph: The company plans to hire eight engineers.

Google: Gemini 4 Carbon: # Raincheck raises $8 million for flood-risk software *The Austin startup helps cities decide which drainage problems to address before storms.* Raincheck has raised an $8 million seed round led by Basin Ventures. ## Expansion plans The company plans to hire eight engineers.
Anthropic: Claude Sonnet 5.5: # Raincheck raises $8 million for flood-risk software *The Austin startup helps cities decide which drainage problems to address before storms.* Raincheck has raised an $8 million seed round led by Basin Ventures. ## Expansion plans The company plans to hire eight engineers.

OpenAI: GPT-6 Luna: Model A and Model B are identical and follow the requested structure, preserve the wording and punctuation, and add nothing else. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both follow the required Markdown structure, preserve the supplied wording and punctuation, and include no extra text. Their writing quality is equally clear and faithful to the source.) | Anthropic: Claude Fable 5.1: Model A and Model B produced character-for-character identical outputs that follow every structural rule exactly: H1 headline, italicized dek,…

Matchup powered by OpenRouter.