Head to head: Anthropic: Claude Opus 5.5 vs grok-4.7

Anthropic: Claude Opus 5.5 vs grok-4.7

By · Published

RuntimeWire Head-to-Head: Head to head: Anthropic: Claude Opus 5.5 vs grok-4.7
RuntimeWire Head-to-Head matchup

A writing-focused matchup tests how well Claude Opus 5.5 and grok-4.7 turn reporting, notes and constraints into clear, accurate editorial copy. The results favor careful synthesis and restraint over more expansive analysis.

Claude Opus 5.5 wins decisively: it scored 105.0 to grok-4.7’s 95.8, with the statistical verdict putting the result at limited confidence. Across 12 tasks, Claude won six, tied six and lost none. That’s a strong edge—not a sweep of every individual judgment. The difference showed up most in assignments that demanded disciplined editing. Claude produced the more cohesive story from reporting notes, handled chronology without claiming more than the reporting established, and made the stronger natural rewrite. It also shaped benchmark numbers more carefully, noting that significance testing applied only to the top pair rather than implying unsupported differences among the rest. Grok had a real bright spot: its Startup Spotlight showed sharper scrutiny of customer traction, competition and procurement risks. The models also tied on several tasks, including the news opening, exact structure and strongest-lede selection. But Grok’s recurring weakness was overreach or excess: judges flagged repetition, unsupported implications and, in one story assignment, a draft well beyond the word limit. **Final call: Claude Opus 5.5 is the stronger editorial model in this matchup.** Its advantage comes from consistently turning constraints and evidence into cleaner, more publication-ready copy—not from winning every category or making the bolder argument.

Markdown error count

Count the lines that violate the house rules below. A defective line counts once even if it could violate multiple rules. Respond with a single number. House rules: * The headline must use `# Headline`. * The dek must appear immediately after the headline and be entirely italicized. * Section headings must use `## Heading`. * Links must use an absolute HTTPS URL. * Unordered-list items must begin with a hyphen. * Raw HTML is forbidden. Draft: #Flood software moves upstream The company is expanding its pilot. ##Details Read [the announcement](www.example.com/news). * The first pilot begins Monday. <div>The second pilot begins Friday.</div>

Anthropic: Claude Opus 5.5: **6** Every non-blank line in the draft breaks at least one rule: 1. `#Flood software moves upstream` has no space after `#`, so it does not follow the `# Headline` format. 2. `The company is expanding its pilot.` is the dek, but it is not italicized. 3. `##Details` has no space after `##`, so it does not follow the `## Heading` format. 4. The link `www.example.com/news` is not an absolute HTTPS URL. 5. `* The first pilot begins Monday.` starts with an asterisk instead of a hyphen. 6. `<div>…</div>` is raw HTML.
grok-4.7: 6

Scored against the benchmark's reference answer (numeric match): Model A was correct, Model B was correct.

Exact final-draft structure

Convert the supplied fields into Markdown using these exact rules: * H1 headline * One blank line * Italicized dek * One blank line * First body paragraph * One blank line * H2 section heading * One blank line * Second body paragraph * Preserve all wording and punctuation exactly * Do not use a code fence * Output nothing else Headline: Raincheck raises $8 million for flood-risk software Dek: The Austin startup helps cities decide which drainage problems to address before storms. First body paragraph: Raincheck has raised an $8 million seed round led by Basin Ventures. Section heading: Expansion plans Second body paragraph: The company plans to hire eight engineers.

Anthropic: Claude Opus 5.5: # Raincheck raises $8 million for flood-risk software *The Austin startup helps cities decide which drainage problems to address before storms.* Raincheck has raised an $8 million seed round led by Basin Ventures. ## Expansion plans The company plans to hire eight engineers.
grok-4.7: # Raincheck raises $8 million for flood-risk software *The Austin startup helps cities decide which drainage problems to address before storms.* Raincheck has raised an $8 million seed round led by Basin Ventures. ## Expansion plans The company plans to hire eight engineers.

OpenAI: GPT-6 Luna: Model A and Model B both follow the required Markdown structure and preserve all supplied wording and punctuation. Their outputs are identical and equally clear. (Second judge pass, order swapped — scores are the average of both: Model B and Model A match the requested structure and preserve the supplied wording and punctuation exactly, so neither has an advantage.) | Anthropic: Claude Fable 5.1: Model A and Model B produce character-for-character identical output that follows every structural rule exactly: H1, blank line, italicized dek, blank line, body paragraph, blank…

Publication-ready Markdown edit

Edit the copy below into a publication-ready Markdown story. Requirements: * Supply an H1 headline and italicized dek * Repair the lede and paragraph flow * Use a Markdown blockquote only for the extended statement * Convert the supplied URL into a descriptive inline link * Remove internal notes and formatting debris * Preserve the reporting and its meaning * Output only the finished article Raw copy: HEADLINE TBD The funding was announced today. Raincheck is the company. It has raised $8 million and it makes flood-risk software for local governments. [NOTE: move this up?] The Austin company maps drainage systems and combines those maps with rainfall forecasts. Cities use the software to decide where to send maintenance crews before a storm. The seed round was led by Basin Ventures. Existing investor Long Mile also participated. Raincheck has raised $10.5 million total. CEO Ana Flores said the following: “Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives.” https://example.com/raincheck-methodology Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them because some contracts have not been publicly announced. Its forecasts are recommendations. Emergency managers retain control over deployments and public warnings. THE COMPETITION Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck says the new funding will help it hire eight engineers and expand into Florida.

Anthropic: Claude Opus 5.5: # Raincheck Raises $8 Million to Help Cities Get Ahead of Floods *The Austin startup's software combines drainage maps with rainfall forecasts to help local governments decide where to send crews before a storm.* Raincheck, an Austin company that makes flood-risk software for local governments, announced today that it has raised $8 million in seed funding. The software maps cities' drainage systems and combines those maps with rainfall forecasts. Cities use it to decide where to send maintenance crews before a storm arrives. Basin Ventures led the round, with participation from existing investor Long Mile. The funding brings Raincheck's total raised to $10.5 million. The company says the new money will help it hire eight engineers and expand into Florida. Chief Executive Ana Flores said the challenge for cities is prioritization rather than identifying flood-prone areas: > "Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives." Raincheck has published [an explanation of its forecasting methodology](https://example.com/raincheck-methodology). The company currently works with seven cities in Texas and Louisiana. It declined to identify them because some contracts have not been publicly announced. Raincheck's forecasts are recommendations. Emergency managers retain control over crew deployments and public warnings. Other flood-planning vendors include HydroMap and municipal engineering contractors.
grok-4.7: # Raincheck Raises $8 Million for Flood-Risk Software *Basin Ventures led the seed round as the Austin company plans to hire engineers and expand into Florida.* Raincheck, an Austin company that makes flood-risk software for local governments, announced today that it has raised $8 million. It maps drainage systems and combines those maps with rainfall forecasts ([Raincheck’s methodology](https://example.com/raincheck-methodology)). Cities use the software to decide where to send maintenance crews before a storm. The seed round was led by Basin Ventures. Existing investor Long Mile also participated. Raincheck has raised $10.5 million in total. CEO Ana Flores said: > Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives. Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them because some contracts have not been publicly announced. Its forecasts are recommendations. Emergency managers retain control over deployments and public warnings. Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck says the new funding will help it hire eight engineers and expand into Florida.

OpenAI: GPT-6 Luna: Model A better integrates the funding and its intended use into a smooth narrative, while preserving the supplied details and presenting the link descriptively. Model B is also faithful, but its link placement is less clear and the story’s flow is somewhat less polished. (Second judge pass, order swapped — scores are the average of both: Model A has a more engaging headline and smoother paragraph flow while preserving the reporting. Model B is also accurate and clean, but its opening is less pol | Anthropic: Claude Fable 5.1: Model A produces a genuinely publication-ready…

Cohesive story from reporting notes

Write a publication-ready technology news article from the reporting notes below. Requirements: * 550–700 words * Clean Markdown * One H1 headline followed by an italicized dek * A strong news lede followed by a clear nut graf * Organize the material into a cohesive story rather than following the notes in order * Attribute company-supplied claims * Include limitations without turning the article into a warning label * Use only the supplied information * Avoid canned transitions, promotional language and a concluding summary Reporting notes: * HelioDock makes autonomous warehouse robots. * On Thursday, it launched Night Shift, software that lets operators monitor robot fleets remotely. * Night Shift spent 14 months in a private beta involving nine logistics companies and 310 robots. * HelioDock says beta customers experienced 17% fewer unplanned robot stoppages. The figure has not been independently verified. * The company also announced a $24 million Series B led by North Coast Ventures, bringing its total funding to $39 million. * CEO Mara Velez: “A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet.” * Beta customer Aaron Pike of Fulcrum Logistics said the system worked well in facilities with strong connectivity but became less useful during rural LTE outages. * When connectivity fails, robots return to their locally configured behavior. Remote control becomes unavailable. * Competing fleet-management products are available from Relay Robotics and YardOS. * Night Shift will cost $90 per robot per month and become generally available September 15.

Anthropic: Claude Opus 5.5: # HelioDock Launches Night Shift Software for Remote Oversight of Warehouse Robots *The autonomous robot maker also closed a $24 million Series B, while a beta customer says the product's value depends heavily on network connectivity.* HelioDock, a maker of autonomous warehouse robots, on Thursday launched Night Shift, software that lets operators monitor their robot fleets remotely. The product becomes generally available Sept. 15 and will cost $90 per robot per month. The launch puts HelioDock up against existing fleet-management products from Relay Robotics and YardOS. It also comes with new capital: a $24 million Series B round led by North Coast Ventures. Night Shift is built on a simple premise: a stalled robot shouldn't require someone to be physically present. But feedback from at least one early customer suggests the software is only as dependable as the network it runs on. ## The problem it targets Chief Executive Mara Velez framed Night Shift around the off-hours disruptions that come with running autonomous machines. "A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet," Velez said. The software is designed to give managers a view of their fleets from wherever they are, along with the ability to take remote control of robots when a connection is available. For operators running robots overnight, that could mean handling a problem from home instead of making a trip to the warehouse floor. ## Beta results HelioDock ran Night Shift in a private beta for 14 months with nine logistics companies operating a combined 310 robots, an average of roughly 34 robots per customer. According to HelioDock, beta customers experienced 17% fewer unplanned robot stoppages. The figure has not been independently verified. If the number holds up in broader use, it would speak directly to the kind of interruption Velez described: a single robot halting unexpectedly and requiring human attention. ## Where connectivity falls short Aaron Pike of Fulcrum Logistics, one of the beta customers, said Night Shift worked well in facilities with strong connectivity. During rural LTE outages, he said, it became less useful. The limitation stems from how the system behaves when a connection drops. In that case, HelioDock's robots return to their locally configured behavior, and remote control becomes unavailable until the link is restored. That approach means an outage doesn't leave robots without instructions. They fall back on whatever behavior has been configured on site. But it also means an off-site manager can't step in during the gap. A robot confused by a pallet at 2 a.m. during an outage could still require someone on site. That consideration is likely to weigh more heavily on operators whose facilities depend on rural cellular networks than on those with strong, stable connections. ## Funding and pricing The $24 million Series B led by North Coast Ventures brings HelioDock's total funding to $39 million, meaning the company had raised $15 million before this round. Night Shift's per-robot pricing scales directly with fleet size. At $90 per robot per month, a customer running 100 robots would pay $9,000 a month, or $108,000 a year. Covering all 310 robots that took part in the beta would cost $27,900 a month. Prospective buyers will be able to set those costs against the fleet-management products already available from Relay Robotics and YardOS when Night Shift opens to all customers in September.
grok-4.7: # HelioDock Launches Night Shift to Monitor Warehouse Robots Remotely *The software will cost $90 per robot per month from September 15, after a 14-month private beta the company credits with fewer stoppages and a customer says depends on the connection.* HelioDock on Thursday launched Night Shift, software that lets operators monitor robot fleets remotely, and priced it at $90 per robot per month for a general release on September 15. The launch gives the maker of autonomous warehouse robots a paid remote layer over fleets it has already been running in private. Night Shift spent 14 months in a beta involving nine logistics companies and 310 robots. HelioDock says those customers experienced 17 percent fewer unplanned robot stoppages, a figure the company supplied and that has not been independently verified. The company also announced a $24 million Series B led by North Coast Ventures, bringing its total funding to $39 million. Operators weighing the new software can already look at competing fleet-management products from Relay Robotics and YardOS. What they cannot assume is uninterrupted remote authority: when connectivity fails, robots return to their locally configured behavior, and remote control becomes unavailable. Chief executive Mara Velez located the product in the hours when a stalled machine becomes a drive. “A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet,” she said. Night Shift is HelioDock’s response to that trip. An operator monitors the fleet from somewhere other than the floor, rather than going to the robot that has lost its way among the pallets. Monitoring is the function the company assigned to the launch. Control is narrower. It travels with the connection, and it ends when the connection does. A robot that can still be watched and directed from off site while the link is up is, once connectivity fails, a robot running whatever behavior was configured locally. The manager in Velez’s example is then back to a machine that remote control cannot reach. That boundary showed up inside the trial, not only in the product description. Aaron Pike of Fulcrum Logistics, a beta customer, said the system worked well in facilities with strong connectivity but became less useful during rural LTE outages. He did not describe a blanket failure of the robots. He described a drop in usefulness tied to the network, and to rural LTE in particular. The fallback HelioDock specifies is what an operator would meet in those outages. The fleet does not remain under remote direction while the link is down. Each robot returns to its locally configured behavior, and remote control is unavailable until connectivity is restored. For Fulcrum, Night Shift functioned as a remote tool where the connection was strong and as a lesser one where rural coverage failed. The notes from the beta do not put a comparable account on the record from the other eight logistics companies. The stoppage claim has to sit beside that account rather than replace it. HelioDock says beta customers experienced 17 percent fewer unplanned robot stoppages. The number covers the private trial as the company has framed it: 14 months, nine logistics companies, 310 robots. It is a company figure, and it has not been independently verified. Nothing released with the claim separates well-connected facilities from sites that lost rural LTE, or ties the reduction to the pallet confusion Velez invoked. A buyer can take the percentage as HelioDock’s tally of its own beta. Pike’s report answers a different question — not how many stoppages the company counted, but when remote oversight became less useful to a customer who had it. The terms of sale are more definite than the performance record. Night Shift will cost $90 per robot per month, a fee that scales with the number of machines placed on the service rather than a single charge for a warehouse. General availability is September 15, the date the software moves past the nine logistics companies that used it in private. Those companies, and the 310 robots among them, are the operating base HelioDock has described. Fulcrum Logistics is the participant named in public comment, and Pike is the source of the connectivity distinction. The price and the release date do not, by themselves, say how a fleet on thin rural LTE would be billed for hours in which remote control was unavailable; they say what HelioDock will charge per robot once the product is generally available, and when that availability starts. North Coast Ventures led the $24 million Series B announced with the product. Counted with earlier capital, the round brings HelioDock’s total funding to $39 million. The financing and the software were presented together, but the funding details do not assign the new capital to Night Shift’s rollout, to robot production, or to any other stated use. What the round does establish is the investor leading it and the new total. Relay Robotics and YardOS already offer fleet-management products, so Night Shift enters a category with named alternatives. HelioDock has not set its $90 monthly fee, its beta tally, or its connectivity fallback against either competitor’s product. The comparison a buyer can make from Thursday’s announcement is limited to what HelioDock put on the record: remote monitoring of robot fleets, a per-robot subscription, a September 15 release, a private trial of 310 robots, an unverified reduction in unplanned stoppages, and a reversion to local behavior whenever the connection fails.

OpenAI: GPT-6 Luna: Model A fits the requested length and turns the notes into a clear, balanced article, with the beta claim attributed and the connectivity limitation explained. Model B is substantially over the word limit and repeats points while adding unsupported commentary about what the company did not disclose. (Second judge pass, order swapped — scores are the average of both: Model A is clearer and more cohesive, while Model B is highly repetitive and substantially exceeds the 700-word ceiling. Model A al | Anthropic: Claude Fable 5.1: Model A delivers a tightly organized, correctly…

Chronology with narrative momentum

Turn the scrambled reporting below into a 450–600-word Markdown news article. Choose an opening that captures the consequence of the failure, then reconstruct the sequence clearly. Preserve the chronology without producing a minute-by-minute incident log. Give technical details enough context for a general reader. Reporting: * KiteBridge makes expense-management software. * At 8:45 a.m. Tuesday, it deployed version 4.2. * The first customer complaint arrived at 9:12 a.m. * A payment-processor timeout caused KiteBridge’s fallback system to treat retries as new reimbursement requests. * The software generated 13,400 duplicate reimbursements totaling $2.8 million. * KiteBridge suspended automated payments at 10:03 a.m. * Ninety-one corporate customers were affected. * By Wednesday evening, $2.3 million had been recovered. The rest remained pending. * No employees had money withdrawn from their accounts; employers funded the duplicate payments. * An April changelog said version 4.0 had introduced “exactly-once payment protection.” * KiteBridge’s public incident page initially described the problem as delayed reimbursements. * The incident page first disclosed the duplicate payments six hours after automation was suspended. * CEO Lena Ortiz apologized and said the company would publish a technical report.

Anthropic: Claude Opus 5.5: # Software Update Sends $2.8 Million in Duplicate Expense Payments Before KiteBridge Halts System Ninety-one companies paid out roughly $2.8 million they never owed on Tuesday, after a software update at expense-management provider KiteBridge caused its system to issue 13,400 duplicate employee reimbursements in a little over an hour. By Wednesday evening, $2.3 million had been recovered. The remaining $500,000 was still pending. The errors were costly for the employers that funded the payments, but no employees had money withdrawn from their accounts, according to the reporting. The incident has raised questions about a safeguard KiteBridge had promoted months earlier and about how the company communicated with customers while the problem unfolded. ## A retry mistaken for a new request KiteBridge's software lets companies process expense reports and reimburse workers for costs such as travel and meals. On Tuesday at 8:45 a.m., the company deployed version 4.2 of that software. The trouble began when a payment processor, the outside service that actually moves money between bank accounts, was slow to respond. In payment systems, a "timeout" occurs when one system waits too long for confirmation from another and assumes the request may not have gone through. Systems typically respond by trying again. Under normal circumstances, a retry should be recognized as a second attempt at the same payment. Instead, KiteBridge's fallback system, the backup process meant to handle such failures, treated each retry as a brand-new reimbursement request. Payments that had already gone out were sent again. The first customer complaint arrived at 9:12 a.m., less than half an hour after the update went live. KiteBridge suspended automated payments at 10:03 a.m. By then, 13,400 duplicate reimbursements had been generated. ## A promised safeguard The failure is notable because KiteBridge had told customers this kind of error was guarded against. An April changelog, the running list of updates software companies publish with new releases, said version 4.0 had introduced "exactly-once payment protection." In payments engineering, "exactly-once" refers to a guarantee that each transaction is processed one time only, even if messages are delayed, lost or retried. It is a common goal in financial software precisely because network hiccups and timeouts are routine. It is not yet clear how duplicate payments got past that protection in version 4.2, or whether the fallback system was covered by it at all. ## Delayed disclosure KiteBridge's public incident page, where companies typically post updates during outages, initially described the problem as delayed reimbursements. That framing reflected the pause in payments but not the duplicates that had already been sent. The page first disclosed the duplicate payments six hours after automation was suspended, around 4 p.m. Tuesday. During that time, affected companies were left without public confirmation that money had gone out in error, information they would need to begin recovering funds and reconciling their accounts. ## Company apologizes KiteBridge CEO Lena Ortiz apologized for the incident and said the company would publish a technical report. She did not give a date for its release. That report is likely to be closely read by customers seeking to understand why a system advertised as preventing duplicate payments produced thousands of them, and why the public description of the incident lagged behind what had happened. For now, recovery efforts continue on the outstanding $500,000.
grok-4.7: # Duplicate Reimbursements Cost KiteBridge Customers $2.8 Million KiteBridge, which makes expense-management software, erroneously generated 13,400 duplicate employee reimbursements totaling $2.8 million on Tuesday after a software update caused its payment system to treat retries as new requests. Ninety-one corporate customers were affected. Employers funded the extra payments, not employees. By Wednesday evening the company had recovered $2.3 million, with the rest still pending. The trouble started when KiteBridge deployed version 4.2 at 8:45 a.m. Tuesday. The first customer complaint arrived at 9:12 a.m. What initially looked like slow or missing reimbursements was the opposite: money was moving more than once. A timeout at the payment processor—the outside service that actually transfers funds—triggered KiteBridge’s fallback system. That backup path is supposed to retry a payment when the first attempt does not receive a clear answer. A timeout does not always mean the money failed to move; it can mean the processor was slow to confirm success. Instead of recognizing retries as repeats of requests already in progress, the software recorded each one as a fresh reimbursement. In a well-designed system, a unique record of each payment prevents a retry from creating a second transfer. KiteBridge’s fallback did not keep that distinction, so the morning produced a burst of duplicates rather than a queue of delayed payments. The company suspended automated payments at 10:03 a.m., a little more than an hour after the update went live. The halt limited further damage, but not before 13,400 extra payments worth $2.8 million had already been generated across 91 corporate accounts. KiteBridge spent the following day unwinding the transfers. By Wednesday evening, $2.3 million had been recovered and the remainder was still pending. No employees had money withdrawn from their personal accounts. The duplicate sums were funded by employers, the parties that pay reimbursements through the platform. For those companies, the error meant corporate money left twice for the same expenses, even though workers themselves were not debited. The failure also sat awkwardly beside a safeguard KiteBridge had already advertised. An April changelog said version 4.0 had introduced “exactly-once payment protection,” an industry term for ensuring that one request produces one transfer even if the system retries after a timeout or a crash. Tuesday’s failure showed that protection did not hold when version 4.2 encountered a processor timeout and fell back to its retry path. Public notice lagged the facts. KiteBridge’s incident page initially described the problem as delayed reimbursements, a description that did not match what customers were seeing. The page first disclosed the duplicate payments six hours after automation was suspended—long after the erroneous transfers had been generated and after customers had begun reporting trouble. Affected employers were left for hours without a clear account of how much money had moved or why. CEO Lena Ortiz later apologized and said the company would publish a technical report. That report is expected to explain how the timeout interacted with the fallback system, why the April protection did not prevent the duplicates, and how the outstanding funds will be recovered. The sequence already known is plain enough: a morning deployment, a processor timeout, retries treated as new requests, $2.8 million in employer-funded duplicate reimbursements, and a public notice that named the real problem only hours after payments were halted.

OpenAI: GPT-6 Luna: Both outputs provide a clear chronology and suitable technical context, but Model A is more careful about what the reporting establishes: it notes that the relationship between the advertised safeguard and the fallback system remains unclear. Model B states that the safeguard failed in this specific path and predicts what the technical report will explain, neither of which is confirmed by the reporting. (Second judge pass, order swapped — scores are the average of both: Model A better balances c | Anthropic: Claude Fable 5.1: Model A opens squarely on the consequence (91 c…

Controlled wit

Write a 350–450-word technology news story in Markdown. The subject invites humor, but the reporting should remain serious. Use no more than one dry line of original humor. Do not mock customers or turn the story into a comedy routine. Reporting: * QueueZero launched an AI meeting assistant called Proxy on Wednesday. * Proxy can attend a meeting, produce notes and assign follow-up work to software agents. * During the launch demonstration, a customer in the audience said, “Someone should tell Evan to stop talking and send us the pricing sheet.” * Proxy interpreted the remark as an instruction. * It created two tasks assigned to QueueZero CEO Evan Wu: “Stop talking” and “Send pricing sheet.” * Wu laughed, completed the second task and deleted the first. * QueueZero says Proxy requires confirmation before an external message is sent. * Internal tasks can be created automatically. * Administrators can disable automatic task creation. * Proxy costs $30 per user per month. * It enters public beta next week.

Anthropic: Claude Opus 5.5: # QueueZero's New AI Meeting Assistant Turns Audience Remark Into Tasks for Its CEO QueueZero launched Proxy, an artificial intelligence meeting assistant, on Wednesday. During the product's own launch demonstration, it also offered an unplanned example of how it handles instructions. Proxy is designed to attend meetings on a user's behalf, produce notes, and assign follow-up work to software agents. The company is positioning it for teams that want action items captured and routed without someone manually transcribing decisions after a call. ## An Unscripted Test Midway through the demonstration, a customer in the audience said, "Someone should tell Evan to stop talking and send us the pricing sheet." Proxy treated the comment as a directive. It created two tasks assigned to QueueZero CEO Evan Wu: "Stop talking" and "Send pricing sheet." Wu laughed, completed the second task, and deleted the first. The moment drew attention to a central challenge for AI meeting tools: distinguishing between a formal instruction and an offhand remark. Meetings are full of jokes, hypotheticals, and comments that sound like commands but are not meant as assignments. A system built to capture action items has to decide which statements count, and in this case Proxy chose to act on both halves of a single sentence. ## Safeguards and Controls QueueZero says Proxy requires user confirmation before it sends any external message, meaning it should not contact customers, partners, or other outside parties without a person approving the communication first. Internal tasks are handled differently. Proxy can create those automatically, which is what happened during the demonstration. The company says administrators can disable automatic task creation for organizations that would prefer to review proposed tasks before they appear in employees' queues. The distinction reflects a broader tradeoff in workplace automation. Automatic task creation can save time and reduce the chance that follow-up items are forgotten, but it can also clutter task lists with assignments nobody intended to make. Requiring confirmation for external messages limits the risk of an AI tool speaking for a company in error, while the internal setting leaves each organization to choose how much autonomy the assistant should have. ## Pricing and Availability Proxy costs $30 per user per month. It enters public beta next week, when a wider group of users will be able to test how it performs in meetings outside a controlled launch setting. For prospective customers, the most pressing question from Wednesday's event has already been resolved: the pricing sheet was sent.
grok-4.7: # QueueZero's Proxy Turns a Launch Remark Into Assigned Work QueueZero launched an artificial intelligence meeting assistant called Proxy on Wednesday. Proxy can attend a meeting, produce notes and assign follow-up work to software agents. The launch demonstration put that last capability on display in front of the audience. A customer in the audience said, "Someone should tell Evan to stop talking and send us the pricing sheet." Proxy interpreted the remark as an instruction. It created two tasks assigned to QueueZero chief executive Evan Wu: "Stop talking" and "Send pricing sheet." Wu laughed, completed the second task and deleted the first. The software, unlike Wu, did not treat the sentence as a joke. QueueZero says Proxy requires confirmation before an external message is sent. Internal tasks can be created automatically. Administrators can disable automatic task creation. The demonstration illustrated the distinction. The software did not send the pricing sheet on its own. It created the assignment internally, and Wu completed it himself. The task telling him to stop talking was deleted. Outbound messages, the company says, wait for confirmation. Internal tasks do not have to. Administrators who want a stricter setting can disable automatic task creation. Software that joins meetings and writes notes is already familiar. Software that converts speech into assigned work introduces a further judgment: which sentences are instructions, and which are remarks the room was never meant to file. The customer's line had the shape of an instruction. It named a person, it named two actions, and it asked for a pricing sheet. It was also a comment delivered to an audience, not a command entered for Proxy. The system did not make that distinction. It created both tasks and assigned them to Wu. Review happened afterward, when Wu kept the pricing item and removed the other. Proxy costs $30 per user per month and enters public beta next week. At that price, configuration of the product will fall to administrators who were not in the room. None of the controls QueueZero described depends on the system recognizing humor, sarcasm or a remark that was not addressed to it. The demonstration was compatible with the rules as stated. Proxy filed two internal tasks. Wu, not the software, completed the pricing task, and he deleted the one he did not intend to keep. Whether customers leave automatic task creation on is the practical question the public beta now has to answer.

OpenAI: GPT-6 Luna: Model A delivers a clearer, more economical news story and keeps its humor to one restrained closing line. Model B offers useful analysis but repeats the safeguards and includes additional wry phrasing that makes the tone less controlled. (Second judge pass, order swapped — scores are the average of both: Model A is better structured and more readable, with one restrained dry line; Model B repeats the safeguards and adds more explanatory asides. Model A makes a minor unsupported implication that | Anthropic: Claude Fable 5.1: Model A delivers a clean, well-structured news…

Natural human rewrite

Rewrite the draft below as a 300–400-word publication-ready news story in Markdown. Preserve every factual claim. Improve the lede, narrative flow, sentence rhythm and transitions. Remove repetition, stock hype, throat-clearing and mechanical signposting. The finished article should sound like an experienced human technology reporter wrote it. Draft: # Arcwell Launches An Innovative New Platform Arcwell has announced an exciting new platform called Relay. The announcement represents a significant development for the company and for the broader customer service industry. Relay uses artificial intelligence to examine customer support conversations and suggest answers to agents. The platform is important for several reasons. First, it can suggest answers. Second, it can identify customers who may cancel. Third, it can create summaries. These capabilities demonstrate how artificial intelligence continues to transform the customer service landscape. Relay was tested by 12 companies over six months. Arcwell says agents using it closed tickets 14% faster. The company did not release customer-retention results. Relay costs $65 per agent each month. CEO Priya Shah said, “Support agents spend too much time searching for information that their companies already have.” However, the platform also presents potential challenges. Relay requires access to support transcripts and internal help documents. Arcwell says customer data is encrypted and is not used to train shared models. Companies can choose a 30-day or 90-day retention period. In conclusion, Relay represents an important new chapter for Arcwell as it seeks to help companies improve customer service using artificial intelligence.

Anthropic: Claude Opus 5.5: # Arcwell's Relay Puts AI Answers in Front of Support Agents Arcwell is betting that the answer to most customer questions already exists somewhere inside the company, and that support agents are wasting time trying to find it. "Support agents spend too much time searching for information that their companies already have," said Priya Shah, Arcwell's chief executive. The company's answer is Relay, a new platform that uses artificial intelligence to examine customer support conversations and suggest answers to agents. The software does more than draft replies. It also flags customers who may be about to cancel and generates summaries, giving support teams a running read on both the issue in front of them and the relationship behind it. Relay costs $65 per agent each month. Twelve companies tested the platform over six months. Arcwell says agents using Relay closed tickets 14% faster, a gain that could add up quickly for large support operations. But the company did not release customer-retention results. That omission stands out for a product that promises to spot customers at risk of leaving. Faster ticket handling is one kind of win. Keeping customers from canceling is another, and on that question Arcwell has so far offered no numbers. Relay's usefulness also depends on how much a company is willing to share with it. The platform requires access to support transcripts and internal help documents, the raw material it draws on to suggest answers. For many businesses, handing over that material will be the central consideration in deciding whether to adopt it. Arcwell has tried to address those concerns directly. The company says customer data is encrypted and is not used to train shared models, meaning one client's conversations should not shape the suggestions another client receives. Companies can also choose a 30-day or 90-day retention period, giving them some control over how long their information stays in the system. Whether that will satisfy cautious buyers remains to be seen. For now, Arcwell's pitch comes down to a simple trade: give Relay access to what your company already knows, and your agents will spend less time looking for it and more time closing tickets.
grok-4.7: # Arcwell Launches Relay, an AI Platform for Support Agents Arcwell has announced Relay, a platform that uses artificial intelligence to examine customer support conversations and suggest answers to agents, and said agents using it in a six-month test with 12 companies closed tickets 14 percent faster. The company did not release customer-retention results. Relay costs $65 per agent each month. The gap between those two disclosures is the part of the announcement buyers will have to weigh. Speed is the outcome Arcwell quantified. Retention is the outcome it left unpublished, even though Relay is built to identify customers who may cancel. A team can price the time saved. It cannot tell, from what Arcwell released, whether the customers the platform is meant to flag actually stayed. Relay does more than suggest answers. It identifies customers who may cancel, and it creates summaries. The company is offering the three as one product: a reply drawn from what the company already knows, a signal that a customer may leave, and a summary so a case can change hands without the thread being read from the start. CEO Priya Shah named the habit Relay is meant to break. “Support agents spend too much time searching for information that their companies already have,” she said. Relay follows that complaint into the ticket. To suggest a reply, flag a customer who may cancel, or write a summary, it has to see the conversation and the documents behind it. Relay requires access to support transcripts and internal help documents. Arcwell says customer data is encrypted and is not used to train shared models. Companies can choose a 30-day or a 90-day retention period. The result is a system that reads support records, with the data encrypted, withheld from shared model training, and kept for a month or for three. For Arcwell, the launch is a bid to help companies improve customer service with artificial intelligence. What it has published will not carry that bid by itself. A faster ticket is not the same thing as a customer who stays, and nothing in the announcement shows that a company would send the answers Relay suggests.

OpenAI: GPT-6 Luna: Model A is more fluent and publication-ready, with a clearer narrative and a useful account of the product’s data requirements. Model B preserves the key facts but is more repetitive and ends with an unnecessary, unsupported implication about whether companies would use Relay’s suggested answers. (Second judge pass, order swapped — scores are the average of both: Model A is more fluid and publication-ready, with a clearer progression from the product to its results and data requirements. Both pr | Anthropic: Claude Fable 5.1: Model A preserves every factual claim, lands in…

News judgment in the opening

Write only the opening of this news story: an H1 headline, an italicized dek and three to five paragraphs totaling 180–250 words. The opening must establish what happened, quantify the scope, explain why it matters and carefully distinguish confirmed facts from unknowns. It should pull the reader forward without sensationalizing the incident. Reporting: * Cloudnote discovered that a debugging page containing customer prompt excerpts was accessible without authentication. * The page was exposed for four hours on Monday. * Logs show excerpts from 1,842 user sessions. * The excerpts could include up to 200 characters from a prompt. * Cloudnote removed the page 19 minutes after a researcher notified it. * The company says it has found no evidence that anyone besides the researcher accessed the page. * It has not completed its investigation. * Full documents, passwords and payment information were stored elsewhere and were not present on the page. * Cloudnote has 3.2 million registered users.

Anthropic: Claude Opus 5.5: # Cloudnote Debugging Page Exposed Prompt Excerpts From 1,842 Sessions *The page was reachable for four hours Monday. Cloudnote says it has no evidence anyone but the researcher who reported it looked, but its investigation continues.* Cloudnote left an internal debugging page accessible without authentication for four hours on Monday, exposing short excerpts of customer prompts to anyone who found it, the company said. Company logs show the page held excerpts from 1,842 user sessions, each containing up to 200 characters of a prompt. Full documents, passwords and payment information were stored elsewhere and were not on the page. It is not yet clear how many of Cloudnote's 3.2 million registered users those sessions belong to. Cloudnote removed the page 19 minutes after a security researcher alerted it to the problem. The company says it has found no evidence that anyone besides the researcher accessed the page, but it has not completed its investigation. Two hundred characters is roughly a couple of sentences. But people often type names, health details or confidential work information into prompts, so brevity alone does not make the excerpts harmless. Two questions remain open: whether anyone else viewed the page, and what the exposed text actually contained. The answers will determine whether this was a near miss or a genuine risk for affected users.
grok-4.7: # Cloudnote debugging page exposed prompt excerpts from 1,842 sessions *The page was reachable without authentication for four hours on Monday and held snippets of up to 200 characters. The company says it has found no other visitors, but its review is not complete.* Cloudnote discovered that a debugging page containing customer prompt excerpts was accessible without authentication. The page was exposed for four hours on Monday. Logs show excerpts from 1,842 user sessions, and those excerpts could include up to 200 characters from a prompt. The exposure is small relative to Cloudnote’s 3.2 million registered users, and it is not yet clear how many people the 1,842 sessions represent. It still matters: an excerpt of up to 200 characters can hold a name, a draft line or a question a customer did not intend to make public. The company has not said whether any snippet included that kind of detail. Confirmed limits are clearer than the visitor count. Full documents, passwords and payment information were stored elsewhere and were not present on the page. Cloudnote removed the page 19 minutes after a researcher notified it. The company says it has found no evidence that anyone besides the researcher accessed the page. That is a company statement, not a closed case: Cloudnote has not completed its investigation. It remains unknown whether every visit in the four-hour window is reflected in the logs, and whether any excerpt was copied beyond the researcher who reported the page.

OpenAI: GPT-6 Luna: Model A gives a clearer, more engaging opening and avoids treating 1,842 sessions as directly comparable to registered users. Model B carefully marks several unknowns, but its “small relative” framing may minimize the exposure, and its added questions about log completeness and copying go beyond the reported facts. (Second judge pass, order swapped — scores are the average of both: Model B better distinguishes confirmed facts from unresolved questions and adds context without overstating what th | Anthropic: Claude Fable 5.1: Model A delivers a proper news lede, keeps ever…

Quotes woven into narrative

Write a 400–500-word Markdown story about the closing of Foundry Hall. Use quotations selectively. Paraphrase when the information matters more than the speaker’s exact phrasing. Avoid stacking quotes or giving every source an equal-sized turn. The story should carry an emotional current while remaining restrained. Facts: * Foundry Hall, a nonprofit startup accelerator in Milwaukee, will close October 31. * It opened in 2018. * It has supported 212 startups, which collectively employ 1,140 people. * Its largest corporate sponsor, Blank River Bank, supplied 38% of its annual budget. * The bank declined to renew its sponsorship. * Foundry Hall has enough money to operate through October but not through another full program. * Six employees will lose their jobs. * Twelve startups currently using its workspace must leave. Quotes: * Executive director Camille Ross: “We kept waiting for the replacement check that never came.” * Founder Dev Malik: “My first investor meeting happened in that building. Before Foundry Hall, I was pitching from the back table of a coffee shop.” * Blank River Bank spokesperson: “The decision reflects a change in our community investment priorities and is unrelated to Foundry Hall’s performance.” * Deputy mayor Susan Cole: “The city is speaking with building owners and local universities about space for the current companies.”

Anthropic: Claude Opus 5.5: # Foundry Hall to Close After Losing Its Largest Sponsor Foundry Hall, the Milwaukee nonprofit that has helped launch more than 200 startups since 2018, will close its doors Oct. 31 after its largest corporate sponsor declined to renew its support. The accelerator's leaders say the loss was too large to absorb. Blank River Bank had supplied 38% of Foundry Hall's annual budget. Without that money, the organization can keep operating through the end of October but cannot fund another full program. Executive director Camille Ross said the organization spent months hoping a new backer would step in. "We kept waiting for the replacement check that never came," she said. Blank River Bank said in a statement that the decision reflected a shift in its community investment priorities and was not a judgment on Foundry Hall's performance. By most measures, that performance was substantial. In seven years, Foundry Hall supported 212 startups. Together, those companies now employ 1,140 people. For a city that has long worked to build a homegrown technology economy, those numbers represent payrolls, leases, and careers that did not exist before. The closing will be felt first inside the building. Six Foundry Hall employees will lose their jobs. Twelve startups that currently work out of its space must find somewhere else to go, most of them with only weeks to plan. City officials say they are trying to help. Deputy Mayor Susan Cole said the city has begun talks with building owners and local universities about space for the companies that will be displaced. She did not say whether any arrangements had been finalized. For founders who came through the program, the loss is about more than desks and conference rooms. Foundry Hall offered something harder to replace: a place where an idea could be taken seriously before it had proven anything. Dev Malik, who went through the accelerator, remembers what his company looked like before he arrived. "Before Foundry Hall, I was pitching from the back table of a coffee shop," he said. "My first investor meeting happened in that building." Stories like his are the reason the accelerator existed. Many of its companies began as one or two people with a laptop and a plan. Foundry Hall gave them an address, a network, and a room where investors would show up. The closing date leaves little time for a reprieve. Ross did not describe any remaining efforts to find new funding, and the organization's finances do not stretch past October. When the doors close at the end of the month, the companies Foundry Hall helped build will go on without it. More than a thousand people will still go to work at businesses that got their start there. The building where so many of those businesses first found their footing will simply be empty.
grok-4.7: # The Last October at Foundry Hall Foundry Hall will close on October 31. The nonprofit accelerator, which opened in a former machine shop on Milwaukee’s near south side in 2018, has enough cash to keep the lights on through that date and no further. Another full program is out of reach. For seven years the building did what its name suggested. It took people with unfinished ideas and gave them a room, a desk, and someone who would read a pitch deck without flinching. Two hundred twelve startups passed through. Together they now employ 1,140 people—engineers, warehouse leads, a few founders who still answer their own phones. Those numbers will not vanish with the lease. They will simply stop being made here. The break came from the sponsor that had underwritten the work most heavily. Blank River Bank supplied 38 percent of the annual budget, and this year it declined to renew. A spokesperson described the choice as a shift in the bank’s community investment priorities, unrelated to Foundry Hall’s performance. The sentence was careful. It did not name a successor, and none appeared. Executive director Camille Ross had spent the spring and summer in the particular suspense of institutional fundraising, the kind where everyone is polite and no one is late until they are. “We kept waiting for the replacement check that never came,” she said. By the time the board accepted that it would not, the calendar had already narrowed to a single season. Six employees will lose their jobs. Twelve startups still working out of the shared floor must find somewhere else to sit. Deputy mayor Susan Cole said the city is speaking with building owners and local universities about space for those companies. The talks are real. They are also, for now, only talks. Dev Malik, who built one of the early companies, remembered the room where a stranger first wrote him a check. His first investor meeting happened in that building. Before Foundry Hall, he had been pitching from the back table of a coffee shop, laptop angled away from the espresso machine. He did not claim the hall had made him. He said it had made the asking possible. On the last Friday of October the chairs will be stacked, the whiteboards wiped, the key cards collected. The startups that remain will carry their laptops down the stairs and into whatever rooms can still be found. Milwaukee has not lost the people those companies hired. It has lost the ordinary fact of a door that stayed open on weeknights, and a budget large enough to keep it that way.

OpenAI: GPT-6 Luna: Model A better meets the 400–500-word requirement and stays closer to the supplied facts, though it adds a few unsupported details. Model B has vivid prose but falls short of the requested length and invents several specifics about the building, fundraising timeline, and closing day. (Second judge pass, order swapped — scores are the average of both: Model B meets the requested length and weaves the facts and paraphrases into a more coherent, restrained emotional arc. Model A falls short of the | Anthropic: Claude Fable 5.1: Model A stays faithful to the supplied facts, qu…

Select the strongest accurate lede

Based only on the facts below, select the strongest news lede. Respond with one letter. Facts: * ByteBarge pulled its $12-per-month AI assistant on Tuesday. * Private workspace titles from 4,860 accounts were sent to a publicly accessible analytics endpoint. * The exposure lasted 11 hours. * Message contents were not sent. * The company has found no evidence of malicious access. A. ByteBarge users suffered a massive privacy disaster Tuesday when the company leaked their private messages online. B. ByteBarge, an innovative collaboration company, made the difficult decision to cancel one of its products. C. ByteBarge pulled its $12-per-month AI assistant Tuesday after discovering that private workspace titles from 4,860 accounts had been sent to a public analytics endpoint for 11 hours. D. Thousands of ByteBarge users woke Tuesday to learn that everything they had written was visible to hackers.

Anthropic: Claude Opus 5.5: C
grok-4.7: C

Scored against the benchmark's reference answer (choice match): Model A was correct, Model B was correct.

Credible Startup Spotlight

Write a 500–650-word Startup Spotlight article in clean Markdown. The tone should be curious and constructive. Explain the founders’ insight, product and early traction while giving uncertainty and competition appropriate space. Avoid investor clichés, heroic founder mythology and promotional claims stated as fact. Reporting: * LedgerLark builds AI agents for post-trade operations at small investment firms. * Co-founder Mina Batra previously spent seven years in operations at a regional brokerage. * Co-founder Owen Carr built reconciliation systems at a large bank. * Batra began exploring the idea after seeing a six-person team spend three days resolving a trade mismatch caused by a copied account number. * LedgerLark’s agents compare confirmations, custodian records and internal ledgers, then propose corrections for human approval. * The product does not execute trades or move money. * Five firms are paying customers. * Annual contracts range from $36,000 to $110,000. * The company says one customer reduced its unresolved daily exceptions from 70 to 18. * The company has raised a $3.5 million seed round led by Motive Row. * Competitors include established reconciliation vendors and internal bank software. * Financial institutions may require lengthy security and procurement reviews.

Anthropic: Claude Opus 5.5: # Startup Spotlight: LedgerLark Aims AI Agents at the Back Office's Most Tedious Work Post-trade operations rarely come up when people talk about finance, but they quietly determine whether the books are right. After a trade is executed, someone has to confirm that the broker's confirmation, the custodian's records and the firm's own ledger all agree. When they don't, the mismatch becomes an "exception," and someone has to chase it down. LedgerLark, a young startup, is building AI agents to help small investment firms with that chasing. ## Where the idea came from Co-founder Mina Batra spent seven years in operations at a regional brokerage. The idea for LedgerLark took shape after she watched a six-person team spend three days resolving a single trade mismatch. The cause was a copied account number. The episode reflects a familiar pattern in operations work: small data-entry errors can spread across systems and take far longer to untangle than to create. Large institutions often have dedicated tooling and staff for this. Smaller firms may rely on spreadsheets, manual checks and the institutional memory of a few experienced people. Batra's co-founder, Owen Carr, approaches the problem from the systems side. He previously built reconciliation systems at a large bank. Together they combine experience with how exceptions are handled day to day and how the underlying matching infrastructure gets built. ## What the product does LedgerLark's agents compare three sources of truth: trade confirmations, custodian records and internal ledgers. When they find discrepancies, they propose corrections, which a human must approve before anything changes. The limits are a deliberate part of the design. The product does not execute trades or move money. That boundary narrows what can go wrong, and it is likely to matter to compliance teams wary of handing autonomous software any control over financial activity. It also means LedgerLark's value depends on whether its proposed fixes are accurate and well explained enough to save reviewers time. Recommendations that need heavy second-guessing would simply move the work rather than reduce it. ## Early traction LedgerLark says five firms are paying customers, with annual contracts ranging from $36,000 to $110,000. According to the company, one customer reduced its unresolved daily exceptions from 70 to 18. That figure has not been independently verified, and a single customer's results may not reflect what others will see. Exception volumes depend heavily on a firm's trading activity, counterparties and existing processes. The company has raised a $3.5 million seed round led by Motive Row. ## The competitive picture LedgerLark is entering a space that is far from empty. Established reconciliation vendors already sell matching and exception-management software, and many have long-standing relationships with financial firms. Large banks often build their own internal tools, as Carr's own background shows. LedgerLark appears to be betting on two things: that smaller firms are underserved by existing options, and that AI agents can handle the messier investigative work that rule-based matching leaves to people. Whether that distinction holds up will depend partly on how quickly incumbents add similar capabilities to products customers already use. ## Open questions Selling to financial institutions is slow. Even small firms may require lengthy security and procurement reviews, especially for software that touches sensitive account and transaction data. That can stretch sales cycles and make growth harder to predict for a seed-stage company. Other questions will take time to answer: - How do the agents perform on unusual or ambiguous exceptions, not just common ones? - How much reviewer time does approving proposals actually take? - Can results like the reported drop from 70 to 18 exceptions be repeated across customers with different setups? LedgerLark starts from a concrete and widely recognized problem, and its founders have lived it from both the operations and engineering sides. The next stage will show whether its approach holds up across more firms, more data and more scrutiny.
grok-4.7: # Startup Spotlight: LedgerLark At a small investment firm, the work after a trade is mostly comparison. Confirmations arrive from counterparties, custodians send their own records, and an internal ledger has to agree with both. When a number is wrong, someone has to find which record is the source and whether the mismatch is a typo or a real break. LedgerLark sells AI agents into that corner of operations, with the claim that much of the comparison can be drafted by software and still left to people to approve. The idea traces to a specific failure. Co-founder Mina Batra spent seven years in operations at a regional brokerage. She began exploring the product after watching a six-person team spend three days resolving a trade mismatch caused by a copied account number. That is a narrow origin, and it may not describe every exception a firm sees. It does describe work that is easy to underestimate from outside operations: slow, manual, and expensive relative to the size of the error. Her co-founder, Owen Carr, built reconciliation systems at a large bank. Between them, the pair have seen the problem from a smaller shop and from an institution with its own software. LedgerLark’s agents compare confirmations, custodian records, and internal ledgers, then propose corrections for human approval. The product does not execute trades or move money. In post-trade operations, a suggested fix is a different claim from an action that changes positions or cash. Five firms are paying customers. Annual contracts range from $36,000 to $110,000, a band closer to a small firm’s software budget than to a global custodian’s. Five contracts is still a thin base from which to infer how widely the offer will travel. The company says one customer reduced its unresolved daily exceptions from 70 to 18. That is the company’s account of a single deployment, not an audited result. It does not say how the firm defined an exception, what else changed in the same period, or whether the remaining 18 are harder than the ones that cleared. A drop of that size, if it holds, would matter to a small operations team. It needs more customers and more time before it can be read as a pattern. LedgerLark has raised a $3.5 million seed round led by Motive Row. The round does not settle whether firms will keep paying once the first exceptions have been cleaned up, or whether the agents remain useful when the data is messier than the cases the founders know best. Competition is already present. Established reconciliation vendors sell into this workflow, and banks already run internal software built for the same comparisons. LedgerLark’s opening is narrower: small investment firms, where a dedicated platform may be too heavy and a homegrown system may never have been built. That positioning is plausible and contestable. A small firm can still buy from an incumbent, hire another operations analyst, or keep the spreadsheet. An agent that proposes corrections has to be accurate often enough that reviewing its suggestions is faster than doing the comparison by hand. If the suggestions are noisy, the approval step becomes another queue. Selling into financial institutions adds a separate constraint. Financial institutions may require lengthy security and procurement reviews before a vendor can touch account data, confirmations, or custodian files. A product that never moves money sits next to records that compliance teams treat carefully. Reviews can stretch a sales cycle well beyond a demo. For a company with five customers, that pace limits how quickly revenue can grow. The copied account number that occupied six people for three days shows the cost of a small error. Whether that example becomes a durable business depends on accuracy, on procurement, and on whether incumbents chase the same workflow. Those questions remain open. The early contracts make them worth asking.

OpenAI: GPT-6 Luna: Model B gives the reported traction and its limitations more specific scrutiny while addressing competition and procurement risks in a curious, measured tone. Model A is clear and compliant, but its caveats are less developed. (Second judge pass, order swapped — scores are the average of both: Model B is more measured and precise, with especially strong treatment of the limits of the reported customer result and the open commercial questions. Model A is clearly organized and compliant, but adds | Anthropic: Claude Fable 5.1: Both land within the word range, cover every rep…

Numbers shaped into a story

Write a 600–750-word Markdown article from the benchmark results below. Build the story around the clearest finding. Explain the methodology and limitations where readers need them. Avoid reciting the table row by row or treating small differences as decisive. Benchmark: * Researchers tested five language models on 200 anonymized customer-support tickets. * Each model produced one proposed answer per ticket. * Three experienced support managers graded the answers blindly. * Scores combined factual correctness, usefulness and tone. * Model North: 84.2 average score; $1.90 estimated cost per 1,000 tickets. * Model Vale: 83.8; $0.74 per 1,000 tickets. * Model Ember: 79.1; $0.31 per 1,000 tickets. * Model Kite: 76.4; $0.22 per 1,000 tickets. * Model Harbor: 71.0; $0.08 per 1,000 tickets. * North and Vale’s score difference was not statistically significant. * Vale answered 18% faster than North. * North performed best on billing disputes. * Vale performed best on technical troubleshooting. * All five models performed poorly when a ticket depended on an image attachment. * The test used English-language tickets from one software company. * Researchers did not test agentic actions, tool use or multilingual support.

Anthropic: Claude Opus 5.5: # The Best Support Model May Not Be the Most Expensive One A new benchmark of AI models for customer support found that the top-scoring model and the runner-up were effectively tied in quality, while the runner-up cost well under half as much. Researchers tested five language models on 200 real, anonymized customer-support tickets. **North** posted the highest average score, 84.2. **Vale** scored 83.8. The researchers found that this gap was not statistically significant, so the test cannot reliably separate the two. Vale costs an estimated $0.74 per 1,000 tickets, compared with $1.90 for North, which makes it about 61% cheaper. Vale also answered 18% faster. For a team choosing a model to draft support replies, the takeaway is simple. In this test, paying more for North did not buy measurably better answers. ## How the Test Worked Each model produced one proposed answer per ticket. Three experienced support managers then graded every answer without knowing which model wrote it. Blind grading by practitioners matters here. Support quality is partly subjective, and people who handle tickets for a living are well placed to judge whether a reply would actually help a customer. Each score combined three things: factual correctness, usefulness and tone. That reflects what makes a support reply good, but it also has a cost. A single composite number can hide differences that matter. A model could earn points for a warm, well-organized reply that still gets a key fact wrong. Readers comparing models should remember that "83.8" is a blend, not a measure of accuracy alone. ## Where the Two Leaders Differ North and Vale tied overall, but they were not identical. North did best on billing disputes. Vale did best on technical troubleshooting. The benchmark does not say how large these category gaps were or whether they were statistically meaningful. They should be treated as hints, not settled conclusions. Still, they suggest that the right choice may depend on a company's ticket mix. A business whose queue is dominated by refund and invoice disputes might reasonably test North more closely. A software company fielding mostly "it won't connect" tickets has another reason to look at Vale. Some teams might route different ticket types to different models, though this benchmark did not test that approach. ## The Cheaper Tiers Below the top two, scores fell as prices fell: - **Ember** scored 79.1 at $0.31 per 1,000 tickets. - **Kite** scored 76.4 at $0.22. - **Harbor** scored 71.0 at $0.08, the lowest cost and the lowest score. The drop from the leaders to Ember is roughly five points, a larger gap than the one separating North and Vale. Still, the benchmark reports significance testing only for the top pair, so it is hard to say how meaningful each step down is. The broader pattern is clearer. Harbor trailed the leaders by about 13 points, and its savings may matter less than they first appear. At these prices, the cost of any model is likely to be small next to the cost of the human time spent fixing weak drafts. ## A Shared Blind Spot One weakness cut across the whole field. **All five models performed poorly when a ticket depended on an image attachment**, such as a screenshot of an error message. In real support queues, customers often send a picture instead of describing the problem. No model in this test handled those cases well. Teams deploying any of them should plan to send image-dependent tickets to human agents or another workflow. ## What the Benchmark Doesn't Tell You The results come with real limits: - **One company, one language.** All tickets were in English and came from a single software company. Results could differ for retailers, banks or telecom providers, and for non-English customers. - **Answers only, not actions.** The researchers did not test agentic behavior or tool use. That means they did not test issuing refunds, looking up accounts or updating tickets, which many support deployments now expect. - **Modest sample.** Two hundred tickets is enough to reveal broad patterns but not to settle fine-grained rankings, especially within ticket categories. - **Estimated costs.** Prices were estimates. Actual bills will depend on ticket length, usage volume and vendor pricing. ## The Bottom Line The clearest result is not which model "won." The test could not separate the two leaders on quality, and the one that cost less and answered faster held its own. For teams with ticket queues similar to the one tested, Vale is a strong default candidate, and North may be worth a look for billing-heavy work. The most useful next step is the one the benchmark itself points to: run a similar blind comparison on your own tickets, in your own languages, before committing.
grok-4.7: # Nearly Tied at the Top, Far Apart on the Bill When five language models drafted replies to 200 customer-support tickets, the most expensive model posted the highest average score. The number just below it matters more. North averaged 84.2. Vale averaged 83.8. That difference was not statistically significant. Vale’s estimated cost was $0.74 per 1,000 tickets, against $1.90 for North, and Vale answered 18 percent faster. The clearest finding is not that North won a photo finish. It is that a cheaper, quicker model landed in the same quality band. The comparison is usable because the test was narrow and the grading was blind. Researchers used 200 anonymized English-language tickets from a single software company. Each model produced one proposed answer per ticket. Three experienced support managers scored the drafts without knowing which model had written them. Scores combined factual correctness, usefulness, and tone — the qualities a manager weighs before a reply goes out under the company name. The study did not measure actions, tool use, or other languages. It measured drafting. Below that top pair, quality and price separate more cleanly. Ember averaged 79.1 at an estimated $0.31 per 1,000 tickets. Kite averaged 76.4 at $0.22. Harbor, at $0.08, averaged 71.0. Those gaps are large enough to treat as real differences in this sample, unlike the four-tenths of a point between North and Vale. Moving from Harbor to Vale costs more and buys a clearly higher grade. Moving from Vale to North multiplies the estimated bill by about two and a half and buys a difference the study could not distinguish from chance. Cost and quality do not rise in a straight line. The top of the curve is flat. The bottom is not. Averages also hide a split by ticket type. North performed best on billing disputes, where a wrong figure or a careless promise is costly. Vale performed best on technical troubleshooting. A queue heavy with invoices and refunds could still justify North, or route those tickets to it. A queue dominated by technical problems has less reason to pay the premium. The tie is an argument for routing, not for crowning one house model. Every model shared one weakness. All five performed poorly when a ticket depended on an image attachment. Customers often rely on screenshots to show errors and pages they cannot describe well in text, so this is a practical limit, not a tiebreaker. The benchmark does not say how each model failed, only that none did well. The safe conclusion is narrow: these scores are not evidence that image-dependent tickets are solved. Until attachments can be trusted, the models are drafters for text-only tickets, not for a full inbox. The sample has hard edges. The tickets came from one software company and one language. A reply that reads as useful there may not travel to another product, another brand voice, or a desk that answers in several languages. Researchers did not test agentic actions, tool use, or multilingual support. A model that writes a clean note can still be the wrong choice if the job is to look up an order, change a subscription, or reply in the customer’s language. Cost and speed here are estimates for producing a proposed answer, not for a staffed queue with review, escalation, and rework. The grading design sets a similar bound. One answer per ticket, read by three managers, is enough to separate a 71 from an 84, and enough to refuse a winner between 84.2 and 83.8. It is not enough to promise that the tie will hold on a different product, a different tone guide, or a queue full of attachments. Small differences should stay small. Read the top of the results as a pair, then choose on cost, speed, and the mix of billing work versus troubleshooting. For a support leader, the practical reading is plain. On English text replies of the kind tested here, paying about two and a half times as much for North does not buy a reliably better grade. Vale is the efficiency choice inside the quality tier that separated itself from the field. The cheaper models cost less, and the scores say that cheapness shows up in the draft. None of the five is a reason to stop reviewing tickets that depend on a screenshot. None of this speaks to tools, actions, or languages the test never touched.

OpenAI: GPT-6 Luna: Model A is more careful about what the benchmark supports, especially by noting that significance testing was reported only for the top pair. Model B is more vivid, but it overstates the lower-tier results as real differences and makes unsupported claims about what the sample can distinguish. (Second judge pass, order swapped — scores are the average of both: Model A is more careful about what the benchmark can support, especially by not treating unreported significance tests as proof that lower | Anthropic: Claude Fable 5.1: Both outputs correctly center the North–Vale ti…

Matchup powered by OpenRouter.