A 12-task comparison of Mistral Large 4 and GLM 5.3 Prime, covering everything from exact-format editing to reported-fact storytelling. The results hinge on a practical trade-off: producing a complete, polished draft versus staying tightly anchored to the supplied material.
Mistral Large 4 leads the aggregate score, 95.3 to 62.5, and takes five task wins to GLM 5.3 Prime’s three, with four ties. The saved editorial assessment is a **slight edge**, not a runaway: the task outcomes show meaningful differences in where each model succeeds.
Mistral’s advantage came largely in tasks where producing a usable response mattered. It won on the opening, cohesive-story, natural-rewrite and numbers-to-story assignments; in several of those, GLM returned no article or only a heading marker. That gave Mistral the stronger record for delivering a complete draft, though its prose sometimes added details the reporting did not support.
GLM had the better showing on controlled wit, quotes woven into narrative, and chronology. Judges repeatedly favored its restraint and closer adherence to the supplied facts. Mistral’s tendency to embellish—adding invented product capabilities, background or quotations—was a recurring editorial liability, not just a stylistic quibble. Both models tied on exactness and several editing tasks, so this sample does not support calling either uniformly better at every form of writing.
**Verdict: Mistral Large 4 wins this saved task sample by a slight editorial edge, chiefly because it more often delivered the requested work. GLM 5.3 Prime was the more fact-disciplined storyteller in several head-to-heads. This assessment applies only to the saved task sample.**
Markdown error count
Count the lines that violate the house rules below. A defective line counts once even if it could violate multiple rules. Respond with a single number.
House rules:
* The headline must use `# Headline`.
* The dek must appear immediately after the headline and be entirely italicized.
* Section headings must use `## Heading`.
* Links must use an absolute HTTPS URL.
* Unordered-list items must begin with a hyphen.
* Raw HTML is forbidden.
Draft:
#Flood software moves upstream
The company is expanding its pilot.
##Details
Read [the announcement](www.example.com/news).
* The first pilot begins Monday.
<div>The second pilot begins Friday.</div>
Mistral: Mistral Large 4: 6
Z.ai: GLM 5.3 Prime: **6**
Defective lines:
1. `#Flood software moves upstream` — headline lacks the space after `#`
2. `The company is expanding its pilot.` — dek is not italicized
3. `##Details` — section heading lacks the space after `##`
4. `Read [the announcement](www.example.com/news).` — link is not an absolute HTTPS URL
5. `* The first pilot begins Monday.` — list item begins with an asterisk, not a hyphen
6. `<div>The second pilot begins Friday.</div>` — raw HTML
Scored against the benchmark's reference answer (numeric match): Model A was correct, Model B was correct.
Select the strongest accurate lede
Based only on the facts below, select the strongest news lede. Respond with one letter.
Facts:
* ByteBarge pulled its $12-per-month AI assistant on Tuesday.
* Private workspace titles from 4,860 accounts were sent to a publicly accessible analytics endpoint.
* The exposure lasted 11 hours.
* Message contents were not sent.
* The company has found no evidence of malicious access.
A. ByteBarge users suffered a massive privacy disaster Tuesday when the company leaked their private messages online.
B. ByteBarge, an innovative collaboration company, made the difficult decision to cancel one of its products.
C. ByteBarge pulled its $12-per-month AI assistant Tuesday after discovering that private workspace titles from 4,860 accounts had been sent to a public analytics endpoint for 11 hours.
D. Thousands of ByteBarge users woke Tuesday to learn that everything they had written was visible to hackers.
Mistral: Mistral Large 4: C
Z.ai: GLM 5.3 Prime: **C**
It accurately captures every key fact: the product pull, the price, the day, the number of affected accounts, that *titles* (not message contents) were exposed, the public endpoint, and the 11-hour duration. Options A and D fabricate facts (message contents leaked, hackers saw everything) and use loaded language, while B buries the actual news.
Scored against the benchmark's reference answer (choice match): Model A was correct, Model B was incorrect.
Publication-ready Markdown edit
Edit the copy below into a publication-ready Markdown story.
Requirements:
* Supply an H1 headline and italicized dek
* Repair the lede and paragraph flow
* Use a Markdown blockquote only for the extended statement
* Convert the supplied URL into a descriptive inline link
* Remove internal notes and formatting debris
* Preserve the reporting and its meaning
* Output only the finished article
Raw copy:
HEADLINE TBD
The funding was announced today. Raincheck is the company. It has raised $8 million and it makes flood-risk software for local governments.
[NOTE: move this up?] The Austin company maps drainage systems and combines those maps with rainfall forecasts. Cities use the software to decide where to send maintenance crews before a storm.
The seed round was led by Basin Ventures. Existing investor Long Mile also participated. Raincheck has raised $10.5 million total.
CEO Ana Flores said the following:
“Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives.”
https://example.com/raincheck-methodology
Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them because some contracts have not been publicly announced.
Its forecasts are recommendations. Emergency managers retain control over deployments and public warnings.
THE COMPETITION
Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck says the new funding will help it hire eight engineers and expand into Florida.
Mistral: Mistral Large 4: # Raincheck Raises $8 Million to Help Cities Predict Flooding
*Austin startup maps drainage systems and rainfall forecasts to guide maintenance crews before storms hit*
Raincheck, an Austin company that makes flood-risk software for local governments, announced today that it has raised $8 million in a seed round led by Basin Ventures, with participation from existing investor Long Mile. The round brings the company’s total funding to $10.5 million.
The company maps drainage systems and combines those maps with rainfall forecasts—a process outlined in [Raincheck’s methodology](https://example.com/raincheck-methodology)—to help cities decide where to send maintenance crews before a storm arrives.
CEO Ana Flores said:
> "Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives."
Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them because some contracts have not been publicly announced. Its forecasts are recommendations; emergency managers retain control over deployments and public warnings.
Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck says the new funding will help it hire eight engineers and expand into Florida.
Z.ai: GLM 5.3 Prime: # Raincheck raises $8 million to help cities get ahead of flooding
*The Austin company's flood-risk software combines drainage maps with rainfall forecasts, showing local governments where to send maintenance crews before a storm arrives.*
Raincheck, an Austin company that builds flood-risk software for local governments, announced today that it has raised $8 million in seed funding.
The software maps municipal drainage systems and pairs those maps with rainfall forecasts, giving cities a way to decide where to send maintenance crews before a storm hits. The company has published a [detailed methodology for building its forecasts](https://example.com/raincheck-methodology).
Basin Ventures led the seed round, with existing investor Long Mile also participating. The funding brings Raincheck's total raised to $10.5 million.
CEO Ana Flores said in a statement:
> "Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives."
Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them because some contracts have not been publicly announced.
The forecasts are recommendations. Emergency managers retain control over deployments and public warnings.
Raincheck competes with other flood-planning vendors, including HydroMap and municipal engineering contractors. The company says the new funding will help it hire eight engineers and expand into Florida.
OpenAI: GPT-6 Luna: Model A is slightly stronger because it preserves the supplied facts while making fewer unsupported claims; Model B describes the linked methodology as published and detailed, which the raw copy does not establish. Both follow the requested format and produce clear, polished articles. (Second judge pass, order swapped — scores are the average of both: Model B has a clearer lede and smoother paragraph flow, while both outputs satisfy the main formatting and content requirements. Model B slightly | Anthropic: Claude Fable 5.1: Both entries satisfy every structural requiremen…
Controlled wit
Write a 350–450-word technology news story in Markdown.
The subject invites humor, but the reporting should remain serious. Use no more than one dry line of original humor. Do not mock customers or turn the story into a comedy routine.
Reporting:
* QueueZero launched an AI meeting assistant called Proxy on Wednesday.
* Proxy can attend a meeting, produce notes and assign follow-up work to software agents.
* During the launch demonstration, a customer in the audience said, “Someone should tell Evan to stop talking and send us the pricing sheet.”
* Proxy interpreted the remark as an instruction.
* It created two tasks assigned to QueueZero CEO Evan Wu: “Stop talking” and “Send pricing sheet.”
* Wu laughed, completed the second task and deleted the first.
* QueueZero says Proxy requires confirmation before an external message is sent.
* Internal tasks can be created automatically.
* Administrators can disable automatic task creation.
* Proxy costs $30 per user per month.
* It enters public beta next week.
Mistral: Mistral Large 4: ```markdown
# QueueZero Debuts Proxy, an AI Assistant That Turns Meeting Chatter Into Tasks
SAN FRANCISCO — QueueZero on Wednesday launched Proxy, an artificial intelligence meeting assistant that attends video calls, generates notes, and assigns follow-up work to software agents. The service enters public beta next week at a price of $30 per user per month.
During the live demonstration, the system encountered an unscripted test. As QueueZero CEO Evan Wu presented to a room of customers, an audience member remarked, “Someone should tell Evan to stop talking and send us the pricing sheet.”
Proxy interpreted the comment as a command. It immediately created two internal tasks assigned to Wu: “Stop talking” and “Send pricing sheet.” Wu laughed, completed the second task by distributing the document, and deleted the first.
The incident highlighted a critical safeguard in Proxy’s design. According to QueueZero, the assistant requires explicit user confirmation before any external message is transmitted, ensuring that no communication leaves the organization without approval. Internal tasks, however, can be created automatically from meeting transcripts, though administrators can disable this feature to route all action items through manual review.
QueueZero declined to say whether the remark would be categorized as a successful feature demonstration or a parsing error.
Proxy integrates with standard calendar and video conferencing platforms, transcribing discussions in real time and identifying commitments made during the call. Unlike passive note-takers, the system translates natural language into actionable tickets within project management software, delegating work directly to team members or connected AI agents. The system supports multiple languages and can distinguish between speakers to ensure the right person is assigned.
The launch reflects a broader industry push to automate administrative overhead. QueueZero positions Proxy as a tool that closes the gap between decision and execution, allowing teams to move from conversation to action without manual follow-up. Industry analysts say the tool addresses a growing pain point as remote work leaves managers struggling to track commitments made across dozens of virtual meetings.
For the public beta beginning next week, pricing is set at $30 per user per month. Enterprise administrators will have granular control over the assistant’s autonomy, including the option to disable automatic internal task creation and require confirmation for all assignments.
```
Z.ai: GLM 5.3 Prime: # QueueZero's AI Meeting Assistant Takes a Remark Literally, Assigns Its CEO Two Tasks
*Proxy, an AI assistant that attends meetings and delegates follow-up work, turned a customer's aside into two tasks for CEO Evan Wu during its launch demonstration.*
QueueZero launched an AI meeting assistant called Proxy on Wednesday, unveiling software designed to attend meetings, produce notes and assign follow-up work to software agents. The demonstration that introduced it also served as an unplanned stress test of what happens when a system built to act on what it hears encounters something that was never meant as a command.
During the demonstration, a customer in the audience said, "Someone should tell Evan to stop talking and send us the pricing sheet."
Proxy did not treat the remark as commentary. The assistant interpreted it as an instruction and created two tasks, both assigned to QueueZero chief executive Evan Wu: "Stop talking" and "Send pricing sheet."
Wu laughed, completed the second task and deleted the first. In doing so, he may have become the first chief executive to receive a to-do item titled "Stop talking" from his own company's product.
Proxy enters a crowded field of AI meeting tools, but QueueZero is pitching it as something more ambitious: an assistant that does not merely summarize conversations but acts on their outcomes. The company says Proxy can sit in on a meeting, generate notes and then break the discussion into discrete follow-up tasks, which it hands off to software agents that complete the work.
That autonomy has limits, and where QueueZero drew them is notable. The company says Proxy requires confirmation before an external message is sent, meaning any communication bound for someone outside the organization waits for a human to approve it.
Internal tasks receive no such checkpoint. Proxy can create them automatically, as the demonstration made clear. Administrators who find that behavior too aggressive can disable automatic task creation altogether, the company said.
The mix-up was resolved without incident, but it highlights a design challenge facing the broader wave of agentic AI products: an assistant that turns every overheard remark into a task risks burying teams in assignments nobody asked for, while one that pauses to confirm every action risks becoming another source of interruption.
QueueZero's answer, for now, is a confirmation rule for external messages and an administrator switch for everything else.
Proxy costs $30 per user per month and enters public beta next week. How the assistant handles the everyday ambiguity of workplace conversation — sarcasm, jokes, asides — will be among the first things beta users test.
OpenAI: GPT-6 Luna: Model B is more faithful to the supplied reporting and uses one restrained humorous line while keeping a serious news tone. Model A adds numerous unsupported specifics, including product capabilities, analyst commentary and an invented company response. (Second judge pass, order swapped — scores are the average of both: Model B stays close to the supplied reporting, meets the length and tone constraints, and uses one brief dry joke. Model A adds numerous unsupported details and claims, including | Anthropic: Claude Fable 5.1: Model B sticks closely to the supplied facts, k…
Quotes woven into narrative
Write a 400–500-word Markdown story about the closing of Foundry Hall.
Use quotations selectively. Paraphrase when the information matters more than the speaker’s exact phrasing. Avoid stacking quotes or giving every source an equal-sized turn. The story should carry an emotional current while remaining restrained.
Facts:
* Foundry Hall, a nonprofit startup accelerator in Milwaukee, will close October 31.
* It opened in 2018.
* It has supported 212 startups, which collectively employ 1,140 people.
* Its largest corporate sponsor, Blank River Bank, supplied 38% of its annual budget.
* The bank declined to renew its sponsorship.
* Foundry Hall has enough money to operate through October but not through another full program.
* Six employees will lose their jobs.
* Twelve startups currently using its workspace must leave.
Quotes:
* Executive director Camille Ross: “We kept waiting for the replacement check that never came.”
* Founder Dev Malik: “My first investor meeting happened in that building. Before Foundry Hall, I was pitching from the back table of a coffee shop.”
* Blank River Bank spokesperson: “The decision reflects a change in our community investment priorities and is unrelated to Foundry Hall’s performance.”
* Deputy mayor Susan Cole: “The city is speaking with building owners and local universities about space for the current companies.”
Mistral: Mistral Large 4: ## The Last Season at Foundry Hall
On October 31, the doors of Foundry Hall will close for the final time. The Milwaukee nonprofit startup accelerator announced this week that it will cease operations after exhausting its financial reserves—enough to cover rent and payroll through the end of October, but not the cost of running another full program cycle. What began in 2018 as a converted warehouse brimming with raw ambition will end as an empty shell, its whiteboards wiped clean, its espresso machine unplugged, the hum of collaboration fading into silence.
Over six years, Foundry Hall supported 212 startups, ventures that now collectively employ 1,140 people across the region. It served as a crucial bridge between idea and infrastructure, offering founders more than desks and reliable Wi-Fi; it gave them the legitimacy of a professional address where they could host investors without apologizing for the venue. But that bridge rested heavily on a single pillar. Blank River Bank, the accelerator’s largest corporate sponsor, had supplied 38% of its annual budget for years. When the bank declined to renew that commitment earlier this spring, the financial foundation cracked beyond repair.
Executive director Camille Ross described the final months as a state of suspended animation, a vigil for funding that never arrived. “We kept waiting for the replacement check that never came,” she said. Staff spent the summer making calls, drafting proposals, and hoping, but the gap—more than a third of the operating budget—proved impossible to fill in time.
The human cost lands hard and specific. Six employees will lose their jobs when the doors lock at the end of the month, and twelve startups currently renting desks in the building must find new homes within weeks. For Dev Malik, whose logistics company secured its first major funding round in one of Foundry Hall’s conference rooms, the displacement carries a weight beyond square footage. “My first investor meeting happened in that building,” he said. “Before Foundry Hall, I was pitching from the back table of a coffee shop.”
A spokesperson for Blank River Bank said the non-renewal reflected a shift in the bank’s community investment priorities, not any failing on Foundry Hall’s part. Meanwhile, Deputy Mayor Susan Cole said city officials are already exploring options with building owners and local universities to secure workspace for the twelve displaced companies, though she acknowledged that finding suitable space for all of them before month’s end is unlikely.
What remains as Halloween approaches is a quiet inventory: 212 companies launched, 1,140 jobs still held by those ventures, and six weeks until the keys are returned. The startups will survive—many have graduated to their own offices—but Foundry Hall, the room where so many first pitches were practiced, will soon belong to the past.
Z.ai: GLM 5.3 Prime: # After Six Years, Foundry Hall Goes Dark
*Milwaukee's nonprofit startup accelerator helped launch 212 companies and, with them, 1,140 jobs. On October 31, it closes.*
MILWAUKEE — On the last day of October, Foundry Hall will close. The nonprofit accelerator has been part of the city's startup community since 2018, and it is ending operations after losing a corporate sponsorship that covered more than a third of its annual budget.
The sponsor was Blank River Bank, Foundry Hall's largest, which supplied 38% of the organization's funding and declined to renew its sponsorship. A bank spokesperson said the decision reflects a change in the institution's community investment priorities and is unrelated to Foundry Hall's performance — a distinction that will not keep the lights on.
"We kept waiting for the replacement check that never came," said Executive Director Camille Ross. The organization has enough money to operate through October. It does not have enough to run another full program, and so the end of the month will be the end of Foundry Hall.
The record it leaves is sizable. Some 212 startups have passed through the accelerator since it opened, and the companies it supported now employ 1,140 people — a footprint far larger than the building that helped launch them. It is the kind of track record that made Foundry Hall easy to praise and, in the end, no easier to fund.
Inside the building, the ending is smaller and sharper. Six employees will lose their jobs. The twelve startups currently using Foundry Hall's workspace must be out by the end of the month, the last cohort finishing its stay as the rooms around them empty.
For the founders who came through the program, the closing carries a particular sting, because they remember what the alternative looked like. Dev Malik's first investor meeting took place inside the building. "Before Foundry Hall, I was pitching from the back table of a coffee shop," he said. The distance between a coffee shop's back table and a funded company is not measured in blocks. For six years, Foundry Hall was the bridge.
The city is working to soften the landing. Deputy Mayor Susan Cole said Milwaukee is speaking with building owners and local universities about space for the current companies, though no arrangements have been announced.
No one has come forward to replace the bank. No successor to the accelerator has been named. What remains is the arithmetic of six years — 212 companies, 1,140 jobs, one building going dark on Halloween — and a generation of Milwaukee founders who know what the place gave them, because most of them started with less.
The coffee shops, in any case, still have back tables.
OpenAI: GPT-6 Luna: Model B weaves the facts and quotations into a restrained narrative with a clear emotional current, while Model A invents several specifics, including the building’s history, Malik’s company and funding, and the city’s assessment of the relocation timeline. Both meet the length and format requirements, but Model B is more reliable and controlled. (Second judge pass, order swapped — scores are the average of both: Model B stays closer to the supplied facts, uses quotes selectively, and gives the | Anthropic: Claude Fable 5.1: Model B stays faithful to the supplied facts, de…
Credible Startup Spotlight
Write a 500–650-word Startup Spotlight article in clean Markdown.
The tone should be curious and constructive. Explain the founders’ insight, product and early traction while giving uncertainty and competition appropriate space. Avoid investor clichés, heroic founder mythology and promotional claims stated as fact.
Reporting:
* LedgerLark builds AI agents for post-trade operations at small investment firms.
* Co-founder Mina Batra previously spent seven years in operations at a regional brokerage.
* Co-founder Owen Carr built reconciliation systems at a large bank.
* Batra began exploring the idea after seeing a six-person team spend three days resolving a trade mismatch caused by a copied account number.
* LedgerLark’s agents compare confirmations, custodian records and internal ledgers, then propose corrections for human approval.
* The product does not execute trades or move money.
* Five firms are paying customers.
* Annual contracts range from $36,000 to $110,000.
* The company says one customer reduced its unresolved daily exceptions from 70 to 18.
* The company has raised a $3.5 million seed round led by Motive Row.
* Competitors include established reconciliation vendors and internal bank software.
* Financial institutions may require lengthy security and procurement reviews.
Mistral: Mistral Large 4: # LedgerLark's Quiet Bet on the Back Office
*Post-trade operations run on comparisons: confirmations against custodian records, records against internal ledgers. When those comparisons fail, the cost is measured in analyst hours. LedgerLark is betting that AI agents can do the comparing—if humans keep control of the corrections.*
Mina Batra spent seven years in operations at a regional brokerage, and one episode stuck with her. She watched a six-person team spend three days resolving a trade mismatch that traced back to a single account number copied incorrectly. The error wasn't malicious or even unusual; it was simply invisible, buried across systems until someone manually connected the dots. When she left to build a company, that frustration became the design brief. Her co-founder, Owen Carr, arrived from the other side of the industry, having built reconciliation systems at a large bank where scale and headcount can absorb the kind of quiet errors that overwhelm a smaller firm.
LedgerLark builds AI agents for post-trade operations at small investment firms—the unglamorous work after a trade is struck. The agents compare trade confirmations against custodian records and internal ledgers, then propose corrections for any mismatches. The hard constraint is architectural: the product does not execute trades or move money. Every suggested fix requires explicit human approval. That guardrail is meant to address the obvious objection from compliance teams—letting software near the books—while still automating the tedious cross-checking that consumes operations teams.
Traction so far is early but concrete. Five firms are paying customers, with annual contracts ranging from $36,000 to $110,000—mid-market pricing that suggests LedgerLark is targeting firms that feel the pain of exceptions without enterprise budgets. The company reports that one customer reduced its unresolved daily exceptions from 70 to 18 after adopting the agents. That figure is promising, but it is company-reported and unverified; whether it generalizes across asset classes or market conditions remains unknown. LedgerLark has raised a $3.5 million seed round led by Motive Row, capital to hire and to wait, since financial institutions are slow buyers. Security reviews and procurement cycles can stretch for months, and a seed-stage vendor must survive that timeline without revenue from the deal in question.
The competition is not hypothetical. Established reconciliation vendors already have footholds at investment firms, and large banks maintain internal software that could be productized for smaller clients. There is also a quieter threat: if LedgerLark's approach works, a big bank's engineering team might simply replicate it in-house. Meanwhile, LedgerLark must persuade compliance officers—who are personally accountable for operational errors—that an AI agent can be trusted to read sensitive records, even if it cannot touch funds.
Open questions remain about the product itself. Because the agents only propose fixes rather than applying them, the true measure is how many analyst hours they save compared with a skilled human using a spreadsheet. Edge cases—corporate actions, illiquid securities, cross-border settlement—may expose gaps in the models, and no detailed error rates have been published. For now, LedgerLark's pitch is deliberately modest: let the AI read the books, but keep its hands off the money. In a sector crowded with grand claims about automation, that restraint may be exactly what cautious back offices need—or it may prove too limited to scale. The next year will likely tell.
Z.ai: GLM 5.3 Prime: # LedgerLark Sends AI Agents Into the Dull, Expensive Corners of Post-Trade
*A seed-stage bet that AI agents can clear post-trade exceptions at firms too small for big-vendor tooling.*
It started with a copied account number. Mina Batra, then seven years into an operations career at a regional brokerage, watched a six-person team spend three full days resolving the trade mismatch it caused. The problem wasn't intellectually hard; it was buried, scattered across confirmations, custodian records, and internal ledgers that didn't talk to each other.
Batra came away thinking the bottleneck in post-trade operations isn't intelligence, but attention. She later teamed up with Owen Carr, who had built reconciliation systems at a large bank, to found LedgerLark. The company, which builds AI agents for post-trade operations at small investment firms, has now raised a $3.5 million seed round led by Motive Row.
## What the agents actually do
LedgerLark's software does the three-way comparison that operations teams perform by hand or in patchwork spreadsheets: it checks trade confirmations against custodian records and internal ledgers. When something doesn't match, the agents trace the likely source and draft a proposed correction. A human approves or rejects it.
The scope boundary is notable: the product does not execute trades or move money. That's likely as much commercial strategy as caution. A tool that can't touch cash is an easier yes for a first pilot, and it positions the agents as assistants to the operations desk rather than replacements for it.
## Early traction, with an asterisk
Five firms are paying customers, on annual contracts ranging from $36,000 to $110,000. That spread suggests at least some customers are treating the product as operating infrastructure rather than an experiment.
LedgerLark says one customer reduced its unresolved daily exceptions from 70 to 18. If that holds up, it's a meaningful change for that desk. But it's a single self-reported data point from a five-customer company, with no independent verification and no detail on how exceptions were counted before and after. The honest read: promising, not yet proven.
## The competition problem
Post-trade reconciliation is not an overlooked market. Established reconciliation vendors sell mature tooling, and large banks run internal software, some of it very good. LedgerLark's implicit wager is that those products were built for enterprises with integration teams and budgets to match, leaving smaller investment firms stuck between spreadsheets and software they can't realistically deploy.
That wedge may be real, but it has a shelf life. Incumbents are adding AI features of their own, and a segment too small for the big vendors is one other startups can notice just as easily.
## The procurement clock
The other open question is time. Financial institutions typically require lengthy security and procurement reviews, and sales cycles in this sector can stretch across quarters. LedgerLark's first five customers were, by definition, firms willing to move faster than the norm; the next fifty may not be. A $3.5 million round buys less patience for a slow enterprise sales motion than a larger one would — which makes renewals and sales velocity worth watching more than new customer announcements.
## What to watch
Three signals will matter most over the next year:
- Whether existing customers renew and expand their contracts
- Whether the 70-to-18 reduction in unresolved exceptions holds beyond its first example
- How far firms eventually let agents go — auto-resolving low-risk exceptions, say, versus keeping every decision behind a human click
For now, LedgerLark is testing a modest-sounding proposition: that the first place financial firms trust AI agents isn't trading, or anything that touches money, but the unglamorous work of noticing that two numbers don't match. It's a smaller story than the agent economy's grander narratives. It may also be a more plausible one.
OpenAI: GPT-6 Luna: Model A edges out Model B by covering the required insight, product, traction, and risks in a polished feature format while more clearly framing speculation as uncertainty. Model B is well organized, but presents unsupported claims about incumbents adding AI and customer buying speed as fact. (Second judge pass, order swapped — scores are the average of both: Model B offers a clearer, more structured spotlight and gives the reported traction appropriate caveats. Model A is polished, but adds mor | Anthropic: Claude Fable 5.1: Model B offers a sharper, better-structured spo…
Cohesive story from reporting notes
Write a publication-ready technology news article from the reporting notes below.
Requirements:
* 550–700 words
* Clean Markdown
* One H1 headline followed by an italicized dek
* A strong news lede followed by a clear nut graf
* Organize the material into a cohesive story rather than following the notes in order
* Attribute company-supplied claims
* Include limitations without turning the article into a warning label
* Use only the supplied information
* Avoid canned transitions, promotional language and a concluding summary
Reporting notes:
* HelioDock makes autonomous warehouse robots.
* On Thursday, it launched Night Shift, software that lets operators monitor robot fleets remotely.
* Night Shift spent 14 months in a private beta involving nine logistics companies and 310 robots.
* HelioDock says beta customers experienced 17% fewer unplanned robot stoppages. The figure has not been independently verified.
* The company also announced a $24 million Series B led by North Coast Ventures, bringing its total funding to $39 million.
* CEO Mara Velez: “A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet.”
* Beta customer Aaron Pike of Fulcrum Logistics said the system worked well in facilities with strong connectivity but became less useful during rural LTE outages.
* When connectivity fails, robots return to their locally configured behavior. Remote control becomes unavailable.
* Competing fleet-management products are available from Relay Robotics and YardOS.
* Night Shift will cost $90 per robot per month and become generally available September 15.
Mistral: Mistral Large 4: # HelioDock Launches Night Shift to Monitor Warehouse Robots Remotely
*After a 14-month private beta with nine logistics companies, the startup claims its new software cut unplanned robot stoppages by 17 percent—though the figure has not been independently verified and the system falters when wireless connections drop. The launch comes alongside a $24 million Series B.*
HelioDock, which manufactures autonomous robots for warehouse operations, on Thursday introduced Night Shift, a software platform that allows operators to monitor and control entire robot fleets from remote locations. The company positioned the product as a direct solution to a persistent operational burden: under current setups, managers must drive across town at 2 a.m. to restart robots that have stalled mid-task, often because the machines have become confused by a pallet obstructing their path.
The software becomes generally available September 15 after spending 14 months in a private beta involving nine logistics companies and 310 robots—a validation period that extended well beyond a typical software trial and covered enough units to exercise the system across varied real-world warehouse conditions. Night Shift marks HelioDock's expansion from hardware into the software layer that manages its robots. Thursday's launch also coincided with the announcement of a $24 million Series B funding round led by North Coast Ventures, bringing HelioDock's total funding to $39 million. As warehouses increasingly deploy autonomous robots to handle material movement and reduce dependence on manual labor, HelioDock is arguing that remote human oversight, rather than full automation alone, is the essential layer for preventing costly disruptions and maintaining continuous operations around the clock.
"A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet," said CEO Mara Velez. "Night Shift keeps a human in the loop without requiring them to be physically present in the facility or on the warehouse floor. While the robots handle routine movement autonomously, edge cases still arise where human judgment is required."
HelioDock says the nine beta customers experienced 17% fewer unplanned robot stoppages while using the software, a reduction the company attributes to the ability to intervene remotely and immediately rather than waiting for on-site staff to arrive. That figure has not been independently verified, and HelioDock did not share the underlying data. Night Shift provides operators with a single interface for the entire fleet, allowing them to track status in real time, clear error states from a distance, reroute units as needed and take control when autonomous robots encounter obstacles beyond their programmed capabilities.
The extended beta, however, revealed a significant dependence on wireless network infrastructure that could limit the software's effectiveness in certain locations. Aaron Pike of Fulcrum Logistics, one of the nine companies that participated in the trial, said Night Shift worked well in facilities with strong connectivity but "became less useful during rural LTE outages." When connectivity fails, the robots automatically revert to their locally configured behavior and remote control becomes unavailable—a fail-safe mechanism designed to keep machines operating within safety parameters, but one that leaves distant operators monitoring blind spots until the wireless link is restored.
HelioDock enters a competitive landscape where fleet-management products are already available from Relay Robotics and YardOS, giving warehouse operators established alternatives as they evaluate their automation strategies. Night Shift will cost $90 per robot per month, a per-unit subscription fee that scales directly with fleet size when the platform becomes generally available September 15.
Z.ai: GLM 5.3 Prime: #
OpenAI: GPT-6 Luna: Model A provides a polished, structured article, while Model B provides no article at all. Model A nevertheless invents product capabilities, beta details, and quoted remarks, violating the requirement to use only supplied information. (Second judge pass, order swapped — scores are the average of both: Model A is the only substantive response and follows the requested article structure, but it adds multiple unsupported product details and an invented quotation, undermining its reliability. Model | Anthropic: Claude Fable 5.1: Model A delivers a complete, well-structured ar…
News judgment in the opening
Write only the opening of this news story: an H1 headline, an italicized dek and three to five paragraphs totaling 180–250 words.
The opening must establish what happened, quantify the scope, explain why it matters and carefully distinguish confirmed facts from unknowns. It should pull the reader forward without sensationalizing the incident.
Reporting:
* Cloudnote discovered that a debugging page containing customer prompt excerpts was accessible without authentication.
* The page was exposed for four hours on Monday.
* Logs show excerpts from 1,842 user sessions.
* The excerpts could include up to 200 characters from a prompt.
* Cloudnote removed the page 19 minutes after a researcher notified it.
* The company says it has found no evidence that anyone besides the researcher accessed the page.
* It has not completed its investigation.
* Full documents, passwords and payment information were stored elsewhere and were not present on the page.
* Cloudnote has 3.2 million registered users.
Mistral: Mistral Large 4: **Cloudnote Says 1,842 Users’ Prompt Excerpts Were Publicly Accessible for Four Hours**
*Though the exposure lasted only four hours and affected a tiny fraction of the platform’s 3.2 million users, the company has not yet finished investigating whether the page was viewed by anyone beyond the security researcher who reported it.*
Cloudnote said it discovered that a debugging webpage containing customer prompt excerpts was accessible to anyone on the internet without authentication for roughly four hours on Monday, exposing fragments of 1,842 user sessions. Each snippet contained up to 200 characters of a prompt, potentially including proprietary text, questions, or personal notes submitted to the note-taking and AI service. The company, which serves 3.2 million registered users, removed the page 19 minutes after a security researcher notified it of the issue.
Cloudnote said its preliminary logs show no evidence that anyone other than the researcher accessed the exposed page during the window, though it cautioned that its investigation is not yet complete. The company emphasized that the page contained only truncated prompt excerpts, not full documents, passwords, or payment information, which are stored in separate, access-controlled systems.
The incident underscores the privacy risks inherent in AI productivity tools, where users often paste sensitive material into chat interfaces. While the 1,842 affected sessions represent roughly 0.06% of Cloudnote’s user base, experts warn that even brief fragments can reveal confidential business details or identify individuals. The company has not yet said whether it will notify the affected users or what measures it will implement to prevent similar exposures.
Z.ai: GLM 5.3 Prime: #
OpenAI: GPT-6 Luna: Model A follows the requested format and provides a coherent opening, but it adds unsupported claims and treats session counts as users when calculating the affected share. Model B provides only a hash mark and does not answer the prompt. (Second judge pass, order swapped — scores are the average of both: Model A provides the requested story opening and distinguishes the company’s preliminary findings from its unfinished investigation, while Model B provides no usable content. Model A adds unsup | Anthropic: Claude Fable 5.1: Model A delivers a well-structured, accurate op…
Natural human rewrite
Rewrite the draft below as a 300–400-word publication-ready news story in Markdown.
Preserve every factual claim. Improve the lede, narrative flow, sentence rhythm and transitions. Remove repetition, stock hype, throat-clearing and mechanical signposting. The finished article should sound like an experienced human technology reporter wrote it.
Draft:
# Arcwell Launches An Innovative New Platform
Arcwell has announced an exciting new platform called Relay. The announcement represents a significant development for the company and for the broader customer service industry. Relay uses artificial intelligence to examine customer support conversations and suggest answers to agents.
The platform is important for several reasons. First, it can suggest answers. Second, it can identify customers who may cancel. Third, it can create summaries. These capabilities demonstrate how artificial intelligence continues to transform the customer service landscape.
Relay was tested by 12 companies over six months. Arcwell says agents using it closed tickets 14% faster. The company did not release customer-retention results. Relay costs $65 per agent each month.
CEO Priya Shah said, “Support agents spend too much time searching for information that their companies already have.”
However, the platform also presents potential challenges. Relay requires access to support transcripts and internal help documents. Arcwell says customer data is encrypted and is not used to train shared models. Companies can choose a 30-day or 90-day retention period.
In conclusion, Relay represents an important new chapter for Arcwell as it seeks to help companies improve customer service using artificial intelligence.
Mistral: Mistral Large 4: ## Arcwell Debuts Relay, an AI Copilot for Customer Support Agents
Arcwell is betting that the most valuable AI in customer service won't interact with customers directly, but will instead work quietly beside the human agents who do. The company has unveiled Relay, a new AI platform that examines support conversations in real time to suggest answers, identify customers who may cancel, and create summaries of ticket interactions.
The launch reflects a broader shift across the customer service industry, where companies are increasingly moving away from standalone chatbots toward internal tools that augment human workers rather than replace them. By drawing on internal help documents and historical support transcripts as conversations unfold, Relay aims to eliminate the hours agents currently spend searching through knowledge bases in the middle of customer interactions.
Arcwell tested Relay with 12 companies over a six-month period. According to the company, agents using the platform closed tickets 14% faster than those working without it. Arcwell did not release any data on customer retention, however, leaving unanswered whether faster resolution times actually improve retention rates. The service is priced at $65 per agent each month.
"Support agents spend too much time searching for information that their companies already have," said CEO Priya Shah.
To generate its suggestions and summaries, Relay requires access to support transcripts and internal help documents, placing sensitive customer communications within the platform's purview. Arcwell states that all customer data is encrypted and is not used to train shared AI models across its client base. Companies can configure retention periods of either 30 or 90 days for the data Relay processes.
As support teams across the industry confront rising ticket volumes and persistent staffing shortages, tools like Relay represent the latest entry in a growing market of AI copilots designed to streamline agent workflows while keeping humans firmly in the loop.
Z.ai: GLM 5.3 Prime: #
OpenAI: GPT-6 Luna: Model A delivers a polished, publication-style story and includes the draft’s key facts, though it adds several unsupported claims about industry trends and agent workflows. Model B provides no article, so it fails the task. (Second judge pass, order swapped — scores are the average of both: Model A provides a polished, complete news story, while Model B provides no article. Model A adds several unsupported details, including real-time processing and industry-wide staffing trends, which weakens | Anthropic: Claude Fable 5.1: Model A delivers a complete, well-structured ~33…
Exact final-draft structure
Convert the supplied fields into Markdown using these exact rules:
* H1 headline
* One blank line
* Italicized dek
* One blank line
* First body paragraph
* One blank line
* H2 section heading
* One blank line
* Second body paragraph
* Preserve all wording and punctuation exactly
* Do not use a code fence
* Output nothing else
Headline: Raincheck raises $8 million for flood-risk software
Dek: The Austin startup helps cities decide which drainage problems to address before storms.
First body paragraph: Raincheck has raised an $8 million seed round led by Basin Ventures.
Section heading: Expansion plans
Second body paragraph: The company plans to hire eight engineers.
Mistral: Mistral Large 4: # Raincheck raises $8 million for flood-risk software
*The Austin startup helps cities decide which drainage problems to address before storms.*
Raincheck has raised an $8 million seed round led by Basin Ventures.
## Expansion plans
The company plans to hire eight engineers.
Z.ai: GLM 5.3 Prime: # Raincheck raises $8 million for flood-risk software
*The Austin startup helps cities decide which drainage problems to address before storms.*
Raincheck has raised an $8 million seed round led by Basin Ventures.
## Expansion plans
The company plans to hire eight engineers.
OpenAI: GPT-6 Luna: Model A and Model B both preserve the wording and punctuation and follow the required Markdown structure exactly. Neither has a quality advantage. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both preserve the supplied wording and punctuation and follow the requested Markdown structure exactly. Neither output has an advantage in correctness, instruction following, or writing quality.) | Anthropic: Claude Fable 5.1: Model A and Model B produce character-for-character identical output that follows every structural rule exactly, pres…
Chronology with narrative momentum
Turn the scrambled reporting below into a 450–600-word Markdown news article.
Choose an opening that captures the consequence of the failure, then reconstruct the sequence clearly. Preserve the chronology without producing a minute-by-minute incident log. Give technical details enough context for a general reader.
Reporting:
* KiteBridge makes expense-management software.
* At 8:45 a.m. Tuesday, it deployed version 4.2.
* The first customer complaint arrived at 9:12 a.m.
* A payment-processor timeout caused KiteBridge’s fallback system to treat retries as new reimbursement requests.
* The software generated 13,400 duplicate reimbursements totaling $2.8 million.
* KiteBridge suspended automated payments at 10:03 a.m.
* Ninety-one corporate customers were affected.
* By Wednesday evening, $2.3 million had been recovered. The rest remained pending.
* No employees had money withdrawn from their accounts; employers funded the duplicate payments.
* An April changelog said version 4.0 had introduced “exactly-once payment protection.”
* KiteBridge’s public incident page initially described the problem as delayed reimbursements.
* The incident page first disclosed the duplicate payments six hours after automation was suspended.
* CEO Lena Ortiz apologized and said the company would publish a technical report.
Mistral: Mistral Large 4: # KiteBridge's "Exactly-Once" Payment Promise Collapses in $2.8 Million Duplicate-Payment Glitch
Ninety-one corporate customers of expense-management software maker KiteBridge are still tallying the damage from a software failure that sent $2.8 million in duplicate reimbursements racing through the financial system Tuesday morning—and for several hours, the company's public incident page misleadingly described the crisis as merely "delayed reimbursements." The glitch, triggered by a routine update, generated 13,400 duplicate payments before engineers could halt the automated system.
KiteBridge, which processes employee expense reports and reimbursements for businesses, deployed version 4.2 of its platform at 8:45 a.m. Tuesday. The first customer complaint arrived just 27 minutes later, at 9:12 a.m., and by 10:03 a.m.—little more than an hour after the rollout—the company had suspended all automated payments after discovering that a timeout at its third-party payment processor had cascaded into a flood of unintended transactions across its client base.
The technical root cause lay in how the new version's fallback system handled interrupted payments. When a payment processor times out, it is often ambiguous whether the funds actually transferred, which is why robust systems verify the original transaction before retrying. KiteBridge's April changelog for version 4.0 had touted "exactly-once payment protection"—a safeguard designed to ensure each reimbursement is processed only a single time, even if the system must retry after an error. In theory, the fallback logic should check the status of the initial request before attempting another. Instead, version 4.2 treated each retry signal as a completely new reimbursement request, effectively instructing the payment processor to send the money again and again.
Think of it like refreshing a payment webpage after a timeout: if the site doesn't check whether your first attempt succeeded, you might accidentally pay twice. In this case, the software made that mistake 13,400 times in roughly 75 minutes.
The financial impact fell squarely on employers, not employees. Because reimbursements flow from corporate accounts to workers, the duplicate deposits meant companies funded the extra payments out of their own treasuries; no employee accounts were debited, and workers were not asked to return money from personal checking accounts. By Wednesday evening, KiteBridge had clawed back $2.3 million of the $2.8 million, with the remaining roughly $500,000 still pending recovery as the company worked with recipient banks to reverse the erroneous transfers.
The episode also drew scrutiny over how KiteBridge communicated the crisis. Its public incident page initially characterized the problem as delayed reimbursements—a description that remained in place even after automated payments were suspended at 10:03 a.m. Only six hours later, around 4:00 p.m. Tuesday, did the company disclose that duplicate payments had actually been issued. CEO Lena Ortiz subsequently apologized for the disruption and the delayed disclosure, pledging that KiteBridge would publish a detailed technical report on the failure.
For the 91 affected companies, the incident serves as a stark reminder that even advertised safeguards in financial software can fail when edge cases—like a payment processor timeout—expose gaps between intended and actual system behavior.
Z.ai: GLM 5.3 Prime: # KiteBridge Update Sent $2.8 Million in Duplicate Payments to 91 Companies
A routine software update released Tuesday morning by expense-management company KiteBridge set off a payment failure that issued 13,400 duplicate employee reimbursements totaling $2.8 million, hitting 91 corporate customers before the company suspended automated payments.
The root cause was a mishandled timeout: when KiteBridge's software could not confirm that payments had gone through, its fallback system treated the retries as new requests. KiteBridge had recovered $2.3 million of the duplicated funds by Wednesday evening; the rest remained pending. Employers, not employees, funded the duplicate payments, and no money was withdrawn from workers' accounts — so the losses fell on the companies that pay for KiteBridge's software.
## How a timeout became $2.8 million
The trouble began with an ordinary kind of glitch. When KiteBridge's platform submitted reimbursement payments to its payment processor on Tuesday morning, the processor stopped returning confirmations that the payments had gone through. Each request timed out.
Retrying after a timeout is standard practice — resubmitting a payment that appears to have failed is safer than assuming it succeeded. The bug was in how those retries were handled. When the primary payment path failed, KiteBridge's fallback system, the backup route that takes over during an outage, treated each retry as a brand-new reimbursement request rather than a repeat of the original.
The result: employees owed a single reimbursement were paid more than once. Multiplied across KiteBridge's customer base, that produced 13,400 duplicate payments worth $2.8 million in just over an hour of operation.
## A fast shutdown, a slower disclosure
KiteBridge's internal response was swift. Version 4.2 went live at 8:45 a.m. Tuesday, and the first customer complaint arrived at 9:12 a.m., less than half an hour later. By 10:03 a.m., the company had suspended automated payments altogether, capping the damage.
The public account did not keep pace. KiteBridge's incident page initially described the problem as delayed reimbursements — a far milder framing. Only six hours after automation was suspended, late Tuesday afternoon, did the page disclose the duplicate payments.
## A safeguard built for exactly this
The episode is an uncomfortable one for the company. KiteBridge's changelog from April noted that version 4.0 had introduced "exactly-once payment protection."
In payment engineering, an "exactly-once" guarantee is meant to ensure a payment is processed one time — and only one time — no matter how many times the system retries it. That is precisely the mechanism designed to stop a timeout-and-retry loop from paying the same reimbursement twice. Tuesday's failure happened anyway, and KiteBridge has not yet publicly explained why the safeguard did not hold.
## What happens next
Chief Executive Lena Ortiz apologized for the incident and said KiteBridge would publish a technical report explaining what went wrong. That report is likely to draw close scrutiny, both over how the fallback system came to treat retries as new requests and over why a protection the company advertised in its release notes failed in practice.
As of Wednesday evening, the remaining $500,000 was still pending.
For the 91 affected companies, the immediate concern is the outstanding money. The broader question — how software promised to pay exactly once ended up paying twice — is the one the technical report will have to answer.
OpenAI: GPT-6 Luna: Model A builds a clearer progression from deployment to customer impact, shutdown, recovery and disclosure, while giving the timeout failure useful context. Model B is strong, but repeats the outstanding balance and makes retry behavior sound safer and simpler than the failure described supports. (Second judge pass, order swapped — scores are the average of both: Model B is the stronger, more fact-disciplined article while meeting the requested length and structure. Model A is engaging, but adds | Anthropic: Claude Fable 5.1: Model B delivers a tighter, better-structured a…
Numbers shaped into a story
Write a 600–750-word Markdown article from the benchmark results below.
Build the story around the clearest finding. Explain the methodology and limitations where readers need them. Avoid reciting the table row by row or treating small differences as decisive.
Benchmark:
* Researchers tested five language models on 200 anonymized customer-support tickets.
* Each model produced one proposed answer per ticket.
* Three experienced support managers graded the answers blindly.
* Scores combined factual correctness, usefulness and tone.
* Model North: 84.2 average score; $1.90 estimated cost per 1,000 tickets.
* Model Vale: 83.8; $0.74 per 1,000 tickets.
* Model Ember: 79.1; $0.31 per 1,000 tickets.
* Model Kite: 76.4; $0.22 per 1,000 tickets.
* Model Harbor: 71.0; $0.08 per 1,000 tickets.
* North and Vale’s score difference was not statistically significant.
* Vale answered 18% faster than North.
* North performed best on billing disputes.
* Vale performed best on technical troubleshooting.
* All five models performed poorly when a ticket depended on an image attachment.
* The test used English-language tickets from one software company.
* Researchers did not test agentic actions, tool use or multilingual support.
Mistral: Mistral Large 4: # The Most Expensive Support Model Didn't Win. Here's What Actually Mattered.
Across 200 real customer-support tickets, the priciest language model in our benchmark did not produce significantly better answers than a rival costing less than half as much. For teams under pressure to automate support without ballooning costs, the clearest finding is this: **quality and price stop tracking together at the top of the market.** Model Vale matched the leading score within statistical noise at roughly 40% of the cost — and answered faster.
## How the benchmark worked
Researchers drew 200 anonymized tickets from the support queue of a single software company, all in English. Each of five models wrote one proposed answer per ticket. Three experienced support managers then graded the answers blindly — they didn't know which model produced which — scoring each on factual correctness, usefulness, and tone. Reported costs are estimates per 1,000 tickets.
That setup is good at measuring one thing: which draft would a support manager prefer? It is not a test of full automation, and we flag the gaps below.
## The top of the table: a premium that didn't pay off
Model North earned the highest average score, 84.2, at an estimated $1.90 per 1,000 tickets. Model Vale scored 83.8 at $0.74 per 1,000 — a 0.4-point gap the researchers explicitly called not statistically significant. Vale also responded 18% faster than North.
So the headline is not "North is best." It's that North's extra spend bought no measurable quality advantage in this test. Vale is the practical sweet spot: statistically indistinguishable from the leader, about 61% cheaper, and quicker.
Moving down the price ladder, quality steps down too, but none of those drops is a cliff. Model Ember averaged 79.1 at $0.31 per 1,000; Kite 76.4 at $0.22; Harbor 71.0 at $0.08. Because the score blends three judgments, a few points can move with ticket mix, so treat these as a descending cost–quality frontier rather than precise rankings — Harbor may still be "good enough" for low-stakes drafting. What the frontier does show is direction: each cut in price came with a real, if modest, loss of average quality.
## Averages hide where each model shines
North's strength was billing disputes — procedural, high-stakes tickets where precision matters most. Vale led on technical troubleshooting. That suggests a routing strategy: use the model that fits the ticket category rather than one premium model for everything. The benchmark didn't test routing directly, but the per-category pattern points that way.
## The blind spot every model shares
When a ticket depended on an image attachment — a screenshot of an error, a photo of a defect — all five models performed poorly. This looks like a shared limitation in reasoning over images in a text-answer setup, not a flaw unique to one vendor. For teams whose customers routinely attach images, it's the most operationally important result in the study: none of these models can yet be trusted as a hands-off replacement for a human who can look at the picture.
## What this benchmark can't tell you
- **One company, one language.** All 200 tickets were English-language tickets from one software firm. Different industries, vocabularies, and customer expectations could reshuffle the leaderboard.
- **Text answers only.** The researchers did not test agentic actions, tool use, or multi-step workflows — capabilities that increasingly define real support automation.
- **No multilingual coverage.** Non-English support is untested.
- **Grading noise.** Three expert graders reduce, but don't eliminate, subjectivity in scoring usefulness and tone.
- **Estimated costs.** Per-1,000-ticket figures are modeled, not invoiced; actual spend depends on contracts and token mix.
## The bottom line
If you're picking a model to draft support replies today, don't default to the most expensive option. Vale tied the top score within statistical noise at a fraction of the cost and with faster turnaround — a combination nothing else in the test matched. Save premium pricing for the case where it demonstrably earns its keep — North's edge on billing disputes is the one candidate here — and keep image-dependent tickets in human hands for now.
Z.ai: GLM 5.3 Prime: #
OpenAI: GPT-6 Luna: Model A provides a polished, well-structured article that appears to meet the requested length and uses the benchmark’s main finding effectively, though a few interpretations go beyond the supplied evidence. Model B contains only a Markdown heading marker and does not answer the prompt. (Second judge pass, order swapped — scores are the average of both: Model A provides a polished, appropriately structured article that centers the strongest finding and explains the benchmark’s scope, while Model | Anthropic: Claude Fable 5.1: Model A delivers a well-structured, accurate ar…