Head to head: Qwen: Qwen3.8 Max (0902) vs grok-4.7

Qwen: Qwen3.8 Max (0902) vs grok-4.7

By · Published · Updated

RuntimeWire Head-to-Head: Head to head: Qwen: Qwen3.8 Max (0902) vs grok-4.7
RuntimeWire Head-to-Head matchup

A newsroom-writing matchup across reported features, news openings, rewrites, numerical analysis and formatting. The tasks reward not just polish, but staying within the brief and keeping claims anchored to the supplied material.

Qwen: Qwen3.8 Max (0902) takes the saved result, with an aggregate score of **100.2** to grok-4.7’s **94.9**. It won **6** tasks to Grok’s **2**, with **4** ties—a clear editorial result on this sample, not a statistical claim. Qwen’s edge came in several demanding reporting tasks: weaving quotes into a narrative, handling a sensitive news opening, building chronology, shaping numbers into a story and editing copy for publication. The judges often favored its control of length and its restraint around what the reporting did—and did not—establish. It also won the natural-rewrite task, though the reviews noted that Grok was more faithful to the supplied facts there. Grok’s two wins were meaningful rather than cosmetic: it produced the stronger cohesive story from reporting notes and the better example of controlled wit. The models tied on exact structure, startup coverage, Markdown error counting and lede selection, so this was not a sweep—and Grok could be the better fit when narrative momentum or a dry, disciplined voice matters most. The verdict applies only to this saved task sample; it does not establish a general capability ranking. **Call: Qwen wins on this sample, chiefly through more consistent execution of constrained newsroom assignments.**

Quotes woven into narrative

Write a 400–500-word Markdown story about the closing of Foundry Hall. Use quotations selectively. Paraphrase when the information matters more than the speaker’s exact phrasing. Avoid stacking quotes or giving every source an equal-sized turn. The story should carry an emotional current while remaining restrained. Facts: * Foundry Hall, a nonprofit startup accelerator in Milwaukee, will close October 31. * It opened in 2018. * It has supported 212 startups, which collectively employ 1,140 people. * Its largest corporate sponsor, Blank River Bank, supplied 38% of its annual budget. * The bank declined to renew its sponsorship. * Foundry Hall has enough money to operate through October but not through another full program. * Six employees will lose their jobs. * Twelve startups currently using its workspace must leave. Quotes: * Executive director Camille Ross: “We kept waiting for the replacement check that never came.” * Founder Dev Malik: “My first investor meeting happened in that building. Before Foundry Hall, I was pitching from the back table of a coffee shop.” * Blank River Bank spokesperson: “The decision reflects a change in our community investment priorities and is unrelated to Foundry Hall’s performance.” * Deputy mayor Susan Cole: “The city is speaking with building owners and local universities about space for the current companies.”

Qwen: Qwen3.8 Max (0902): # The Last Lights at Foundry Hall By late October, the desks in Foundry Hall will be cleared, the whiteboards wiped, and the front door locked for the final time. The Milwaukee nonprofit startup accelerator, which opened in 2018 with the promise of giving early-stage companies a place to build, will close on October 31. The ending did not arrive all at once. It came in meetings, forecasts, and a gap that never closed. Blank River Bank, Foundry Hall’s largest corporate sponsor, provided 38% of the organization’s annual budget. When the bank chose not to renew its sponsorship, the Hall was left with enough money to operate through October, but not enough to carry another full program. Executive director Camille Ross put it plainly: “We kept waiting for the replacement check that never came.” That absence now moves through the building in small, practical ways. Six employees will lose their jobs. Twelve startups that have been using the workspace must find somewhere else to go. The companies are not failing in the abstract; they are losing phone booths, conference rooms, a mailing address, and the casual collisions that often turn into advice, introductions, or rescue. Still, the numbers argue that Foundry Hall was more than a room with fast internet. Since opening, it has supported 212 startups, which collectively employ 1,140 people. Those figures describe payroll and products, but they also describe a pattern: founders who once worked alone learning to build in public, surrounded by others doing the same. Dev Malik, one of the founders who passed through the Hall, remembered how different his company looked before it had a place. His first investor meeting happened inside the building. Before that, he had been pitching from the back table of a coffee shop, trying to make a noisy room feel like a boardroom. Blank River Bank said its decision reflected a change in community investment priorities and was not related to Foundry Hall’s performance. The explanation may be accurate, but it does not soften the result. A budget line moved, and an institution that had helped anchor a local ecosystem is disappearing. City officials are trying to soften the landing. Deputy mayor Susan Cole said the city is talking with building owners and local universities about space for the current companies. That may help some startups keep working, perhaps in scattered corners rather than under one roof. For now, the Hall remains open until Halloween. The lights still come on. The coffee is still made. Founders still arrive with laptops and unfinished plans. But the countdown is visible, and everyone inside knows it. The building was never just square footage. It was proof that Milwaukee had a place where new companies could be taken seriously. Soon, that proof will have to live somewhere else.
grok-4.7: Foundry Hall will close on October 31, ending a run that began in 2018 in a former machine shop on Milwaukee’s near south side. For seven years the nonprofit accelerator was the room where early companies practiced sounding like companies. The lights will go out with the books still in order and the work unfinished. In that time the hall supported 212 startups. Those firms now employ 1,140 people—coders, line supervisors, founders who still answer their own phones. The figures belong on an annual report. They do not capture a Tuesday afternoon, bad coffee cooling on a windowsill while someone in the corridor tried a sentence until it held. The closing is a budget fact, not a verdict. Blank River Bank, the largest sponsor, had supplied 38 percent of the yearly budget and declined to renew. A spokesperson described the choice as a change in the bank’s community investment priorities, unrelated to Foundry Hall’s performance. Executive director Camille Ross was plainer about what followed: “We kept waiting for the replacement check that never came.” No check arrived. The organization can operate through October. It cannot carry another full program. Six employees will lose their jobs. Twelve startups still using the workspace must leave. Deputy mayor Susan Cole has said the city is speaking with building owners and local universities about space for those companies. The talks are underway; they are not yet a key. Founders have started measuring desks against the trunks of cars and deciding which whiteboards are worth keeping. Dev Malik, from an early cohort, remembers his first investor meeting in the building. Before Foundry Hall, he was pitching from the back table of a coffee shop, hoping his name would not be called mid-sentence. He did not call the place a rescue. He called it the difference between a pitch and a conversation someone might stay for. On the last afternoons the lights stay on past the hour the work requires. Half-erased market sizes remain on the boards. A plant by the window has not been claimed. Milwaukee will keep most of its young companies, scattered into borrowed rooms and university corners. What it will not keep is this particular quiet, where 212 beginnings were allowed, for a while, to sound as if they might last.

OpenAI: GPT-6 Luna: Model A meets the requested length, selectively weaves the supplied quotes into the narrative, and keeps the emotional tone restrained. Model B is vivid, but it falls short of the word-count requirement and adds unsupported details, including a new statement attributed to Dev Malik. (Second judge pass, order swapped — scores are the average of both: Model A better meets the 400–500-word requirement while weaving the key facts into a restrained narrative. Model B is evocative, but it falls short | Anthropic: Claude Fable 5.1: Model A hits the word range, uses Markdown, quot…

Exact final-draft structure

Convert the supplied fields into Markdown using these exact rules: * H1 headline * One blank line * Italicized dek * One blank line * First body paragraph * One blank line * H2 section heading * One blank line * Second body paragraph * Preserve all wording and punctuation exactly * Do not use a code fence * Output nothing else Headline: Raincheck raises $8 million for flood-risk software Dek: The Austin startup helps cities decide which drainage problems to address before storms. First body paragraph: Raincheck has raised an $8 million seed round led by Basin Ventures. Section heading: Expansion plans Second body paragraph: The company plans to hire eight engineers.

Qwen: Qwen3.8 Max (0902): # Raincheck raises $8 million for flood-risk software *The Austin startup helps cities decide which drainage problems to address before storms.* Raincheck has raised an $8 million seed round led by Basin Ventures. ## Expansion plans The company plans to hire eight engineers.
grok-4.7: # Raincheck raises $8 million for flood-risk software *The Austin startup helps cities decide which drainage problems to address before storms.* Raincheck has raised an $8 million seed round led by Basin Ventures. ## Expansion plans The company plans to hire eight engineers.

OpenAI: GPT-6 Luna: Model A and Model B both follow the requested structure, preserve the wording and punctuation, and output nothing extra. Their writing quality is equally strong for this formatting task. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both follow the requested structure and preserve all wording and punctuation exactly. Their outputs are identical, so neither is better.) | Anthropic: Claude Fable 5.1: Model A and Model B produced byte-identical outputs that follow every structural rule exactly: H1, blank line, italic dek, blank lines…

Credible Startup Spotlight

Write a 500–650-word Startup Spotlight article in clean Markdown. The tone should be curious and constructive. Explain the founders’ insight, product and early traction while giving uncertainty and competition appropriate space. Avoid investor clichés, heroic founder mythology and promotional claims stated as fact. Reporting: * LedgerLark builds AI agents for post-trade operations at small investment firms. * Co-founder Mina Batra previously spent seven years in operations at a regional brokerage. * Co-founder Owen Carr built reconciliation systems at a large bank. * Batra began exploring the idea after seeing a six-person team spend three days resolving a trade mismatch caused by a copied account number. * LedgerLark’s agents compare confirmations, custodian records and internal ledgers, then propose corrections for human approval. * The product does not execute trades or move money. * Five firms are paying customers. * Annual contracts range from $36,000 to $110,000. * The company says one customer reduced its unresolved daily exceptions from 70 to 18. * The company has raised a $3.5 million seed round led by Motive Row. * Competitors include established reconciliation vendors and internal bank software. * Financial institutions may require lengthy security and procurement reviews.

Qwen: Qwen3.8 Max (0902): # Startup Spotlight: LedgerLark LedgerLark is building AI agents for a part of finance that rarely gets attention: post-trade operations. The startup’s software reviews trade records across confirmations, custodian records and internal ledgers, flags mismatches, and proposes corrections for human approval. It does not execute trades or move money. The insight comes from a familiar operational failure. Co-founder Mina Batra spent seven years in operations at a regional brokerage, where she saw how small errors could consume outsized effort. In one case, a copied account number created a trade mismatch that kept a six-person team working for three days. That experience suggested a narrower, more practical use for AI than replacing finance workers: helping them sort through exceptions that are tedious, time-sensitive and expensive to leave unresolved. Batra co-founded LedgerLark with Owen Carr, who built reconciliation systems at a large bank. Their combined backgrounds point to a problem that is less about model novelty than about workflow fit. Post-trade operations depend on checking data across systems, formats and counterparties. Much of the work is repetitive, but it also requires judgment and auditability. LedgerLark’s approach keeps people in the loop by presenting proposed fixes rather than acting independently. The product is aimed at small investment firms, where operations teams may be lean and legacy tools may not justify large implementation projects. The agents compare records, identify discrepancies and prepare corrections for review. If the system works as intended, it could reduce the amount of time staff spend chasing breaks before they become larger problems. Early traction is modest but measurable. LedgerLark has five paying customers, with annual contracts ranging from $36,000 to $110,000. The company says one customer reduced unresolved daily exceptions from 70 to 18. That figure comes from the company, and it is too early to know how broadly similar results will apply across different firms, data environments and asset classes. Still, the metric points to a practical measure of value: fewer open items requiring human attention at the end of each day. The startup has also raised a $3.5 million seed round led by Motive Row. The funding could support product development, security reviews and customer implementation, though the company will need to show that its agents can handle the messy variety of real-world records without creating new operational risk. Competition is an important part of the picture. Reconciliation is not an empty category. Established vendors already serve financial institutions, and some banks rely on internal software developed over many years. LedgerLark’s challenge will be to prove that AI agents can improve on these alternatives in accuracy, speed or cost, especially in environments where mistakes can be expensive. Sales cycles may also slow progress. Financial institutions typically require careful security reviews, compliance checks and procurement processes. For a young company selling to regulated firms, each new customer may require more time and integration effort than a standard software contract would suggest. Those constraints do not weaken the opportunity; they define it. Post-trade operations are not glamorous, but they sit close to real costs, real errors and real regulatory exposure. If LedgerLark can make exception handling faster and more reliable while keeping humans responsible for final decisions, it may have a useful role in a market that tends to reward incremental, provable improvement over bold automation claims. For now, the company’s promise is specific enough to test: can AI agents reduce the backlog of trade exceptions without taking control away from operations teams? The early customer numbers suggest interest, but the harder question is whether LedgerLark can repeat that result as it moves into more complex firms and workflows.
grok-4.7: # Startup Spotlight: LedgerLark A copied account number should not take three days to unwind. Mina Batra watched it happen anyway. During seven years in operations at a regional brokerage, she saw a six-person team spend three days tracing a trade mismatch back to that single error. The records did not agree, the exception stayed open, and the work of finding the break fell to people who already had a full queue. That episode is where Batra says she began exploring what became LedgerLark, which builds AI agents for post-trade operations at small investment firms. Her co-founder, Owen Carr, comes from the other side of the same problem: he built reconciliation systems at a large bank. Between them, the pair has seen both the manual grind of a smaller shop and the heavier software that larger institutions can afford to maintain. The question they are testing is narrower than a general claim about AI in finance. It is whether software can reliably compare records that already exist and hand a proposed fix to a person who still has to approve it. LedgerLark’s agents compare trade confirmations, custodian records, and internal ledgers, then propose corrections for human approval. The company is explicit about a boundary that matters in this market: the product does not execute trades or move money. In post-trade work, the costly part is often not the original order but the cleanup — mismatched account details, breaks between a custodian’s report and an internal book, exceptions that sit until someone has time to chase them. An agent that drafts a correction is a different proposition from one allowed to act on it. That limit may make a pilot easier to accept. It also means LedgerLark is selling assistance, not a closed loop, and customers still need staff who can judge whether a suggestion is right. Early commercial evidence is real and still small. Five firms are paying customers. Annual contracts range from $36,000 to $110,000, a spread that suggests either uneven scope or a price that has not yet settled. The company says one customer reduced its unresolved daily exceptions from 70 to 18. If that drop holds, it is material. It is also a company-reported figure from a single account, not a result measured across the base. Five customers cannot show whether the same pattern appears under different custodians, asset mixes, or exception types. The contracts indicate that some firms will pay for the workflow. They do not yet show that the category is settled. LedgerLark has raised a $3.5 million seed round led by Motive Row. For a product aimed at regulated buyers, that capital is as likely to go toward security work, implementation, and support as toward model development. Small investment firms are a practical first market: many lack the engineering teams that larger banks use to maintain their own tools. They are not free of procurement friction. Financial institutions may require lengthy security and vendor reviews before software is allowed near client records, even when it never moves cash. Competition is already present. Established reconciliation vendors have sold into this workflow for years, with integrations, audit trails, and relationships a young company does not yet have. Internal bank software is the other alternative — familiar from Carr’s own background — and some firms will prefer to extend what they already run. LedgerLark’s bet is that agents reading across confirmations, custodian files, and ledgers can take on work those tools still leave to people, and that a human-approval step is enough oversight for cautious buyers. Whether that bet survives a long sales cycle is open. A three-day hunt for a copied account number is a clear argument that the work is brittle. It is not proof that an AI agent is the durable fix, or that five early contracts will become a business established vendors cannot match. The more useful test is quieter: whether proposed corrections keep being accepted, whether exception counts fall at more than one firm, and whether LedgerLark can get through the reviews that stand between a pilot and a renewal.

OpenAI: GPT-6 Luna: Model A better meets the requested length and stays closer to the supplied reporting while covering traction, uncertainty, and competition. Model B is more vivid, but appears to exceed the word limit and adds several plausible details not established in the prompt. (Second judge pass, order swapped — scores are the average of both: Model B gives a more specific and balanced account of the product’s limits, early evidence, competition, and sales friction, while remaining within the requested form | Anthropic: Claude Fable 5.1: Model B delivers a sharper, more specific piece…

Markdown error count

Count the lines that violate the house rules below. A defective line counts once even if it could violate multiple rules. Respond with a single number. House rules: * The headline must use `# Headline`. * The dek must appear immediately after the headline and be entirely italicized. * Section headings must use `## Heading`. * Links must use an absolute HTTPS URL. * Unordered-list items must begin with a hyphen. * Raw HTML is forbidden. Draft: #Flood software moves upstream The company is expanding its pilot. ##Details Read [the announcement](www.example.com/news). * The first pilot begins Monday. <div>The second pilot begins Friday.</div>

Qwen: Qwen3.8 Max (0902): 6
grok-4.7: 6

Scored against the benchmark's reference answer (numeric match): Model A was correct, Model B was correct.

Cohesive story from reporting notes

Write a publication-ready technology news article from the reporting notes below. Requirements: * 550–700 words * Clean Markdown * One H1 headline followed by an italicized dek * A strong news lede followed by a clear nut graf * Organize the material into a cohesive story rather than following the notes in order * Attribute company-supplied claims * Include limitations without turning the article into a warning label * Use only the supplied information * Avoid canned transitions, promotional language and a concluding summary Reporting notes: * HelioDock makes autonomous warehouse robots. * On Thursday, it launched Night Shift, software that lets operators monitor robot fleets remotely. * Night Shift spent 14 months in a private beta involving nine logistics companies and 310 robots. * HelioDock says beta customers experienced 17% fewer unplanned robot stoppages. The figure has not been independently verified. * The company also announced a $24 million Series B led by North Coast Ventures, bringing its total funding to $39 million. * CEO Mara Velez: “A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet.” * Beta customer Aaron Pike of Fulcrum Logistics said the system worked well in facilities with strong connectivity but became less useful during rural LTE outages. * When connectivity fails, robots return to their locally configured behavior. Remote control becomes unavailable. * Competing fleet-management products are available from Relay Robotics and YardOS. * Night Shift will cost $90 per robot per month and become generally available September 15.

Qwen: Qwen3.8 Max (0902): # HelioDock Launches Night Shift Remote Fleet Monitoring and Raises $24 Million *The robotics company says Night Shift was tested for 14 months with nine logistics companies and 310 robots before its Sept. 15 general availability.* HelioDock, a maker of autonomous warehouse robots, on Thursday launched Night Shift, software that lets operators monitor robot fleets remotely, and announced a $24 million Series B led by North Coast Ventures. The announcement brings together HelioDock’s product and funding news. The company is moving Night Shift from private beta to a paid commercial service, while the new round increases its total funding to $39 million. HelioDock said beta customers experienced 17% fewer unplanned robot stoppages. The figure was supplied by the company and has not been independently verified. The 17% claim is tied to the beta period and to customers using Night Shift with HelioDock robots. Night Shift is priced at $90 per robot per month and is scheduled to become generally available Sept. 15. The per-robot model means the software cost rises with fleet size, rather than being tied to one building or one software license. Because the software is sold per robot, operators with larger fleets will see the subscription cost scale directly with the number of robots covered. HelioDock said Night Shift spent 14 months in private beta with nine logistics companies and 310 robots. The beta gave the company customer feedback across multiple logistics operations before the public release. Nine logistics companies participated in the beta, according to HelioDock, and Fulcrum Logistics is one beta customer identified by name. CEO Mara Velez described the software as a way to reduce the need for managers to respond in person to isolated robot problems outside normal working hours. “A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet,” Velez said. Velez’s remark identifies a specific operational issue in robot fleets: human time spent responding to a single machine. Night Shift’s remote monitoring is meant to keep that response from always requiring a physical visit. Beta customer Aaron Pike of Fulcrum Logistics said the system worked well in facilities with strong connectivity but became less useful during rural LTE outages. HelioDock said that when connectivity fails, robots return to their locally configured behavior. Remote control becomes unavailable. Under those conditions, operators can no longer use remote control through the software, while the robots continue operating according to local configuration. The company’s description and Pike’s feedback both point to network access as a factor in Night Shift’s usefulness. Facilities with strong connectivity can support the remote-monitoring functions, while rural LTE outages can reduce the value of remote oversight. The Series B provides funding at the point where HelioDock is converting beta use into a generally available product. The company said the $24 million round was led by North Coast Ventures and brings total funding to $39 million. Before the round, total funding stood at $15 million, based on the company’s new $39 million total. HelioDock is not the only company offering fleet-management software for logistics robots. Relay Robotics and YardOS both have competing products, giving operators other options for monitoring and managing robot fleets. The presence of Relay Robotics and YardOS means operators evaluating Night Shift can compare it with products already available in the market. General availability is set for Sept. 15. At $90 per robot per month, the price works out to $1,080 per robot per year, $450 per month for a five-robot fleet and $900 per month for a 10-robot fleet.
grok-4.7: # HelioDock Launches Night Shift Remote Monitoring as Series B Closes *Warehouse-robot maker cites a 17 percent drop in unplanned stoppages from a 14-month beta while a customer flags rural connectivity gaps.* HelioDock on Thursday released Night Shift, software that lets operators monitor fleets of its autonomous warehouse robots from remote locations. The announcement arrived alongside a $24 million Series B led by North Coast Ventures, lifting the company’s total funding to $39 million. The product spent 14 months in a private beta that covered nine logistics companies and 310 robots. HelioDock says those customers experienced 17 percent fewer unplanned robot stoppages. The company supplied the figure; it has not been independently verified. Night Shift will be priced at $90 per robot per month and is scheduled to become generally available on September 15. Chief executive Mara Velez described the operational problem the software is intended to address. “A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet,” she said. Night Shift is built to give that manager a remote view and a means of intervention instead of an immediate site visit. Beta results were uneven across network conditions. Aaron Pike of Fulcrum Logistics, one of the nine participating companies, said the system worked well in facilities with strong connectivity but became less useful during rural LTE outages. When a connection drops, the robots return to their locally configured behavior and remote control becomes unavailable. The machines continue under whatever rules were set on site; the remote layer simply ceases to function until connectivity returns. That fallback is part of the product’s design rather than an unstated failure mode. Operators who adopt Night Shift therefore retain local robot operation during an outage, yet they lose the very remote monitoring the subscription is meant to supply. Pike’s account supplies the only customer detail available from the beta: performance tracked connectivity quality, holding in well-linked buildings and declining where rural LTE proved unreliable. Fleet-management software is already sold by Relay Robotics and by YardOS. Night Shift enters that set of options tied to HelioDock’s own robots and billed monthly per machine. Buyers comparing the offerings can weigh the $90 rate, the September 15 availability date, the company-reported stoppage reduction, and the documented reversion to local behavior against whatever remote features the two competitors provide. The Series B supplies capital as the software moves from the closed group of nine logistics firms toward general release. During the preceding 14 months the 310 robots ran under Night Shift while HelioDock recorded the unplanned-stoppage data it now publishes. Because that 17 percent claim rests solely on the company’s measurement, prospective customers have no third-party confirmation of the result or of how it might translate outside the beta cohort. Velez’s example of a confused robot and a cross-town drive at 2 a.m. sets a concrete use case for remote access. Pike’s experience indicates the use case depends on the network path between operator and facility. In locations where connectivity stayed strong, Fulcrum Logistics found the system serviceable. In rural settings subject to LTE outages, the same system lost its remote function and left the robots on local configuration alone. HelioDock has not published additional customer comments or a breakdown of how many of the 310 robots operated under strong versus intermittent links. The funding total of $39 million includes the new North Coast Ventures-led round and all prior capital. Nothing in the announcement specifies how the proceeds will be allocated between robot hardware, the Night Shift software, or other work. The commercial terms that are specified remain the per-robot monthly price and the mid-September general-availability date. Operators evaluating the launch therefore have a narrow set of disclosed facts: a remote-monitoring layer for HelioDock robots, a company assertion of fewer unplanned stoppages across nine beta customers, an explicit loss of remote control when connectivity fails, a contrasting report from Fulcrum Logistics on rural LTE, existing fleet tools from Relay Robotics and YardOS, and a $90 monthly charge beginning September 15. The 17 percent figure stands as HelioDock’s own accounting of the beta, unverified by any outside party, while the connectivity limit stands as a condition already encountered by at least one participating logistics company.

OpenAI: GPT-6 Luna: Model B has the stronger news structure and more natural prose, though it repeats the connectivity caveat and ends with a summary-like paragraph. Model A is more mechanical and padded, and both introduce some details or interpretations beyond the notes. (Second judge pass, order swapped — scores are the average of both: Model B better satisfies the 550–700-word requirement and includes the key reporting details, though it repeats points and ends with a summary-like passage. Model A is more conci | Anthropic: Claude Fable 5.1: Model B delivers a stronger lede, a real nut gr…

News judgment in the opening

Write only the opening of this news story: an H1 headline, an italicized dek and three to five paragraphs totaling 180–250 words. The opening must establish what happened, quantify the scope, explain why it matters and carefully distinguish confirmed facts from unknowns. It should pull the reader forward without sensationalizing the incident. Reporting: * Cloudnote discovered that a debugging page containing customer prompt excerpts was accessible without authentication. * The page was exposed for four hours on Monday. * Logs show excerpts from 1,842 user sessions. * The excerpts could include up to 200 characters from a prompt. * Cloudnote removed the page 19 minutes after a researcher notified it. * The company says it has found no evidence that anyone besides the researcher accessed the page. * It has not completed its investigation. * Full documents, passwords and payment information were stored elsewhere and were not present on the page. * Cloudnote has 3.2 million registered users.

Qwen: Qwen3.8 Max (0902): # Cloudnote Says Debugging Page Exposed Prompt Excerpts for Four Hours *A researcher found an unauthenticated page showing snippets from 1,842 sessions; the company says no broader access has been confirmed, and its investigation continues.* Cloudnote, which has 3.2 million registered users, said Monday that a debugging page containing customer prompt excerpts was publicly accessible without authentication for four hours. The company said it removed the page 19 minutes after a security researcher notified it of the exposure. Logs reviewed by Cloudnote show excerpts from 1,842 user sessions. Each excerpt could include as many as 200 characters from a prompt, a limited but potentially sensitive portion of a request. The company said full documents, passwords and payment information were stored elsewhere and were not present on the page. Cloudnote said it has found no evidence that anyone besides the researcher accessed the page, but it has not completed its investigation. That leaves open the question of whether any other person reached it during the exposure window, and no additional access has been verified yet by the company. The incident shows how internal debugging tools can create privacy risk when they contain user-submitted text from customer accounts. Cloudnote has not said which prompts were exposed, whether affected users will be notified, or what steps will prevent a similar recurrence.
grok-4.7: # Cloudnote debugging page exposed prompt excerpts without a login *An unauthenticated debugging page was reachable for four hours Monday and showed excerpts from 1,842 sessions. The company says only a researcher accessed it; the review is unfinished.* Cloudnote said a debugging page containing customer prompt excerpts was accessible without authentication for four hours on Monday. Logs show excerpts from 1,842 user sessions. Each excerpt could include up to 200 characters from a prompt. The company said full documents, passwords and payment information were stored elsewhere and were not present on the page. Cloudnote has 3.2 million registered users. The figure in the logs is 1,842 sessions, not the full user base, and those logs do not show how many people opened the page. A researcher notified Cloudnote, and the company removed the page 19 minutes later. Cloudnote says it has found no evidence that anyone besides the researcher accessed the page. It has not completed its investigation, so that statement is not yet independently confirmed. Whether anyone else viewed the excerpts, and what they contained, is still unknown. The exposure matters because even a short prompt can include private or sensitive text, and a page without login can be opened by anyone who finds the address. Confirmed so far are the four-hour window, excerpts from 1,842 sessions, a limit of 200 characters and the absence of full documents, passwords and payment data from the page. Not confirmed is whether anyone other than the researcher reached it.

OpenAI: GPT-6 Luna: Both entries meet the requested format and length, but Model A more carefully preserves the distinction between no evidence of further access and proof that none occurred. Model B’s dek overstates the company’s finding as saying only a researcher accessed the page, and it adds claims about what the logs show and who could open the page that are not established in the reporting. (Second judge pass, order swapped — scores are the average of both: Model A is more concise and gives the opening a cle | Anthropic: Claude Fable 5.1: Model A reads like a polished news opening that…

Select the strongest accurate lede

Based only on the facts below, select the strongest news lede. Respond with one letter. Facts: * ByteBarge pulled its $12-per-month AI assistant on Tuesday. * Private workspace titles from 4,860 accounts were sent to a publicly accessible analytics endpoint. * The exposure lasted 11 hours. * Message contents were not sent. * The company has found no evidence of malicious access. A. ByteBarge users suffered a massive privacy disaster Tuesday when the company leaked their private messages online. B. ByteBarge, an innovative collaboration company, made the difficult decision to cancel one of its products. C. ByteBarge pulled its $12-per-month AI assistant Tuesday after discovering that private workspace titles from 4,860 accounts had been sent to a public analytics endpoint for 11 hours. D. Thousands of ByteBarge users woke Tuesday to learn that everything they had written was visible to hackers.

Qwen: Qwen3.8 Max (0902): C
grok-4.7: C

Scored against the benchmark's reference answer (choice match): Model A was correct, Model B was correct.

Controlled wit

Write a 350–450-word technology news story in Markdown. The subject invites humor, but the reporting should remain serious. Use no more than one dry line of original humor. Do not mock customers or turn the story into a comedy routine. Reporting: * QueueZero launched an AI meeting assistant called Proxy on Wednesday. * Proxy can attend a meeting, produce notes and assign follow-up work to software agents. * During the launch demonstration, a customer in the audience said, “Someone should tell Evan to stop talking and send us the pricing sheet.” * Proxy interpreted the remark as an instruction. * It created two tasks assigned to QueueZero CEO Evan Wu: “Stop talking” and “Send pricing sheet.” * Wu laughed, completed the second task and deleted the first. * QueueZero says Proxy requires confirmation before an external message is sent. * Internal tasks can be created automatically. * Administrators can disable automatic task creation. * Proxy costs $30 per user per month. * It enters public beta next week.

Qwen: Qwen3.8 Max (0902): # QueueZero Launches AI Meeting Assistant That Mistakes an Audience Quip for Work Orders QueueZero on Wednesday launched Proxy, an artificial-intelligence meeting assistant designed to attend meetings, take notes and assign follow-up work to software agents. The product’s first public demonstration quickly provided a case study in how literal such systems can be. During the launch event, a customer in the audience said, “Someone should tell Evan to stop talking and send us the pricing sheet.” Proxy interpreted the remark as an instruction rather than a casual comment. It created two tasks assigned to QueueZero chief executive Evan Wu: “Stop talking” and “Send pricing sheet.” Wu laughed at the episode, completed the second task and deleted the first. The moment gave the company an unplanned example of Proxy’s core function: listening to meeting language and converting it into actionable items. QueueZero said Proxy requires confirmation before any external message is sent. Internal tasks, however, can be created automatically, a setting that reflects how many companies want AI agents to reduce administrative work without waiting for human approval at every step. The company also said administrators can disable automatic task creation, giving organizations more control over how aggressively the software acts inside their workflows. The incident underscores a broader issue facing AI assistant makers as they push products into meetings, email threads and business software. Users may speak in shorthand, sarcasm or offhand remarks, while systems trained to identify tasks may treat every sentence as a potential command. For QueueZero, the challenge is to make Proxy useful enough to save time without creating work that no one intended. Proxy is priced at $30 per user per month, placing it in the competitive market for AI productivity tools that promise to handle note-taking, follow-ups and task management. The product is aimed at teams that spend large amounts of time in meetings and want a consistent record of decisions and next steps. QueueZero said Proxy will enter public beta next week. That release will give a wider group of customers access to the assistant and likely provide more examples of how it handles ambiguous workplace language. For now, the company’s launch demonstration offered a reminder that when software is built to take initiative, it may take every word literally.
grok-4.7: # QueueZero’s Meeting Assistant Files Two Tasks for Its CEO QueueZero on Wednesday launched Proxy, an artificial intelligence meeting assistant that can attend a meeting, produce notes and assign follow-up work to software agents. The demonstration that accompanied the launch showed the assignment function acting on a sentence that was not offered as a command. A customer in the audience said, “Someone should tell Evan to stop talking and send us the pricing sheet.” Proxy interpreted the remark as an instruction. It created two tasks, both assigned to QueueZero chief executive Evan Wu: “Stop talking” and “Send pricing sheet.” Wu laughed, completed the second task and deleted the first. The software had matched a name, divided a compound remark and written work against the person it identified. It had not paused to ask whether either item should have been created. QueueZero says Proxy requires confirmation before an external message is sent. Internal tasks can be created automatically. Administrators can disable automatic task creation, an option that leaves the assistant able to take notes without opening work on its own. The controls separate listening from outward action: speech inside a meeting can become an internal assignment without a prompt, while a message bound for someone outside the company is supposed to wait for a person to approve it. That separation is the question Proxy is walking into. Meeting tools have spent years summarizing calls and extracting action items. Assigning follow-up to software agents goes further: the record of a meeting becomes work that other programs, as well as people, may be expected to carry out. A pricing request captured cleanly is the intended use. An instruction to stop talking, filed because it shared a sentence with that request, is the miss. Wednesday’s error stayed inside the company. The two tasks were internal assignments, not outbound messages, which is consistent with QueueZero’s statement that external messages require confirmation. Wu was the assignee because the remark used his name. He accepted the pricing task as work and removed the other. Proxy had not drawn that distinction on its own. Public beta begins next week, moving Proxy from a launch demonstration into ordinary meetings, where asides and half-finished thoughts are common. Buyers will be weighing a listed price of $30 per user per month against how much automatic task creation they are willing to leave on. Disabling that feature is available. Leaving it on means accepting that an agent may write work from language it was never directly given. The speaking ban was the item that did not remain on the list.

OpenAI: GPT-6 Luna: Model B is more precise about the incident and its safeguards, and its single dry closing line fits the requested controlled wit. Model A is clear and compliant, but adds broader claims that are not supported by the reporting provided. (Second judge pass, order swapped — scores are the average of both: Model B delivers a serious, well-reported story within the requested length and limits its humor to one dry closing line. Model A is polished but falls short of the word-count requirement and adds | Anthropic: Claude Fable 5.1: Model B hews tightly to the reported facts, kee…

Chronology with narrative momentum

Turn the scrambled reporting below into a 450–600-word Markdown news article. Choose an opening that captures the consequence of the failure, then reconstruct the sequence clearly. Preserve the chronology without producing a minute-by-minute incident log. Give technical details enough context for a general reader. Reporting: * KiteBridge makes expense-management software. * At 8:45 a.m. Tuesday, it deployed version 4.2. * The first customer complaint arrived at 9:12 a.m. * A payment-processor timeout caused KiteBridge’s fallback system to treat retries as new reimbursement requests. * The software generated 13,400 duplicate reimbursements totaling $2.8 million. * KiteBridge suspended automated payments at 10:03 a.m. * Ninety-one corporate customers were affected. * By Wednesday evening, $2.3 million had been recovered. The rest remained pending. * No employees had money withdrawn from their accounts; employers funded the duplicate payments. * An April changelog said version 4.0 had introduced “exactly-once payment protection.” * KiteBridge’s public incident page initially described the problem as delayed reimbursements. * The incident page first disclosed the duplicate payments six hours after automation was suspended. * CEO Lena Ortiz apologized and said the company would publish a technical report.

Qwen: Qwen3.8 Max (0902): # KiteBridge Update Created $2.8 Million in Duplicate Reimbursements A software update at expense-management company KiteBridge turned an ordinary Tuesday morning payment run into a costly duplication failure, producing 13,400 duplicate reimbursements totaling $2.8 million for 91 corporate customers. The incident forced the company to suspend automated payments, prompted a delayed public explanation, and raised questions about a safeguard KiteBridge had previously advertised as protection against repeated payments. The trouble began after KiteBridge deployed version 4.2 of its software at 8:45 a.m. Tuesday. The company, which helps businesses manage employee expense claims and reimbursements, had promoted the update as routine. At 9:12 a.m., the first customer complaint arrived, signaling that something in the payment workflow was misbehaving. According to the company’s account, the failure started when a payment processor stopped responding long enough to trigger a timeout. In payment systems, a timeout is a safeguard that prevents software from waiting indefinitely for an answer. When that happens, a fallback process may step in to decide whether a transaction needs to be retried, ignored, or flagged for review. In this case, KiteBridge’s fallback system mishandled the situation. Instead of recognizing that the original reimbursement requests were still in progress or had already been sent, it treated the automatic retries as fresh, separate requests. The result was that the same reimbursement was paid more than once, creating duplicate transfers tied to expense claims. For a general user, the error was similar to submitting one expense report, having the system lose track of the first attempt, and then paying the same claim again because it assumed no payment had been made. By 10:03 a.m., KiteBridge had suspended automated payments in an effort to stop the duplicate transactions from spreading. The move froze the immediate flow of money but did not immediately clarify the scale of the problem. The company’s public incident page initially described the issue as delayed reimbursements, a characterization that understated what had already occurred. It was not until six hours after automated payments were suspended that the incident page disclosed that duplicate payments had been issued. By then, the company had identified 13,400 repeated transactions and estimated their value at $2.8 million. Ninety-one corporate customers were affected. KiteBridge said no employees had money withdrawn from their accounts. The duplicate payments were funded by employers, meaning the error created excess payouts from company accounts rather than taking money away from workers. That distinction may have spared employees direct financial disruption, but it left finance teams to reconcile duplicate deposits, reverse transactions, and adjust expense records. The failure also drew attention to KiteBridge’s earlier promises. In an April changelog, the company said version 4.0 had introduced “exactly-once payment protection,” a feature intended to prevent a reimbursement from being paid multiple times. The duplicate payments under version 4.2 suggest that either the protection failed, did not cover the failure path created by the payment-processor timeout, or was bypassed by the fallback system. By Wednesday evening, KiteBridge said it had recovered $2.3 million of the duplicated funds. The remainder was still pending, leaving some customers waiting for resolution. Chief Executive Lena Ortiz apologized for the incident and said the company would publish a technical report explaining what went wrong. For KiteBridge, the next challenge will be showing that its promised safeguards can do more than appear in a changelog.
grok-4.7: # KiteBridge Error Issued $2.8 Million in Duplicate Reimbursements A flaw in KiteBridge’s expense-management software sent 13,400 duplicate reimbursement payments totaling $2.8 million to employees on Tuesday, leaving 91 corporate customers to unwind charges their finance teams had never approved twice. The company deployed version 4.2 at 8:45 a.m. The first customer complaint arrived at 9:12 a.m. Behind those early reports was a narrow but costly failure in how the system handled uncertainty. A payment processor timed out, meaning KiteBridge did not receive a timely confirmation that a transfer had succeeded. Timeouts are common in online payments: the absence of a reply does not prove the money failed to move, only that the sender did not hear back. KiteBridge’s fallback was supposed to retry the same request. Instead, it treated each retry as a new reimbursement, and the software issued a fresh payment for an expense that had already been submitted. Automated payments continued until 10:03 a.m., when KiteBridge suspended them. In a little more than an hour, the error had multiplied across 91 employers. Employees did not have money withdrawn from their accounts. In this kind of software, the employer funds the reimbursement; the employee receives it. The duplicates were therefore extra charges against corporate customers, not deductions from workers. The failure sits awkwardly beside a promise made only months earlier. An April changelog said version 4.0 had introduced “exactly-once payment protection,” a safeguard meant to ensure that one approved expense produced one payment even if a network problem forced a retry. Version 4.2 was deployed as a later release of that product. On Tuesday, the timeout-and-retry path appears to have bypassed the protection the company had advertised. Public description of the incident lagged the engineering response. KiteBridge’s incident page initially called the problem delayed reimbursements, a framing that suggested money was slow to arrive rather than arriving twice. The page first disclosed the duplicate payments six hours after automation was suspended, well into Tuesday afternoon. By then, finance teams at affected companies would already have been reconciling unexpected transfers. Chief executive Lena Ortiz later apologized and said the company would publish a technical report. Recovery moved faster than that first public account. By Wednesday evening, KiteBridge had recovered $2.3 million of the $2.8 million. The remaining $500,000 was still pending, tied up in transfers not yet reversed or not yet confirmed by customers. The company has not said how many of the 13,400 duplicate transactions remain open, or whether any employers spent or reallocated the extra funds before the error was identified. For readers outside finance software, the mechanics are simpler than the product. Payment systems often cannot know, at the moment a request is sent, whether the other side received it. A careful design records a unique identifier for each intended payment and refuses to create a second one when it retries. KiteBridge’s fallback, under the conditions triggered Tuesday, did the opposite: it minted new reimbursement requests, and the payment pipeline honored them. Suspending automation stopped that multiplication, but it also froze legitimate reimbursements until engineers could separate real requests from duplicates. That is why customers first experienced the outage as delay, and why an incident page written in those terms was not wholly false. It was incomplete. Duplicate charges and slow payments are different problems, and treasury teams plan for them differently. Ortiz’s promised technical report will be read against the April claim of exactly-once protection. Customers will want to know whether version 4.2 reopened a path that version 4.0 was supposed to close, whether the processor timeout was unusual or routine, and why the public account changed only after automation had already been off for six hours. Until that report appears, the settled facts are these: employers funded the duplicates, most of the money has been pulled back, and a safeguard advertised as exactly-once did not hold when the network went quiet and the software tried again.

OpenAI: GPT-6 Luna: Model A gives a clear chronological account within the requested length and mostly keeps its technical explanation grounded in the reporting. Model B has a strong opening, but it appears to exceed the word limit and adds several unsupported details about payment safeguards, recovery, and customer operations. (Second judge pass, order swapped — scores are the average of both: Model A gives a clear chronological account within the requested length; Model B offers useful technical context but likel | Anthropic: Claude Fable 5.1: Model A stays within the 450–600-word limit, pr…

Natural human rewrite

Rewrite the draft below as a 300–400-word publication-ready news story in Markdown. Preserve every factual claim. Improve the lede, narrative flow, sentence rhythm and transitions. Remove repetition, stock hype, throat-clearing and mechanical signposting. The finished article should sound like an experienced human technology reporter wrote it. Draft: # Arcwell Launches An Innovative New Platform Arcwell has announced an exciting new platform called Relay. The announcement represents a significant development for the company and for the broader customer service industry. Relay uses artificial intelligence to examine customer support conversations and suggest answers to agents. The platform is important for several reasons. First, it can suggest answers. Second, it can identify customers who may cancel. Third, it can create summaries. These capabilities demonstrate how artificial intelligence continues to transform the customer service landscape. Relay was tested by 12 companies over six months. Arcwell says agents using it closed tickets 14% faster. The company did not release customer-retention results. Relay costs $65 per agent each month. CEO Priya Shah said, “Support agents spend too much time searching for information that their companies already have.” However, the platform also presents potential challenges. Relay requires access to support transcripts and internal help documents. Arcwell says customer data is encrypted and is not used to train shared models. Companies can choose a 30-day or 90-day retention period. In conclusion, Relay represents an important new chapter for Arcwell as it seeks to help companies improve customer service using artificial intelligence.

Qwen: Qwen3.8 Max (0902): # Arcwell launches Relay, an AI platform that suggests answers to support agents Arcwell has launched Relay, an artificial intelligence platform that reviews customer support conversations and suggests answers to agents as they work. The product aims to reduce the time agents spend hunting through internal documents and past tickets, while also giving managers an early warning when customers appear likely to cancel. Relay examines support interactions and offers recommended responses during conversations. It can also identify customers who may cancel and automatically create summaries of support cases. The idea is to turn those conversations into operational signals for supervisors and agents, not just a record of complaints. By analyzing conversations as they unfold, Arcwell is betting support software can intervene earlier in the customer relationship. That approach may help supervisors spot problems before they become cancellations, but it also raises the stakes for accuracy, since agents may rely on the system during live interactions. The company said Relay was tested by 12 companies over six months. During that period, agents using the platform closed tickets 14% faster, according to Arcwell. The company did not release customer-retention results, leaving unclear how much impact its cancellation predictions had on keeping customers. CEO Priya Shah said the product was designed around a persistent problem in support operations. “Support agents spend too much time searching for information that their companies already have,” Shah said. Relay is priced at $65 per agent each month. To work, it requires access to support transcripts and internal help documents, raising questions about how sensitive customer information is handled. Arcwell said customer data is encrypted and is not used to train shared models. Companies can choose a 30-day or 90-day retention period. The launch marks Arcwell’s attempt to sell a layer of analysis across support conversations, rather than a single automation feature. Its challenge will be convincing customers that the productivity gains justify the monthly cost and that its safeguards are strong enough for data that includes private conversations and internal knowledge.
grok-4.7: # Arcwell Announces Relay, an AI Platform for Support Agents Arcwell has announced Relay, a platform that uses artificial intelligence to examine customer support conversations and suggest answers to agents. CEO Priya Shah cast it as a fix for time lost inside the company, not for a lack of information. "Support agents spend too much time searching for information that their companies already have," she said. Suggested answers are only part of what Relay does. The platform also identifies customers who may cancel and creates summaries. Arcwell tested it with 12 companies over six months and says agents using it closed tickets 14 percent faster. The company did not release customer-retention results, so the cancellation capability arrives without a published account of whether the customers it identified stayed. Relay costs $65 per agent each month. The number Arcwell is willing to stand behind is speed. Twelve companies, six months, tickets closed 14 percent faster: that is the test record as the company tells it. Retention, the outcome most closely tied to identifying customers who may cancel, is the result it left out. Price does not have that ambiguity. Access does not, either. Relay requires access to support transcripts and internal help documents — the conversations it examines and the material Shah says agents already spend too long searching for. Arcwell says customer data is encrypted and is not used to train shared models. Companies can choose a 30-day or a 90-day retention period. Those conditions sit at the center of the product, not beside it. Suggested answers, summaries, and the identification of customers who may cancel depend on the transcripts and help documents Relay is built to examine. Encryption, and a stated bar on using customer data to train shared models, are the limits Arcwell attaches to that access. How long the data is kept is the choice it leaves open: 30 days or 90. Companies deciding whether a 14 percent gain in closing tickets is worth $65 an agent each month will be deciding, at the same time, whether those terms are enough — and whether they can accept a cancellation feature whose retention results were not released.

OpenAI: GPT-6 Luna: Model B stays closer to the supplied facts, while Model A adds unsupported specifics about live analysis, automatic summaries and supervisory alerts. Model B is notably repetitive, but its stronger factual restraint gives it the edge for this task. (Second judge pass, order swapped — scores are the average of both: Model B is more faithful to the draft, while Model A adds unsupported details about real-time use, managers and supervisors, and automated summaries. Model B’s repeated test results a | Anthropic: Claude Fable 5.1: Model A delivers a clean, naturally paced news…

Numbers shaped into a story

Write a 600–750-word Markdown article from the benchmark results below. Build the story around the clearest finding. Explain the methodology and limitations where readers need them. Avoid reciting the table row by row or treating small differences as decisive. Benchmark: * Researchers tested five language models on 200 anonymized customer-support tickets. * Each model produced one proposed answer per ticket. * Three experienced support managers graded the answers blindly. * Scores combined factual correctness, usefulness and tone. * Model North: 84.2 average score; $1.90 estimated cost per 1,000 tickets. * Model Vale: 83.8; $0.74 per 1,000 tickets. * Model Ember: 79.1; $0.31 per 1,000 tickets. * Model Kite: 76.4; $0.22 per 1,000 tickets. * Model Harbor: 71.0; $0.08 per 1,000 tickets. * North and Vale’s score difference was not statistically significant. * Vale answered 18% faster than North. * North performed best on billing disputes. * Vale performed best on technical troubleshooting. * All five models performed poorly when a ticket depended on an image attachment. * The test used English-language tickets from one software company. * Researchers did not test agentic actions, tool use or multilingual support.

Qwen: Qwen3.8 Max (0902): # For Support Tickets, Value Beats a Narrow Lead When teams compare language models for customer support, the temptation is to crown the model with the highest average score. In this benchmark of five models on 200 anonymized customer-support tickets, that instinct would point to Model North, which averaged 84.2. But the more important result is that Model Vale, at 83.8, was not statistically different from North while costing far less and answering faster. For most support organizations, Vale looks like the more practical default. The benchmark asked each model to produce one proposed answer for each ticket. Three experienced support managers then graded the responses without knowing which model wrote them. Their scores combined factual correctness, usefulness, and tone—three qualities that matter more than raw fluency in a real support queue. The result is not a test of which model sounds clever; it is a test of whether the reply would help a customer and protect the company’s relationship. Against that bar, the top two models formed a clear leading group. North and Vale both scored well above the rest of the field, and their difference was too small to treat as meaningful. In practical terms, a support team should not pay a premium for North solely because its average is 0.4 points higher. The benchmark does not show that North is decisively better. Where the models did differ was in economics and speed. Vale’s estimated cost was $0.74 per 1,000 tickets, compared with $1.90 for North. Vale also answered 18% faster. In a high-volume queue, those advantages matter. Faster responses can improve first-response targets, and lower costs make it easier to route more tickets through AI assistance before a human reviews them. Vale’s combination of comparable quality, lower price, and greater speed is the clearest finding in the data. That does not mean North is irrelevant. It performed best on billing disputes, a category where accuracy, policy awareness, and careful tone can be especially important. If a company’s support mix is heavily billing-focused, North may deserve a closer look, even at a higher price. Vale, meanwhile, performed best on technical troubleshooting, which often rewards clear step-by-step reasoning. The better choice may depend on the shape of the ticket backlog, not on the overall average alone. The lower-cost models—Ember, Kite, and Harbor—show the expected tradeoff. Their scores decline as price falls, with Harbor posting the lowest average but also the lowest estimated cost. Still, the gap between the top pair and the rest is large enough that buyers should be cautious about choosing purely on price. A cheaper model may save money on inference while increasing the cost of human correction, customer frustration, or escalations. Readers should also keep the methodology in mind. Two hundred tickets is enough to spot broad patterns, but it is not large enough to prove that one model is superior on every type of request. The grading process was thoughtful—blind review by experienced managers is a strength—but human scoring still reflects judgment. The benchmark also used English-language tickets from one software company, so the findings may not transfer cleanly to other products, customer bases, or support styles. One limitation is especially important: all five models struggled when a ticket depended on an image attachment. That weakness cuts across the entire field. For organizations where screenshots, receipts, diagrams, or error images are central to support, these results suggest that current models may need strong human oversight or additional tooling before they can be trusted. The benchmark also did not test agentic actions, tool use, or multilingual support. A model that looks good at drafting replies may behave differently when asked to look up account details, call internal tools, or operate in another language. The takeaway is not that Model North is bad or that Model Vale is universally superior. It is that the benchmark’s most useful signal is a value signal: Vale delivers statistically comparable support quality to North at a much lower cost and faster speed. For teams building a first-response system, triage assistant, or agent draft tool, that is a compelling starting point. The next step would be a narrower evaluation on a company’s own tickets, especially billing disputes, technical issues, and attachment-heavy cases, before making a final deployment decision.
grok-4.7: # The Expensive Model Did Not Win On a blind review of 200 customer-support tickets, the costliest language model did not earn a meaningful quality edge. Model North scored 84.2. Model Vale scored 83.8. That gap was not statistically significant. Vale, however, cost about 60 percent less—$0.74 versus $1.90 per 1,000 tickets—and answered 18 percent faster. For a support operation that cares about usable replies more than leaderboard decimals, that is the result that matters. The rest of the field fell away more clearly. Ember averaged 79.1 at $0.31 per 1,000 tickets, Kite 76.4 at $0.22, and Harbor 71.0 at $0.08. Those drops are large enough to treat as real differences in this test, not noise. Quality did not collapse in a straight line with price, but the cheapest options were also the weakest. The interesting choice is not “best score at any cost” versus “cheapest model available.” It is whether a near-tie at the top justifies paying more than twice as much. ## How the scores were made Researchers gave each of five models one proposed answer per ticket. The tickets were anonymized and drawn from a single software company, all in English. Three experienced support managers graded the answers without knowing which model wrote them. Each score combined factual correctness, usefulness, and tone, then those judgments were averaged. That design is stronger than a single automated metric. Blind human grading reduces the chance that reviewers favored a familiar brand, and three raters are better than one. It is still a narrow test. One answer per ticket does not show whether a model would recover after a follow-up, ask a better clarifying question, or stay consistent across a thread. Estimated cost per 1,000 tickets is a planning figure, not an invoice. Latency is reported only as Vale’s 18 percent advantage over North, so speed comparisons among the other models are unknown. Small score gaps should be read cautiously. North’s 0.4-point lead over Vale did not clear the bar the researchers set for statistical significance. A manager who picked North solely because 84.2 looks higher than 83.8 would be over-reading the table. The larger gaps—North and Vale in the low-to-mid 80s, then Ember near 79, Kite in the mid-70s, Harbor at 71—are the ones that change a deployment decision. ## Where each leader actually helped Averages also hide a split in the work. North was strongest on billing disputes. Vale was strongest on technical troubleshooting. The study does not publish the size of those advantages, so they should not be treated as a routing algorithm. They do suggest that “overall quality” is not the same as “best on the tickets that cost the most to get wrong.” A team whose queue is heavy with refunds, invoices, and plan changes might still prefer North if billing errors are expensive and the extra cost is tolerable. A team whose queue is mostly setup problems, error messages, and how-to questions has a clearer case for Vale: similar overall scores, lower cost, faster replies, and the better showing on that kind of work. Mixing both models by ticket type is an obvious next experiment. This benchmark did not run it. The cheaper models are harder to recommend from these numbers alone. Ember sits in a middle band: noticeably below the leaders, still well above Harbor, and much less expensive than North. Whether a four- or five-point drop is acceptable depends on what the graders were marking down—wrong facts, vague steps, or tone. The published scores do not separate those components, so a buyer cannot tell whether Ember’s discount buys mostly harmless stiffness or occasional incorrect policy. ## The failure they all shared Every model did poorly when a ticket depended on an image attachment. That shared miss is more operationally important than the ranking among text-only replies. Screenshots of error dialogs, billing pages, and broken layouts are ordinary in software support. A model that writes a fluent answer from the ticket text while missing the picture can sound helpful and still be wrong. The test does not say whether the models received the images, only that performance was poor when the ticket depended on one. Teams should not assume a higher score on text tickets will carry over to visual cases. Until a model is evaluated on the actual attachment workflow—image in, answer out, graded for whether it used what was on screen—image-heavy tickets should stay with people, or with a process that does not pretend the model saw the evidence. ## What this does not prove The sample is 200 English-language tickets from one software company. Wording, product surface, and policy will differ elsewhere. Results may not transfer to other industries, other languages, or even another software firm with a different help center. The researchers did not test agentic actions, tool use, or multilingual support. A model that only drafts a reply is a different product from one that looks up an order, issues a refund, or resets an account. Nothing here measures those capabilities, or the new failure modes they introduce. Human grades of a single draft also miss downstream cost. A slightly weaker answer that forces a second contact can erase a per-ticket saving. A faster answer that is wrong on billing can be more expensive than a slower one that is right. The study scores correctness, usefulness, and tone together, which is the right instinct, but it does not track reopen rates, handle time after the draft, or customer satisfaction on the live ticket. ## The decision the numbers support Use Vale as the default draft model unless a specific slice of work shows otherwise. North did not separate itself on overall quality, and it costs more while answering more slowly. Keep North in mind for billing-heavy queues, and test that claim on your own disputes before paying the premium. Treat Ember, Kite, and Harbor as quality steps down, not as free substitutes, unless a separate review shows their errors are cheap to catch. Do not automate tickets whose answer lives in a screenshot. And do not read this benchmark as a verdict on agents, tools, or languages it never touched. The clearest finding is modest and useful: among these five models, the top of the quality range was a tie, and the cheaper, faster side of that tie is the one worth buying first.

OpenAI: GPT-6 Luna: Model A stays within the requested length and clearly centers the statistically comparable quality of North and Vale, while explaining relevant limitations. Model B is well written but substantially exceeds the word limit and overstates that score gaps among the lower-ranked models are real differences. (Second judge pass, order swapped — scores are the average of both: Model A better fits the requested length and centers the clearest finding: Vale’s statistically comparable score, lower cost, a | Anthropic: Claude Fable 5.1: Model A stays close to the 600–750-word limit,…

Publication-ready Markdown edit

Edit the copy below into a publication-ready Markdown story. Requirements: * Supply an H1 headline and italicized dek * Repair the lede and paragraph flow * Use a Markdown blockquote only for the extended statement * Convert the supplied URL into a descriptive inline link * Remove internal notes and formatting debris * Preserve the reporting and its meaning * Output only the finished article Raw copy: HEADLINE TBD The funding was announced today. Raincheck is the company. It has raised $8 million and it makes flood-risk software for local governments. [NOTE: move this up?] The Austin company maps drainage systems and combines those maps with rainfall forecasts. Cities use the software to decide where to send maintenance crews before a storm. The seed round was led by Basin Ventures. Existing investor Long Mile also participated. Raincheck has raised $10.5 million total. CEO Ana Flores said the following: “Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives.” https://example.com/raincheck-methodology Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them because some contracts have not been publicly announced. Its forecasts are recommendations. Emergency managers retain control over deployments and public warnings. THE COMPETITION Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck says the new funding will help it hire eight engineers and expand into Florida.

Qwen: Qwen3.8 Max (0902): # Raincheck Raises $8 Million to Help Cities Prepare for Flooding *The Austin company uses drainage maps and rainfall forecasts to help local governments prioritize work before storms arrive.* Raincheck, a startup that makes flood-risk software for local governments, said today that it has raised $8 million in seed funding. The company maps drainage systems and combines those maps with rainfall forecasts, giving cities a way to decide where to send maintenance crews before a storm arrives. The seed round was led by Basin Ventures, with participation from existing investor Long Mile. Raincheck has now raised $10.5 million in total. CEO Ana Flores said the challenge for many customers is not identifying where flooding occurs, but determining which fixes to address first. > “Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives.” Raincheck currently works with seven cities in Texas and Louisiana, though the company declined to identify them because some contracts have not been publicly announced. More details about its approach are available in [Raincheck’s methodology](https://example.com/raincheck-methodology). The company said its forecasts are recommendations, not directives, and that emergency managers retain control over deployments and public warnings. Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck said the new funding will be used to hire eight engineers and expand into Florida.
grok-4.7: # Raincheck Raises $8 Million for Flood-Risk Software *The Austin company maps drainage systems against rainfall forecasts so cities can send maintenance crews before a storm.* Raincheck, an Austin company that makes flood-risk software for local governments, announced today that it has raised $8 million. The company maps drainage systems and combines those maps with rainfall forecasts. Cities use the software to decide where to send maintenance crews before a storm. Raincheck publishes its [flood-risk methodology](https://example.com/raincheck-methodology). The seed round was led by Basin Ventures. Existing investor Long Mile also participated. Raincheck has raised $10.5 million total. CEO Ana Flores said: > Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives. Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them because some contracts have not been publicly announced. Its forecasts are recommendations. Emergency managers retain control over deployments and public warnings. Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck says the new funding will help it hire eight engineers and expand into Florida.

OpenAI: GPT-6 Luna: Model A has smoother paragraph flow and a more effective lede while preserving the key reporting. Model B is also clean and compliant, but its claim that Raincheck publishes the methodology is not stated in the source copy. (Second judge pass, order swapped — scores are the average of both: Model B preserves the reported facts and presents the material cleanly, while Model A repeats the CEO’s point in a paraphrase immediately before the full quote. Both meet the main formatting requirements, tho | Anthropic: Claude Fable 5.1: Model A delivers a genuinely repaired lede and…

Matchup powered by OpenRouter.