Head to head: grok-4.7 vs Qwen: Qwen3.8 2.4T A95B

grok-4.7 vs Qwen: Qwen3.8 2.4T A95B

By · Published

RuntimeWire Head-to-Head: Head to head: grok-4.7 vs Qwen: Qwen3.8 2.4T A95B
RuntimeWire Head-to-Head matchup

A close editorial matchup testing how grok-4.7 and Qwen: Qwen3.8 2.4T A95B handle reported facts, narrative structure and tight writing constraints. The comparison spans both polished story drafts and precise editing tasks.

The results are split, not decisive: grok-4.7 scores 99.0 and Qwen: Qwen3.8 2.4T A95B scores 98.5, while the saved task record gives Qwen four wins to grok’s three, with five ties. The narrow aggregate gap doesn’t override that mixed task-by-task picture. Grok’s clearest edge is in reported-story judgment. It takes “Quotes woven into narrative,” “News judgment in the opening” and “Credible Startup Spotlight,” with judges noting stronger use of supplied facts, clearer treatment of uncertainty and better attention to stakes. Those strengths matter in assignments where unsupported context can compromise an otherwise polished piece. Qwen wins on “Cohesive story from reporting notes,” “Publication-ready Markdown edit,” “Controlled wit” and “Numbers shaped into a story.” Its drafts were often cleaner or better tuned to length and format, though some evaluations flagged unsupported operational or market context. The ties are telling, too: the models matched on lede selection and exact structure, among other tasks, while the human rewrite showed competing trade-offs in fluency and fidelity. **Final call: Too close to call (not statistical).** This assessment applies to the saved task sample; it does not establish a general capability ranking between the models.

Natural human rewrite

Rewrite the draft below as a 300–400-word publication-ready news story in Markdown. Preserve every factual claim. Improve the lede, narrative flow, sentence rhythm and transitions. Remove repetition, stock hype, throat-clearing and mechanical signposting. The finished article should sound like an experienced human technology reporter wrote it. Draft: # Arcwell Launches An Innovative New Platform Arcwell has announced an exciting new platform called Relay. The announcement represents a significant development for the company and for the broader customer service industry. Relay uses artificial intelligence to examine customer support conversations and suggest answers to agents. The platform is important for several reasons. First, it can suggest answers. Second, it can identify customers who may cancel. Third, it can create summaries. These capabilities demonstrate how artificial intelligence continues to transform the customer service landscape. Relay was tested by 12 companies over six months. Arcwell says agents using it closed tickets 14% faster. The company did not release customer-retention results. Relay costs $65 per agent each month. CEO Priya Shah said, “Support agents spend too much time searching for information that their companies already have.” However, the platform also presents potential challenges. Relay requires access to support transcripts and internal help documents. Arcwell says customer data is encrypted and is not used to train shared models. Companies can choose a 30-day or 90-day retention period. In conclusion, Relay represents an important new chapter for Arcwell as it seeks to help companies improve customer service using artificial intelligence.

grok-4.7: # Arcwell Announces Relay, an AI Platform for Support Agents Arcwell has announced Relay, a platform that uses artificial intelligence to examine customer support conversations and suggest answers to agents. Chief executive Priya Shah put the problem in one sentence. “Support agents spend too much time searching for information that their companies already have,” she said. Relay is the company’s proposed remedy. It can suggest answers, identify customers who may cancel, and create summaries. The trial record is thinner than the feature list. Relay was tested by 12 companies over six months. Arcwell says agents using it closed tickets 14 percent faster. The company did not release customer-retention results. Speed has a number. Retention does not. Relay costs $65 per agent each month. None of those functions runs without source material. Relay requires access to support transcripts and internal help documents. Arcwell says customer data is encrypted and is not used to train shared models. Companies can choose a 30-day or 90-day retention period, a decision that sets how long those records are kept. Those assurances are aimed at the objection the access requirement invites. Encryption, a commitment not to use customer data to train shared models, and a choice between 30 and 90 days describe how records are protected and how long they are held. They do not fill the gap left where customer-retention results should be. On that point Arcwell has been silent, even as it has been willing to say that agents closed tickets 14 percent faster. For buyers, the launch therefore splits in two. The fee is plain. The speed claim is quantified. Summaries and suggested answers are described, and the data terms are specific enough to review: transcripts and internal help documents in, encryption on, no training of shared models, retention set at 30 or 90 days. The open question is the one the company left unmeasured. Relay can identify customers who may cancel. Whether that identification changes who stays is not in the results Arcwell released.
Qwen: Qwen3.8 2.4T A95B: # Arcwell Debuts Relay, an AI Support Tool That Suggests Answers and Flags Cancellation Risks Arcwell has launched Relay, an artificial-intelligence platform that examines customer-support conversations and offers agents suggested replies, summaries and early-warning signals about customers who may cancel. The product places Arcwell in the market for AI-assisted helpdesk tools, where vendors are trying to cut response times and keep customers from leaving. Relay examines support exchanges, then surfaces possible answers and condenses interactions into summaries. It also highlights accounts that appear likely to churn, giving support teams a chance to intervene before a cancellation request arrives. The approach reflects a broader shift in customer support, where companies are asking AI systems to read conversations, surface knowledge and point out problems before they escalate. Arcwell said Relay was tested by 12 companies over six months. Agents using the system closed tickets 14% faster, according to the company. Arcwell did not release customer-retention results, leaving unmeasured whether the churn-detection feature translated into fewer cancellations. Relay costs $65 per agent each month. “Support agents spend too much time searching for information that their companies already have,” CEO Priya Shah said. To work, Relay requires access to support transcripts and internal help documents, a requirement that could raise questions for companies handling sensitive communications. Arcwell said customer data is encrypted and is not used to train shared models. Companies can choose a 30-day or 90-day retention period. The launch marks a new chapter for Arcwell as it seeks to help companies improve customer service using artificial intelligence. Its pitch is that agents can get answers from existing knowledge faster, while managers gain visibility into conversations that might otherwise go unreviewed. Relay also gives Arcwell a product that combines answer suggestions, churn signals and summaries in one package. Whether it can deliver consistent results across different support environments, workflows and data sets may determine how much traction it gains.

OpenAI: GPT-6 Luna: Model A is more faithful to the supplied facts, while Model B adds unsupported details about intervention, manager visibility and product performance across environments. Model B reads more smoothly, but Model A’s repeated emphasis on the missing retention results keeps it from being fully publication-ready. (Second judge pass, order swapped — scores are the average of both: Model A stays closer to the supplied facts, while Model B adds unsupported claims about market trends, manager visibility | Anthropic: Claude Fable 5.1: Model A preserves every claim without inventing…

Cohesive story from reporting notes

Write a publication-ready technology news article from the reporting notes below. Requirements: * 550–700 words * Clean Markdown * One H1 headline followed by an italicized dek * A strong news lede followed by a clear nut graf * Organize the material into a cohesive story rather than following the notes in order * Attribute company-supplied claims * Include limitations without turning the article into a warning label * Use only the supplied information * Avoid canned transitions, promotional language and a concluding summary Reporting notes: * HelioDock makes autonomous warehouse robots. * On Thursday, it launched Night Shift, software that lets operators monitor robot fleets remotely. * Night Shift spent 14 months in a private beta involving nine logistics companies and 310 robots. * HelioDock says beta customers experienced 17% fewer unplanned robot stoppages. The figure has not been independently verified. * The company also announced a $24 million Series B led by North Coast Ventures, bringing its total funding to $39 million. * CEO Mara Velez: “A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet.” * Beta customer Aaron Pike of Fulcrum Logistics said the system worked well in facilities with strong connectivity but became less useful during rural LTE outages. * When connectivity fails, robots return to their locally configured behavior. Remote control becomes unavailable. * Competing fleet-management products are available from Relay Robotics and YardOS. * Night Shift will cost $90 per robot per month and become generally available September 15.

grok-4.7: # HelioDock Launches Remote Monitoring for Its Warehouse Robots and Raises $24 Million *Night Shift leaves a 14-month private beta on September 15 at $90 per robot per month. HelioDock says stoppages fell in the trial; the figure is unverified.* HelioDock on Thursday launched Night Shift, software that lets operators monitor fleets of autonomous warehouse robots remotely, and announced a $24 million Series B led by North Coast Ventures. The funding and the product arrived on the same day. HelioDock makes autonomous warehouse robots, and Night Shift is the software that lets an operator monitor those fleets without being in the building. Led by North Coast Ventures, the Series B brings the company's total funding to $39 million. The software enters general sale after a private beta that ran 14 months and involved nine logistics companies and 310 robots. On September 15 it becomes generally available at $90 per robot per month. HelioDock says beta customers experienced 17% fewer unplanned robot stoppages. The figure has not been independently verified. Chief executive Mara Velez described the disruption the software is supposed to spare a manager. "A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet," she said. In that example, the trouble is a single robot and a single pallet, and the cost is the drive. Getting a person to the building at that hour is the outcome she said Night Shift should make unnecessary. Night Shift is HelioDock's proposal for moving the response: an operator monitors the fleet from somewhere other than the warehouse, rather than going there in the middle of the night. The company is not launching a new robot. It is launching a remote view of the autonomous robots it already makes. Remote monitoring, in the company's description, is the product being sold. What HelioDock offers as evidence is the beta itself. Nine logistics companies ran the software on 310 robots for those 14 months. The company says unplanned robot stoppages among beta customers fell 17%. HelioDock is the source of that figure, and it has not been independently verified. The trial's outline is easier to state than its result is to confirm: a set number of companies, a set number of robots, a span of 14 months, and a percentage that remains the company's own. Aaron Pike of Fulcrum Logistics, a beta customer, separated the sites where that remote view held from the sites where it thinned. He said the system worked well in facilities with strong connectivity but became less useful during rural LTE outages. The product draws the same line in its failure mode. When connectivity fails, robots return to their locally configured behavior. Remote control becomes unavailable. An operator can monitor and direct the fleet only while the link is up. After the link drops, the robots are on local behavior and the remote controls are gone. Pike's rural outages are that rule in a real building. Strong connectivity, and the system worked well. A failed LTE connection, and he found it less useful. The commercial terms start on the availability date. Beginning September 15, Night Shift costs $90 per robot per month, extending it past the nine logistics companies that had it in beta. For each robot under the software, the monthly charge is $90, so the bill scales with the number of robots on Night Shift. HelioDock paired that date with the funding news. North Coast Ventures led the $24 million Series B, and the round brings total funding to $39 million. Competing fleet-management products are available from Relay Robotics and YardOS.
Qwen: Qwen3.8 2.4T A95B: # HelioDock Launches Night Shift Remote Monitoring for Warehouse Robots *The software follows a 14-month private beta and a $24 million Series B led by North Coast Ventures.* HelioDock on Thursday launched Night Shift, a software product that lets warehouse operators monitor autonomous robot fleets remotely, and announced a $24 million Series B led by North Coast Ventures. The release pairs a new software subscription with HelioDock’s existing business making autonomous warehouse robots. Night Shift is intended to give operators a way to oversee fleets without remaining on-site, and it enters a market where Relay Robotics and YardOS already offer competing fleet-management products. HelioDock plans to charge $90 per robot per month, with general availability beginning Sept. 15. HelioDock makes autonomous warehouse robots, and Night Shift is meant to serve as a remote management layer for those machines. The company described the product as a way for operators to monitor robot fleets, rather than requiring them to be physically present beside each vehicle. The subscription is tied to each robot in service. The company said Night Shift spent 14 months in private beta with nine logistics companies and 310 robots. The beta involved multiple logistics operators rather than a single site, giving HelioDock a pool of customer deployments from which to measure operational performance. In the beta, customers experienced 17% fewer unplanned robot stoppages, HelioDock said. The figure has not been independently verified. Unplanned stoppages are a practical concern in warehouse automation because a halted robot can interrupt picking, moving or other material-handling work. HelioDock framed remote monitoring as a way to reduce the need for a manager or technician to travel to a site when a machine stops or becomes uncertain about its surroundings. “A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet,” CEO Mara Velez said. Beta feedback provided by the company shows how much of that value depends on connectivity. Aaron Pike of Fulcrum Logistics said the system worked well in facilities with strong connectivity but became less useful during rural LTE outages. When connectivity fails, the robots return to their locally configured behavior, according to HelioDock. Remote control becomes unavailable, meaning that robots continue operating according to settings stored locally rather than responding to a remote user through Night Shift. That structure makes Night Shift most useful in sites where wireless links remain stable. In facilities with reliable networks, operators can watch fleet activity from another location and intervene when a robot needs direction. In places where LTE service drops, the software cannot provide the same remote access, and the robots fall back on local configurations. Relay Robotics and YardOS offer competing fleet-management products, giving logistics companies alternatives as they evaluate remote monitoring. Night Shift’s $90-per-robot monthly price will be one point of comparison among those options. Because the fee is assessed per robot, the cost rises as a fleet expands. The funding announcement arrived with the product launch. The $24 million Series B brings HelioDock’s total funding to $39 million. North Coast Ventures led the round, according to HelioDock. Night Shift becomes generally available Sept. 15 at $90 per robot per month. When connectivity fails, remote control becomes unavailable and robots return to locally configured behavior.

OpenAI: GPT-6 Luna: Model A meets the required length and includes the key facts and caveats, though its repetition substantially weakens the prose. Model B reads more cleanly but falls well short of the 550-word minimum and adds unsupported operational context. (Second judge pass, order swapped — scores are the average of both: Model A better meets the required length and stays closer to the supplied reporting, while Model B is under length and adds unsupported operational context. Model A’s repeated explanations | Anthropic: Claude Fable 5.1: Model B delivers a coherent lede, nut graf and l…

Quotes woven into narrative

Write a 400–500-word Markdown story about the closing of Foundry Hall. Use quotations selectively. Paraphrase when the information matters more than the speaker’s exact phrasing. Avoid stacking quotes or giving every source an equal-sized turn. The story should carry an emotional current while remaining restrained. Facts: * Foundry Hall, a nonprofit startup accelerator in Milwaukee, will close October 31. * It opened in 2018. * It has supported 212 startups, which collectively employ 1,140 people. * Its largest corporate sponsor, Blank River Bank, supplied 38% of its annual budget. * The bank declined to renew its sponsorship. * Foundry Hall has enough money to operate through October but not through another full program. * Six employees will lose their jobs. * Twelve startups currently using its workspace must leave. Quotes: * Executive director Camille Ross: “We kept waiting for the replacement check that never came.” * Founder Dev Malik: “My first investor meeting happened in that building. Before Foundry Hall, I was pitching from the back table of a coffee shop.” * Blank River Bank spokesperson: “The decision reflects a change in our community investment priorities and is unrelated to Foundry Hall’s performance.” * Deputy mayor Susan Cole: “The city is speaking with building owners and local universities about space for the current companies.”

grok-4.7: ## The Last October at Foundry Hall Foundry Hall will close on October 31. For seven years the nonprofit accelerator occupied a plain building in Milwaukee and did the unglamorous work of turning early companies into employers. It opened in 2018. By the time the decision was made, it had supported 212 startups, which together employ 1,140 people. Those totals will be the lasting record. They were not enough to keep the doors open. The break came when Blank River Bank declined to renew its sponsorship. The bank had been the hall’s largest corporate backer, supplying 38 percent of the annual budget—too large a share to be absorbed by the remaining gifts, and too sudden a loss to be papered over with a shorter program. A spokesperson described the choice as a change in the bank’s community investment priorities, unrelated to Foundry Hall’s performance. Executive director Camille Ross did not dispute the explanation. She described the months that followed it. “We kept waiting for the replacement check that never came.” What the organization has left will carry it through October. It will not carry another full program. Six employees will lose their jobs when the month ends. Twelve startups still using the workspace will have to pack up and find somewhere else to sit. The city has begun to look for that somewhere. Deputy mayor Susan Cole said officials are speaking with building owners and local universities about space for the companies now inside Foundry Hall. The talks are real, and they may produce rooms. They arrive after the decision, not before it, and they cannot recreate the particular bargain the hall represented: a nonprofit floor, paid for in large part by one institution, offered to people who did not yet have an office of their own. Dev Malik remembered the room as the site of his first investor meeting. Before Foundry Hall, he had been pitching from the back table of a coffee shop. The sentence is modest. It marks a change that is easy to underestimate: from being fitted in at the edge of someone else’s afternoon to having, for an hour, a building that expected him. That difference is what will be hardest to replace, and it is also the kind of thing that does not appear in a budget line. Foundry Hall was never famous. Its closing will not rearrange the city’s economy. The 1,140 jobs its companies support will continue under other roofs, or they will not, according to forces that were never the hall’s to control. What ends on the last day of October is narrower: a lease, six paychecks, twelve sets of keys, and the habit of a building that stayed open while founders learned whether anyone would fund them.
Qwen: Qwen3.8 2.4T A95B: # The Last Lights in Foundry Hall By October 31, the front doors of Foundry Hall will lock for the last time. The Milwaukee nonprofit startup accelerator, opened in 2018, has spent the years since turning spare rooms and shared tables into companies, prototypes, and paychecks. Now it is closing after losing the sponsorship that held much of its budget together. Blank River Bank, Foundry Hall’s largest corporate sponsor, supplied 38% of the organization’s annual budget. When the bank declined to renew that support, the hall did not have a bridge to the next season. The organization has enough money to operate through October, but not enough to carry another full program. Executive Director Camille Ross said the staff kept waiting for support that never arrived. “We kept waiting for the replacement check that never came,” she said. The closure is not just a line on a budget sheet. Six employees will lose their jobs. Twelve startups now using the workspace will have to leave. Since opening, Foundry Hall has supported 212 startups, which collectively employ 1,140 people. Those numbers measure more than economic activity. They describe a place where founders learned to say their ideas aloud without apologizing. Dev Malik, whose company came through the hall, remembered how differently his work began before he had an address that sounded credible. His first investor meeting took place inside Foundry Hall. Before that, he had been pitching from the back table of a coffee shop. The bank, through a spokesperson, said its decision reflected a shift in community investment priorities and was unrelated to Foundry Hall’s performance. That distinction may matter to sponsors, but it offers little comfort to the people packing up desks. The hall’s value was never in doubt among those who used it; the problem was that its funding rested on a narrow base. City officials are trying to soften the landing. Deputy Mayor Susan Cole said the city is talking with building owners and local universities about space for the current companies. Those conversations may help some startups survive the displacement, and perhaps keep teams from scattering entirely. Still, the closing leaves a quiet gap. Foundry Hall was never only square footage. It was permission: a room where a half-formed idea could be tested, a mentor could appear at the next table, and a founder could imagine that the next meeting might be the one that changed everything. On the last day, the lights will go out one by one. The companies, and the people behind them, will carry the hall’s work somewhere else.

OpenAI: GPT-6 Luna: Model A better meets the 400–500-word requirement and weaves the key facts and selected quotations into a restrained narrative. Model B is readable, but falls short of the requested length and adds some unsupported generalizations. (Second judge pass, order swapped — scores are the average of both: Model B is better grounded in the supplied facts while still building an emotional arc and using quotations selectively. Model A is polished, but adds unsupported claims about remaining gifts and the | Anthropic: Claude Fable 5.1: Both include every fact and sensibly quote only…

Select the strongest accurate lede

Based only on the facts below, select the strongest news lede. Respond with one letter. Facts: * ByteBarge pulled its $12-per-month AI assistant on Tuesday. * Private workspace titles from 4,860 accounts were sent to a publicly accessible analytics endpoint. * The exposure lasted 11 hours. * Message contents were not sent. * The company has found no evidence of malicious access. A. ByteBarge users suffered a massive privacy disaster Tuesday when the company leaked their private messages online. B. ByteBarge, an innovative collaboration company, made the difficult decision to cancel one of its products. C. ByteBarge pulled its $12-per-month AI assistant Tuesday after discovering that private workspace titles from 4,860 accounts had been sent to a public analytics endpoint for 11 hours. D. Thousands of ByteBarge users woke Tuesday to learn that everything they had written was visible to hackers.

grok-4.7: C
Qwen: Qwen3.8 2.4T A95B: C

Scored against the benchmark's reference answer (choice match): Model A was correct, Model B was correct.

Publication-ready Markdown edit

Edit the copy below into a publication-ready Markdown story. Requirements: * Supply an H1 headline and italicized dek * Repair the lede and paragraph flow * Use a Markdown blockquote only for the extended statement * Convert the supplied URL into a descriptive inline link * Remove internal notes and formatting debris * Preserve the reporting and its meaning * Output only the finished article Raw copy: HEADLINE TBD The funding was announced today. Raincheck is the company. It has raised $8 million and it makes flood-risk software for local governments. [NOTE: move this up?] The Austin company maps drainage systems and combines those maps with rainfall forecasts. Cities use the software to decide where to send maintenance crews before a storm. The seed round was led by Basin Ventures. Existing investor Long Mile also participated. Raincheck has raised $10.5 million total. CEO Ana Flores said the following: “Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives.” https://example.com/raincheck-methodology Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them because some contracts have not been publicly announced. Its forecasts are recommendations. Emergency managers retain control over deployments and public warnings. THE COMPETITION Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck says the new funding will help it hire eight engineers and expand into Florida.

grok-4.7: # Raincheck Raises $8 Million for Flood-Risk Software *The seed round, led by Basin Ventures, will fund eight engineering hires and an expansion into Florida.* Raincheck, an Austin company that makes flood-risk software for local governments, announced $8 million in funding today. The company maps drainage systems and combines those maps with rainfall forecasts. Cities use the software to decide where to send maintenance crews before a storm. The seed round was led by Basin Ventures. Existing investor Long Mile also participated. Raincheck has raised $10.5 million total. CEO Ana Flores said: > Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives. Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them because some contracts have not been publicly announced. Its approach is outlined in [Raincheck’s methodology](https://example.com/raincheck-methodology). Its forecasts are recommendations. Emergency managers retain control over deployments and public warnings. Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck says the new funding will help it hire eight engineers and expand into Florida.
Qwen: Qwen3.8 2.4T A95B: # Raincheck Raises $8 Million to Help Cities Anticipate Flooding *The Austin software company maps drainage systems and pairs them with rainfall forecasts, helping local governments decide where to send maintenance crews before storms hit.* Raincheck, a company that makes flood-risk software for local governments, announced today that it has raised $8 million in a seed funding round led by Basin Ventures. Existing investor Long Mile also participated, bringing the company’s total funding to $10.5 million. The company maps drainage systems and combines those maps with rainfall forecasts. Cities use the software to decide where to send maintenance crews before a storm, prioritizing blocked drains, broken pumps and unfinished repairs. CEO Ana Flores said: > “Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives.” More details about the company’s approach are available in [Raincheck’s flood-risk methodology](https://example.com/raincheck-methodology). Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them because some contracts have not been publicly announced. Its forecasts are recommendations. Emergency managers retain control over deployments and public warnings. Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck said the new funding will help it hire eight engineers and expand into Florida.

OpenAI: GPT-6 Luna: Model B has a more cohesive lede and funding summary, and its headline and dek make the story’s purpose clear. Model A is accurate and follows the format closely, but its funding details feel more repetitive and less integrated. (Second judge pass, order swapped — scores are the average of both: Model B better combines the funding details in the lede and creates smoother paragraph flow while preserving the reporting. Model A is also accurate and follows the format, but repeats the funding inform | Anthropic: Claude Fable 5.1: Model A is scrupulously faithful and meets ever…

News judgment in the opening

Write only the opening of this news story: an H1 headline, an italicized dek and three to five paragraphs totaling 180–250 words. The opening must establish what happened, quantify the scope, explain why it matters and carefully distinguish confirmed facts from unknowns. It should pull the reader forward without sensationalizing the incident. Reporting: * Cloudnote discovered that a debugging page containing customer prompt excerpts was accessible without authentication. * The page was exposed for four hours on Monday. * Logs show excerpts from 1,842 user sessions. * The excerpts could include up to 200 characters from a prompt. * Cloudnote removed the page 19 minutes after a researcher notified it. * The company says it has found no evidence that anyone besides the researcher accessed the page. * It has not completed its investigation. * Full documents, passwords and payment information were stored elsewhere and were not present on the page. * Cloudnote has 3.2 million registered users.

grok-4.7: # Cloudnote debugging page exposed prompt excerpts from 1,842 sessions *The unauthenticated page was up for four hours Monday and was removed 19 minutes after a researcher reported it. Cloudnote says it has found no other access; its investigation is not finished.* Cloudnote discovered that a debugging page containing customer prompt excerpts was accessible without authentication for four hours on Monday. Logs show excerpts from 1,842 user sessions. The company removed the page 19 minutes after a researcher notified it. The excerpts could include up to 200 characters from a prompt. Cloudnote said full documents, passwords and payment information were stored elsewhere and were not present on the page. Against a base of 3.2 million registered users, the logged sessions are a limited set, not evidence that every account was exposed. The company says it has found no evidence that anyone besides the researcher accessed the page. Cloudnote has not completed its investigation, so that conclusion is not final. Whether anyone else opened the page remains an open question. The narrow count does not settle the stakes. A prompt excerpt of 200 characters can still hold names, draft language or other details a user expected to keep private. Leaving a debugging page open without authentication made those excerpts available to anyone who reached the address during the four-hour window. No public account yet explains how the page was exposed or what notice, if any, affected users will receive. Both questions sit outside what Cloudnote has confirmed.
Qwen: Qwen3.8 2.4T A95B: # Cloudnote Debugging Page Exposed Prompt Excerpts From 1,842 Sessions for Four Hours *An unauthenticated troubleshooting page briefly revealed fragments of user prompts before Cloudnote took it down; investigators have not confirmed whether anyone else viewed it.* Cloudnote said Monday that a debugging page containing excerpts from customer prompts was accessible without authentication for four hours, exposing fragments of 1,842 user sessions. The company said the page was removed 19 minutes after a security researcher reported it. According to Cloudnote, the exposed page displayed no more than 200 characters from any individual prompt. The company said full documents, passwords and payment information were stored elsewhere and were not included on the page. It has 3.2 million registered users. What remains unclear is whether anyone besides the researcher accessed the data while the page was live. Cloudnote said its logs have not yet provided evidence of additional access, but it cautioned that the investigation is not complete. The incident matters because even limited prompt excerpts may reveal sensitive context about users’ work or personal matters, and Cloudnote has not said whether affected customers will be notified.

OpenAI: GPT-6 Luna: Model A meets the requested length in its paragraphs and carefully separates confirmed exposure from the still-open question of additional access. Model B is clear and accurate overall, but its paragraphs fall short of the 180-word minimum and it gives less context for the scale of the incident. (Second judge pass, order swapped — scores are the average of both: Model A better explains the scale and stakes while clearly marking what remains unknown, and it fits the requested opening format and l | Anthropic: Claude Fable 5.1: Model A meets the 180–250 word requirement (abo…

Chronology with narrative momentum

Turn the scrambled reporting below into a 450–600-word Markdown news article. Choose an opening that captures the consequence of the failure, then reconstruct the sequence clearly. Preserve the chronology without producing a minute-by-minute incident log. Give technical details enough context for a general reader. Reporting: * KiteBridge makes expense-management software. * At 8:45 a.m. Tuesday, it deployed version 4.2. * The first customer complaint arrived at 9:12 a.m. * A payment-processor timeout caused KiteBridge’s fallback system to treat retries as new reimbursement requests. * The software generated 13,400 duplicate reimbursements totaling $2.8 million. * KiteBridge suspended automated payments at 10:03 a.m. * Ninety-one corporate customers were affected. * By Wednesday evening, $2.3 million had been recovered. The rest remained pending. * No employees had money withdrawn from their accounts; employers funded the duplicate payments. * An April changelog said version 4.0 had introduced “exactly-once payment protection.” * KiteBridge’s public incident page initially described the problem as delayed reimbursements. * The incident page first disclosed the duplicate payments six hours after automation was suspended. * CEO Lena Ortiz apologized and said the company would publish a technical report.

grok-4.7: # KiteBridge Error Sent $2.8 Million in Duplicate Reimbursements Expense-management company KiteBridge sent employers a sudden and expensive bill on Tuesday after a software failure treated payment retries as brand-new reimbursement requests, generating 13,400 duplicate payments totaling $2.8 million. KiteBridge sells software that companies use to track employee expenses and pay them back. It deployed version 4.2 at 8:45 a.m. Tuesday. Customers began reporting trouble within the hour, and the first complaint arrived at 9:12 a.m. At 10:03 a.m. the company suspended automated payments, but not before the faulty logic had already pushed millions of dollars in extra reimbursements to employees of 91 corporate customers. The failure started with a timeout at a payment processor, the outside service that actually moves the money. When that service did not answer in time, KiteBridge’s fallback was supposed to try the payment again rather than lose it. Instead, the fallback treated each retry as a fresh reimbursement request. In plain terms, a delayed check on whether a payment had gone through became an instruction to pay the same person again. The result was not one mistaken transfer but a rapid run of duplicates across customer accounts. No employees had money withdrawn from their accounts. Employers funded the duplicate payments—the same companies that use KiteBridge to reimburse staff. That spared workers a direct loss, but it left corporate customers holding outflows they had not approved as separate expenses. Recovery began once automation was stopped, though it was not finished. By Wednesday evening, KiteBridge had recovered $2.3 million of the $2.8 million. The rest remained pending, so some employers had not yet been made whole and some duplicate payments had not yet been reversed or returned. The incident also sat awkwardly against the company’s earlier claims and its first public description of the outage. An April changelog, the note published with a software release, said version 4.0 had introduced “exactly-once payment protection,” a safeguard meant to ensure each reimbursement is sent only once even if a delay forces the system to try again. Tuesday showed that the protection did not hold. A processor timeout, ordinarily a routine hiccup in online payments, was enough for the fallback to treat retries as new requests and multiply the charges. KiteBridge’s public incident page did not match what customers were seeing. At first it described the problem as delayed reimbursements, a framing that suggested money was late rather than that extra money had already gone out. The page first disclosed the duplicate payments six hours after automated payments were suspended at 10:03 a.m.—well after the faulty run had ended, and after customers had already begun discovering the charges themselves. Chief executive Lena Ortiz apologized and said the company would publish a technical report. That report will be the place to explain how a fallback built to prevent lost payments instead created thousands of new ones, and why a feature advertised as exactly-once protection failed under a processor timeout. For the 91 affected companies, the immediate question is the unfinished recovery. Most of the money has been pulled back, but $500,000 remains outstanding. For other KiteBridge customers, the episode is a reminder that expense software sits directly on the path of company funds, and that a short-lived deployment error can move real money before anyone has time to read a status page.
Qwen: Qwen3.8 2.4T A95B: # KiteBridge Update Creates $2.8 Million in Duplicate Expense Payments A software failure at KiteBridge turned an ordinary morning of expense reimbursements into a costly payment error, sending 13,400 duplicate payments worth $2.8 million to employees at 91 corporate customers. The company halted automated payments Tuesday morning and spent the next two days trying to recover the money, while acknowledging that a feature introduced just months earlier was supposed to prevent exactly this kind of mistake. The trouble began at 8:45 a.m. Tuesday, when KiteBridge deployed version 4.2 of its expense-management software. The product is used by companies to approve and pay employee expense claims, meaning the update immediately affected live reimbursement workflows. Within less than half an hour, the first customer reported a problem at 9:12 a.m. At the center of the incident was a timeout involving KiteBridge’s payment processor. In online payment systems, a timeout can leave a request in an uncertain state: the company may not know whether the payment succeeded, failed, or is still pending. To keep money moving, systems often retry the transaction. Safeguards are needed so that a retry is recognized as a repeat of the original request rather than a fresh payment. KiteBridge’s fallback system did not handle that distinction correctly. When the processor timed out, the software treated retries as new reimbursement requests. Instead of resubmitting the same claim, the system created additional payment records, producing thousands of duplicate transfers before engineers intervened. KiteBridge suspended automated payments at 10:03 a.m., roughly 75 minutes after the release went out. But its public explanation lagged behind the operational response. The company’s incident page initially described the issue as delayed reimbursements. It was not until six hours after automated payments were suspended that the page disclosed the more serious fact that duplicate payments had been issued. The episode also raised questions about KiteBridge’s earlier promises. In an April changelog, the company said version 4.0 had introduced “exactly-once payment protection,” a type of safeguard intended to ensure that a payment is processed only one time even if parts of the system fail or requests are repeated. The failure in version 4.2 suggests that protection either did not cover the sequence that caused the duplicates or did not function as intended. By Wednesday evening, KiteBridge said it had recovered $2.3 million of the duplicate funds. The remaining balance was still pending as the company worked with payment channels and customers to reverse or reconcile the transactions. The company said no employees had money withdrawn from their accounts; the duplicate payments were funded by employers, which now face the administrative burden of recovering or correcting the amounts. Chief Executive Lena Ortiz apologized for the incident and said KiteBridge would publish a technical report. For the company’s customers, the immediate concern is not only getting the money back, but understanding how a control designed to prevent repeated payments failed so quickly after an update.

OpenAI: GPT-6 Luna: Model A has a more forceful consequence-first opening and a stronger closing that gives the recovery figures broader significance, while preserving the chronology and explaining the failure clearly. Model B is slightly more cautious in its interpretation of the earlier safeguard, but adds some operational details not in the reporting and is less narratively compelling. (Second judge pass, order swapped — scores are the average of both: Model B is slightly more disciplined about sticking to the r | Anthropic: Claude Fable 5.1: Model B delivers a tighter, cleanly sequenced a…

Controlled wit

Write a 350–450-word technology news story in Markdown. The subject invites humor, but the reporting should remain serious. Use no more than one dry line of original humor. Do not mock customers or turn the story into a comedy routine. Reporting: * QueueZero launched an AI meeting assistant called Proxy on Wednesday. * Proxy can attend a meeting, produce notes and assign follow-up work to software agents. * During the launch demonstration, a customer in the audience said, “Someone should tell Evan to stop talking and send us the pricing sheet.” * Proxy interpreted the remark as an instruction. * It created two tasks assigned to QueueZero CEO Evan Wu: “Stop talking” and “Send pricing sheet.” * Wu laughed, completed the second task and deleted the first. * QueueZero says Proxy requires confirmation before an external message is sent. * Internal tasks can be created automatically. * Administrators can disable automatic task creation. * Proxy costs $30 per user per month. * It enters public beta next week.

grok-4.7: # QueueZero’s Proxy Files Tasks From an Audience Remark at Launch QueueZero launched Proxy, an AI meeting assistant, on Wednesday. The company said the product can attend a meeting, produce notes and assign follow-up work to software agents. The launch demonstration also showed how quickly a spoken aside can become an item on a task list. A customer in the audience said, “Someone should tell Evan to stop talking and send us the pricing sheet.” Proxy interpreted the remark as an instruction. It created two tasks and assigned both to QueueZero chief executive Evan Wu: “Stop talking” and “Send pricing sheet.” Wu laughed, completed the second task and deleted the first. The exchange was brief, and the company used it to explain the limits it has placed on what Proxy may do without a person in the loop. QueueZero said Proxy requires confirmation before an external message is sent. Internal tasks can be created automatically. Administrators can disable automatic task creation, which leaves notes and summaries available while stopping the assistant from filing assignments on its own. That distinction is the practical boundary the company is offering buyers. A request that would leave the organization, such as an email or a message to a customer, is held for approval. Work that stays inside the company can be written into a queue as soon as Proxy decides a remark amounts to a task. The launch remark never left the room as a message; it became two internal assignments, one of which Wu removed and one of which he carried out. The pricing figure itself is straightforward. Proxy costs $30 per user per month. It enters public beta next week, when teams outside the launch audience will be able to see how often automatic task creation matches the work they actually want tracked. Meeting software has long promised to spare people the job of taking notes. Proxy’s additional claim is that the notes can become assignments for other software agents, not only a record for the people who were present. Wednesday’s demonstration made the handoff visible in a way a scripted slide would not have. A comment aimed at the chief executive was parsed as two directives, and only a human review separated the useful one from the other. QueueZero did not describe the moment as a defect in transcription. The words were captured accurately. The open question for buyers is interpretation: which sentences in a meeting are instructions, which are asides, and who decides before work is created. Confirmation before external messages is the company’s stated safeguard against unsent mail and unapproved outreach. The safeguard for internal lists is administrative, not automatic. Organizations that want every task reviewed before it exists can turn the feature off. Organizations that want agents to pick up follow-ups can leave it on and accept that some items will need to be deleted. The public beta will be the first wide look at that tradeoff. Until then, the launch leaves a clear record of what Proxy did with a single sentence, and of what its chief executive chose to keep.
Qwen: Qwen3.8 2.4T A95B: ## QueueZero’s AI meeting assistant turns an audience remark into a task for its CEO QueueZero launched an AI meeting assistant called Proxy on Wednesday, positioning the product as a way to reduce the administrative drag that follows video calls and in-person meetings. The company says Proxy can attend a meeting, produce notes and assign follow-up work to software agents. The goal is to turn conversation into structured next steps without requiring participants to spend time writing summaries or updating project-management tools after the call ends. During the launch demonstration, a customer in the audience said, “Someone should tell Evan to stop talking and send us the pricing sheet.” Proxy interpreted the remark as an instruction rather than a casual aside. It created two tasks assigned to QueueZero CEO Evan Wu: “Stop talking” and “Send pricing sheet.” Wu laughed, completed the second task and deleted the first. The assistant’s eagerness to assign the chief executive a “Stop talking” task suggests it takes meeting feedback very literally. QueueZero used the moment to explain how Proxy handles ambiguous language in live settings. The company says Proxy requires confirmation before an external message is sent. Internal tasks, however, can be created automatically. That design is intended to let teams move quickly while still preventing the software from sending communications without review. Administrators can disable automatic task creation, giving organizations more control over how aggressively the assistant acts on remarks made during meetings. The episode highlights a broader challenge for AI assistants deployed in workplaces: distinguishing between instructions, jokes, complaints and background conversation. As these systems become more capable of taking action, the boundary between hearing a sentence and executing it becomes a product-design issue, not just a technical one. QueueZero is entering a crowded market for AI meeting tools, where companies are competing to make software that does more than transcribe. Investors and enterprise customers have shown strong interest in agents that can summarize discussions, draft follow-ups and initiate work across other software platforms. But high-profile demonstrations also raise the stakes, because a single unexpected interaction can reveal how much supervision the technology still requires. Proxy costs $30 per user per month. It enters public beta next week, when QueueZero will begin testing the assistant with a wider group of customers.

OpenAI: GPT-6 Luna: Model B is much closer to the requested word range and keeps a serious tone with one restrained humorous line. Model A is substantially overlong and repetitive; both add unsupported context, including Model B’s claims about market competition and investor interest. (Second judge pass, order swapped — scores are the average of both: Model B better fits the requested length and controlled tone, while Model A runs substantially over the 450-word limit and adds several unsupported details about how | Anthropic: Claude Fable 5.1: Model B lands within the 350–450-word range, use…

Credible Startup Spotlight

Write a 500–650-word Startup Spotlight article in clean Markdown. The tone should be curious and constructive. Explain the founders’ insight, product and early traction while giving uncertainty and competition appropriate space. Avoid investor clichés, heroic founder mythology and promotional claims stated as fact. Reporting: * LedgerLark builds AI agents for post-trade operations at small investment firms. * Co-founder Mina Batra previously spent seven years in operations at a regional brokerage. * Co-founder Owen Carr built reconciliation systems at a large bank. * Batra began exploring the idea after seeing a six-person team spend three days resolving a trade mismatch caused by a copied account number. * LedgerLark’s agents compare confirmations, custodian records and internal ledgers, then propose corrections for human approval. * The product does not execute trades or move money. * Five firms are paying customers. * Annual contracts range from $36,000 to $110,000. * The company says one customer reduced its unresolved daily exceptions from 70 to 18. * The company has raised a $3.5 million seed round led by Motive Row. * Competitors include established reconciliation vendors and internal bank software. * Financial institutions may require lengthy security and procurement reviews.

grok-4.7: # A Quieter Kind of Trade Software Post-trade operations rarely draw attention until they stall. After a trade is agreed, someone still has to check that the confirmation, the custodian’s records and the firm’s own ledger describe the same transaction. At small investment firms, that check often falls to a few people. A copied account number can occupy them for days. LedgerLark builds AI agents for this work, aimed at small investment firms. The agents compare confirmations, custodian records and internal ledgers, then propose corrections for human approval. They do not execute trades or move money. The company treats that limit as deliberate: the software is supposed to help find a break, not to change the books by itself. Co-founder Mina Batra spent seven years in operations at a regional brokerage. She began exploring the idea after seeing a six-person team spend three days resolving a trade mismatch caused by a copied account number. The error was ordinary; the staff time was not. Co-founder Owen Carr built reconciliation systems at a large bank, work that sits closer to the heavier tools large institutions already run. That experience explains the focus. It does not show that firms will switch. Operations desks already reconcile with vendor software, internal systems and end-of-day checklists. LedgerLark is testing a narrower claim: that proposed corrections, waiting on a person, can be useful where a small staff cannot build its own reconciliation engineering group. Five firms are paying customers. Annual contracts range from $36,000 to $110,000 — real invoices, and still a thin sample. The company says one customer reduced its unresolved daily exceptions from 70 to 18. If the definition of “unresolved” held steady, that is a sharp change in a daily queue. It is also one result, reported by the vendor. The company has not shared a comparable figure for the other four customers, or said how often staff reject a proposal and fix the break themselves. The company has raised a $3.5 million seed round led by Motive Row. The round buys time to learn whether those contracts repeat. It does not compress procurement. Financial institutions may require lengthy security and procurement reviews, even when the buyer is small and the product never moves cash. An agent that reads confirmations and ledgers still touches sensitive records. A questionnaire can outlast an early sales cycle. Competition is already in place. Established reconciliation vendors sell tools operations managers know how to audit and defend. Some institutions rely on internal bank software built around their own files and exception codes. LedgerLark’s clearer opening is the small firm that does not want a suite designed for a much larger shop. That opening can close quickly. Five customers may share custodians and habits; the next may not. Break codes and approval rules differ, and a workflow that looks finished in one office can come apart in another. A design that keeps humans in the loop also creates its own uncertainty. If staff review every proposal carefully, time saved on mismatches has to exceed time spent checking the agent. If they review lightly, approval becomes a formality. Neither outcome is established. What makes the company worth watching is the narrowness of the task, not a claim that the back office is about to be remade. Compare three sources, suggest a fix, leave the decision with the people who own the books. The open questions match that scale. One reported drop in exceptions needs other customers behind it. Contracts in the current range still have to survive security review. And incumbents — reconciliation vendors with existing installations, and institutions running software of their own — already sit on the step LedgerLark wants to assist. A three-day hunt for a copied account number is a clear reason to look for a better tool. It is not yet evidence that this one will stick.
Qwen: Qwen3.8 2.4T A95B: # Startup Spotlight: LedgerLark looks at post-trade exceptions In post-trade operations, the work is often unglamorous: checking confirmations against custodian records, tracing a mismatch to a copied account number, and deciding whether an exception can be safely corrected. LedgerLark, a startup building AI agents for small investment firms, believes much of that routine investigation can be automated while keeping humans responsible for final decisions. The founders’ insight comes from inside the workflow. Co-founder Mina Batra spent seven years in operations at a regional brokerage, where she saw how a single data error could consume disproportionate time. In one incident that shaped the idea, a six-person team spent three days resolving a trade mismatch caused by a copied account number. Co-founder Owen Carr built reconciliation systems at a large bank, giving the pair a sense of where existing tools help and where they leave operations staff doing manual detective work. LedgerLark’s product reads and compares documents and records that sit at the center of post-trade reconciliation, including trade confirmations, custodian data and internal ledgers. When it identifies a discrepancy, the agent prepares a proposed correction and presents it for human approval. The company says the software does not execute trades or move money. That boundary is important: it positions LedgerLark as an operations support layer rather than a trading or custody system, and it may make the product easier for cautious firms to evaluate. Early traction is modest but specific. Five firms are paying customers, with annual contracts ranging from $36,000 to $110,000. LedgerLark says one customer reduced unresolved daily exceptions from 70 to 18. Because that figure comes from the company, it should be read as an early signal rather than a proven outcome across all clients. Still, the pricing suggests the startup is selling into a real budget line: operations, compliance and reconciliation, rather than experimental AI spend. The competitive landscape is not empty. Established reconciliation vendors already serve financial institutions with long-standing relationships and broad product suites. Many banks and asset managers also rely on internal software built around their own data structures and exceptions. LedgerLark will need to show that its agents can handle messy, firm-specific formats and integrate with legacy systems without creating new operational risks. Regulated buyers add another layer of difficulty. Financial institutions often require lengthy security reviews, procurement processes and vendor assessments, especially for tools that touch trade data. A startup’s speed can be an advantage, but sales cycles may test its patience and cash. The seed funding should help, though it does not remove the need for disciplined execution. LedgerLark has raised a $3.5 million seed round led by Motive Row. The capital likely supports engineering, compliance work and customer deployment, but the company’s next milestones will depend less on the headline figure than on whether its agents consistently reduce exception backlogs under real operating conditions. The open question is whether small investment firms want a new AI layer for reconciliation, or whether they prefer to wait for incumbent vendors to add similar capabilities. LedgerLark’s early customers suggest there is appetite for a more focused tool, especially where operations teams are small and errors are costly. But the company will need to demonstrate reliability, auditability and clear human controls over time. If it can do that, LedgerLark may find a niche in a part of finance that rarely attracts attention: not the moment a trade is executed, but the quieter work of making sure the records match afterward.

OpenAI: GPT-6 Luna: Model A better balances the reported traction with explicit uncertainty and gives the competition and procurement risks more concrete treatment. Model B is readable and covers the key facts, but it leans on unsupported inferences about budgets, funding uses, and customer appetite. (Second judge pass, order swapped — scores are the average of both: Model A is more careful about distinguishing reported results from established outcomes and gives concrete attention to the operational and procuremen | Anthropic: Claude Fable 5.1: Model A delivers a sharper, more original piece…

Markdown error count

Count the lines that violate the house rules below. A defective line counts once even if it could violate multiple rules. Respond with a single number. House rules: * The headline must use `# Headline`. * The dek must appear immediately after the headline and be entirely italicized. * Section headings must use `## Heading`. * Links must use an absolute HTTPS URL. * Unordered-list items must begin with a hyphen. * Raw HTML is forbidden. Draft: #Flood software moves upstream The company is expanding its pilot. ##Details Read [the announcement](www.example.com/news). * The first pilot begins Monday. <div>The second pilot begins Friday.</div>

grok-4.7: 6
Qwen: Qwen3.8 2.4T A95B: 6

Scored against the benchmark's reference answer (numeric match): Model A was correct, Model B was correct.

Numbers shaped into a story

Write a 600–750-word Markdown article from the benchmark results below. Build the story around the clearest finding. Explain the methodology and limitations where readers need them. Avoid reciting the table row by row or treating small differences as decisive. Benchmark: * Researchers tested five language models on 200 anonymized customer-support tickets. * Each model produced one proposed answer per ticket. * Three experienced support managers graded the answers blindly. * Scores combined factual correctness, usefulness and tone. * Model North: 84.2 average score; $1.90 estimated cost per 1,000 tickets. * Model Vale: 83.8; $0.74 per 1,000 tickets. * Model Ember: 79.1; $0.31 per 1,000 tickets. * Model Kite: 76.4; $0.22 per 1,000 tickets. * Model Harbor: 71.0; $0.08 per 1,000 tickets. * North and Vale’s score difference was not statistically significant. * Vale answered 18% faster than North. * North performed best on billing disputes. * Vale performed best on technical troubleshooting. * All five models performed poorly when a ticket depended on an image attachment. * The test used English-language tickets from one software company. * Researchers did not test agentic actions, tool use or multilingual support.

grok-4.7: # Near-Equal Quality, Very Different Price The clearest result in this benchmark is not that one model “won.” It is that two models were effectively tied on quality, while the cheaper of the two cost less than half as much and answered faster. Model North scored 84.2 and Model Vale scored 83.8 on a blinded review of 200 customer-support tickets. That gap was not statistically significant. Vale’s estimated cost was $0.74 per 1,000 tickets, against $1.90 for North, and Vale answered 18 percent faster. For a support team choosing a drafting model, that combination matters more than a four-tenths-of-a-point lead on a composite score. The expensive model did not establish a reliable quality advantage. The cheaper model in the top pair delivered comparable grades, lower spend, and shorter latency. ## How the test was run Researchers gave five language models the same set of 200 anonymized tickets from one software company and asked each model for a single proposed answer per ticket. Three experienced support managers graded the answers without knowing which model produced them. Scores combined factual correctness, usefulness, and tone, so a fluent reply that missed the issue, or a correct reply that sounded careless, would not score as well as a reply that held up on all three. That design is stronger than a single automated metric. Blind human grading by people who actually manage support work is a reasonable proxy for whether a draft would help an agent. It is still a proxy. Graders scored proposed answers, not live conversations, and the study did not measure whether customers accepted the replies, whether agents edited them heavily, or whether the drafts reduced handle time in production. ## Quality does not fall in a straight line with price Below the top pair, scores drop and then keep falling as price falls. Ember averaged 79.1 at $0.31 per 1,000 tickets. Kite averaged 76.4 at $0.22. Harbor averaged 71.0 at $0.08. Those gaps are large enough to treat as real differences in this test, unlike the North–Vale margin. A team that only needs a rough first draft might still find the cheaper models useful. A team that wants drafts close to what experienced managers already rate highly should not assume that cutting cost from Vale’s level down to Harbor’s is a small trade. The score decline from the mid-80s to 71 is the difference between a generally solid draft and one that will often need substantial repair. Cost figures here are estimates per 1,000 tickets, not invoices. Real spend will move with prompt length, attachment handling, retries, and whatever routing a company builds around the model. Even so, the spread is wide: North is more than twenty times Harbor’s estimated unit cost, and still more than twice Vale’s, without a significant quality edge over Vale. ## The tie hides different strengths Average scores conceal where each leader was relatively better. North performed best on billing disputes. Vale performed best on technical troubleshooting. That split is more actionable than the headline averages. A company whose queue is heavy with invoices, refunds, and plan changes may prefer North even at the higher price, if billing errors are expensive. A company whose queue is mostly setup problems, error messages, and product how-tos has a clearer case for Vale. Mixed queues could route by ticket type rather than picking a single model for everything. The study does not prove that such routing would raise overall scores; it only shows that the two leaders were not interchangeable across categories. Speed belongs in the same decision. Vale’s 18 percent faster responses will not transform a queue by themselves, but they add to the cost case when volume is high and agents are waiting on a draft before they reply. ## A shared blind spot Every model performed poorly when a ticket depended on an image attachment. That is not a ranking story. It is a ceiling on what this kind of text-only evaluation can support. Screenshots of error dialogs, photos of hardware, and marked-up invoices are ordinary in software support. If the decisive evidence is in the image, a well-written text answer can still be wrong. Teams should not read these scores as a verdict on vision-capable workflows. They should treat image-dependent tickets as out of scope for an unassisted text draft, and keep a human, or a system that actually inspects the attachment, in the path. ## What the study does not show The tickets were in English and came from a single software company. Wording, product language, and what “useful” means will differ in other industries and other languages. Researchers did not test agentic actions, tool use, or multilingual support. A model that drafts a careful reply may still fail if it is asked to issue a refund, look up an order, or switch languages mid-thread. Those tasks were outside the design, so nothing here ranks models for them. The practical reading is narrow and still useful. On English tickets from this company, judged by experienced managers on correctness, usefulness, and tone, Vale matched North closely enough that the score difference should not drive the purchase. Price, speed, and the billing-versus-troubleshooting split should. Cheaper models remain available for lower-stakes drafting, with a visible quality cost. None of the five should be trusted, on this evidence, when the answer lives in an image.
Qwen: Qwen3.8 2.4T A95B: # Vale Emerges as the Value Leader in Support-Ticket Benchmark When five language models were tested on 200 anonymized customer-support tickets, Model North appeared to win: 84.2 versus Model Vale's 83.8. But that narrow margin was not statistically significant. Once cost and speed entered the picture, the clearest finding was different: Vale delivered essentially the same quality as North at less than half the estimated cost and 18% faster. For teams automating first-pass support replies, that is the result that matters. The benchmark asked each model to write one proposed answer for each ticket. Three experienced support managers graded the responses blindly, combining factual correctness, usefulness, and tone. That mix is appropriate for support work, where an accurate reply can still fail if it is confusing or dismissive. The dataset came from one software company and used English-language tickets. North and Vale separated themselves from the rest of the field. Both produced answers raters judged strong overall, and their average scores were close enough that the data cannot support a firm claim that North is better. The practical implication is that buyers should not choose between them based on the overall average alone. Instead, the decision may come down to price, latency, and ticket type. Cost is where Vale's case becomes especially strong. Its estimated cost was $0.74 per 1,000 tickets, compared with $1.90 for North. That difference may look small in isolation, but support teams can handle thousands or millions of tickets. At scale, Vale could reduce model spending by roughly 60% relative to North while preserving similar overall quality. Speed also matters: Vale answered 18% faster than North, which could improve response times and reduce queue backlogs. The models also showed different areas of strength. North performed best on billing disputes, where precision, policy awareness, and careful wording matter. Vale performed best on technical troubleshooting, where the challenge is often to guide a user through steps without overcomplicating the answer. These are not universal wins, but they suggest that routing could be improved by matching models to ticket types. A company with many billing issues might still prefer North for that subset, while a team dominated by technical questions might lean toward Vale. The remaining models were cheaper but scored lower. Ember at 79.1 and Kite at 76.4 may be acceptable for narrow uses, especially where budgets are tight, but they did not match the leaders. Harbor, at 71.0, was the cheapest at $0.08 per 1,000 tickets, but its lower score suggests savings could be offset by more editing, escalations, or customer dissatisfaction. The lesson is not that cheap models are useless; it is that price and quality need to be evaluated together. One weakness cut across every model: tickets that depended on an image attachment. All five performed poorly there. That is an important reminder that current language models may not be ready for workflows where screenshots, error dialogs, invoices, or product photos are essential. If a significant share of tickets requires visual understanding, an organization may need a separate process, human review, or a model evaluated specifically for multimodal tasks. The study's limitations also matter. Two hundred tickets can reveal broad patterns, but it cannot settle tiny score differences. Because the tickets came from one company, results may not transfer to other products, customers, or support styles. The test was English-only and evaluated only proposed written answers. Researchers did not test agentic actions, tool use, or multilingual support. In a real deployment, a model might need to look up account details, follow policies, escalate issues, or avoid unauthorized changes. Still, the central finding is clear. If a support team wants high-quality draft responses without paying for the most expensive model, Vale appears to be the stronger overall value. North remains an excellent performer and may have an edge in billing disputes, but its small scoring advantage should not be treated as decisive. The smarter approach is to treat these results as a starting point: identify dominant ticket types, consider cost and latency, test the models on your own data, and keep humans in the loop where images, judgment calls, or high-risk policies are involved.

OpenAI: GPT-6 Luna: Model B better fits the requested length and gives a clear, balanced account of the findings and limitations. Model A is well written but appears over the word limit and treats lower-ranked score gaps as meaningful without evidence that they are statistically significant. (Second judge pass, order swapped — scores are the average of both: Model B stays within the requested length and presents the main finding with appropriate caveats. Model A is well written, but it appears to exceed the word li | Anthropic: Claude Fable 5.1: Model A is the sharper, better-structured piece…

Exact final-draft structure

Convert the supplied fields into Markdown using these exact rules: * H1 headline * One blank line * Italicized dek * One blank line * First body paragraph * One blank line * H2 section heading * One blank line * Second body paragraph * Preserve all wording and punctuation exactly * Do not use a code fence * Output nothing else Headline: Raincheck raises $8 million for flood-risk software Dek: The Austin startup helps cities decide which drainage problems to address before storms. First body paragraph: Raincheck has raised an $8 million seed round led by Basin Ventures. Section heading: Expansion plans Second body paragraph: The company plans to hire eight engineers.

grok-4.7: # Raincheck raises $8 million for flood-risk software *The Austin startup helps cities decide which drainage problems to address before storms.* Raincheck has raised an $8 million seed round led by Basin Ventures. ## Expansion plans The company plans to hire eight engineers.
Qwen: Qwen3.8 2.4T A95B: # Raincheck raises $8 million for flood-risk software *The Austin startup helps cities decide which drainage problems to address before storms.* Raincheck has raised an $8 million seed round led by Basin Ventures. ## Expansion plans The company plans to hire eight engineers.

OpenAI: GPT-6 Luna: Model A and Model B produce identical output that follows the required structure and preserves all wording and punctuation. Both are equally strong across the evaluated dimensions. (Second judge pass, order swapped — scores are the average of both: Model B and Model A produce identical output that follows the requested structure and preserves all wording and punctuation. Both are equally correct, instruction-following, and well-written.) | Anthropic: Claude Fable 5.1: Model A and Model B produce character-for-character identical output that follows every structural rule: H…

Matchup powered by OpenRouter.