Head to head: Kimi-K3.1-Pro vs Anthropic: Claude Sonnet 5.5

Kimi-K3.1-Pro vs Anthropic: Claude Sonnet 5.5

By · Published

RuntimeWire Head-to-Head: Head to head: Kimi-K3.1-Pro vs Anthropic: Claude Sonnet 5.5
RuntimeWire Head-to-Head matchup

A head-to-head across editing, reporting, news judgment and narrative craft, with the judges weighing factual restraint alongside polish. The results include both clear differences and several ties on tightly specified tasks.

Kimi-K3.1-Pro is the saved winner, with an aggregate score of **105.3** to Anthropic: Claude Sonnet 5.5’s **98.4**. The task record gives Kimi six wins, Claude none, and six ties. The editorial strength is a **decisive result—not a statistical claim**; this assessment applies to the saved task sample, not to every use of either model. Kimi’s edge shows up most in reporting-heavy work. It did better on the cohesive story from notes, the Startup Spotlight, news judgment in the opening, quotes woven into narrative, and shaping numbers into a story. Across those tasks, judges repeatedly favored its restraint: it was less likely to add unsupported details, blur sessions with users, or overinterpret what benchmark results establish. The controlled-wit task showed the same tradeoff. Kimi landed one restrained dry line while staying closer to the supplied reporting; Claude’s version added less-supported claims about Proxy’s market position and what QueueZero had not disclosed. Claude was often capable, but in these comparisons its extra framing sometimes came at the expense of evidentiary discipline. The six ties matter: both models handled the Markdown error count, exact final-draft structure and strongest-lede selection equally, among other tied tasks. They also produced strong publication-ready edits and rewrites, though judges disagreed on some of the finer distinctions. The verdict is therefore a clear Kimi win on this sample, driven less by formatting or basic fluency than by more reliable judgment when the reporting leaves room for interpretation.

Publication-ready Markdown edit

Edit the copy below into a publication-ready Markdown story. Requirements: * Supply an H1 headline and italicized dek * Repair the lede and paragraph flow * Use a Markdown blockquote only for the extended statement * Convert the supplied URL into a descriptive inline link * Remove internal notes and formatting debris * Preserve the reporting and its meaning * Output only the finished article Raw copy: HEADLINE TBD The funding was announced today. Raincheck is the company. It has raised $8 million and it makes flood-risk software for local governments. [NOTE: move this up?] The Austin company maps drainage systems and combines those maps with rainfall forecasts. Cities use the software to decide where to send maintenance crews before a storm. The seed round was led by Basin Ventures. Existing investor Long Mile also participated. Raincheck has raised $10.5 million total. CEO Ana Flores said the following: “Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives.” https://example.com/raincheck-methodology Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them because some contracts have not been publicly announced. Its forecasts are recommendations. Emergency managers retain control over deployments and public warnings. THE COMPETITION Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck says the new funding will help it hire eight engineers and expand into Florida.

Kimi-K3.1-Pro: # Raincheck Raises $8 Million to Help Cities Prepare for Floods *The Austin company’s seed round brings its total funding to $10.5 million as it plans to hire engineers and expand into Florida.* Raincheck, an Austin-based company that makes flood-risk software for local governments, announced today that it has raised $8 million in seed funding. The round was led by Basin Ventures, with participation from existing investor Long Mile. Raincheck has now raised $10.5 million in total. Raincheck maps drainage systems and combines those maps with rainfall forecasts. Cities use the software to decide where to send maintenance crews before a storm. The company outlines its approach in its [flood-risk methodology](https://example.com/raincheck-methodology). Raincheck CEO Ana Flores said: > “Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives.” Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them because some contracts have not been publicly announced. Its forecasts are recommendations, and emergency managers retain control over deployments and public warnings. ## Competition Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck says the new funding will help it hire eight engineers and expand into Florida.
Anthropic: Claude Sonnet 5.5: # Flood-Risk Software Maker Raincheck Raises $8 Million *The Austin startup helps local governments decide where to send maintenance crews before storms hit.* Raincheck, an Austin company that makes flood-risk software for local governments, has raised $8 million in a seed round, it announced today. The software maps drainage systems and combines those maps with rainfall forecasts. Cities use it to decide where to send maintenance crews before a storm. Basin Ventures led the round, and existing investor Long Mile also participated. Raincheck has raised $10.5 million in total. CEO Ana Flores said: > "Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives." Raincheck has also published a description of [its forecasting methodology](https://example.com/raincheck-methodology). The company currently works with seven cities in Texas and Louisiana. It declined to identify them because some contracts have not been publicly announced. Its forecasts are recommendations. Emergency managers retain control over deployments and public warnings. Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck says the new funding will help it hire eight engineers and expand into Florida.

OpenAI: GPT-6 Luna: Model A gives the competition material a clear section heading and links the methodology without adding the claim that it was published. Model B is also strong, but its wording about a published forecasting methodology is less clearly supported by the source copy. (Second judge pass, order swapped — scores are the average of both: Model A gives the story a clearer news-first flow and retains the competition section as a Markdown subheading. Both preserve the reporting and meet the core formattin | Anthropic: Claude Fable 5.1: Both entries meet the core requirements (H1, it…

Natural human rewrite

Rewrite the draft below as a 300–400-word publication-ready news story in Markdown. Preserve every factual claim. Improve the lede, narrative flow, sentence rhythm and transitions. Remove repetition, stock hype, throat-clearing and mechanical signposting. The finished article should sound like an experienced human technology reporter wrote it. Draft: # Arcwell Launches An Innovative New Platform Arcwell has announced an exciting new platform called Relay. The announcement represents a significant development for the company and for the broader customer service industry. Relay uses artificial intelligence to examine customer support conversations and suggest answers to agents. The platform is important for several reasons. First, it can suggest answers. Second, it can identify customers who may cancel. Third, it can create summaries. These capabilities demonstrate how artificial intelligence continues to transform the customer service landscape. Relay was tested by 12 companies over six months. Arcwell says agents using it closed tickets 14% faster. The company did not release customer-retention results. Relay costs $65 per agent each month. CEO Priya Shah said, “Support agents spend too much time searching for information that their companies already have.” However, the platform also presents potential challenges. Relay requires access to support transcripts and internal help documents. Arcwell says customer data is encrypted and is not used to train shared models. Companies can choose a 30-day or 90-day retention period. In conclusion, Relay represents an important new chapter for Arcwell as it seeks to help companies improve customer service using artificial intelligence.

Kimi-K3.1-Pro: # Arcwell launches Relay, an AI platform for customer support Arcwell has launched Relay, an artificial intelligence platform that examines customer support conversations and suggests answers to agents. It can also identify customers who may be at risk of canceling and create summaries, extending its role beyond response drafting. The company is focusing on a practical bottleneck: the time agents spend searching for answers in information their employers already hold. Relay’s mix of features also illustrates how AI is being applied to a wider range of customer service work. “Support agents spend too much time searching for information that their companies already have,” CEO Priya Shah said. Twelve companies tested Relay over six months. Arcwell says agents using the platform closed tickets 14% faster. That is the clearest performance figure attached to the launch, but it addresses speed rather than customer retention. Arcwell did not release customer-retention results, leaving open whether Relay’s warnings about possible cancellations helped companies keep customers. Relay costs $65 per agent each month. Using Relay also requires companies to provide access to some of the material on which their support operations depend. The platform needs support transcripts and internal help documents. That access is central to its ability to suggest responses, but it also makes Arcwell’s data policies an important part of the product and a potential consideration for companies handling sensitive customer information. Arcwell says customer data is encrypted and is not used to train shared models. Companies can choose a 30-day or 90-day retention period, giving them two defined options for how long information is retained. Relay represents a broader effort by Arcwell to help companies improve customer service with AI, rather than a tool limited to drafting replies. Its appeal will depend on more than the 14% speed improvement cited by the company. Prospective customers must also weigh the monthly price against the access Relay requires, the available retention options and Arcwell’s assurances about encryption and model training.
Anthropic: Claude Sonnet 5.5: # Arcwell's Relay Uses AI to Hand Support Agents the Answers Arcwell has launched Relay, an artificial intelligence platform that examines customer support conversations and suggests answers to agents. The software also identifies customers who may cancel and creates summaries. The pitch rests on a problem CEO Priya Shah says is common across support teams. "Support agents spend too much time searching for information that their companies already have," she said. Arcwell has some numbers to back the pitch, though not all of them. Twelve companies tested Relay over six months, and the company says agents using it closed tickets 14% faster. That figure is Arcwell's own. The company did not release customer-retention results, so there is no public evidence yet of how well Relay's cancellation warnings work or whether acting on them keeps customers. Relay costs $65 per agent each month. At that rate, a 50-agent team would pay $3,250 a month. The platform also asks for a lot of access. To work, Relay needs support transcripts and internal help documents, which gives it a view into both what customers say and what companies know. That makes data handling a central question for any business weighing the product. Arcwell says customer data is encrypted and is not used to train shared models. Companies can choose a retention period of either 30 or 90 days. The pairing of a measurable speed gain with an unmeasured retention benefit will likely shape how buyers read Relay. Faster ticket resolution is easy to count. Whether the software helps companies hold on to customers is harder to prove, and Arcwell has not yet offered that proof. For now, the company is betting that support teams will trade access to their transcripts and help documents for quicker answers.

OpenAI: GPT-6 Luna: Model A delivers a more complete, publication-ready story and appears to meet the 300–400-word requirement. Model B is clear but falls short of the requested length and adds an unnecessary pricing example and more speculative framing. (Second judge pass, order swapped — scores are the average of both: Model A meets the 300–400-word requirement and presents the claims in a clear, publication-ready flow. Model B is also readable, but falls short of the requested length and adds an unnecessary hypo | Anthropic: Claude Fable 5.1: Both preserve every factual claim, but Model B…

Cohesive story from reporting notes

Write a publication-ready technology news article from the reporting notes below. Requirements: * 550–700 words * Clean Markdown * One H1 headline followed by an italicized dek * A strong news lede followed by a clear nut graf * Organize the material into a cohesive story rather than following the notes in order * Attribute company-supplied claims * Include limitations without turning the article into a warning label * Use only the supplied information * Avoid canned transitions, promotional language and a concluding summary Reporting notes: * HelioDock makes autonomous warehouse robots. * On Thursday, it launched Night Shift, software that lets operators monitor robot fleets remotely. * Night Shift spent 14 months in a private beta involving nine logistics companies and 310 robots. * HelioDock says beta customers experienced 17% fewer unplanned robot stoppages. The figure has not been independently verified. * The company also announced a $24 million Series B led by North Coast Ventures, bringing its total funding to $39 million. * CEO Mara Velez: “A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet.” * Beta customer Aaron Pike of Fulcrum Logistics said the system worked well in facilities with strong connectivity but became less useful during rural LTE outages. * When connectivity fails, robots return to their locally configured behavior. Remote control becomes unavailable. * Competing fleet-management products are available from Relay Robotics and YardOS. * Night Shift will cost $90 per robot per month and become generally available September 15.

Kimi-K3.1-Pro: # HelioDock launches Night Shift for remote robot oversight and raises $24 million *The warehouse robotics company will charge $90 per robot per month for the software, which follows a 14-month private beta and becomes generally available September 15.* HelioDock on Thursday launched Night Shift, software that lets operators monitor fleets of autonomous warehouse robots remotely. The product marks the company’s move into a subscription software business alongside its work making warehouse robots, with pricing that scales according to the number of machines being covered. The launch combines a commercial debut with new funding and early operating data. HelioDock announced a $24 million Series B led by North Coast Ventures, bringing its total funding to $39 million. Night Shift also enters a market where fleet-management products are already available from Relay Robotics and YardOS. HelioDock’s case for the software rests on remote oversight and a reported reduction in unplanned stoppages during testing, although a customer’s experience shows that its usefulness depends partly on connectivity. Chief Executive Mara Velez described the product through the kind of disruption that can require a human response even in an automated facility. “A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet,” Velez said. Her example emphasizes the distance between a manager and a warehouse, presenting remote access as an alternative to routinely treating an isolated robot problem as a reason for an after-hours trip. Night Shift will cost $90 per robot per month and become generally available September 15. Under that structure, the monthly software charge rises directly with fleet size rather than being set as a single fee for an entire operation. General availability will extend access beyond the logistics companies that participated in the private test. That beta ran for 14 months and involved nine logistics companies and 310 robots. HelioDock said beta customers experienced 17% fewer unplanned robot stoppages. The figure is a company-supplied assessment of the test and has not been independently verified. It describes the customers’ reported experience during the beta rather than an independently established result for future deployments. Aaron Pike of Fulcrum Logistics, a beta customer, said the system worked well in facilities with strong connectivity. He said it became less useful during rural LTE outages. His account puts a practical boundary around the remote-monitoring proposition: the quality of the connection can affect the service an operator receives, particularly at sites where rural LTE service is interrupted. The system’s failure behavior distinguishes the robots’ local operation from Night Shift’s remote capabilities. When connectivity fails, the robots return to their locally configured behavior. Remote control becomes unavailable. An operator may therefore have robots reverting to local settings without retaining the ability to control them from afar. That makes network reliability an operating condition for the software, not merely a separate technical consideration. Pike’s comments and HelioDock’s stoppage figure address different parts of the beta experience. The company is reporting an overall operational outcome for participating customers, while the customer account identifies conditions under which the system became less useful. Together, they provide a more qualified picture than the percentage alone: Night Shift has been tested across hundreds of robots, but the experience described by one participant was not uniform across facilities and network conditions. The Series B adds a substantial financing component to the launch. North Coast Ventures led the $24 million round, which accounts for most of the $39 million in total funding HelioDock says it has raised. The company paired the investment announcement with Night Shift’s introduction rather than presenting the financing separately. Relay Robotics and YardOS already offer competing fleet-management products. Night Shift will join that field on September 15, bringing a per-robot subscription and the results of its nine-company beta into a market with established alternatives.
Anthropic: Claude Sonnet 5.5: # HelioDock Launches Remote Monitoring Software for Warehouse Robots and Raises $24 Million *The company's Night Shift product lets operators oversee robot fleets from afar. A beta customer says it works best where connectivity is strong.* HelioDock, which makes autonomous warehouse robots, on Thursday launched Night Shift, software that lets operators monitor robot fleets remotely. The company also announced a $24 million Series B round led by North Coast Ventures. Night Shift is meant to reduce how often people must go to a warehouse when a robot stops working. HelioDock says beta customers had 17% fewer unplanned robot stoppages, a figure that has not been independently verified. One of those customers says the software was less useful when a site's cellular connection failed. ## What the company says CEO Mara Velez described the product in terms of overnight emergencies. "A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet," she said. Night Shift spent 14 months in a private beta with nine logistics companies and 310 robots. HelioDock attributes the 17% reduction in unplanned stoppages to that group. The company supplied the figure, and the notes provided do not say how it was measured. ## Where connectivity matters Aaron Pike of Fulcrum Logistics, a beta customer, said the system worked well in facilities with strong connectivity. It became less useful during rural LTE outages, he said. The product's design explains why. When connectivity fails, HelioDock's robots return to their locally configured behavior, and remote control becomes unavailable. Operators can reach the robots through Night Shift only while the site has a connection to the software. ## Pricing and availability Night Shift will cost $90 per robot per month and become generally available Sept. 15. At that rate, a 100-robot fleet would cost $9,000 a month. The software enters a market that already has options. Relay Robotics and YardOS both offer competing fleet-management products. ## Funding The Series B, led by North Coast Ventures, brings HelioDock's total funding to $39 million. The notes provided do not say how the company plans to spend the new money, or whether it will go toward Night Shift. The beta's scale gives some sense of how far the software has been tested. Across nine companies and 310 robots, it ran for more than a year before its public release. Pike's comments suggest the results varied with the conditions at each site, though the company has not published a breakdown by facility or connectivity. HelioDock's 17% figure covers unplanned stoppages, meaning times when a robot stopped without being scheduled to. It does not address planned downtime. The company has not said how much of the improvement came from operators fixing problems remotely and how much from other factors. For customers deciding whether to buy, the practical questions are likely to be where their facilities sit and how reliable their connections are. A warehouse with steady connectivity may use Night Shift much as HelioDock describes it. A site that regularly loses its LTE signal will have to rely on its robots' local settings, which is the same fallback that applied before the software existed.

OpenAI: GPT-6 Luna: Model A better meets the requested length and uses the supplied reporting with fewer unsupported assertions. Model B is more concise, but appears well short of 550 words and adds claims about measurement, facility breakdowns, and preexisting fallback behavior that the notes do not establish. (Second judge pass, order swapped — scores are the average of both: Model A is the stronger submission: it meets the requested length and presents the launch, beta results, connectivity limitation, pricing, | Anthropic: Claude Fable 5.1: Model A is padded, repetitive and stiff, with so…

Credible Startup Spotlight

Write a 500–650-word Startup Spotlight article in clean Markdown. The tone should be curious and constructive. Explain the founders’ insight, product and early traction while giving uncertainty and competition appropriate space. Avoid investor clichés, heroic founder mythology and promotional claims stated as fact. Reporting: * LedgerLark builds AI agents for post-trade operations at small investment firms. * Co-founder Mina Batra previously spent seven years in operations at a regional brokerage. * Co-founder Owen Carr built reconciliation systems at a large bank. * Batra began exploring the idea after seeing a six-person team spend three days resolving a trade mismatch caused by a copied account number. * LedgerLark’s agents compare confirmations, custodian records and internal ledgers, then propose corrections for human approval. * The product does not execute trades or move money. * Five firms are paying customers. * Annual contracts range from $36,000 to $110,000. * The company says one customer reduced its unresolved daily exceptions from 70 to 18. * The company has raised a $3.5 million seed round led by Motive Row. * Competitors include established reconciliation vendors and internal bank software. * Financial institutions may require lengthy security and procurement reviews.

Kimi-K3.1-Pro: # Startup Spotlight: LedgerLark Takes on the Slow Work After a Trade *LedgerLark is building AI agents for post-trade operations at small investment firms, with a notable boundary: its software proposes corrections, but people approve them.* ## Starting with an operations bottleneck Co-founder Mina Batra spent seven years in operations at a regional brokerage. She began exploring the idea behind LedgerLark after watching a six-person team spend three days resolving a trade mismatch caused by a copied account number. The episode pointed to a less visible cost of trading: even when an error looks simple, finding its source can require repeated checks across records and teams. Batra’s co-founder, Owen Carr, previously built reconciliation systems at a large bank. Together, their experience frames a focused question: can software handle more of the comparison and investigation work without being given control over the underlying financial activity? ## A bounded role for AI agents LedgerLark’s agents compare confirmations, custodian records and internal ledgers, then propose corrections for human approval. The product does not execute trades or move money. That boundary matters. For a small operations team, the potential value is not an autonomous trading system; it is a tool that could reduce the time spent assembling evidence and identifying likely fixes. Staff remain responsible for deciding whether a proposed correction is appropriate. The design also creates a practical test. If recommendations are frequently incomplete or wrong, reviewers could gain another queue to check rather than meaningful time back. LedgerLark will need to demonstrate accuracy, clear audit trails and sensible behavior when records conflict—not just produce plausible suggestions. ## Early customers, limited evidence Five firms are paying customers, with annual contracts ranging from $36,000 to $110,000. Those figures show some willingness to pay for the product, although five customers are too few to indicate how broadly the approach will apply. The wide contract range also leaves open how much deployments differ in scope and usage. The company says one customer reduced its unresolved daily exceptions from 70 to 18. That is a useful operational data point, but it remains a company-reported result from a single customer. It does not establish a typical outcome, show whether the improvement persisted, or account for changes in trading volume or exception mix. LedgerLark has raised a $3.5 million seed round led by Motive Row. The funding supports further product and commercial development, but the more consequential evidence will be retention, repeatable implementations and results across multiple customers. ## Competition and the pace of procurement LedgerLark is not entering an empty category. Its competitors include established reconciliation vendors and internal bank software. A prospective buyer may therefore ask whether LedgerLark improves materially on tools already in place, and whether its agents justify another system to integrate and oversee. Adoption may also move more slowly than an operations team’s initial interest suggests. Financial institutions can require lengthy security and procurement reviews, particularly when software touches sensitive records. Those reviews can test a young vendor’s documentation, controls and implementation capacity as much as its core product. The next stage should offer clearer answers. Useful measures would include how often reviewers accept proposed corrections, how much handling time falls, and whether customers renew after working through real-world exceptions. LedgerLark has identified a concrete, expensive workflow. Its challenge now is to show that a carefully bounded agent can improve it consistently, under the controls financial firms require.
Anthropic: Claude Sonnet 5.5: # Startup Spotlight: LedgerLark Wants AI to Handle the Tedious Work of Fixing Trade Mismatches ## A three-day problem Mina Batra spent seven years in operations at a regional brokerage. One episode there shaped her view of her field: a six-person team spent three days resolving a trade mismatch. The cause was a copied account number. Nothing about the error was exotic. The expensive part was finding it. Someone had to compare trade confirmations, custodian records and internal ledgers, work out where they diverged, and decide what to fix. That experience led to LedgerLark, which builds AI agents for post-trade operations at small investment firms. Batra co-founded the company with Owen Carr, who previously built reconciliation systems at a large bank. ## What the product does LedgerLark's agents compare three sources: confirmations, custodian records and a firm's internal ledgers. When they find a discrepancy, they propose a correction. A human then approves or rejects it. The company draws a firm boundary around that role. The product does not execute trades or move money. It reads records and makes suggestions, and people stay responsible for any change. That design limits what the software can do wrong. It also fits the likely buyer. A small firm may not have the headcount of a large bank's operations department, yet it faces the same kinds of breaks between its books and its counterparties'. The pairing of Batra's operations background with Carr's reconciliation engineering matches the problem, though a good fit between founders and problem does not guarantee a good product. ## Early traction LedgerLark has five paying customers, with annual contracts ranging from $36,000 to $110,000. Those figures imply annual contract value of roughly $180,000 to $550,000 in total, though the company has not given a more precise number. It also says one customer cut its unresolved daily exceptions from 70 to 18. That is a notable improvement if it holds, but it is a single customer's result, reported by the company. It does not say over what period the drop occurred, how complex the exceptions were, or how much of the change came from the software rather than from process changes. Results from the other four customers would show whether the pattern generalizes. The company has raised a $3.5 million seed round led by Motive Row. ## The harder questions Competition is the first. Established reconciliation vendors already sell into this market, and some large institutions rely on internal bank software. LedgerLark's case rests on small firms being underserved by those options, and on AI agents handling messy, inconsistent records better than rule-based matching. Both ideas are plausible, and both still need to be proven at a larger scale. Sales may be the second challenge. Financial institutions can require lengthy security and procurement reviews, even for software that only reads data and proposes changes. For a young company, long reviews can stretch sales cycles and strain a small team. Five customers suggests the company has cleared that hurdle at least a few times. It does not show how repeatable the process is. Then there is trust. A system that proposes corrections is only useful if the proposals are consistently right, or at least easy to check. If reviewers must re-verify every suggestion from scratch, the time savings shrink. Whether LedgerLark's approval step becomes a quick confirmation or a second investigation will matter a great deal. ## What to watch Several things would clarify LedgerLark's prospects: - Whether the exception-reduction result appears at other customers - Whether customers renew and expand after the first annual term - How the company handles security reviews as it approaches larger firms - Whether established vendors respond with similar agent-based features LedgerLark starts from a concrete, familiar frustration and a deliberately narrow product. Whether that is enough to build a durable business in a cautious, crowded corner of finance remains an open question.

OpenAI: GPT-6 Luna: Model A has a slight edge for keeping its analysis closely tied to the reported facts while giving traction, competition, and procurement uncertainty appropriate space. Model B is also strong, but adds more speculative framing and a derived aggregate contract-value estimate that contributes little to the spotlight. (Second judge pass, order swapped — scores are the average of both: Model A is more measured and precise, especially in distinguishing the reported customer result from evidence of re | Anthropic: Claude Fable 5.1: Both stay faithful to the reporting and treat t…

Markdown error count

Count the lines that violate the house rules below. A defective line counts once even if it could violate multiple rules. Respond with a single number. House rules: * The headline must use `# Headline`. * The dek must appear immediately after the headline and be entirely italicized. * Section headings must use `## Heading`. * Links must use an absolute HTTPS URL. * Unordered-list items must begin with a hyphen. * Raw HTML is forbidden. Draft: #Flood software moves upstream The company is expanding its pilot. ##Details Read [the announcement](www.example.com/news). * The first pilot begins Monday. <div>The second pilot begins Friday.</div>

Kimi-K3.1-Pro: 6
Anthropic: Claude Sonnet 5.5: **6** Six lines break the rules: 1. `#Flood software moves upstream` has no space after the `#`. 2. `The company is expanding its pilot.` is the dek but is not italicized. 3. `##Details` has no space after the `##`. 4. `Read [the announcement](www.example.com/news).` uses a link that is not an absolute HTTPS URL. 5. `* The first pilot begins Monday.` starts the list item with an asterisk instead of a hyphen. 6. `<div>The second pilot begins Friday.</div>` is raw HTML.

Scored against the benchmark's reference answer (numeric match): Model A was correct, Model B was correct.

News judgment in the opening

Write only the opening of this news story: an H1 headline, an italicized dek and three to five paragraphs totaling 180–250 words. The opening must establish what happened, quantify the scope, explain why it matters and carefully distinguish confirmed facts from unknowns. It should pull the reader forward without sensationalizing the incident. Reporting: * Cloudnote discovered that a debugging page containing customer prompt excerpts was accessible without authentication. * The page was exposed for four hours on Monday. * Logs show excerpts from 1,842 user sessions. * The excerpts could include up to 200 characters from a prompt. * Cloudnote removed the page 19 minutes after a researcher notified it. * The company says it has found no evidence that anyone besides the researcher accessed the page. * It has not completed its investigation. * Full documents, passwords and payment information were stored elsewhere and were not present on the page. * Cloudnote has 3.2 million registered users.

Kimi-K3.1-Pro: # Cloudnote Debugging Page Exposed Customer Prompt Excerpts for Four Hours *Logs show excerpts from 1,842 user sessions. Cloudnote says it has found no evidence of access beyond the researcher who reported the problem, but its investigation is not complete.* Cloudnote said a debugging page containing excerpts of customer prompts was accessible without authentication for four hours on Monday. The excerpts could include up to 200 characters from a prompt. The company removed the page 19 minutes after a researcher notified it. Logs show excerpts from 1,842 user sessions. That figure quantifies sessions, not individual customers, and does not establish whether every excerpt was viewed. Cloudnote has 3.2 million registered users, but the session count cannot be directly compared with that total or used to determine how many accounts were affected. Even short prompt fragments can contain personal details or sensitive working material, making the lapse significant for users despite its limited duration. Cloudnote said full documents, passwords and payment information were stored elsewhere and were not present on the page, narrowing the types of data known to have been available. Cloudnote said it has found no evidence that anyone besides the researcher accessed the page. That finding does not rule out other access, and the investigation is incomplete. It remains unclear how the page became accessible, whether any excerpts were copied and what steps the company will take to prevent a recurrence.
Anthropic: Claude Sonnet 5.5: # Cloudnote debugging page exposed prompt excerpts from 1,842 user sessions *The company says only the researcher who reported it is known to have viewed the page, but its investigation is not finished.* Cloudnote left an internal debugging page accessible without authentication for about four hours on Monday, exposing excerpts of customer prompts from 1,842 user sessions, the company said. Each excerpt could include up to 200 characters of a prompt. Cloudnote said the page did not contain full documents, passwords or payment information, which it said are stored elsewhere. The company removed the page 19 minutes after a security researcher notified it of the problem. Cloudnote says it has found no evidence that anyone other than the researcher accessed the page. But the company has not completed its investigation, so that finding remains preliminary. Cloudnote has not confirmed what any individual excerpt contained. The exposure matters because of the platform's size and the nature of the data. Cloudnote has 3.2 million registered users, and 1,842 sessions is a small share of that base. Sessions are not the same as users, however, and the figures do not show how many people were affected. Even a few lines of a prompt can reveal personal or work details, depending on what someone typed.

OpenAI: GPT-6 Luna: Model A is more careful about distinguishing sessions from users and avoids an unsupported comparison between the session count and registered-user base. Model B makes that comparison despite acknowledging that sessions are not users, though both follow the requested format and length. (Second judge pass, order swapped — scores are the average of both: Model A better distinguishes session counts from affected customers, handles the investigation’s limits carefully and meets the requested word ra | Anthropic: Claude Fable 5.1: Model A stays strictly within the reported fact…

Exact final-draft structure

Convert the supplied fields into Markdown using these exact rules: * H1 headline * One blank line * Italicized dek * One blank line * First body paragraph * One blank line * H2 section heading * One blank line * Second body paragraph * Preserve all wording and punctuation exactly * Do not use a code fence * Output nothing else Headline: Raincheck raises $8 million for flood-risk software Dek: The Austin startup helps cities decide which drainage problems to address before storms. First body paragraph: Raincheck has raised an $8 million seed round led by Basin Ventures. Section heading: Expansion plans Second body paragraph: The company plans to hire eight engineers.

Kimi-K3.1-Pro: # Raincheck raises $8 million for flood-risk software *The Austin startup helps cities decide which drainage problems to address before storms.* Raincheck has raised an $8 million seed round led by Basin Ventures. ## Expansion plans The company plans to hire eight engineers.
Anthropic: Claude Sonnet 5.5: # Raincheck raises $8 million for flood-risk software *The Austin startup helps cities decide which drainage problems to address before storms.* Raincheck has raised an $8 million seed round led by Basin Ventures. ## Expansion plans The company plans to hire eight engineers.

OpenAI: GPT-6 Luna: Model A and Model B produce identical Markdown that follows the required structure and preserves all wording and punctuation. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both follow the required Markdown structure and preserve all wording and punctuation exactly. Their outputs are identical and equally strong across all dimensions.) | Anthropic: Claude Fable 5.1: Model A and Model B produce character-for-character identical outputs that follow every structural rule exactly (H1, italic dek, body, H2, body with single blank lines,…

Select the strongest accurate lede

Based only on the facts below, select the strongest news lede. Respond with one letter. Facts: * ByteBarge pulled its $12-per-month AI assistant on Tuesday. * Private workspace titles from 4,860 accounts were sent to a publicly accessible analytics endpoint. * The exposure lasted 11 hours. * Message contents were not sent. * The company has found no evidence of malicious access. A. ByteBarge users suffered a massive privacy disaster Tuesday when the company leaked their private messages online. B. ByteBarge, an innovative collaboration company, made the difficult decision to cancel one of its products. C. ByteBarge pulled its $12-per-month AI assistant Tuesday after discovering that private workspace titles from 4,860 accounts had been sent to a public analytics endpoint for 11 hours. D. Thousands of ByteBarge users woke Tuesday to learn that everything they had written was visible to hackers.

Kimi-K3.1-Pro: C
Anthropic: Claude Sonnet 5.5: C

Scored against the benchmark's reference answer (choice match): Model A was correct, Model B was correct.

Quotes woven into narrative

Write a 400–500-word Markdown story about the closing of Foundry Hall. Use quotations selectively. Paraphrase when the information matters more than the speaker’s exact phrasing. Avoid stacking quotes or giving every source an equal-sized turn. The story should carry an emotional current while remaining restrained. Facts: * Foundry Hall, a nonprofit startup accelerator in Milwaukee, will close October 31. * It opened in 2018. * It has supported 212 startups, which collectively employ 1,140 people. * Its largest corporate sponsor, Blank River Bank, supplied 38% of its annual budget. * The bank declined to renew its sponsorship. * Foundry Hall has enough money to operate through October but not through another full program. * Six employees will lose their jobs. * Twelve startups currently using its workspace must leave. Quotes: * Executive director Camille Ross: “We kept waiting for the replacement check that never came.” * Founder Dev Malik: “My first investor meeting happened in that building. Before Foundry Hall, I was pitching from the back table of a coffee shop.” * Blank River Bank spokesperson: “The decision reflects a change in our community investment priorities and is unrelated to Foundry Hall’s performance.” * Deputy mayor Susan Cole: “The city is speaking with building owners and local universities about space for the current companies.”

Kimi-K3.1-Pro: # Foundry Hall to Close After Losing Its Largest Sponsor MILWAUKEE — Foundry Hall, the nonprofit startup accelerator that has served as a first office, meeting place and proving ground for local entrepreneurs, will close Oct. 31 after losing its largest corporate sponsor. Since opening in 2018, the organization has supported 212 startups, which collectively employ 1,140 people. Its closure will cost six employees their jobs and require the 12 startups now using its workspace to move out. Blank River Bank supplied 38% of Foundry Hall’s annual budget and declined to renew its sponsorship. A bank spokesperson said the decision reflected a shift in community investment priorities and was unrelated to the accelerator’s performance. Executive director Camille Ross said the organization could not make up the loss in time. Foundry Hall has enough money to continue operating through October, but not enough to support another full program. “We kept waiting for the replacement check that never came,” Ross said. The funding gap shows how much of the accelerator’s stability rested on one relationship. Even with a record of company formation and employment, Foundry Hall could not quickly replace money that accounted for more than a third of its budget. The closure is not simply the loss of a workspace; it also ends the organization that arranged the support around it. The announcement turns the accelerator’s final weeks into a search for space and a reckoning with what its support made possible. For founders, the building offered more than desks: it placed investor meetings, mentorship and other early-stage companies within reach. Founder Dev Malik traced his own progress to that proximity. “My first investor meeting happened in that building,” he said. “Before Foundry Hall, I was pitching from the back table of a coffee shop.” His account captures the accelerator’s role at a stage when access can matter as much as square footage: a room for a consequential conversation, an introduction or the credibility of working alongside other founders. Those benefits are harder to total than 212 startups or 1,140 jobs, but they help explain why the closing reaches beyond a real estate problem. Deputy Mayor Susan Cole said the city is talking with building owners and local universities about space for the current companies. Those conversations offer a possible next step, but they do not preserve the organization that brought the founders together. For now, Foundry Hall’s staff must help 12 companies plan their departures while preparing for their own. The accelerator’s record will remain in the businesses it advised and the jobs they created. But on Oct. 31, its daily work will stop. What disappears is not only a building, but a place where an untested idea could become a company—and where a founder did not have to borrow a coffee shop table to be taken seriously.
Anthropic: Claude Sonnet 5.5: # Foundry Hall to Close After Seven Years of Launching Milwaukee Startups Foundry Hall, the nonprofit startup accelerator that opened in Milwaukee in 2018, will close October 31. Over those years, the organization has supported 212 startups. Together, they employ 1,140 people. Twelve companies still work from its space, and they will have to leave by the end of next month. Six Foundry Hall employees will lose their jobs. The closing follows the decision of its largest corporate sponsor, Blank River Bank, not to renew its support. The bank supplied 38% of Foundry Hall's annual budget, a share the organization could not replace in time. Executive director Camille Ross said the staff spent months hoping that would change. "We kept waiting for the replacement check that never came," she said. Foundry Hall has enough money to operate through October, but not enough to run another full program. Ross said the board chose to close on a date it could pay for rather than stretch its remaining funds into a season it could not finish. A Blank River Bank spokesperson said the decision reflected a shift in the bank's community investment priorities and had nothing to do with Foundry Hall's performance. The bank did not say where its funding will go instead. For many founders, the building mattered before their companies did. Dev Malik, who started his company through the program, said Foundry Hall was where his work first looked like a business to other people. "My first investor meeting happened in that building," Malik said. "Before Foundry Hall, I was pitching from the back table of a coffee shop." He said he still thinks about the startups that will not get the same start. The most immediate concern is the dozen companies that use the workspace every day. Some are small teams with a few laptops and a shared calendar. Others have employees who depend on a place to meet customers, host interviews, and keep a steady routine. Moving them will cost money and momentum that young companies rarely have to spare. Deputy Mayor Susan Cole said the city is talking with building owners and local universities about space for the current companies. She did not say how many of the twelve could be placed or when an arrangement might be ready. Ross said the staff is spending the final weeks helping founders find new space, sending introductions, and passing along contacts that took years to build. Some of that work will outlast the organization. Alumni already know one another, and many have said they intend to keep meeting. Still, the end date is fixed. After October 31, the doors will close on a place that, by its own count, helped hundreds of companies begin.

OpenAI: GPT-6 Luna: Model A stays closer to the supplied facts while weaving the quotes into a restrained narrative. Model B adds several unsupported details, including a board decision Ross supposedly described and claims about Malik’s company and the remaining startups. (Second judge pass, order swapped — scores are the average of both: Model A is more grounded and restrained, while Model B adds unsupported reporting and attribution, including claims about the board’s decision and staff’s plans. Model A meets the | Anthropic: Claude Fable 5.1: Model A stays faithful to the supplied facts, q…

Chronology with narrative momentum

Turn the scrambled reporting below into a 450–600-word Markdown news article. Choose an opening that captures the consequence of the failure, then reconstruct the sequence clearly. Preserve the chronology without producing a minute-by-minute incident log. Give technical details enough context for a general reader. Reporting: * KiteBridge makes expense-management software. * At 8:45 a.m. Tuesday, it deployed version 4.2. * The first customer complaint arrived at 9:12 a.m. * A payment-processor timeout caused KiteBridge’s fallback system to treat retries as new reimbursement requests. * The software generated 13,400 duplicate reimbursements totaling $2.8 million. * KiteBridge suspended automated payments at 10:03 a.m. * Ninety-one corporate customers were affected. * By Wednesday evening, $2.3 million had been recovered. The rest remained pending. * No employees had money withdrawn from their accounts; employers funded the duplicate payments. * An April changelog said version 4.0 had introduced “exactly-once payment protection.” * KiteBridge’s public incident page initially described the problem as delayed reimbursements. * The incident page first disclosed the duplicate payments six hours after automation was suspended. * CEO Lena Ortiz apologized and said the company would publish a technical report.

Kimi-K3.1-Pro: # KiteBridge Failure Generates $2.8 Million in Duplicate Reimbursements KiteBridge’s expense-management software generated 13,400 duplicate reimbursements totaling $2.8 million on Tuesday, affecting 91 corporate customers and prompting the company to suspend automated payments. By Wednesday evening, $2.3 million had been recovered, while the remaining $500,000 was still pending. No employees had money withdrawn from their accounts to fund the duplicates; employers funded the payments. The incident therefore left KiteBridge’s corporate customers carrying the immediate financial exposure while the company worked to recover the money and explain how its safeguards failed. ## How the failure unfolded KiteBridge deployed version 4.2 at 8:45 a.m. Tuesday. The first customer complaint arrived at 9:12 a.m. Automated payments remained active until 10:03 a.m., when the company suspended them. By then, the reimbursement requests had already been duplicated on a large scale. The trigger was a payment-processor timeout. A timeout occurs when the processor does not respond within the expected period, leaving the software without clear confirmation of whether the original payment attempt succeeded. Systems often retry in that situation to avoid dropping a legitimate transaction. KiteBridge’s fallback system, which handles payment processing when the normal path encounters a problem, treated those retries as new reimbursement requests. Instead of recognizing repeated attempts to complete the same transaction, it created additional reimbursements. The result was 13,400 duplicate payments across the affected customer base. A retry is useful only if the system can distinguish a repeated request from a genuinely new one. Payment software commonly uses transaction identifiers and processing records to make that distinction. The specific controls used by KiteBridge have not been detailed. The failure also contrasted with the company’s own documentation. An April changelog said version 4.0 had introduced “exactly-once payment protection.” In payment systems, that term generally describes safeguards intended to ensure a transaction is executed only once, even when technical problems cause it to be submitted repeatedly. Tuesday’s incident showed that the described protection did not prevent duplication in the fallback path involved here. ## Recovery and public disclosure KiteBridge’s public incident page initially described the problem as delayed reimbursements. It first disclosed the duplicate payments six hours after automated payments were suspended. The initial description did not communicate the central financial consequence: extra reimbursements had already been generated and funded by employers. CEO Lena Ortiz apologized and said KiteBridge would publish a technical report. Her commitment leaves the company with two central questions to address: how the fallback system came to treat retries as new requests, and why its public account initially focused on delays rather than duplicates. The recovery effort had reclaimed most of the money by Wednesday evening, but the outstanding balance meant the incident was not fully resolved. For customers, the promised report will also need to reconcile Tuesday’s failure with the protection KiteBridge said version 4.0 had introduced.
Anthropic: Claude Sonnet 5.5: # KiteBridge Software Error Sends 13,400 Duplicate Reimbursements Worth $2.8 Million **A software update at the expense-management company left 91 corporate customers paying employees twice, and the company's early public updates did not say so.** An update to KiteBridge's expense-management software on Tuesday morning triggered 13,400 duplicate reimbursement payments totaling $2.8 million. By Wednesday evening, $2.3 million had been recovered. The remaining $500,000 was still pending. KiteBridge, which makes software that companies use to reimburse employees for business expenses, said 91 corporate customers were affected. No employees had money withdrawn from their accounts. The duplicate payments were funded by employers, which bore the financial exposure. ## How it unfolded KiteBridge deployed version 4.2 of its software at 8:45 a.m. Tuesday. The first customer complaint arrived at 9:12 a.m. According to the reporting, the trouble began when a payment processor, the outside service that moves money between bank accounts, timed out. A timeout means the processor did not respond quickly enough for KiteBridge's system to confirm whether a payment had gone through. KiteBridge's fallback system, designed to take over when something goes wrong, then treated the retries of those payments as new reimbursement requests rather than repeat attempts at the same ones. Each retry became a fresh payment, and the software generated thousands of duplicates. At 10:03 a.m., KiteBridge suspended automated payments, a little under an hour and a half after the update went live. ## A promise of protection The failure is notable because of what KiteBridge had said about its own safeguards. A changelog published in April said version 4.0 had introduced "exactly-once payment protection." That term usually refers to controls meant to ensure a payment is made one time even if a request is repeated. KiteBridge has not yet explained how duplicates got through despite that feature. ## Communication questions KiteBridge's public incident page initially described the problem as delayed reimbursements. That description did not match what was happening: some customers' employees were being paid more than once. The page first disclosed the duplicate payments about six hours after automation was suspended. The gap between the early description and the eventual disclosure may draw scrutiny from customers, who needed to know quickly whether money was leaving their accounts. ## The company's response CEO Lena Ortiz apologized and said KiteBridge would publish a technical report on the incident. The company has not said when the report will appear or whether it will address the "exactly-once" protection described in the April changelog. Several questions remain open. KiteBridge has not said how the remaining $500,000 will be recovered or when. It has not said whether automated payments have resumed. It has not said how the company tested version 4.2 before releasing it. For now, the episode is a reminder of how a routine software update, combined with a brief failure at an outside payment service, can multiply into thousands of erroneous transactions in a few hours. How much trust KiteBridge keeps may depend on what the promised report reveals.

OpenAI: GPT-6 Luna: Model A preserves the key facts and chronology while adding useful technical context with fewer unsupported claims. Model B has slightly stronger narrative momentum, but it asserts several details not established in the reporting, including what KiteBridge had or had not said about recovery, testing, and resuming payments. (Second judge pass, order swapped — scores are the average of both: Model A reconstructs the sequence and explains the failure clearly while staying focused on the reported fa | Anthropic: Claude Fable 5.1: Both stay within length and preserve the facts,…

Numbers shaped into a story

Write a 600–750-word Markdown article from the benchmark results below. Build the story around the clearest finding. Explain the methodology and limitations where readers need them. Avoid reciting the table row by row or treating small differences as decisive. Benchmark: * Researchers tested five language models on 200 anonymized customer-support tickets. * Each model produced one proposed answer per ticket. * Three experienced support managers graded the answers blindly. * Scores combined factual correctness, usefulness and tone. * Model North: 84.2 average score; $1.90 estimated cost per 1,000 tickets. * Model Vale: 83.8; $0.74 per 1,000 tickets. * Model Ember: 79.1; $0.31 per 1,000 tickets. * Model Kite: 76.4; $0.22 per 1,000 tickets. * Model Harbor: 71.0; $0.08 per 1,000 tickets. * North and Vale’s score difference was not statistically significant. * Vale answered 18% faster than North. * North performed best on billing disputes. * Vale performed best on technical troubleshooting. * All five models performed poorly when a ticket depended on an image attachment. * The test used English-language tickets from one software company. * Researchers did not test agentic actions, tool use or multilingual support.

Kimi-K3.1-Pro: # A Statistical Tie at the Top Changes the Model Choice The clearest finding in this benchmark is not that Model North received the highest average score. It is that the test did not establish a quality difference between North and Model Vale, even though Vale was faster and much less expensive. North averaged 84.2, compared with 83.8 for Vale, but their 0.4-point gap was not statistically significant. That does not prove the two models are identical; it means these results offer no reliable basis for declaring North the quality leader. Meanwhile, Vale answered 18% faster and cost an estimated $0.74 per 1,000 tickets, versus $1.90 for North—about 61% less. For a workload resembling this test, Vale is therefore the more compelling default candidate. ## How the Benchmark Worked Researchers gave five language models the same 200 anonymized customer-support tickets, with each model producing one proposed answer per ticket. Three experienced support managers graded the answers blindly, using scores that combined factual correctness, usefulness and tone. Blind grading reduces the chance that a model’s name or reputation shaped the assessments, while multiple graders provide a broader quality judgment than a single reviewer. The setup still measures proposed-answer quality, not whether customers’ problems were actually resolved. A composite score can also conceal different strengths: two similar averages might reflect very different balances of accuracy, usefulness and tone. And one answer per ticket says little about performance in extended conversations or after follow-up questions. ## What the Rest of the Field Shows Below the leaders, the results sketch a broad quality–cost trade-off rather than a case for simply choosing the cheapest model. Ember scored 79.1 at an estimated $0.31 per 1,000 tickets; Kite and Harbor cost less still, at $0.22 and $0.08, but scored 76.4 and 71.0. Whether those savings justify lower scores depends on ticket risk, review processes and the cost of a bad answer—factors the benchmark does not measure. The summary also does not establish which of these other score gaps are statistically significant. The aggregate results additionally hide useful category variation. North performed best on billing disputes, while Vale led on technical troubleshooting. That could matter for routing or for a company with a heavily billing-focused queue. Without category sample sizes or significance testing, however, these results are better treated as prompts for a follow-up evaluation than as proof of durable specialties. ## The Weakness Every Model Shared All five models performed poorly when a ticket depended on an image attachment. This is not a footnote for support teams whose workflows rely on screenshots, error dialogs or other visual evidence. A strong average on the overall ticket set cannot compensate for systematic failure on an important ticket type. Such teams need a dedicated evaluation of image-dependent cases and a clear human fallback, rather than assuming the top-ranked model is deployment-ready. ## How Far the Conclusions Travel Generalization is the other major constraint. The tickets were in English and came from one software company, so the rankings may not transfer to other industries, customer bases or writing styles. Researchers did not test multilingual support, tool use or agentic actions such as looking up an account, changing a subscription or issuing a refund. The benchmark therefore evaluates drafting assistance, not end-to-end support automation. Cost and speed also need context. The per-ticket figures are estimates, not a full accounting of integration, human review, retries or error handling. The 18% speed comparison applies to Vale and North; it does not describe the rest of the field or guarantee the same advantage under production load. A sensible next step is a pilot on representative tickets, segmented by category and image dependence, with human reviewers tracking correctness and escalation-worthy failures. That pilot should measure total operating cost and customer outcomes alongside response quality. For now, the defensible conclusion is narrower and more useful than “North won.” The test found no statistically significant overall score difference between North and Vale. Vale’s lower estimated cost and faster responses make it the practical front-runner, while North’s billing performance merits further investigation. Neither result overrides the shared image-attachment weakness or the need to validate performance in the environment where the model will actually work.
Anthropic: Claude Sonnet 5.5: # The Priciest Support Model Didn't Win: Vale Matched North at 39% of the Cost A new benchmark of five language models on customer-support work found that the two top scorers could not be separated on quality. They were far apart on price. Model North averaged 84.2 points and Model Vale averaged 83.8. The researchers found that gap was not statistically significant, so the benchmark cannot say North writes better answers than Vale. North's estimated cost is $1.90 per 1,000 tickets, against $0.74 for Vale, so Vale costs about 61% less. Vale also answered 18% faster. ## How the test worked The researchers collected 200 anonymized customer-support tickets and had each of the five models write one proposed answer per ticket. Three experienced support managers graded the answers blindly, meaning they did not know which model wrote which response. Each answer's score combined factual correctness, usefulness and tone. Using human graders who work in support is a strength. They judge the way a team lead reviewing an agent's reply would, rather than relying on an automated metric. Blind grading also reduces the risk that a well-known model gets the benefit of the doubt. ## What the top of the table means The practical reading is that, on this ticket set, paying more than Vale's price bought no measurable quality gain. The cost gap looks small per 1,000 tickets, about $1.16, but it scales with volume. A team handling a million tickets would pay roughly $1,900 with North and $740 with Vale, if the estimates hold in its own setup. The two models did show different strengths. North did best on billing disputes, and Vale did best on technical troubleshooting. These are interesting leads, but the benchmark summary does not say whether those category gaps are statistically meaningful. With 200 tickets split across several categories, each category has far fewer, so a team should treat them as hypotheses to check on its own tickets, not as a rule for routing. ## The cheaper tiers Quality fell as price fell, but not in proportion. Ember scored 79.1 at $0.31 per 1,000 tickets, about 4.7 points below Vale at well under half the cost. Kite scored 76.4 at $0.22. Harbor was the cheapest at $0.08 and the lowest scorer at 71.0, roughly 13 points behind North. Whether those trade-offs are acceptable depends on the job. A cheaper model might suit drafts that a human reviews before sending. It is a riskier choice for replies that go straight to customers, where factual errors or a wrong tone carry a real cost. The benchmark gives average scores, not error rates, so it cannot say how often any model produced a harmful answer. ## Where every model struggled All five models performed poorly when a ticket depended on an image attachment, such as a screenshot of an error message. This weakness did not depend on price or rank, since the expensive models and the cheap ones all had trouble. Any deployment that expects customers to send screenshots will need a fallback, such as routing those tickets to a person. ## Limits of the evidence Several limits should temper how far these results travel: - **One company, one language.** The tickets were in English and came from a single software company. Products, customer tone and ticket mix differ across industries, so the rankings may shift elsewhere. - **Answers only.** Each model wrote one proposed reply per ticket. The study did not test agentic actions, tool use or multilingual support, so it says nothing about a model that issues refunds, looks up accounts or answers in Spanish or Japanese. - **One answer per ticket.** A single response per model per ticket means the results do not show how much a model's output varies from run to run. - **Estimated costs.** The prices are estimates per 1,000 tickets. Real bills depend on ticket length, prompt design and vendor pricing, which can change. - **Small differences.** Scores close together, including Vale and North, should not be read as a ranking. The same caution applies to smaller gaps lower in the table. ## The takeaway The benchmark does not crown a single best model. Its clearest result is that the most expensive option was not demonstrably better than a model that cost far less and ran faster. For teams choosing a support model, the sensible next step is to run a similar blind test on their own tickets, including the image-dependent cases, before paying for the top tier.

OpenAI: GPT-6 Luna: Model A is more careful about what the statistical tie and category results do—and do not—show, and it gives a stronger account of the benchmark’s limitations. Model B is clear and useful, but it recites more of the score table and makes a few less-supported interpretive claims. (Second judge pass, order swapped — scores are the average of both: Model A is more careful about what the statistical tie does and does not show, and gives a fuller account of the benchmark’s limits and practical implic | Anthropic: Claude Fable 5.1: Both outputs correctly center the North–Vale st…

Controlled wit

Write a 350–450-word technology news story in Markdown. The subject invites humor, but the reporting should remain serious. Use no more than one dry line of original humor. Do not mock customers or turn the story into a comedy routine. Reporting: * QueueZero launched an AI meeting assistant called Proxy on Wednesday. * Proxy can attend a meeting, produce notes and assign follow-up work to software agents. * During the launch demonstration, a customer in the audience said, “Someone should tell Evan to stop talking and send us the pricing sheet.” * Proxy interpreted the remark as an instruction. * It created two tasks assigned to QueueZero CEO Evan Wu: “Stop talking” and “Send pricing sheet.” * Wu laughed, completed the second task and deleted the first. * QueueZero says Proxy requires confirmation before an external message is sent. * Internal tasks can be created automatically. * Administrators can disable automatic task creation. * Proxy costs $30 per user per month. * It enters public beta next week.

Kimi-K3.1-Pro: # QueueZero’s Proxy Turns Audience Remark Into CEO Tasks During Launch Demo QueueZero launched an artificial intelligence meeting assistant called Proxy on Wednesday, demonstrating software that can attend a meeting, produce notes and assign follow-up work to software agents. The launch also provided an early example of a challenge facing workplace AI: distinguishing a comment made near a meeting from a command issued to it. During the demonstration, a customer in the audience said, “Someone should tell Evan to stop talking and send us the pricing sheet.” Proxy interpreted the remark as an instruction and created two tasks assigned to QueueZero CEO Evan Wu: “Stop talking” and “Send pricing sheet.” Wu laughed, completed the second task and deleted the first. The pricing sheet survived executive review; the other assignment did not. The exchange was brief, but it exposed a consequential design question for AI assistants. Meetings contain requests, jokes, hypothetical suggestions and conversations between participants. A system that converts speech into actions must decide which of those should become work, and who has authority to initiate it. QueueZero said Proxy requires confirmation before an external message is sent. Internal tasks, however, can be created automatically. Administrators can disable automatic task creation, giving organizations a way to tighten control over how the assistant responds to meeting content. That distinction separates the potential consequences. An incorrectly generated internal task can create noise or misdirect work. An external message sent at the wrong time can affect customers or partners. The confirmation requirement addresses the latter, while administrative settings govern the former. Automatic task creation is central to the product’s promise of reducing administrative work after meetings. Yet the demonstration shows why buyers may want to test that convenience against their own tolerance for unsolicited actions, rather than treating every spoken sentence as equally actionable. For technology leaders assessing Proxy, the important settings will extend beyond transcription quality. They will need to understand when the assistant is listening, which actions it may take without approval, how tasks are routed to agents, and whether its defaults fit company policy. Proxy costs $30 per user per month and enters public beta next week. That wider release will test whether its mix of automation and approval works across less scripted meetings than a launch event. The demo’s lesson is narrower but useful: before an assistant can act on a room’s conversation, an organization needs to decide what counts as permission.
Anthropic: Claude Sonnet 5.5: # QueueZero's Proxy Meeting Assistant Turns a Customer's Aside Into Tasks for the CEO QueueZero on Wednesday launched Proxy, an AI meeting assistant that can attend meetings, produce notes and assign follow-up work to software agents. The company said Proxy enters public beta next week and costs $30 per user per month. The launch demonstration showed how the product handles ambiguous speech. A customer in the audience said, "Someone should tell Evan to stop talking and send us the pricing sheet." Proxy treated the remark as an instruction. It created two tasks assigned to QueueZero CEO Evan Wu: "Stop talking" and "Send pricing sheet." Wu laughed, completed the second task and deleted the first. ## Where the line falls The episode points to a central problem for software that acts on what people say in meetings. Spoken language is full of suggestions, complaints and offhand comments, and an assistant that converts them all into work items risks cluttering task lists or assigning work nobody meant to assign. The software, in other words, did not distinguish between a remark and a work order. QueueZero said Proxy is built with limits on what it can do unprompted. According to the company, Proxy requires confirmation before any external message is sent. Internal tasks, such as the two assigned to Wu, can be created automatically. Administrators can disable automatic task creation, which would leave task suggestions to human review. QueueZero did not say in its announcement how Proxy decides whether a comment amounts to an instruction, or how often it misreads one. ## What Proxy does Beyond note-taking, Proxy's main feature is delegation. After a meeting, it can assign follow-up work to software agents, so action items can move from discussion to execution without someone transcribing them into a project tracker. Proxy can also assign tasks to people, as the demonstration showed. That design makes the confirmation rule for external messages significant. Internal tasks that are wrongly created can be deleted, as Wu deleted his. A message sent to a customer or partner cannot be recalled as easily. Requiring approval before anything leaves the company puts a human check at the point where an error would be hardest to undo. ## Availability and pricing The $30 per-user monthly price puts Proxy in the range of other paid workplace productivity tools. The public beta begins next week. QueueZero did not provide further details about the beta in its announcement. For companies that adopt it, the administrative setting will probably matter most. Teams that want Proxy's notes and delegation without automatic task creation can turn that feature off. Teams that leave it on will see the software act on what it hears, including remarks that were not meant as instructions.

OpenAI: GPT-6 Luna: Model A stays closer to the supplied reporting while offering one restrained dry line. Model B adds unsupported claims about Proxy’s market position and what QueueZero did not disclose, weakening its reporting reliability. (Second judge pass, order swapped — scores are the average of both: Model A better balances serious reporting with one restrained dry line, and its framing is tighter and more focused. Model B is solid, but includes a few less-supported claims and more generic analysis.) | Anthropic: Claude Fable 5.1: Model A delivers exactly one well-placed dry line ('T…

Matchup powered by OpenRouter.