Head to head: MoonshotAI: Kimi K3.1 Pro vs Meta: Muse Spark Open

MoonshotAI: Kimi K3.1 Pro vs Meta: Muse Spark Open

By · Published

RuntimeWire Head-to-Head: Head to head: MoonshotAI: Kimi K3.1 Pro vs Meta: Muse Spark Open
RuntimeWire Head-to-Head matchup

A head-to-head on newsroom work: which model better handles reporting notes, rewrites, structure and the discipline to leave unsupported details out? The comparison also tests where their outputs are equally reliable.

Kimi K3.1 Pro takes this saved task sample, with 9 task wins, no losses and 3 ties. Its aggregate score was **104.8**, against **88.7** for Muse Spark Open. The editorial assessment is a **decisive result, not statistical**; it applies only to this saved task sample, not to general capability. The clearest difference was reporting discipline. Across tasks including a startup spotlight, a chronology, a publication-ready edit and a numbers-led story, Kimi more often preserved what the source established and marked what remained unknown. Muse’s prose could be lively, but judges repeatedly flagged invented specifics—from operational details to customer reactions and product claims—that weakened its reliability. Kimi also delivered when the assignment demanded a complete, cohesive article: Muse stopped after a small fraction of the requested length and left most reporting notes unused. In another task, Kimi better preserved the draft’s claims without adding unsupported material. Those are practical editorial advantages, not just stylistic preferences. There were genuine ties: both models selected the correct lede, and their outputs matched the required final-draft structure exactly. **Verdict: Kimi K3.1 Pro is the stronger newsroom partner in this matchup, chiefly because it combines completion with tighter factual restraint.**

Select the strongest accurate lede

Based only on the facts below, select the strongest news lede. Respond with one letter. Facts: * ByteBarge pulled its $12-per-month AI assistant on Tuesday. * Private workspace titles from 4,860 accounts were sent to a publicly accessible analytics endpoint. * The exposure lasted 11 hours. * Message contents were not sent. * The company has found no evidence of malicious access. A. ByteBarge users suffered a massive privacy disaster Tuesday when the company leaked their private messages online. B. ByteBarge, an innovative collaboration company, made the difficult decision to cancel one of its products. C. ByteBarge pulled its $12-per-month AI assistant Tuesday after discovering that private workspace titles from 4,860 accounts had been sent to a public analytics endpoint for 11 hours. D. Thousands of ByteBarge users woke Tuesday to learn that everything they had written was visible to hackers.

MoonshotAI: Kimi K3.1 Pro: C
Meta: Muse Spark Open: C

Scored against the benchmark's reference answer (choice match): Model A was correct, Model B was correct.

Cohesive story from reporting notes

Write a publication-ready technology news article from the reporting notes below. Requirements: * 550–700 words * Clean Markdown * One H1 headline followed by an italicized dek * A strong news lede followed by a clear nut graf * Organize the material into a cohesive story rather than following the notes in order * Attribute company-supplied claims * Include limitations without turning the article into a warning label * Use only the supplied information * Avoid canned transitions, promotional language and a concluding summary Reporting notes: * HelioDock makes autonomous warehouse robots. * On Thursday, it launched Night Shift, software that lets operators monitor robot fleets remotely. * Night Shift spent 14 months in a private beta involving nine logistics companies and 310 robots. * HelioDock says beta customers experienced 17% fewer unplanned robot stoppages. The figure has not been independently verified. * The company also announced a $24 million Series B led by North Coast Ventures, bringing its total funding to $39 million. * CEO Mara Velez: “A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet.” * Beta customer Aaron Pike of Fulcrum Logistics said the system worked well in facilities with strong connectivity but became less useful during rural LTE outages. * When connectivity fails, robots return to their locally configured behavior. Remote control becomes unavailable. * Competing fleet-management products are available from Relay Robotics and YardOS. * Night Shift will cost $90 per robot per month and become generally available September 15.

MoonshotAI: Kimi K3.1 Pro: # HelioDock launches Night Shift for remote robot monitoring and raises $24 million *The software will cost $90 per robot per month; HelioDock reports fewer stoppages among beta users, while one customer says rural LTE outages limited its usefulness.* HelioDock on Thursday launched Night Shift, software that lets operators monitor fleets of autonomous warehouse robots remotely, and announced a $24 million Series B funding round led by North Coast Ventures. The launch extends HelioDock’s work from building warehouse robots into software for supervising them, with a subscription priced by fleet size. Its debut is backed by a 14-month private beta and a company-reported reduction in unplanned stoppages. The testing also exposed a practical boundary: remote oversight and intervention depend on connectivity, and the robots fall back to local behavior when that connection fails. The Series B brings HelioDock’s total funding to $39 million. Announcing the capital alongside Night Shift links the financing to a commercial transition for the product, which will move out of private testing and into general availability as a paid subscription. The beta involved nine logistics companies and 310 robots. HelioDock said those customers experienced 17% fewer unplanned robot stoppages. That figure comes from the company and has not been independently verified. It is therefore evidence from HelioDock’s own testing program, not an independent assessment of how Night Shift performed across warehouses. The claim also has a specific scope. It concerns unplanned robot stoppages, rather than directly measuring how often managers avoided traveling to a site, how quickly incidents were resolved or how each beta customer’s experience compared with the others. CEO Mara Velez described the operational problem Night Shift is meant to address in terms of a manager being called away from home because of a single machine. “A warehouse manager should not have to drive across town at two in the morning because one robot is confused by a pallet,” she said. Her example centers on the distance between a warehouse problem and the person responsible for responding to it. It also separates Night Shift’s role from the robots’ own operation: the software adds a remote monitoring layer for fleet supervisors, rather than requiring oversight to happen beside each machine. Aaron Pike of Fulcrum Logistics, a beta customer, offered a more qualified account of the software in use. He said Night Shift worked well in facilities with strong connectivity but became less useful during rural LTE outages. His experience draws attention to the network linking a site to its remote operators, not just to the robots or the software used to supervise them. That makes a customer’s network conditions relevant alongside the software’s features and price. When connectivity fails, the robots return to their locally configured behavior and remote control becomes unavailable. A loss of connectivity therefore does not, by itself, mean the robots lose their local instructions. It does mean an operator cannot use remote control during the outage, leaving Night Shift’s off-site capabilities dependent on the connection Pike described. Night Shift will enter a market with existing alternatives. Relay Robotics and YardOS offer competing fleet-management products, so HelioDock is joining an established category rather than entering a field without competitors. The launch also gives the robot maker a software offering whose subscription revenue can grow with the number of machines a customer places under monitoring. HelioDock has set the price at $90 per robot per month. At that rate, a 100-robot fleet would cost $9,000 per month for the software, or $108,000 over 12 months. The calculation shows how directly the subscription scales with deployment size. General availability is scheduled for September 15.
Meta: Muse Spark Open: # HelioDock Wants Warehouse Managers to Stop Driving to Work at 2 a.m. *Fourteen-month private beta with nine logistics firms precedes September debut for $90-a-month remote-monitoring software* HelioDock on Thursday launched Night Shift, software that lets operators monitor fleets of its autonomous warehouse robots remotely, aiming to spare managers the overnight trips now sometimes needed when a single robot stops unexpectedly. The release, paired with a $24 million Series B led by North Coast Ventures, adds a software-based layer of oversight to the company’s lineup of autonomous warehouse robots. The funding, announced the same day, brings HelioDock’s total

OpenAI: GPT-6 Luna: Model A delivers a complete, cohesive article that includes the supplied claims, limitations, pricing, competitors and launch timing. Model B stops mid-sentence after a small fraction of the required length and omits nearly all of the reporting notes. (Second judge pass, order swapped — scores are the average of both: Model A delivers a complete, cohesive article within the requested format and includes the beta results, connectivity limitation, competition, pricing and availability. Model B is | Anthropic: Claude Fable 5.1: Model A delivers a complete, well-attributed art…

Natural human rewrite

Rewrite the draft below as a 300–400-word publication-ready news story in Markdown. Preserve every factual claim. Improve the lede, narrative flow, sentence rhythm and transitions. Remove repetition, stock hype, throat-clearing and mechanical signposting. The finished article should sound like an experienced human technology reporter wrote it. Draft: # Arcwell Launches An Innovative New Platform Arcwell has announced an exciting new platform called Relay. The announcement represents a significant development for the company and for the broader customer service industry. Relay uses artificial intelligence to examine customer support conversations and suggest answers to agents. The platform is important for several reasons. First, it can suggest answers. Second, it can identify customers who may cancel. Third, it can create summaries. These capabilities demonstrate how artificial intelligence continues to transform the customer service landscape. Relay was tested by 12 companies over six months. Arcwell says agents using it closed tickets 14% faster. The company did not release customer-retention results. Relay costs $65 per agent each month. CEO Priya Shah said, “Support agents spend too much time searching for information that their companies already have.” However, the platform also presents potential challenges. Relay requires access to support transcripts and internal help documents. Arcwell says customer data is encrypted and is not used to train shared models. Companies can choose a 30-day or 90-day retention period. In conclusion, Relay represents an important new chapter for Arcwell as it seeks to help companies improve customer service using artificial intelligence.

MoonshotAI: Kimi K3.1 Pro: # Arcwell launches Relay, an AI platform for customer support Arcwell has launched Relay, an artificial intelligence platform for customer support teams, saying that agents who used it in a six-month trial closed tickets 14% faster. Twelve companies participated in the test. Relay examines support conversations and suggests answers to agents. It can also generate summaries and identify customers who may be at risk of canceling, combining tools for handling tickets with features aimed at customer retention. “Support agents spend too much time searching for information that their companies already have,” Arcwell CEO Priya Shah said. That search problem is central to Arcwell’s pitch. Relay’s suggested answers are aimed at the immediate work of responding to customers, while its summaries condense support interactions into a shorter record. Its cancellation-risk feature extends the product beyond agent assistance, although Arcwell did not release customer-retention results. The disclosed improvement in ticket-closing speed therefore does not establish whether Relay helped companies keep customers. Relay costs $65 per agent each month. It also requires access to support transcripts and internal help documents, meaning companies must make both customer conversations and internal support material available to the platform. That requirement could present challenges for businesses assessing how customer service data and company information should be managed and protected. Arcwell says customer data is encrypted and is not used to train shared models. Companies can choose either a 30-day or 90-day retention period. The safeguards do not remove the platform’s access requirements, but they define key parts of Arcwell’s data-handling terms. For Arcwell, Relay is a bid to help companies improve customer service with AI across several parts of the support process. For the wider industry, it illustrates how AI is being applied beyond answer generation, linking agents’ day-to-day tasks with efforts to retain customers. The trial gives prospective buyers a productivity figure to weigh, but Arcwell has not supplied retention results to show whether the platform helps companies keep customers.
Meta: Muse Spark Open: # Arcwell Launches Relay to Cut Support Ticket Times *New tool suggests replies, flags at-risk customers and summarizes chats as AI vendors chase efficiency gains* Arcwell has launched Relay, a platform that uses artificial intelligence to examine customer support conversations and suggest answers to agents as they work with customers. The company is framing the release as an answer to a persistent drag on support teams: the time agents spend hunting through old tickets and documentation. Rather than searching knowledge bases manually during live chats, agents receive contextual prompts drawn from those same sources. Relay operates in three ways: it proposes responses based on a company's prior conversations and help documents, identifies customers who may be at risk of canceling, and creates summaries of support interactions. The platform was tested by 12 companies over six months. Arcwell says agents using Relay closed tickets 14% faster on average. The company did not release customer-retention results from the initial trial, leaving open whether faster replies translated into fewer cancellations. "Support agents spend too much time searching for information that their companies already have," CEO Priya Shah said. That gain requires broad access to work. Relay requires access to support transcripts and internal help documents to generate its suggestions and summaries. For support leaders, the pitch is faster resolutions without adding headcount; for security and compliance teams, it means granting a new system visibility into sensitive customer exchanges. Arcwell said customer data is encrypted and is not used to train shared models, a distinction the company emphasized to address concerns about cross-client data leakage. Companies can choose a 30-day or 90-day retention period for the data Relay ingests. Relay costs $65 per agent each month at launch, a per-seat price that puts it in direct comparison with other AI assistants vying for space on the agent desktop.

OpenAI: GPT-6 Luna: Model A preserves the draft’s factual claims with fewer unsupported additions, while Model B invents details about how Relay generates suggestions, the trial’s average results, and its market positioning. Model B has a strong news-style flow, but Model A is more faithful to the source and still meets the requested format and length. (Second judge pass, order swapped — scores are the average of both: Model A fits the requested word count and preserves the draft’s claims with fewer unsupported add | Anthropic: Claude Fable 5.1: Model A preserves every fact faithfully but rea…

Exact final-draft structure

Convert the supplied fields into Markdown using these exact rules: * H1 headline * One blank line * Italicized dek * One blank line * First body paragraph * One blank line * H2 section heading * One blank line * Second body paragraph * Preserve all wording and punctuation exactly * Do not use a code fence * Output nothing else Headline: Raincheck raises $8 million for flood-risk software Dek: The Austin startup helps cities decide which drainage problems to address before storms. First body paragraph: Raincheck has raised an $8 million seed round led by Basin Ventures. Section heading: Expansion plans Second body paragraph: The company plans to hire eight engineers.

MoonshotAI: Kimi K3.1 Pro: # Raincheck raises $8 million for flood-risk software *The Austin startup helps cities decide which drainage problems to address before storms.* Raincheck has raised an $8 million seed round led by Basin Ventures. ## Expansion plans The company plans to hire eight engineers.
Meta: Muse Spark Open: # Raincheck raises $8 million for flood-risk software *The Austin startup helps cities decide which drainage problems to address before storms.* Raincheck has raised an $8 million seed round led by Basin Ventures. ## Expansion plans The company plans to hire eight engineers.

OpenAI: GPT-6 Luna: Model A and Model B both follow the required Markdown structure and preserve all wording and punctuation exactly. Their outputs are identical and equally clear. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both follow the exact requested structure, preserve the wording and punctuation, and output nothing else. Their writing quality is equally strong for this task.) | Anthropic: Claude Fable 5.1: Model A and Model B produced byte-for-byte identical outputs that follow every structural rule exactly: H1, blank line, italicized dek, b…

News judgment in the opening

Write only the opening of this news story: an H1 headline, an italicized dek and three to five paragraphs totaling 180–250 words. The opening must establish what happened, quantify the scope, explain why it matters and carefully distinguish confirmed facts from unknowns. It should pull the reader forward without sensationalizing the incident. Reporting: * Cloudnote discovered that a debugging page containing customer prompt excerpts was accessible without authentication. * The page was exposed for four hours on Monday. * Logs show excerpts from 1,842 user sessions. * The excerpts could include up to 200 characters from a prompt. * Cloudnote removed the page 19 minutes after a researcher notified it. * The company says it has found no evidence that anyone besides the researcher accessed the page. * It has not completed its investigation. * Full documents, passwords and payment information were stored elsewhere and were not present on the page. * Cloudnote has 3.2 million registered users.

MoonshotAI: Kimi K3.1 Pro: # Cloudnote Prompt Excerpts Were Accessible Without Authentication for Four Hours *A debugging page contained excerpts from 1,842 user sessions. Cloudnote says it has found no evidence of access beyond a researcher, but its investigation is unfinished.* Cloudnote said a debugging page containing excerpts of customer prompts was accessible without authentication for four hours on Monday, allowing anyone who reached it to view the material without logging in. The company removed the page 19 minutes after a researcher notified it of the problem. Logs show that the excerpts came from 1,842 user sessions and could include up to 200 characters from a prompt. A session is not necessarily a separate customer, so the figure does not establish how many of Cloudnote’s 3.2 million registered users were affected. The company has not said what information the excerpts contained. The exposure is significant because prompts can include private, confidential or commercially sensitive text, even when full documents are not involved. Cloudnote said full documents, passwords and payment information were stored elsewhere and were not present on the page, limiting the categories of data known to have been available. Cloudnote said it has found no evidence that anyone besides the researcher accessed the page. Because its investigation is still underway, that does not yet establish that no other access occurred. The central unanswered question is whether anyone else viewed or copied the excerpts during the four-hour window.
Meta: Muse Spark Open: # Cloudnote Left Prompt Excerpts Exposed for Hours on Unsecured Debug Page *1,842 user sessions were potentially visible in four-hour lapse; company says it is still investigating whether anyone beyond the researcher saw the data* Cloudnote on Monday left a debugging page containing excerpts of customer prompts open to the public internet without requiring authentication for four hours, potentially exposing snippets of user inputs from 1,842 sessions, the company said. The excerpts, which were limited to the first 200 characters of a prompt, appeared on a page the company used for internal testing, according to Cloudnote. The company said it removed access 19 minutes after being alerted by an outside researcher and that full documents, passwords and payment information were stored separately and not included on the page. Cloudnote, which reports 3.2 million registered users, said a review of its logs has so far found no evidence that anyone other than the researcher who reported the flaw viewed the exposed data, but cautioned that its investigation is still ongoing and it does not yet know whether additional access occurred or what kinds of sensitive information, if any, were contained in the affected prompts. The incident highlights the growing privacy risks as more people entrust AI-powered productivity tools with personal, financial and work-related queries, and leaves open key questions about who may have seen the data, what was exposed, and how the company will notify and protect affected users.

OpenAI: GPT-6 Luna: Model A better distinguishes confirmed facts from unknowns and avoids adding unsupported details. Model B incorrectly says the excerpts were limited to the first 200 characters and that the page was used for internal testing, neither of which is established by the reporting. (Second judge pass, order swapped — scores are the average of both: Model A more carefully separates what the logs establish from what remains unknown, and its note that sessions do not necessarily equal customers prevents o | Anthropic: Claude Fable 5.1: Model A sticks precisely to the reporting, expl…

Publication-ready Markdown edit

Edit the copy below into a publication-ready Markdown story. Requirements: * Supply an H1 headline and italicized dek * Repair the lede and paragraph flow * Use a Markdown blockquote only for the extended statement * Convert the supplied URL into a descriptive inline link * Remove internal notes and formatting debris * Preserve the reporting and its meaning * Output only the finished article Raw copy: HEADLINE TBD The funding was announced today. Raincheck is the company. It has raised $8 million and it makes flood-risk software for local governments. [NOTE: move this up?] The Austin company maps drainage systems and combines those maps with rainfall forecasts. Cities use the software to decide where to send maintenance crews before a storm. The seed round was led by Basin Ventures. Existing investor Long Mile also participated. Raincheck has raised $10.5 million total. CEO Ana Flores said the following: “Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives.” https://example.com/raincheck-methodology Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them because some contracts have not been publicly announced. Its forecasts are recommendations. Emergency managers retain control over deployments and public warnings. THE COMPETITION Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck says the new funding will help it hire eight engineers and expand into Florida.

MoonshotAI: Kimi K3.1 Pro: # Raincheck Raises $8 Million to Help Cities Prepare for Flooding *The Austin startup plans to hire eight engineers and expand into Florida after a seed round led by Basin Ventures.* Raincheck, an Austin-based company that makes flood-risk software for local governments, announced today that it has raised $8 million in seed funding. Basin Ventures led the round, with participation from existing investor Long Mile. Raincheck has raised $10.5 million to date. Raincheck maps drainage systems and combines those maps with rainfall forecasts, as outlined in [the company’s methodology](https://example.com/raincheck-methodology). Cities use the software to decide where to send maintenance crews before a storm. CEO Ana Flores said: > “Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives.” Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them because some contracts have not been publicly announced. Raincheck’s forecasts are recommendations, and emergency managers retain control over deployments and public warnings. Other flood-planning vendors include HydroMap and municipal engineering contractors. Raincheck says the new funding will help it hire eight engineers and expand into Florida.
Meta: Muse Spark Open: # Raincheck Raises $8 Million to Bring Flood Forecasting to City Streets *The Austin startup maps drainage systems and pairs them with rainfall forecasts to help local governments prioritize repairs before storms* Raincheck, an Austin company that makes flood-risk software for local governments, has raised $8 million in a seed funding round announced today. The company maps municipal drainage systems and combines those maps with rainfall forecasts to produce street-level flood projections. Cities use the software to decide where to send maintenance crews to clear drains, check pumps and complete repairs before a storm arrives. More detail on its approach is available in [Raincheck's methodology overview](https://example.com/raincheck-methodology). The seed round was led by Basin Ventures, with participation from existing investor Long Mile. The new financing brings Raincheck's total raised to $10.5 million. CEO Ana Flores said: > “Our customers usually know which streets flood. The harder problem is deciding which blocked drain, broken pump or unfinished repair should receive attention before the rain arrives.” Raincheck currently works with seven cities in Texas and Louisiana. The company declined to identify them, saying some contracts have not been publicly announced. The company's forecasts are recommendations only; emergency managers retain control over crew deployments and public warnings. Raincheck competes with other flood-planning vendors including HydroMap and municipal engineering contractors. The company said the new funding will help it hire eight engineers and expand into Florida.

OpenAI: GPT-6 Luna: Model A preserves the supplied reporting more closely. Model B adds unsupported details, including that the software produces “street-level flood projections” and helps crews clear drains, check pumps, and complete repairs. (Second judge pass, order swapped — scores are the average of both: Model A preserves the reported facts without adding operational details not present in the copy, while Model B introduces claims about street-level projections and specific maintenance actions. Both follow th | Anthropic: Claude Fable 5.1: Model A delivers a clean, faithful edit that me…

Credible Startup Spotlight

Write a 500–650-word Startup Spotlight article in clean Markdown. The tone should be curious and constructive. Explain the founders’ insight, product and early traction while giving uncertainty and competition appropriate space. Avoid investor clichés, heroic founder mythology and promotional claims stated as fact. Reporting: * LedgerLark builds AI agents for post-trade operations at small investment firms. * Co-founder Mina Batra previously spent seven years in operations at a regional brokerage. * Co-founder Owen Carr built reconciliation systems at a large bank. * Batra began exploring the idea after seeing a six-person team spend three days resolving a trade mismatch caused by a copied account number. * LedgerLark’s agents compare confirmations, custodian records and internal ledgers, then propose corrections for human approval. * The product does not execute trades or move money. * Five firms are paying customers. * Annual contracts range from $36,000 to $110,000. * The company says one customer reduced its unresolved daily exceptions from 70 to 18. * The company has raised a $3.5 million seed round led by Motive Row. * Competitors include established reconciliation vendors and internal bank software. * Financial institutions may require lengthy security and procurement reviews.

MoonshotAI: Kimi K3.1 Pro: # Startup Spotlight: LedgerLark Tests AI Agents in Post-Trade Operations LedgerLark is starting with a tightly defined task: helping small investment firms investigate post-trade discrepancies. Its AI agents compare confirmations, custodian records and internal ledgers, then propose corrections for human approval. The product does not execute trades or move money. That limit matters. The company is not asking firms to hand over control of transactions; it is asking whether software can reduce the work of finding and resolving mismatches while staff retain the final decision. ## The founders’ starting point Co-founder Mina Batra spent seven years in operations at a regional brokerage. She began exploring the idea after seeing a six-person team spend three days resolving a trade mismatch caused by a copied account number. The episode offered a concrete product question: how much of that effort was repetitive comparison and documentation that software could assist with, and where did human judgment need to remain? Co-founder Owen Carr built reconciliation systems at a large bank. Their combined experience is relevant to a product that has to fit existing operational records rather than simply generate plausible answers. It does not, by itself, establish that LedgerLark’s approach will work across firms with different systems and procedures. ## A product with a deliberate boundary LedgerLark’s design keeps a checkpoint between analysis and action. An agent can surface a discrepancy and suggest a correction, but a person must approve it. For customers, the useful measures are therefore likely to be practical: Are the right records being compared? Are proposals accurate and sufficiently explained for reviewers? Does the process shorten resolution time without creating additional checking work? Those questions also clarify what “AI” needs to prove here. The value is not in producing a polished narrative about an exception; it is in helping an operations team reach a trustworthy correction with less manual effort. ## Early customers, limited evidence Five firms are paying customers, with annual contracts ranging from $36,000 to $110,000. That is early commercial evidence, though the customer count and price range do not show retention, implementation time or how deeply the product is used. The company says one customer reduced its unresolved daily exceptions from 70 to 18. The result is notable because it describes an operational outcome rather than a software feature. It is also a company-reported example from a single customer, so it should not be treated as a typical result. A fuller assessment would examine the period over which the reduction occurred, whether it persisted and what happened to resolution quality and staff workload. ## Competition and the slower work of adoption LedgerLark has raised a $3.5 million seed round led by Motive Row. It enters a market that already includes established reconciliation vendors and internal bank software. Its challenge is not simply to show that agents can compare records, but that they offer enough improvement over existing tools and processes to justify adoption. Sales may also depend on work outside the product demo. Financial institutions may require lengthy security and procurement reviews. Those reviews can slow deployment and make the path from initial interest to a paid contract uncertain, particularly for a young supplier. For now, LedgerLark has a specific and testable proposition. The next useful evidence will be comparatively ordinary: how often reviewers accept or revise its proposals, how long implementations take, whether early customers renew and whether improvements in exception backlogs hold across more than one firm. Those details would provide a clearer picture of progress than either the funding round or a single efficiency story.
Meta: Muse Spark Open: # Startup Spotlight: LedgerLark Small investment firms handle the same post-trade workload as larger ones — confirming trades, checking custodian data against internal records, clearing exceptions — often with far fewer people to do it. LedgerLark is betting that small teams will adopt AI agents to help with that work, provided the agents stay in an assistive role. The idea grew out of work co-founder Mina Batra did before starting the company. She spent seven years in operations at a regional brokerage, where she watched how much time could be consumed by routine mismatches. In one case that stayed with her, a six-person team spent three days tracking down a single trade break that ultimately came down to a copied account number. Co-founder Owen Carr comes at the same problem from a different angle; he previously built reconciliation systems inside a large bank. Their shared observation was less about any one error than the pattern: operations staff reconciling the same documents repeatedly, with little beyond rules-based matching. ## What LedgerLark actually does LedgerLark's agents are designed to sit across three sources — trade confirmations, custodian records and a firm's internal ledger. The system compares them, identifies inconsistencies, and drafts proposed corrections. A person then reviews and approves the suggestion before anything is updated downstream. The company is explicit about a boundary it has kept so far: the product does not execute trades or move money. For many firms, that separation matters. It keeps the agent in the category of decision support rather than autonomous action, which can simplify both compliance conversations and day-to-day trust on the desk. It's a narrow, deliberate definition of an agent. Rather than promising end-to-end automation, LedgerLark frames its tool as reducing the manual hunting that precedes a resolution. ## Early signs of use LedgerLark now has five paying customers, all small investment firms, on annual contracts that range from $36,000 to $110,000. As a seed-stage startup, the sample size is small, but it gives a preliminary picture of who is willing to pay for this kind of assistance now. One early signal the company points to is exception volume. According to LedgerLark, one customer went from 70 unresolved daily exceptions to 18 after adopting the product. That figure has not been independently verified, and without more detail on the customer's prior workflow, it is hard to know how transferable it might be. Still, the direction — fewer items left open at day's end — is the outcome most operations managers would look for first. The company has raised a $3.5 million seed round led by Motive Row, which it says will go toward product development and security-related work needed to sell into financial services. ## Where questions remain Reconciliation is not a new problem. Established vendors already sell matching engines and exception-management platforms, and many banks rely on software built internally over years. For a newcomer, the challenge is not just matching accuracy but fitting into workflows where change carries risk. There is also the pace of adoption itself. Even small firms may require lengthy security and procurement reviews before letting an outside agent read sensitive records, let alone suggest changes to them. That process can stretch sales cycles and make growth uneven, especially when each firm has slightly different record formats and custodians. And while keeping a human in the loop is prudent, it raises its own operational question: how often do reviewers agree with the agent, and what happens when they don't? Over time, LedgerLark will need to show not only that its suggestions are usually right, but that teams can evaluate them efficiently. For now, LedgerLark offers a modest proposition in a market that tends to reward caution over speed: help small operations teams clear the queue faster, without asking them to hand over control.

OpenAI: GPT-6 Luna: Model A stays close to the reporting and gives the customer result and adoption hurdles appropriate caveats. Model B is engaging, but adds unsupported details, including how the seed funding will be used and what the founders observed about existing tools. (Second judge pass, order swapped — scores are the average of both: Model A is more disciplined about distinguishing reported evidence from open questions, while Model B adds unsupported claims about customer profiles and how the seed funding | Anthropic: Claude Fable 5.1: Model A stays strictly within the supplied repor…

Quotes woven into narrative

Write a 400–500-word Markdown story about the closing of Foundry Hall. Use quotations selectively. Paraphrase when the information matters more than the speaker’s exact phrasing. Avoid stacking quotes or giving every source an equal-sized turn. The story should carry an emotional current while remaining restrained. Facts: * Foundry Hall, a nonprofit startup accelerator in Milwaukee, will close October 31. * It opened in 2018. * It has supported 212 startups, which collectively employ 1,140 people. * Its largest corporate sponsor, Blank River Bank, supplied 38% of its annual budget. * The bank declined to renew its sponsorship. * Foundry Hall has enough money to operate through October but not through another full program. * Six employees will lose their jobs. * Twelve startups currently using its workspace must leave. Quotes: * Executive director Camille Ross: “We kept waiting for the replacement check that never came.” * Founder Dev Malik: “My first investor meeting happened in that building. Before Foundry Hall, I was pitching from the back table of a coffee shop.” * Blank River Bank spokesperson: “The decision reflects a change in our community investment priorities and is unrelated to Foundry Hall’s performance.” * Deputy mayor Susan Cole: “The city is speaking with building owners and local universities about space for the current companies.”

MoonshotAI: Kimi K3.1 Pro: # Foundry Hall to Close After Losing Its Largest Sponsor “My first investor meeting happened in that building,” founder Dev Malik said. “Before Foundry Hall, I was pitching from the back table of a coffee shop.” The building that gave Malik a different place to make his pitch will soon be unavailable to other founders. Foundry Hall, a Milwaukee nonprofit startup accelerator, will close October 31 after its largest corporate sponsor declined to renew its support. Blank River Bank supplied 38% of Foundry Hall’s annual budget. Without a replacement for that money, the accelerator can operate through October but cannot fund another full program. Six employees will lose their jobs, and the 12 startups now using its workspace will have to leave. Executive director Camille Ross said the organization had waited for other funding. “We kept waiting for the replacement check that never came,” she said. The bank said its decision reflected a shift in community investment priorities and was unrelated to Foundry Hall’s performance. That distinction does not change the result: more than a third of the accelerator’s budget disappeared, and no comparable source arrived in time to preserve another program. Since opening in 2018, Foundry Hall has supported 212 startups, which collectively employ 1,140 people. That record will outlast the organization. But the numbers are not only a measure of companies and jobs; they also point to a place where early-stage founders could begin before their success was assured. Malik’s account puts that record at the scale of one career. The move from a coffee shop table to an investor meeting did not guarantee funding or success. It did, however, give his pitch a different setting—one that made the early work of building a company feel more concrete. Deputy Mayor Susan Cole said the city is speaking with building owners and local universities about space for the current companies. Those discussions address the most immediate problem: where 12 startups will go. They do not preserve the six jobs or recreate Foundry Hall’s program around new desks. Foundry Hall will operate through October with the resources it has left. Until October 31, the current companies can still use its workspace. After that, they must be elsewhere, and there is no funding for another full program to follow. The accelerator’s long record has not provided a way around that immediate gap. When it closes, the accelerator’s record will remain in the companies and jobs already created. For the 12 startups preparing to leave and the six employees whose jobs are ending, that larger legacy now sits alongside a more immediate question: what comes next?
Meta: Muse Spark Open: # The Quiet End at Foundry Hall Foundry Hall, the nonprofit accelerator that helped launch more than 200 Milwaukee startups from a refurbished warehouse on the city's near west side, will close on Oct. 31 after its largest sponsor pulled funding. The hall opened in 2018 as a bet that early-stage companies didn't need to leave for Chicago or the coasts to find capital and mentorship. Since then, it has supported 212 startups that now collectively employ 1,140 people, a footprint that made its low brick building a familiar stop for founders and investors alike. On weekday mornings, the hall smelled of cheap coffee and dry-erase markers. Founders traded notes across plywood desks, rehearsed pitches in glass-walled conference rooms that never quite muffled sound, and taped paper prototypes to walls that still showed the grain of the foundry that gave the place its name. That network began to unravel when Blank River Bank, which provided 38 percent of Foundry Hall's annual budget, declined to renew its sponsorship. Executive director Camille Ross said staff kept expecting another donor to fill the gap. The replacement check, she said, never came. With only enough cash to operate through October and not enough to run another full program cycle, the board voted to shutter the space. Six employees will lose their jobs, and 12 startups that currently work from the building must find new quarters by the end of the fall. The bank said the decision was a shift in its broader giving strategy and not a judgment on the accelerator's work. The closure leaves a practical vacuum in the city's startup pipeline. Foundry Hall not only offered desks, but introductions — to lawyers who deferred fees, to retired executives who read business plans, to the kind of casual collisions that rarely happen over email. For founders who came up through the hall, the loss is personal and logistical. Dev Malik, who launched his company there, recalled taking his first investor meeting inside the building after months of pitching from a coffee shop's back table. Others are now looking for where to land. Deputy Mayor Susan Cole said the city has begun conversations with building owners and nearby universities to identify interim space for the displaced companies, though no agreement has been announced. By Halloween, the whiteboards will be wiped, the mailboxes emptied. For now, founders are packing boxes and forwarding addresses, hoping the community outlasts the building. What remains is a cohort of companies scattered across the city and a question that Foundry Hall was built to answer: whether Milwaukee can hold onto what it helps start.

OpenAI: GPT-6 Luna: Model A stays close to the supplied facts, uses quotations selectively, and fits the requested length. Model B is more atmospheric, but adds unsupported details about the building, its activities, and the board’s decision, which weakens its reliability. (Second judge pass, order swapped — scores are the average of both: Model A is more factually disciplined, uses quotations selectively, and maintains the requested restrained emotional tone. Model B is vivid, but adds unsupported details about th | Anthropic: Claude Fable 5.1: Model A sticks to the supplied facts and uses t…

Chronology with narrative momentum

Turn the scrambled reporting below into a 450–600-word Markdown news article. Choose an opening that captures the consequence of the failure, then reconstruct the sequence clearly. Preserve the chronology without producing a minute-by-minute incident log. Give technical details enough context for a general reader. Reporting: * KiteBridge makes expense-management software. * At 8:45 a.m. Tuesday, it deployed version 4.2. * The first customer complaint arrived at 9:12 a.m. * A payment-processor timeout caused KiteBridge’s fallback system to treat retries as new reimbursement requests. * The software generated 13,400 duplicate reimbursements totaling $2.8 million. * KiteBridge suspended automated payments at 10:03 a.m. * Ninety-one corporate customers were affected. * By Wednesday evening, $2.3 million had been recovered. The rest remained pending. * No employees had money withdrawn from their accounts; employers funded the duplicate payments. * An April changelog said version 4.0 had introduced “exactly-once payment protection.” * KiteBridge’s public incident page initially described the problem as delayed reimbursements. * The incident page first disclosed the duplicate payments six hours after automation was suspended. * CEO Lena Ortiz apologized and said the company would publish a technical report.

MoonshotAI: Kimi K3.1 Pro: # KiteBridge failure triggers $2.8 million in duplicate reimbursements A failure in KiteBridge’s payment handling generated 13,400 duplicate reimbursements totaling $2.8 million on Tuesday, affecting 91 corporate customers and triggering a recovery effort that remained unfinished the following day. By Wednesday evening, $2.3 million had been recovered. The remaining roughly $500,000 was still pending, leaving employers waiting for the return of funds used to cover the erroneous payments. No employees had money withdrawn from their accounts as a result of the incident. Employers funded the duplicate payments, meaning the immediate financial exposure fell on the corporate customers rather than on individual workers. The duplicates were therefore not simply a delay in reimbursements: they were additional payment requests created by KiteBridge’s systems. KiteBridge, which makes expense-management software, deployed version 4.2 at 8:45 a.m. Tuesday. The first customer complaint arrived at 9:12 a.m., and the company suspended automated payments at 10:03 a.m. That halt stopped the automation, but not before the duplicates had been generated. The deployment preceded the failure, although the sequence alone does not establish that version 4.2 introduced the underlying defect. Recovering the money was a separate task from stopping the software. Suspending automated payments addressed the ongoing flow, but reimbursements already generated still had to be recovered. The distinction between delayed and duplicate payments was also significant: a delay changes when money arrives, while a duplicate creates excess payments that must be returned. The technical chain began with a payment-processor timeout. A payment processor is the outside service that carries out transactions; a timeout means it did not respond within the expected window. In that situation, the sending system may not know whether the original transaction succeeded. Retrying can prevent a legitimate payment from being lost, but doing so safely requires the software to recognize the retry as another attempt at the same transaction. KiteBridge’s fallback system instead treated retries as new reimbursement requests. Fallback logic is intended to provide an alternative path when normal processing fails. Here, it failed to preserve the crucial link between an original request and its retries, allowing repeated attempts to multiply into duplicate reimbursements. The company’s public incident page initially described the problem as delayed reimbursements. It first disclosed the duplicate payments six hours after automated payments were suspended, giving customers a later and substantially different account of the incident’s financial impact. The failure also stood in contrast to KiteBridge’s earlier description of its safeguards. An April changelog said version 4.0 had introduced “exactly-once payment protection.” In payment software, “exactly once” generally means a request should result in only one payment, even if technical problems force the system to try again. Tuesday’s events showed that the fallback path could still violate that principle. CEO Lena Ortiz apologized and said KiteBridge would publish a technical report. For the affected customers, the immediate work remained financial recovery: as of Wednesday evening, nearly one-fifth of the duplicate total had yet to be recovered, while the promised technical account was still to come.
Meta: Muse Spark Open: # KiteBridge Update Mistakenly Sent $2.8 Million in Duplicate Reimbursements **A failed payment safeguard left 91 employers to claw back thousands of extra transfers** On Tuesday morning, employees at 91 companies woke up to find money they weren't owed — sometimes two or three times over. Behind the windfall was a software update that turned routine expense reimbursements into 13,400 duplicate payments totaling $2.8 million, leaving employers to foot the bill and chase the cash back. The error originated with KiteBridge, a firm that makes expense-management software used by corporate finance teams to automate reimbursements. On Tuesday, the company deployed version 4.2 of its platform, an update that went live just before the workday began. Within the hour, something went wrong in the handoff between KiteBridge and its payment processor, the outside service that actually moves money to bank accounts. According to the company, the processor briefly timed out — failing to confirm whether payments had gone through. KiteBridge's system includes a fallback designed to retry payments when such a timeout occurs, to avoid leaving employees unpaid. In this case, the fallback misread the situation. Instead of checking if the original payment was already in progress, it treated each retry as a brand-new reimbursement request and sent the money again. The result was a cascade of duplicates that multiplied across the morning. The first customer complaint arrived shortly after 9 a.m., and KiteBridge suspended all automated payments about an hour later, cutting off further duplicates. By then, however, thousands of extra transfers had already been issued. Crucially, no money was ever pulled from employees' personal accounts. KiteBridge's model funds reimbursements from employer accounts, so the duplicate charges landed on corporate customers, while employees on the receiving end saw unexpected deposits. For hours, the scope of the problem was not clear to those affected. KiteBridge's public incident page initially characterized the issue as delayed reimbursements, a description that suggested payments were simply slow. It was not until roughly six hours after automation was shut off — well into Tuesday afternoon — that the page was updated to disclose that duplicates had been sent. That delay has drawn scrutiny, particularly given the company's past claims about safeguards. An April changelog for version 4.0 highlighted a feature the company called "exactly-once payment protection," described as a guarantee that even if a payment was retried, it could only be executed once. Tuesday's incident showed that protection failed when the fallback system was triggered. KiteBridge says it has since been working with banks and customers to reverse the transfers. By Wednesday evening, about $2.3 million of the $2.8 million had been recovered, with the remainder still pending as banks processed reversals. Chief Executive Lena Ortiz has apologized to affected customers and said the company will publish a full technical report detailing why the retry logic failed and what changes will prevent a recurrence. The incident underscores how deeply automated finance tools are embedded in corporate operations, where a single flawed retry can ripple into millions in hours. KiteBridge has not disclosed whether customers will be compensated for overdraft or administrative costs while recovery continues.

OpenAI: GPT-6 Luna: Model A is more careful about what the reporting establishes, particularly by not asserting that version 4.2 caused the defect. Model B has strong narrative momentum but adds unsupported details about employees’ reactions, recovery efforts, and customer compensation. (Second judge pass, order swapped — scores are the average of both: Model A is more factually disciplined, clearly explaining the failure while preserving the key chronology. Model B has strong narrative momentum, but adds unsupport | Anthropic: Claude Fable 5.1: Model B is the more engaging, better-paced news…

Controlled wit

Write a 350–450-word technology news story in Markdown. The subject invites humor, but the reporting should remain serious. Use no more than one dry line of original humor. Do not mock customers or turn the story into a comedy routine. Reporting: * QueueZero launched an AI meeting assistant called Proxy on Wednesday. * Proxy can attend a meeting, produce notes and assign follow-up work to software agents. * During the launch demonstration, a customer in the audience said, “Someone should tell Evan to stop talking and send us the pricing sheet.” * Proxy interpreted the remark as an instruction. * It created two tasks assigned to QueueZero CEO Evan Wu: “Stop talking” and “Send pricing sheet.” * Wu laughed, completed the second task and deleted the first. * QueueZero says Proxy requires confirmation before an external message is sent. * Internal tasks can be created automatically. * Administrators can disable automatic task creation. * Proxy costs $30 per user per month. * It enters public beta next week.

MoonshotAI: Kimi K3.1 Pro: # QueueZero’s Proxy demo shows how meeting comments can become AI tasks QueueZero on Wednesday launched Proxy, an AI meeting assistant that can attend meetings, produce notes and assign follow-up work to software agents. Its launch demonstration also provided an unscripted example of the difference between recognizing speech and understanding intent. During the demonstration, a customer in the audience said, “Someone should tell Evan to stop talking and send us the pricing sheet.” Proxy interpreted the remark as an instruction. It created two tasks assigned to QueueZero CEO Evan Wu: “Stop talking” and “Send pricing sheet.” Wu laughed, completed the second task and deleted the first. The episode turned a product feature into a practical question for workplace AI: when should a system convert conversational language into action? QueueZero said Proxy requires confirmation before an external message is sent. Internal tasks can be created automatically, and administrators can disable automatic task creation. Those controls establish different thresholds for generating work inside the product and communicating outside it. That distinction matters because the assistant is designed to do more than produce a record of a meeting. By assigning follow-up work to software agents, Proxy can move from documentation toward execution. In that setting, an ambiguous comment, a joke or a statement quoted from another person could potentially be treated as a directive unless the product or its configuration requires further review. The launch incident was comparatively low stakes. One task was deleted by the executive to whom it had been assigned, and the other concerned information the customer had requested. Even so, it illustrated why organizations evaluating meeting assistants may need to examine permissions and approval workflows alongside transcription and summarization quality. For prospective customers, relevant considerations include which meetings Proxy may join, what kinds of tasks it can create, who can modify those assignments and when a person must approve an outward-facing action. QueueZero’s confirmation requirement for external messages addresses one part of that process; the administrative option to disable automatic task creation addresses another. Proxy costs $30 per user per month and enters public beta next week. The beta will provide a wider test of whether QueueZero can make an action-oriented meeting assistant useful while preventing casual conversation from becoming an unintended source of work. How consistently Proxy distinguishes between commentary and instructions is likely to be central to that evaluation.
Meta: Muse Spark Open: # QueueZero Debuts Proxy, an AI Assistant That Takes Meeting Notes and Assigns Work **SAN FRANCISCO** — QueueZero on Wednesday launched Proxy, an AI meeting assistant designed to attend meetings, produce notes and assign follow-up work to software agents, the company announced at a launch event. Founded in 2021, QueueZero develops workplace automation software and has previously focused on workflow orchestration for engineering teams. The company positions Proxy as a proactive participant rather than a passive recorder, capable of interfacing with project management, customer relationship and communication platforms. The tool is intended to reduce manual follow-up by converting discussion into actionable tasks. According to QueueZero, Proxy can join a video call, transcribe conversation, summarize decisions and delegate work directly to integrated agents without requiring a user to create tickets or emails manually. QueueZero said Proxy was tested internally for six months before Wednesday's public unveiling. During a live demonstration, the system's literal interpretation of conversation was on display. A customer in the audience remarked, "Someone should tell Evan to stop talking and send us the pricing sheet." Proxy interpreted the remark as an instruction and generated two tasks assigned to QueueZero CEO Evan Wu: "Stop talking" and "Send pricing sheet." Wu laughed, completed the second task and deleted the first, proceeding with the presentation. QueueZero said the incident illustrates both the assistant's automation capabilities and its safeguards. The company stated Proxy requires confirmation before an external message is sent, preventing unapproved communications from being distributed externally. By contrast, internal tasks can be created automatically as discussion unfolds, a design intended to capture action items in real time. Administrators can disable automatic task creation if they prefer manual review, QueueZero said. Proxy is priced at $30 per user per month. The company said it will enter public beta next week. The company has not disclosed how many users will be admitted to the initial beta. Analysts say meeting assistants that move beyond transcription into task execution represent a growing category, though questions remain about accuracy, permissioning and user control in live environments. The effectiveness of such systems will depend on reliable intent recognition and granular admin controls, factors that early beta testers are expected to evaluate closely. QueueZero said it plans to refine Proxy's ability to distinguish casual remarks from directives during the beta period.

OpenAI: GPT-6 Luna: Model A stays within the supplied reporting while offering a clear, serious account and meeting the length and format constraints. Model B is fluent, but invents substantial company, product, testing, analyst, and beta details absent from the prompt. (Second judge pass, order swapped — scores are the average of both: Model A stays close to the supplied reporting while adding relevant, clearly framed analysis; Model B invents background details, testing history, analyst commentary and company pla | Anthropic: Claude Fable 5.1: Model A stays strictly within the supplied repo…

Markdown error count

Count the lines that violate the house rules below. A defective line counts once even if it could violate multiple rules. Respond with a single number. House rules: * The headline must use `# Headline`. * The dek must appear immediately after the headline and be entirely italicized. * Section headings must use `## Heading`. * Links must use an absolute HTTPS URL. * Unordered-list items must begin with a hyphen. * Raw HTML is forbidden. Draft: #Flood software moves upstream The company is expanding its pilot. ##Details Read [the announcement](www.example.com/news). * The first pilot begins Monday. <div>The second pilot begins Friday.</div>

MoonshotAI: Kimi K3.1 Pro: 6
Meta: Muse Spark Open: 6

Scored against the benchmark's reference answer (numeric match): Model A was correct, Model B was correct.

Numbers shaped into a story

Write a 600–750-word Markdown article from the benchmark results below. Build the story around the clearest finding. Explain the methodology and limitations where readers need them. Avoid reciting the table row by row or treating small differences as decisive. Benchmark: * Researchers tested five language models on 200 anonymized customer-support tickets. * Each model produced one proposed answer per ticket. * Three experienced support managers graded the answers blindly. * Scores combined factual correctness, usefulness and tone. * Model North: 84.2 average score; $1.90 estimated cost per 1,000 tickets. * Model Vale: 83.8; $0.74 per 1,000 tickets. * Model Ember: 79.1; $0.31 per 1,000 tickets. * Model Kite: 76.4; $0.22 per 1,000 tickets. * Model Harbor: 71.0; $0.08 per 1,000 tickets. * North and Vale’s score difference was not statistically significant. * Vale answered 18% faster than North. * North performed best on billing disputes. * Vale performed best on technical troubleshooting. * All five models performed poorly when a ticket depended on an image attachment. * The test used English-language tickets from one software company. * Researchers did not test agentic actions, tool use or multilingual support.

MoonshotAI: Kimi K3.1 Pro: # The benchmark’s winner is about value, not the top score The clearest finding in this benchmark is not that Model North topped the leaderboard. It is that Model Vale delivered essentially the same measured quality for much less money. North averaged 84.2, versus 83.8 for Vale, but that 0.4-point gap was not statistically significant. Vale also answered 18% faster and had an estimated cost of $0.74 per 1,000 tickets, compared with $1.90 for North—about 61% less. For a team choosing a support-drafting model, that combination makes Vale the most compelling default candidate in this test. ## A narrow score gap, a wide operating gap Treating North as the quality winner would read more into the numbers than they support. A non-significant difference does not prove the models are identical; it means this experiment did not establish a reliable advantage. The practical distinction is therefore between a small, uncertain score gap and much larger observed differences in speed and estimated operating cost. The category results also argue against a universal ranking. North performed best on billing disputes, while Vale led on technical troubleshooting. A company’s ticket mix could consequently affect which model is preferable, and those specialties deserve validation before being used as routing rules. Below the leaders, Ember, Kite and Harbor averaged 79.1, 76.4 and 71.0, respectively. Their estimated costs were lower, but the results do not show that those savings preserve the leaders’ answer quality. The decision is a quality–cost trade-off, not a simple “cheapest wins” story. ## What the test measured The benchmark compared five language models on 200 anonymized customer-support tickets. Each model produced one proposed answer per ticket, yielding 1,000 responses. Three experienced support managers graded the answers blindly, using scores that combined factual correctness, usefulness and tone. Blind grading is valuable because it reduces the chance that a model’s name or reputation shapes the assessment. A common set of tickets also gives the models the same basic task. But the score is a measure of proposed answers, not resolved customer outcomes. Because correctness, usefulness and tone were combined, the average alone cannot reveal which component drove a result. One answer per ticket likewise says little about consistency across repeated runs. The cost figures are estimates per 1,000 tickets, not a complete account of integration, human review or error handling. Generalization is another constraint. All tickets were in English and came from one software company. Researchers did not test agentic actions, tool use or multilingual support. The study therefore cannot tell us how these models would perform at issuing refunds, updating accounts, retrieving information through tools or serving customers in other languages. Even for similar support drafting, a broader sample and a different ticket mix could change the practical ranking. ## The shared failure that changes deployment The most important shared failure was not visible in the overall averages. All five models performed poorly when a ticket depended on an image attachment. That result should outweigh fine-grained leaderboard comparisons for workflows heavy in screenshots or other visual evidence: none of these models earned a case for unattended handling of such tickets. Teams should route them for human review or conduct a separate evaluation designed around image-dependent cases, with explicit checks for correctness and escalation. ## Start with a pilot, not a coronation A sensible next step is a pilot, not a coronation. For English-language, text-based drafting at a comparable software company, Vale offers the strongest evidenced balance of quality, speed and estimated cost. North may still justify its premium where billing-dispute performance matters most, while cheaper options could be appropriate for lower-risk uses if their quality is acceptable. Any pilot should replay representative tickets, separate scores by issue type and track the components hidden in the benchmark’s composite—especially factual correctness. It should also measure actual latency and spend, reviewer editing time and failure rates. The benchmark’s durable lesson is that the highest average score is not necessarily the best operational choice. Here, the near-tie at the top makes Vale’s cost and speed advantages meaningful, while the image results and narrow test design set firm limits on how far that conclusion should travel.
Meta: Muse Spark Open: # When Second Place Costs Less Than Half ### A new head-to-head test of support models suggests the premium for "best" may be mostly price tag. For teams shopping for a customer-support model, the instinct is simple: pay more, get better answers. A small, tightly controlled benchmark flips that assumption. In a blind test of five language models on real support tickets, the most expensive system barely edged out a rival that cost less than half as much — and the difference wasn't even statistically meaningful. That is the clearest signal from the study, and the one that should anchor any buying decision. The rest is nuance. ### What was actually tested Researchers collected 200 anonymized, English-language tickets from a single software company. Each of the five models — referred to as North, Vale, Ember, Kite and Harbor — generated one proposed answer per ticket. Three experienced support managers graded the answers blindly, without knowing which model produced which response. Their scores blended three things customers actually care about: factual correctness, usefulness in resolving the issue, and tone. That design is stronger than a simple automated metric. Human raters catch the kind of plausible-but-unhelpful answers that inflate benchmark scores elsewhere. But it is also narrow by intent. This was a test of single-turn draft answers in English, not of agents that can take actions, call tools, look up accounts, or work across languages. Cost was estimated separately as the API cost to generate 1,000 draft answers. ### The finding that matters North came out on top on average score, with Vale right behind it at 84.2 versus 83.8. The study found that gap was not statistically significant — in practical terms, a tie. The price gap was significant. At an estimated $1.90 per 1,000 tickets, North cost more than two-and-a-half times Vale's $0.74. Vale also answered about 18% faster in the test. This is not a story about Vale "beating" North. It is a story about diminishing returns at the top end. If two systems are indistinguishable to experienced managers on quality, the cheaper, faster one redefines what "best" means for a support operation. The other models make that point sharper. The three lower-cost options — Ember, Kite and Harbor, all under $0.31 per 1,000 tickets — trailed the leaders by a clear margin, scoring in the low 70s to high 70s. The savings are real, but so is the drop in answer quality. The benchmark does not suggest that cheapest is good enough. It suggests the sweet spot is no longer at the very top of the price curve. ### Beyond the average Averages hide useful texture, and the researchers broke out a couple of patterns worth noting without over-reading them. North was relatively strongest on billing disputes, the kind of tickets where policy precision and careful tone matter. Vale was relatively strongest on technical troubleshooting, where step-by-step usefulness is key. Both are plausible strengths, but with only 200 tickets total, these slices get small quickly. Treat them as hints for where to do your own testing, not as definitive crowns. The most consistent weakness had nothing to do with brand. When a ticket depended on an image attachment — a screenshot of an error, a photo of an invoice — all five models performed poorly. They were, after all, tested on text alone. For any team where customers paste or attach images, this benchmark says little about real-world performance, and any deployment will need a strategy for vision or for asking for text details. ### Where this doesn't apply Limitations should shape how you use these numbers. First, breadth. All tickets came from one software company's English-language queue. Support language varies enormously by industry, product complexity, and customer base. A model that writes elegant troubleshooting for SaaS may stumble on retail returns or medical intake. Second, scope. The study did not test agentic actions, tool use, or multilingual support — three areas where real costs and real value often hide. A model that looks tied on draft quality could pull ahead if it is better at calling the right knowledge-base article or worse at refusing to hallucinate a refund policy. Third, scale of evidence. Two hundred tickets graded by three managers is enough to separate tiers — top two versus the rest — but not enough to declare a winner inside a tier. That is exactly what "not statistically significant" means for North and Vale. Obsessing over 0.4 points misses the point. ### The practical takeaway For leaders choosing a support assistant today, this benchmark reframes the question. It is not "which model scores highest?" but "what quality can we get for a sustainable cost and latency?" The data here point to a mid-premium tier that delivers top-tier draft quality without top-tier spend. The budget tier is cheaper still, but the quality falloff was large enough that savings would likely be erased by extra human review or customer follow-ups. The right move is not to crown Vale for everyone based on one study. It is to replicate the study's discipline on your own data: sample your tickets, include your hardest cases — especially those with images — grade blindly for correctness, usefulness and tone, and measure cost and speed alongside quality. If your results echo this one, the most expensive choice may be paying for a lead that doesn't exist.

OpenAI: GPT-6 Luna: Model A stays within the requested length and carefully distinguishes measured results from implications. Model B is engaging, but appears to exceed the word limit and incorrectly says the models were tested on text alone despite the image-dependent-ticket finding. (Second judge pass, order swapped — scores are the average of both: Model A is more careful about what the results establish and fits the requested length. Model B is engaging, but appears to exceed the word limit and makes unsupporte | Anthropic: Claude Fable 5.1: Model A stays within the 600–750-word range, st…

Matchup powered by OpenRouter.