Head to head: Inception: Mercury 3 vs Qwen3.8 Flash
Inception: Mercury 3 vs Qwen3.8 Flash
By Ryan Merket · Published
Mercury 3 and Qwen3.8 Flash face an editorial test of headline writing, earnings copy, source attribution and factual precision.
**Final call: Too close to call (not statistical).** Mercury 3 wrote the sharper copy in two tasks; Qwen3.8 Flash showed better source discipline in one. This assessment applies only to the saved task sample.
26. Funding headline and dek
Use only the source packet below. SOURCE PACKET Holloway Robotics raised a $24 million Series B led by Keel Ventures. Existing investors Northline and Passenger Fund also participated. Holloway makes warehouse inspection robots and says it plans to expand into three additional states. The company currently operates in two states. It did not disclose its valuation, revenue or customer count. Produce: A headline of no more than 12 words. A dek of no more than 28 words. Do not imply that the expansion has already occurred. Do not estimate a valuation, revenue figure or customer count. Avoid promotional adjectives. The response will be evaluated for accuracy, restraint, clarity and compliance with the length limits.
OpenAI: GPT-6 Sol: Model A gives readers the company’s business, funding and planned expansion without repeating information. Model B correctly includes the participating investors but repeats the lead investor and omits what the company makes; both meet the length limits. (Second judge pass, order swapped — scores are the average of both: Model A is clearer and more concise, using the dek to explain what Holloway Robotics makes and its planned expansion. Model B includes the participating investors, but repeats t | Anthropic: Claude Fable 5.1: Both entries are accurate, within length limits,…
15. Growth without obscuring the baseline
Use only the source packet below. SOURCE PACKET Daily active users increased from 3,000 in January to 9,000 in June. Which headline conveys the change most precisely? A. Usage explodes 200% B. Daily users triple from 3,000 to 9,000 C. Platform adds 9,000 daily users D. Platform reaches 12,000 daily users Respond with only the letter.
Scored against the benchmark's reference answer (choice match): Model A was correct, Model B was correct.
19. Unverified superlative
Use only the source packet below. SOURCE PACKET In a press release, Cedar Ledger called its product “the world’s first fully autonomous tax auditor.” The release supplied no independent evidence for that distinction. Other companies previously introduced automated tax-audit products. Which sentence is appropriate? A. Cedar Ledger launched the world’s first autonomous tax auditor. B. Cedar Ledger described its product as the world’s first fully autonomous tax auditor, a claim RuntimeWire could not independently verify. C. Cedar Ledger invented automated tax auditing. D. Independent researchers confirmed Cedar Ledger’s product was the first of its kind. Respond with only the letter.
Scored against the benchmark's reference answer (choice match): Model A was correct, Model B was correct.
20. Amended filing
Use only the source packet below. SOURCE PACKET At 9:02 a.m., a company filing said its CEO would resign immediately. At 10:47 a.m., the company filed an amendment saying the CEO would retire on December 31 and remain an adviser. A spokesperson confirmed that the amended filing corrected the earlier language. Which description should a current article use? A. The CEO resigned immediately. B. The CEO was dismissed at 9:02 a.m. C. The CEO will retire December 31 and remain an adviser. D. The company has issued two conflicting plans and provided no clarification. Respond with only the letter.
Scored against the benchmark's reference answer (choice match): Model A was correct, Model B was correct.
29. Earnings paragraph
Use only the source packet below. SOURCE PACKET Mariner Cloud reported fourth-quarter revenue of $48 million, up from $40 million one year earlier. Its quarterly net loss widened from $3.2 million to $5.6 million. Customer count increased 18% year over year. The company ended the quarter with $71 million in cash. Its CEO said management is targeting break-even during the second half of next year. Write one news paragraph of 80–100 words. Requirements: Accurately describe both revenue growth and the wider loss. Treat break-even as a management target, not a certainty. Do not add analyst expectations, explanations or opinions. The response will be evaluated for numerical accuracy, balance, source fidelity and clarity.
OpenAI: GPT-6 Sol: Model A accurately covers the revenue increase, wider loss and break-even target in a clear, concise paragraph. Model B is also accurate and meets the word count, but its repeated references to the quarter and its explanation that a target is not a certainty make it less polished. (Second judge pass, order swapped — scores are the average of both: Model A accurately covers the revenue increase, wider loss, customer growth, cash balance and break-even target in a clear 81-word paragraph. Model B | Anthropic: Claude Fable 5.1: Model A delivers an accurate, clean 81-word parag…
8. Revenue versus run rate
Use only the source packet below. SOURCE PACKET In April, Northstar Relay generated $275,000 in monthly recurring revenue. It collected $1.1 million in total revenue during the preceding 12 months. Which statement is accurate? A. Northstar Relay earned $3.3 million during the preceding 12 months. B. Northstar Relay’s annual recurring revenue run rate was $1.1 million in April. C. April MRR implied a $3.3 million annual run rate, while actual preceding-12-month revenue was $1.1 million. D. Northstar Relay booked $275,000 in annual recurring revenue. Respond with only the letter.
Scored against the benchmark's reference answer (choice match): Model A was correct, Model B was correct.
17. Survey population
Use only the source packet below. SOURCE PACKET Bright Acre surveyed 620 paying customers through an invitation displayed inside its app. Of those respondents, 68% said they were likely to renew. The survey did not include former customers or members of the general public. Which headline is accurate? A. 68% of surveyed Bright Acre customers say they are likely to renew B. 68% of Americans plan to buy Bright Acre C. 68% of former Bright Acre customers plan to return D. Bright Acre retains 68% of all customers Respond with only the letter.
Scored against the benchmark's reference answer (choice match): Model A was correct, Model B was correct.
1. Funding headline fidelity
Use only the source packet below. SOURCE PACKET AsterVale announced an $18 million Series A led by Northbank Capital, with participation from Delta Shore Ventures. The company said it will use the money to expand manufacturing capacity and hire up to 20 employees. AsterVale currently employs 42 people. The announcement did not disclose a valuation. Which headline is fully supported? A. AsterVale acquires manufacturing rival for $18 million B. AsterVale raises $18 million for manufacturing expansion and hiring C. AsterVale raises $18 million and commits to hiring exactly 20 people D. AsterVale reaches an $18 million valuation Respond with only the letter.
Scored against the benchmark's reference answer (choice match): Model A was correct, Model B was correct.
27. Reconciling disputed incident totals
Use only the source packet below. SOURCE A — REGULATOR The state transportation regulator said it received 23 reports involving unexpected braking by Kestrel Drive vehicles. Its investigation remains open. SOURCE B — COMPANY Kestrel Drive said it identified 17 distinct incidents. It believes six regulator reports duplicate incidents already counted. The company said it is cooperating with the investigation. SOURCE C — REGULATOR FOLLOW-UP The regulator has not determined whether any reports are duplicates. Write a 100–130 word news update explaining the dispute. Requirements: Attribute each figure. Do not select one figure as definitively correct. State that the investigation remains open. Do not characterize the vehicles as defective or cleared. The response will be evaluated for factual accuracy, attribution, handling of uncertainty and readability.
OpenAI: GPT-6 Sol: Model B more closely follows the source packet, attributing both figures and the unresolved duplicate question without adding unsupported details. Model A calls the regulator’s reports “complaints,” says the company is cooperating “fully,” and asserts that regulators have made no finding about the vehicles—none of which the sources establish. (Second judge pass, order swapped — scores are the average of both: Model B attributes both figures, preserves the regulator’s uncertainty about duplicates | Anthropic: Claude Fable 5.1: Model B stays strictly within the source packet,…
7. Percentage points
A manufacturer’s defect rate declined from 5% to 3%. By how many percentage points did the defect rate decline? Respond with the number only.
Scored against the benchmark's reference answer (numeric match): Model A was correct, Model B was correct.
22. Entity disambiguation
Use only the source packet below. SOURCE PACKET Orbit Harbor Inc. is a Portland-based logistics software company. It announced a $4 million seed round Monday. Orbit Harbor Aerospace LLC is a Denver-based propulsion manufacturer. It received a NASA subcontract Tuesday. Which company received the NASA subcontract? Respond with the complete company name exactly as written.
Scored against the benchmark's reference answer (exact match): Model A was correct, Model B was correct.
14. Contract correction
Use only the source packet below. SOURCE PACKET Falcon Ridge received a three-year government contract worth $7.4 million in total. An article incorrectly described it as “a $7.4 million annual contract.” Which replacement is accurate? A. A one-year contract worth $22.2 million B. An annual contract worth $7.4 million C. A three-year contract worth $7.4 million annually D. A three-year contract worth $7.4 million in total Respond with only the letter.
Scored against the benchmark's reference answer (choice match): Model A was correct, Model B was correct.
Matchup powered by OpenRouter.