Head to head: DeepSeek-V4-Pro vs Cohere-command-a-plus-05-2026
DeepSeek-V4-Pro vs Cohere-command-a-plus-05-2026
By Ryan Merket · Published
This matchup turns on strict formatting, localization nuance, and implementation details—not broad capability claims. The raw scores separate the models, but the statistical evidence does not.
DeepSeek-V4-Pro posted the higher aggregate score, 100.8 to 87.6, and accumulated more task-level wins. It was notably steadier on messy-booking extraction, character-limited localization, precise proofreading, dense-passage summaries, release-note bullets, contradiction detection, and LRU-cache implementations. Those wins had a pattern: DeepSeek more often delivered directly usable output and respected fine-grained requirements. Cohere repeatedly leaked wrapper or control tokens into JSON, code, and banner responses; it also exceeded a 60-character localization limit and used linear-time array operations in one supposedly O(1) cache. DeepSeek was not spotless—its character counts were sometimes wrong, and some cache variants mishandled zero capacity—but its underlying answers were usually cleaner. Cohere had real advantages where terseness and temporal fidelity mattered. It handled support-shift assignments more directly, avoiding DeepSeek’s unnecessary reasoning, and better preserved the scheduled-future meaning of an app-maintenance notice. On the outage email, Cohere also had stronger length discipline in some judge passes, although its missing phone number and unsupported assurances weakened the result. The crucial fact overrides the apparent scoreboard gap: the analysis assigns only limited confidence that either model is genuinely better. That makes the 13.2-point aggregate spread descriptive, not decisive; on this evidence, the models are effectively even. **Final call: too close to call—a statistical tie, with no defensible overall winner.**
Constraint satisfaction — support shift assignment
Assign one on-call support shift to each of four engineers for Monday: 09:00, 11:00, 13:00, 15:00. Engineers: Amina, Ben, Carla, Diego. Exactly one engineer per shift and each engineer gets exactly one shift. Constraints: (1) Amina cannot do 09:00. (2) Ben must be earlier than Carla. (3) Diego cannot do 11:00 or 15:00. (4) Carla cannot do 13:00. (5) The 09:00 shift must be either Ben or Diego. (6) Amina must be exactly two hours after Ben. Return only the final assignment as lines in the form "Name — time".
OpenAI: GPT-5.6 Sol Pro: Model B provides a valid assignment concisely, with only the extraneous wrapper tokens detracting from the required format. Model A also reaches a valid assignment but substantially violates the instruction to return only assignment lines and adds an unnecessary arbitrary justification. (Second judge pass, order swapped — scores are the average of both: Model B provides a valid assignment in exactly the requested concise format. Model A also finds a valid assignment and correctly notes that two | Anthropic: Claude Fable 5.1: Both Model A and Model B produce valid assi…
Information extraction — messy conference bookings
Extract the attendee records from this messy text into a JSON array. For each attendee return exactly these fields: full_name, company, ticket_type, workshops (array), dietary_notes, amount_paid_usd, invoice_id. Use null if a field is missing, and preserve original spelling of names. Text: "BlueHarbor Summit reg sheet dump — do not forward. 1) Mara Ionescu / LatticeDock / Pro pass / workshops: 'Forecasting with Tiny Teams', 'Vendor Scorecards' / vegan + no walnuts / paid $349 / inv BH-2048. 2) Kenji Vale (Pine & Orbit) — Standard ticket — workshop: none — diet: n/a — amount 199 USD — invoice BH-2051. 3) S. Alvarez, company=RedMesa Clinic; PRO; workshops=[Incident Review Basics]; note: gluten-free; paid 349; inv# BH-2050. 4) Priya Natarajan | KiteThread | Standard | selected workshops: 'Forecasting with Tiny Teams' and 'Incident Review Basics' | dietary: vegetarian | payment pending | invoice BH-2054. 5) Tom Okafor, North Spur Labs, Speaker comped, workshops none, dairy-free, paid $0, invoice BH-2056."
OpenAI: GPT-5.6 Sol Pro: Model A provides clean, accurate JSON-formatted data, with only a minor normalization of "Standard ticket" to "Standard." Model B preserves that value more exactly but includes extraneous special tokens outside the code block, making the response less compliant and directly usable. (Second judge pass, order swapped — scores are the average of both: Model A is better overall because it avoids the extraneous start/end tokens surrounding Model B’s response and otherwise provides a clean, accurate e | Anthropic: Claude Fable 5.1: Both outputs extract all five records with…
Localization with tone
Translate this app onboarding line into natural, friendly European Spanish suitable for a mobile toast (keep it under 60 characters, no exclamation marks): "You're all set — your first backup starts tonight." Return only the translation, then the character count in parentheses.
OpenAI: GPT-5.6 Sol Pro: Model A is natural and stays under 60 characters, though its stated count is wrong: the translation has 47 characters. Model B is clear but has 62 characters rather than 57, so it exceeds the limit. (Second judge pass, order swapped — scores are the average of both: Model A is natural, friendly, and within the 60-character limit, though its stated count is incorrect: the translation has 47 characters, not 52. Model B uses the more explicit term «copia de seguridad», but its translation has 62 ch | Anthropic: Claude Fable 5.1: Model A delivers a natural, toast-friendly…
Precise proofreading
Correct ONLY the grammar, spelling, and punctuation errors in the sentence below without changing wording, tone, or meaning, then on a second line list each change as 'was -> now'. Sentence: "Its been a long time since we seen there new office, and me and him was suprised how quite it was."
OpenAI: GPT-5.6 Sol Pro: Model A makes all necessary corrections with minimal wording changes and cleanly lists each edit. Model B is grammatically correct, but Model B adds “have” rather than making the more minimal tense correction and includes unrequested markup tokens. (Second judge pass, order swapped — scores are the average of both: Model A makes the idiomatic minimal correction “seen -> saw,” preserving the original wording more closely. Model B unnecessarily inserts “have” and also includes extraneous start/end | Anthropic: Claude Fable 5.1: Both fix every error accurately and list c…
Summarize dense passage
Summarize the passage below in exactly three bullet points, each one sentence, capturing the mechanism, the tradeoff, and the caveat — no jargon a non-specialist couldn't follow. Return only the bullets. Passage: "Speculative decoding pairs a small draft model with a large target model: the draft proposes several tokens, the target verifies them in one pass, and accepted tokens are kept while the first rejection resets to the target's own choice. This can cut latency substantially when acceptance is high, but the draft model's compute is pure overhead when acceptance is low, and gains evaporate on adversarial or highly novel inputs where the draft and target disagree often."
OpenAI: GPT-5.6 Sol Pro: Model A follows the plain-language requirement closely while clearly covering the mechanism, tradeoff, and caveat in exactly three one-sentence bullets. Model B is more precise about resetting after rejection, but it retains jargon such as “tokens” and “latency” and overstates lost speed gains as outright failure. (Second judge pass, order swapped — scores are the average of both: Model A better follows the plain-language requirement while clearly covering the mechanism, tradeoff, and caveat in | Anthropic: Claude Fable 5.1: Model A delivers three plain-language, sing…
Translation & localization — app maintenance banner
Translate this product notice into Brazilian Portuguese for a consumer finance app. Tone: clear, reassuring, conversational. Constraint: maximum 22 words total. Keep the scheduled time, the temporary limitation, and the reassurance intact. Source: "We’re doing scheduled maintenance tonight from 11:30 p.m. to 12:15 a.m. You can still use your card, but transfers may be delayed."
OpenAI: GPT-5.6 Sol Pro: Model B better preserves the scheduled, future nature of the maintenance while clearly retaining the time window, card availability, and possible transfer delays within 22 words. Model A is more naturally localized but “Estamos em manutenção” can imply the maintenance is already underway. (Second judge pass, order swapped — scores are the average of both: Model B better preserves the “tonight” cue with “hoje” and adds explicit reassurance while staying under 22 words. Model A is natural and conc | Anthropic: Claude Fable 5.1: Model B preserves the 'tonight' (hoje) sch…
Professional writing — outage delay email
Draft a customer email from the head of operations at Northline Payroll to HR administrators at small businesses. Situation: a bank file formatting error delayed 14 direct deposits scheduled for Friday, 12 April. Fix is in progress; affected employees should receive funds by Monday, 15 April, 3:00 p.m. local time. Include: apology, plain-English explanation, what customers do and do not need to do, where to contact support ([email protected], 7 a.m.–7 p.m. CT), and a commitment to send another update by 10:00 a.m. CT tomorrow even if there is no change. Tone: accountable and calm, not legalistic. Length: 140–170 words.
OpenAI: GPT-5.6 Sol Pro: Model B is the better fit because Model B stays within the 140–170-word limit while covering the delay, apology, expected resolution, no-action guidance, support email and hours, and promised update. Model A gives clearer guidance about not resubmitting payroll or contacting a bank and has stronger prose, but Model A materially exceeds the requested length; Model B also awkwardly mentions calling without providing a phone number. (Second judge pass, order swapped — scores are the average of both | Anthropic: Claude Fable 5.1: Model A delivers a clearer plain-English e…
LRU cache
Implement a class `LRUCache<K, V>` in TypeScript with a fixed capacity set in the constructor, and O(1) `get(key): V | undefined` and `set(key, value): void`. Accessing or updating a key must mark it most-recently-used; inserting beyond capacity must evict the least-recently-used entry. Return ONLY the class, no prose.
OpenAI: GPT-5.6 Sol Pro: Model A uses Map insertion order to provide average O(1) access, recency updates, and eviction, though zero capacity is not handled correctly. Model B implements the expected eviction behavior but violates the O(1) requirement because indexOf, splice, and shift are linear-time array operations. (Second judge pass, order swapped — scores are the average of both: Model A uses Map insertion order to provide average O(1) access, updates, and eviction, whereas Model B uses array searches, splices, an | Anthropic: Claude Fable 5.1: Model A leverages Map's insertion-order se…
Summarization — release note bullets
Summarize the following internal release notes into exactly 4 bullet points for customer-facing release notes. Each bullet must be one sentence, 12–18 words, plain language, and must not mention internal ticket IDs. Source notes: "v3.18.0 draft: Added CSV export to the Team Activity page (ticket PX-7712). Fixed a bug where timezone changes could duplicate reminder emails for some users (PX-7689). Improved login speed by caching organization metadata; median time down from 2.4s to 1.6s in our tests (PX-7701). Admins can now require 2-step verification only for billing roles, instead of the whole workspace (PX-7664). Known issue: dark mode still shows low-contrast helper text on the Invite Members dialog (PX-7697), not fixed in this release."
OpenAI: GPT-5.6 Sol Pro: Model A accurately summarizes all four released changes using complete, plain-language sentences within the required word range. Model B is accurate overall, but its second bullet has only 10 words, and multiple bullets use sentence fragments. (Second judge pass, order swapped — scores are the average of both: Model A satisfies every formatting constraint, with four clear 12–18-word sentences and accurate customer-facing language. Model B is generally accurate and polished, but its second bullet | Anthropic: Claude Fable 5.1: Model A keeps every bullet within the 12–1…
Nuanced classification
Classify each review's sentiment as "positive", "negative", or "mixed", and give a 6-word-max reason. Return ONLY a JSON array of {"text","label","reason"} in input order. Reviews: ["Fast shipping but the fabric feels cheap.", "Absolutely love it, wearing it daily!", "It broke after a week. Refund was quick and painless though."]
OpenAI: GPT-5.6 Sol Pro: Model A and Model B classify all reviews correctly with concise reasons, but both violate the JSON-only requirement by adding wrapper text. Model A is slightly better because standard code fencing is less intrusive than Model B's non-JSON delimiter tokens, and Model A's reasons read more naturally. (Second judge pass, order swapped — scores are the average of both: Model B and Model A classify all reviews correctly and keep every reason within six words, but both violate the JSON-only requiremen | Anthropic: Claude Fable 5.1: Model A and Model B assign identical, corr…
Find the contradiction
The following spec contains exactly one internal contradiction. Quote the two conflicting sentences verbatim and explain the conflict in one sentence. Do not fix it. Spec: "Free accounts may create up to three projects. Every account, regardless of tier, may archive unlimited projects. Archiving a project does not count against the project limit. Free accounts are limited to three projects total, including archived ones."
OpenAI: GPT-5.6 Sol Pro: Model A identifies and quotes the exact conflicting sentences and explains their incompatibility directly. Model B quotes a pair that is not inherently contradictory and relies on an unquoted third sentence to describe the actual conflict. (Second judge pass, order swapped — scores are the average of both: Model A quotes the directly contradictory sentences and explains their conflict precisely in one sentence. Model B identifies the wrong pair because its first quoted sentence does not specify | Anthropic: Claude Fable 5.1: Model A correctly identifies the two direct…
Algorithm & data structures — LRU cache implementation
Implement an LRU cache in Python 3. Create a class LRUCache with methods __init__(capacity: int), get(key: int) -> int, and put(key: int, value: int) -> None. Requirements: both get and put must run in O(1) average time; if the key is missing, get returns -1; when capacity is exceeded, evict the least recently used key. Use only the standard library. Return code only.
OpenAI: GPT-5.6 Sol Pro: Model A is concise, fully O(1) on average, and safely handles zero capacity by inserting and immediately evicting. Model B generally works but raises KeyError on put when capacity is zero and also includes unused imports. (Second judge pass, order swapped — scores are the average of both: Model A correctly handles a zero-capacity cache by inserting and then immediately evicting, whereas Model B raises KeyError by attempting to evict from an empty cache. Model A is also cleaner, using move_to_end | Anthropic: Claude Fable 5.1: Both implementations satisfy the O(1) requ…
Matchup powered by OpenRouter.