Head to head: DeepSeek-V4-Flash vs Cohere-command-a-plus-05-2026
DeepSeek-V4-Flash vs Cohere-command-a-plus-05-2026
This one isn’t a blowout, but DeepSeek-V4-Flash finishes ahead by being more dependable on code, bug-fixing, and instruction-heavy writing tasks. Cohere-command-a-plus-05-2026 lands some sharper wins on SQL correctness and extraction, yet not enough to erase DeepSeek’s broader edge.
The topline is straightforward: DeepSeek-V4-Flash takes the match, **91.8 to 87.8**, with a **73% confidence** verdict. That is a real lead, but not a rout. The task count tells the same story — **6 wins for DeepSeek, 4 for Cohere, 2 ties** — so the right read is a credible lean, not a demolition. Where DeepSeek earns the verdict is reliability on practical execution. It was clearly better on the **LRU cache** task because Cohere’s answer was truncated and unusable, and it was cleaner on the **concurrency bug fix**, where it cached the in-flight promise exactly as requested and returned minimal, correct code. It also kept edging ahead on instruction-sensitive writing and formatting work: the **delay-update email** was more polished and client-ready, the **speaker bio** followed the constraints more faithfully, and the **Spanish translation** was cleaner and more natural. Cohere’s best case is that some of its wins were substantive, not cosmetic. On **sql-refund-rate**, it avoided DeepSeek’s major double-counting bug — the single biggest correctness miss in the matchup. It also beat DeepSeek on **strict JSON extraction** by normalizing fields more cleanly, on **find the contradiction** by naming the actual conflicting lines verbatim, and on **constraint scheduling** by giving the correct schedule directly while DeepSeek wandered through a wrong intermediate answer. Those are legitimate strengths, especially when precision matters. But DeepSeek was the steadier model across the board. Even in tasks it didn’t dominate, it usually produced something usable, while Cohere had a few avoidable instruction-following and output-quality lapses, including visible token artifacts and one outright broken coding response. The two ties reinforce the pattern: on **reasoning-shift-coverage**, both were equally solid; on **support triage**, both got the classifications right, with each model showing a different formatting weakness. **Final call: DeepSeek-V4-Flash is the better pick here — not because it crushed Cohere, but because it was more consistently dependable where failures are costly. Cohere-command-a-plus-05-2026 has sharper moments, especially on SQL-style correctness, but DeepSeek wins the head-to-head on overall execution.**
reasoning-shift-coverage
A clinic needs at least 2 nurses on duty in every hour from 08:00 to 20:00. Three shift types are available: - Early: 08:00-14:00 - Mid: 11:00-17:00 - Late: 14:00-20:00 Each nurse works exactly one shift. Costs are: Early $108, Mid $126, Late $96. What is the minimum total staffing cost that satisfies coverage for every hour block, and how many nurses should be assigned to each shift type? Give the counts and the total cost.
Model A and Model B both give the correct optimal staffing plan of 2 Early, 0 Mid, and 2 Late for a total cost of $408, and both satisfy the prompt clearly. Model A is slightly more step-by-step, while Model B is slightly more concise and polished, so neither is meaningfully better overall. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both give the correct optimal staffing plan of 2 Early, 0 Mid, and 2 Late for a total cost of $408, and both clearly justify why no cheaper solution exists. Model B is slightly more polished in presentation, while Model A is slightly more explicit in formulating the constraints, so neither is meaningfully better overall.)
professional-writing-delay-update
Draft a customer-facing email from a software vendor to a mid-market client. Situation: a data migration for client "Northline Studios" was planned for Wednesday, 7 May, but must move to Friday, 9 May because validation found 312 duplicate contact records that could cause merge errors. Reassure them no source data was changed, apologize once, and explain the team will send a cleaned preview file by 3:00 PM tomorrow for approval. Tone: calm, accountable, non-defensive. Length: 120-150 words. Include a clear subject line.
Model A is slightly better because it is polished, calm, and clearly accountable while covering every required detail without awkward artifacts. Model B is also strong, but the visible start/end tokens and slightly more mechanical phrasing reduce its professionalism and writing quality. (Second judge pass, order swapped — scores are the average of both: Model A is slightly stronger because it is more polished and client-facing while still covering every required fact: the new date, the 312 duplicate records, reassurance that no source data changed, the single apology, and the 3:00 PM preview file commitment. Model B is also solid, but Model A reads a bit more naturally and confidently for a calm, accountable customer update.)
sql-refund-rate
Tables: customers(id, name, segment) orders(id, customer_id, ordered_at, status) order_items(order_id, sku, quantity, unit_price) refunds(id, order_id, refunded_at, amount) Write a single SQL query to return, for each customer segment, the refund rate for Q1 2025 defined as total refunded amount in orders placed from 2025-01-01 through 2025-03-31 divided by gross merchandise value of those same orders (sum of quantity*unit_price). Include columns: segment, gmv, refunded_amount, refund_rate. Exclude orders whose status = 'cancelled'. Use 0 when a segment has no refunds. Sort by refund_rate descending, then segment ascending.
Model B is better because it avoids the major double-counting bug in Model A, where joining order_items and refunds directly can multiply both GMV and refunded totals when an order has multiple items and/or refunds. Model B is still imperfect because it does not restrict refunds to Q1 2025 if that was intended by the prompt and returns NULL rather than 0 when a segment’s GMV is zero, but overall it is materially more correct and still follows the requested shape well. (Second judge pass, order swapped — scores are the average of both: Model B correctly avoids double-counting GMV by aggregating order_items per order before joining refunds, and it uses 0 for segments with no refunds. Model A incorrectly joins refunds directly to order_items, which can multiply both GMV and refunded amounts, and it also filters refunds by refunded_at in Q1 even though the prompt defines refunds based on orders placed in Q1, not refund date.)
LRU cache
Implement a class `LRUCache<K, V>` in TypeScript with a fixed capacity set in the constructor, and O(1) `get(key): V | undefined` and `set(key, value): void`. Accessing or updating a key must mark it most-recently-used; inserting beyond capacity must evict the least-recently-used entry. Return ONLY the class, no prose.
Model A provides a complete TypeScript class that satisfies the requested API and LRU behavior using Map insertion order for O(1) operations in practice. Model B is truncated and syntactically incomplete, so it does not implement the requested class or follow the output constraints successfully. (Second judge pass, order swapped — scores are the average of both: Model A provides a complete TypeScript class that satisfies the LRU behavior and returns only code, while Model B is truncated, contains invalid/incomplete code, and is not usable as given. Model A is slightly imperfect because it relies on Map insertion order rather than an explicit linked-list design, but it still meets the requested behavior cleanly.)
Concurrency bug fix
This TypeScript function is meant to memoize an async loader but has a race: concurrent callers can each trigger the underlying fetch. Fix it so the fetch runs at most once per key, and a rejected fetch does NOT poison the cache (a later call must retry). Return ONLY the corrected function. ```ts const cache = new Map<string, Promise<string>>(); async function load(key: string, fetcher: (k: string) => Promise<string>) { if (cache.has(key)) return cache.get(key)!; const value = await fetcher(key); cache.set(key, Promise.resolve(value)); return value; } ```
Model A cleanly fixes the race by caching the in-flight promise before awaiting and removes the cache entry on rejection, exactly satisfying the requirements with minimal, correct code. Model B is also functionally correct, but it adds unnecessary commentary and extra conditional logic, which is less aligned with the instruction to return only the corrected function. (Second judge pass, order swapped — scores are the average of both: Model A cleanly fixes the race by storing the in-flight promise before awaiting and deletes the cache entry on rejection so retries work, exactly matching the requirements. Model B is also largely correct, but it adds unnecessary complexity and comments, making it less direct for a prompt that asked to return only the corrected function.)
Strict JSON extraction
Extract every meeting from this text into a JSON array of {"title","day","start24","durationMinutes"}. Use 24-hour "HH:MM" for start24, integer minutes for duration, and title-case titles. Return ONLY valid JSON, no commentary. Text: "Standup is daily at 9am for a quarter hour. The design review runs thursday from 2:30-4pm. Payroll sync — first monday, 11:00 to noon."
Model B is better because it preserves title case consistently for the textual fields, while Model A leaves the day values in lowercase despite the prompt’s normalization style. Both models correctly extract the meetings, times, and durations, but Model B adheres more cleanly to the requested formatting. (Second judge pass, order swapped — scores are the average of both: Model B is better because it correctly normalizes the day values into title case while preserving all meeting details accurately. Model A extracts the same meetings and times correctly, but its day fields remain lowercase, making it less compliant with the formatting implied by the task.)
classification-support-triage
Classify each incoming message into one category: Billing, Technical Issue, Sales Lead, or Account Access. Return exactly 5 lines, one per item, in this format: <number> | <category> | <brief justification of 4-8 words>. 1. "Your April invoice shows 14 seats, but we removed three in March. Please correct the charge." 2. "The export button spins forever in Firefox on our warehouse PCs, but works in Chrome." 3. "We're evaluating vendors for 120 field reps and need SSO plus offline mode pricing by Friday." 4. "I changed phones and no longer receive the verification codes to sign in." 5. "Can you explain why VAT appears on one subsidiary's renewal quote but not the others?"
Model A and Model B both classify all five items correctly, but Model A better follows the required output format. Model B adds extraneous start/end tokens, which violates the instruction to return exactly five lines in the specified format. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both classify all five items correctly, but Model B better follows the required format by keeping each justification within 4-8 words. Model A violates the brevity constraint on multiple lines by using 9-word justifications and adds unnecessary punctuation.)
SQL window query
Given Postgres tables `orders(id, customer_id, created_at, total_cents)` and `customers(id, name)`, write ONE SQL query returning each customer's name and their SECOND-largest order total (in dollars, 2 decimals), excluding customers with fewer than two orders. Order by that value descending. Return ONLY the SQL.
Model A is much closer: it returns the requested columns, excludes customers with fewer than two distinct order totals via the rank filter, and orders correctly, though using DENSE_RANK means ties can yield multiple rows per customer instead of exactly the second order amount. Model B has a serious join bug (`f.name = c.id::text`) that makes the query incorrect, and it is unnecessarily complex despite using ROW_NUMBER for the second order. (Second judge pass, order swapped — scores are the average of both: Model A is closer to the requested single-query solution and correctly returns customers with a second-ranked order total, though it uses DENSE_RANK so it may return multiple rows per customer on ties rather than the second order. Model B has a serious join bug by matching customer name to customer id text, which makes the query incorrect despite using ROW_NUMBER and explicitly filtering to customers with at least two orders.)
Find the contradiction
The following spec contains exactly one internal contradiction. Quote the two conflicting sentences verbatim and explain the conflict in one sentence. Do not fix it. Spec: "Free accounts may create up to three projects. Every account, regardless of tier, may archive unlimited projects. Archiving a project does not count against the project limit. Free accounts are limited to three projects total, including archived ones."
Model B identifies the actual contradictory pair verbatim: one sentence says archived projects do not count against the limit, while the other says archived projects are included in the free-account total. Model A quotes a weaker pair and relies on an extra inference about active versus created projects rather than the direct contradiction in the spec. (Second judge pass, order swapped — scores are the average of both: Model B identifies the actual contradictory pair verbatim and explains the conflict directly. Model A quotes a non-contradictory pair and relies on an unquoted third sentence in its explanation, so it is less correct and less faithful to the task.)
instruction-following-bio
Write a speaker bio for Maren Ilyas, a procurement analytics lead speaking at an internal operations summit. Requirements: - Exactly 4 bullet points - Each bullet must start with "- " - Total word count across all bullets: 52 to 60 words - Include these facts exactly once each: works at Alder Quay Foods; led a 9% reduction in packaging waste; mentors two junior analysts; based in Rotterdam - Do not use the words "passion", "driven", or "expert" - Final bullet must end with the exact text: "Session: Buying smarter, wasting less." - No first-person voice
Model A is better because it satisfies all required facts exactly once, uses exactly four bullets, avoids banned words, and ends the final bullet with the exact required text; its main issue is falling short of the 52–60 word total. Model B also misses the word-count requirement and has a weaker final bullet, since it consists only of the session line and provides less substantive bio content overall. (Second judge pass, order swapped — scores are the average of both: Model A better satisfies the required 52–60 total word count and includes all mandated facts exactly once while ending the final bullet with the exact session text. Model B exceeds the word-count limit and its final bullet contains only the session line, making it less effective as a speaker bio despite otherwise meeting most constraints.)
translation-es-mx
Translate into Mexican Spanish for a friendly but professional notice in a mobile banking app. Keep it to 2 sentences and under 28 words total. Source: "We couldn't verify your new device yet. For your security, transfers above $750 are temporarily paused until 6:00 PM tomorrow."
Model A is a clean, accurate Mexican Spanish translation that stays within 2 sentences and under 28 words. Model B is also accurate and natural, but it includes extraneous markup tokens, which hurts instruction following and overall quality. (Second judge pass, order swapped — scores are the average of both: Model A is slightly better because it stays natural and concise while preserving the meaning, and it fits the two-sentence, under-28-word constraint. Model B is also accurate, but "hasta las 6:00 PM de mañana" sounds less natural in Mexican Spanish and is a bit more awkward for an app notice.)
Constraint scheduling
Four talks (A, B, C, D) fill four 1-hour slots 9,10,11,12. Constraints: A is before D; C is not first; B is immediately after A; D is not at 12. Give the ONE valid schedule as 'slot: talk' lines, then a one-line justification. If impossible, say so and explain.
Model B gives the correct unique schedule cleanly and matches the requested format with a concise justification. Model A eventually reaches the same correct schedule, but it includes an incorrect intermediate schedule, self-contradiction, and extra text that weakens instruction adherence and clarity. (Second judge pass, order swapped — scores are the average of both: Model B gives the unique valid schedule directly and its justification is fully correct and concise. Model A eventually reaches the same schedule, but it first presents an invalid one, includes contradictory self-corrections, and does not cleanly follow the requested output format.)
Matchup powered by OpenRouter.