Head to head: DeepSeek-V4-Flash vs Cohere-command-a-plus-05-2026

DeepSeek-V4-Flash vs Cohere-command-a-plus-05-2026

This one isn’t a blowout, but DeepSeek-V4-Flash finishes ahead by being more dependable on code, bug-fixing, and instruction-heavy writing tasks. Cohere-command-a-plus-05-2026 lands some sharper wins on SQL correctness and extraction, yet not enough to erase DeepSeek’s broader edge.

The topline is straightforward: DeepSeek-V4-Flash takes the match, **91.8 to 87.8**, with a **73% confidence** verdict. That is a real lead, but not a rout. The task count tells the same story — **6 wins for DeepSeek, 4 for Cohere, 2 ties** — so the right read is a credible lean, not a demolition. Where DeepSeek earns the verdict is reliability on practical execution. It was clearly better on the **LRU cache** task because Cohere’s answer was truncated and unusable, and it was cleaner on the **concurrency bug fix**, where it cached the in-flight promise exactly as requested and returned minimal, correct code. It also kept edging ahead on instruction-sensitive writing and formatting work: the **delay-update email** was more polished and client-ready, the **speaker bio** followed the constraints more faithfully, and the **Spanish translation** was cleaner and more natural. Cohere’s best case is that some of its wins were substantive, not cosmetic. On **sql-refund-rate**, it avoided DeepSeek’s major double-counting bug — the single biggest correctness miss in the matchup. It also beat DeepSeek on **strict JSON extraction** by normalizing fields more cleanly, on **find the contradiction** by naming the actual conflicting lines verbatim, and on **constraint scheduling** by giving the correct schedule directly while DeepSeek wandered through a wrong intermediate answer. Those are legitimate strengths, especially when precision matters. But DeepSeek was the steadier model across the board. Even in tasks it didn’t dominate, it usually produced something usable, while Cohere had a few avoidable instruction-following and output-quality lapses, including visible token artifacts and one outright broken coding response. The two ties reinforce the pattern: on **reasoning-shift-coverage**, both were equally solid; on **support triage**, both got the classifications right, with each model showing a different formatting weakness. **Final call: DeepSeek-V4-Flash is the better pick here — not because it crushed Cohere, but because it was more consistently dependable where failures are costly. Cohere-command-a-plus-05-2026 has sharper moments, especially on SQL-style correctness, but DeepSeek wins the head-to-head on overall execution.**

reasoning-shift-coverage

A clinic needs at least 2 nurses on duty in every hour from 08:00 to 20:00. Three shift types are available: - Early: 08:00-14:00 - Mid: 11:00-17:00 - Late: 14:00-20:00 Each nurse works exactly one shift. Costs are: Early $108, Mid $126, Late $96. What is the minimum total staffing cost that satisfies coverage for every hour block, and how many nurses should be assigned to each shift type? Give the counts and the total cost.

DeepSeek-V4-Flash: We need to cover each hour from 08:00 to 20:00 with at least 2 nurses. Let: - \( x \) = number of nurses on Early shift (08:00–14:00) - \( y \) = number of nurses on Mid shift (11:00–17:00) - \( z \) = number of nurses on Late shift (14:00–20:00) --- ### Step 1: Hourly coverage requirements We break the day into hour blocks and list which shifts cover them: - **08:00–11:00**: Only Early shift works. Demand: \( x \geq 2 \) (for hours 08–09, 09–10, 10–11) - **11:00–14:00**: Early and Mid both work. Demand: \( x + y \geq 2 \) - **14:00–17:00**: Mid and Late both work. Demand: \( y + z \geq 2 \) - **17:00–20:00**: Only Late shift works. Demand: \( z \geq 2 \) (for hours 17–18, 18–19, 19–20) --- ### Step 2: Constraints summary \[ x \geq 2 x + y \geq 2 y + z \geq 2 z \geq 2 \] All variables are non-negative integers. --- ### Step 3: Cost minimization Cost = \( 108x + 126y + 96z \). From \( x \geq 2 \) and \( z \geq 2 \): Minimum possible \( x = 2 \), \( z = 2 \) to start. Check \( y + z \geq 2 \): If \( z = 2 \), then \( y \) can be 0 and still satisfy. Check \( x + y \geq 2 \): If \( x = 2 \), then \( y \) can be 0 and satisfy. So \( x=2, y=0, z=2 \) works. --- ### Step 4: Cost calculation \[ \text{Cost} = 108(2) + 126(0) + 96(2) = 216 + 0 + 192 = 408 \] --- **Final answer:** - Early: 2 - Mid: 0 - Late: 2 - Total cost: $408 \[ \boxed{2, 0, 2, 408} \]
Cohere-command-a-plus-05-2026: <|START_TEXT|>**Optimal staffing** | Shift | Nurses required | Cost per nurse | Total cost | |-------|-----------------|----------------|------------| | Early (08:00‑14:00) | **2** | \$108 | \$216 | | Mid (11:00‑17:00) | **0** | \$126 | \$0 | | Late (14:00‑20:00) | **2** | \$96 | \$192 | | **Total** | – | – | **\$408** | **Why this works** - Hours 08:00‑11:00 are covered only by the Early shift → need at least 2 Early nurses. - Hours 17:00‑20:00 are covered only by the Late shift → need at least 2 Late nurses. - Overlapping periods (11:00‑14:00 and 14:00‑17:00) are each covered by two adjacent shifts, so with 2 Early + 2 Late the condition “≥ 2 nurses on duty” is met even when the Mid shift has 0 nurses. Any attempt to reduce the number of Early or Late nurses would leave the early‑only or late‑only hours uncovered, and adding Mid‑shift nurses would only increase cost (Mid is the most expensive shift). Therefore the **minimum total staffing cost is \$408**, achieved with **2 nurses on Early shift, 0 nurses on Mid shift, and 2 nurses on Late shift**.<|END_TEXT|>

Model A and Model B both give the correct optimal staffing plan of 2 Early, 0 Mid, and 2 Late for a total cost of $408, and both satisfy the prompt clearly. Model A is slightly more step-by-step, while Model B is slightly more concise and polished, so neither is meaningfully better overall. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both give the correct optimal staffing plan of 2 Early, 0 Mid, and 2 Late for a total cost of $408, and both clearly justify why no cheaper solution exists. Model B is slightly more polished in presentation, while Model A is slightly more explicit in formulating the constraints, so neither is meaningfully better overall.)

professional-writing-delay-update

Draft a customer-facing email from a software vendor to a mid-market client. Situation: a data migration for client "Northline Studios" was planned for Wednesday, 7 May, but must move to Friday, 9 May because validation found 312 duplicate contact records that could cause merge errors. Reassure them no source data was changed, apologize once, and explain the team will send a cleaned preview file by 3:00 PM tomorrow for approval. Tone: calm, accountable, non-defensive. Length: 120-150 words. Include a clear subject line.

DeepSeek-V4-Flash: **Subject:** Update: Northline Studios Data Migration Rescheduled to May 9 Dear Northline Studios Team, During our final validation for the migration scheduled this Wednesday, May 7, we identified 312 duplicate contact records. Addressing these now is essential to prevent merge errors and ensure data integrity. Please be assured that no source data has been altered—your systems remain unchanged. To proceed carefully, we’ve moved the migration to Friday, May 9. Our team is already cleaning the duplicates and will send you a preview file by 3:00 PM tomorrow for your review and approval. Once you confirm, we’ll finalize the migration as planned. We apologize for the delay and appreciate your understanding. Our priority is delivering a smooth, error-free transition, and this adjustment ensures that result. Please feel free to reply with any questions. We’ll keep you updated every step of the way. Best regards, [Your Name] [Your Title] [Vendor Name]
Cohere-command-a-plus-05-2026: <|START_TEXT|>**Subject:** Update on Data Migration Schedule – Preview File Arriving Tomorrow Dear Northline Studios Team, We wanted to inform you that the data migration originally scheduled for Wednesday, 7 May has been rescheduled to Friday, 9 May. During our validation process we identified 312 duplicate contact records that could lead to merge errors if processed unchanged. I apologize for any inconvenience this may cause. Please rest assured that no source data has been altered; we simply need additional time to clean the records. Our team will deliver a cleaned preview file by 3:00 PM tomorrow for your review and approval. Once approved, we will proceed with the full migration on Friday, ensuring all contacts are correctly merged. If you have any questions, please reply to this email. Thank you, [Your Name] [Vendor Company]<|END_TEXT|>

Model A is slightly better because it is polished, calm, and clearly accountable while covering every required detail without awkward artifacts. Model B is also strong, but the visible start/end tokens and slightly more mechanical phrasing reduce its professionalism and writing quality. (Second judge pass, order swapped — scores are the average of both: Model A is slightly stronger because it is more polished and client-facing while still covering every required fact: the new date, the 312 duplicate records, reassurance that no source data changed, the single apology, and the 3:00 PM preview file commitment. Model B is also solid, but Model A reads a bit more naturally and confidently for a calm, accountable customer update.)

sql-refund-rate

Tables: customers(id, name, segment) orders(id, customer_id, ordered_at, status) order_items(order_id, sku, quantity, unit_price) refunds(id, order_id, refunded_at, amount) Write a single SQL query to return, for each customer segment, the refund rate for Q1 2025 defined as total refunded amount in orders placed from 2025-01-01 through 2025-03-31 divided by gross merchandise value of those same orders (sum of quantity*unit_price). Include columns: segment, gmv, refunded_amount, refund_rate. Exclude orders whose status = 'cancelled'. Use 0 when a segment has no refunds. Sort by refund_rate descending, then segment ascending.

DeepSeek-V4-Flash: Here’s the SQL query that meets your requirements: ```sql SELECT c.segment, COALESCE(SUM(oi.quantity * oi.unit_price), 0) AS gmv, COALESCE(SUM(r.amount), 0) AS refunded_amount, CASE WHEN SUM(oi.quantity * oi.unit_price) > 0 THEN COALESCE(SUM(r.amount), 0) / SUM(oi.quantity * oi.unit_price) ELSE 0 END AS refund_rate FROM customers c JOIN orders o ON c.id = o.customer_id JOIN order_items oi ON o.id = oi.order_id LEFT JOIN refunds r ON o.id = r.order_id AND r.refunded_at BETWEEN '2025-01-01' AND '2025-03-31' WHERE o.ordered_at BETWEEN '2025-01-01' AND '2025-03-31' AND o.status != 'cancelled' GROUP BY c.segment ORDER BY refund_rate DESC, c.segment ASC; ``` **Key points about this query:** 1. **Filters**: Only includes orders placed in Q1 2025 (`ordered_at` between the dates) and excludes cancelled orders 2. **GMV calculation**: `SUM(oi.quantity * oi.unit_price)` from order_items 3. **Refund handling**: LEFT JOIN on refunds so customers with no refunds get 0 4. **Refund Rate**: Refunded amount / GMV with protection against division by zero 5. **Sorting**: By refund_rate descending, then segment ascending 6. **Null handling**: COALESCE ensures 0 instead of NULL for segments with no orders or no refunds
Cohere-command-a-plus-05-2026: Here’s the SQL query to achieve the desired result: ```sql WITH order_gmv AS ( SELECT c.segment, o.id AS order_id, SUM(oi.quantity * oi.unit_price) AS gmv FROM orders o JOIN customers c ON o.customer_id = c.id JOIN order_items oi ON o.id = oi.order_id WHERE o.status != 'cancelled' AND o.ordered_at BETWEEN '2025-01-01' AND '2025-03-31' GROUP BY c.segment, o.id ), refunded_amounts AS ( SELECT og.segment, og.order_id, og.gmv, COALESCE(SUM(r.amount), 0) AS refunded_amount FROM order_gmv og LEFT JOIN refunds r ON og.order_id = r.order_id GROUP BY og.segment, og.order_id, og.gmv ), segment_summary AS ( SELECT segment, SUM(gmv) AS total_gmv, SUM(refunded_amount) AS total_refunded_amount FROM refunded_amounts GROUP BY segment ) SELECT ss.segment, ss.total_gmv AS gmv, ss.total_refunded_amount AS refunded_amount, ROUND(ss.total_refunded_amount / NULLIF(ss.total_gmv, 0), 4) AS refund_rate FROM segment_summary ss ORDER BY refund_rate DESC, segment ASC; ``` ### Explanation: 1. **order_gmv CTE**: - Calculates the gross merchandise value (GMV) for each order placed in Q1 2025, excluding cancelled orders. - Joins `orders`, `customers`, and `order_items` tables to compute `SUM(quantity * unit_price)` per order. 2. **refunded_amounts CTE**: - Joins the `order_gmv` results with `refunds` to calculate the total refunded amount per order. - Uses `COALESCE` to handle orders with no refunds (sets refunded amount to 0). 3. **segment_summary CTE**: - Aggregates GMV and refunded amounts by customer segment. 4. **Final SELECT**: - Computes the refund rate as `refunded_amount / GMV`, using `NULLIF` to avoid division by zero. - Rounds the refund rate to 4 decimal places. - Orders results by `refund_rate` descending and `segment` ascending. This query ensures all requirements are met, including handling segments with no refunds and proper sorting.

Model B is better because it avoids the major double-counting bug in Model A, where joining order_items and refunds directly can multiply both GMV and refunded totals when an order has multiple items and/or refunds. Model B is still imperfect because it does not restrict refunds to Q1 2025 if that was intended by the prompt and returns NULL rather than 0 when a segment’s GMV is zero, but overall it is materially more correct and still follows the requested shape well. (Second judge pass, order swapped — scores are the average of both: Model B correctly avoids double-counting GMV by aggregating order_items per order before joining refunds, and it uses 0 for segments with no refunds. Model A incorrectly joins refunds directly to order_items, which can multiply both GMV and refunded amounts, and it also filters refunds by refunded_at in Q1 even though the prompt defines refunds based on orders placed in Q1, not refund date.)

LRU cache

Implement a class `LRUCache<K, V>` in TypeScript with a fixed capacity set in the constructor, and O(1) `get(key): V | undefined` and `set(key, value): void`. Accessing or updating a key must mark it most-recently-used; inserting beyond capacity must evict the least-recently-used entry. Return ONLY the class, no prose.

DeepSeek-V4-Flash: ```typescript class LRUCache<K, V> { private capacity: number; private cache: Map<K, V>; constructor(capacity: number) { this.capacity = capacity; this.cache = new Map<K, V>(); } get(key: K): V | undefined { if (!this.cache.has(key)) { return undefined; } const value = this.cache.get(key)!; this.cache.delete(key); this.cache.set(key, value); return value; } set(key: K, value: V): void { if (this.cache.has(key)) { this.cache.delete(key); } else if (this.cache.size >= this.capacity) { const oldestKey = this.cache.keys().next().value; if (oldestKey !== undefined) { this.cache.delete(oldestKey); } } this.cache.set(key, value); } } ```
Cohere-command-a-plus-05-2026: <|START_TEXT|>```ts class LRUCache<K, V> { private capacity: number; private cache: Map<K, Node<K, V>>; private head: Node<K, V> | null = null; private tail: Node<K, V> | null = null; constructor(capacity: number) { if (capacity <= 0) throw new Error('Capacity must be positive'); this.capacity = capacity; this.cache = new Map(); } get(key: K): V | undefined { const node = this.cache.get(key); if (!node) return undefined; this.moveToHead(node); return node.value; } set(key: K, value: V): void { let node = this.cache.get(key); if (node) { node.value = value; this.moveToHead(node); } else { if (this.cache.size >= this.capacity) { this.evictLRU(); } const newNode = new this.Node(key, value); this

Model A provides a complete TypeScript class that satisfies the requested API and LRU behavior using Map insertion order for O(1) operations in practice. Model B is truncated and syntactically incomplete, so it does not implement the requested class or follow the output constraints successfully. (Second judge pass, order swapped — scores are the average of both: Model A provides a complete TypeScript class that satisfies the LRU behavior and returns only code, while Model B is truncated, contains invalid/incomplete code, and is not usable as given. Model A is slightly imperfect because it relies on Map insertion order rather than an explicit linked-list design, but it still meets the requested behavior cleanly.)

Concurrency bug fix

This TypeScript function is meant to memoize an async loader but has a race: concurrent callers can each trigger the underlying fetch. Fix it so the fetch runs at most once per key, and a rejected fetch does NOT poison the cache (a later call must retry). Return ONLY the corrected function. ```ts const cache = new Map<string, Promise<string>>(); async function load(key: string, fetcher: (k: string) => Promise<string>) { if (cache.has(key)) return cache.get(key)!; const value = await fetcher(key); cache.set(key, Promise.resolve(value)); return value; } ```

DeepSeek-V4-Flash: ```ts const cache = new Map<string, Promise<string>>(); async function load(key: string, fetcher: (k: string) => Promise<string>) { if (cache.has(key)) return cache.get(key)!; const promise = fetcher(key).then( value => { cache.set(key, Promise.resolve(value)); return value; }, error => { cache.delete(key); throw error; } ); cache.set(key, promise); return promise; } ```
Cohere-command-a-plus-05-2026: ```ts const cache = new Map<string, Promise<string>>(); async function load(key: string, fetcher: (k: string) => Promise<string>) { if (cache.has(key)) { const cachedPromise = cache.get(key)!; // If the cached promise is already resolved or rejected, return it // Otherwise, let the caller await the existing promise return cachedPromise; } // Create a new promise that handles the fetch and cache update const promise = fetcher(key) .then(value => { // Only cache the resolved value if it's not already in the cache if (!cache.has(key) || cache.get(key) === promise) { cache.set(key, Promise.resolve(value)); } return value; }) .catch(error => { // Remove the promise from the cache if it rejects if (cache.get(key) === promise) { cache.delete(key); } throw error; }); // Cache the promise to handle concurrent calls cache.set(key, promise); return promise; } ```

Model A cleanly fixes the race by caching the in-flight promise before awaiting and removes the cache entry on rejection, exactly satisfying the requirements with minimal, correct code. Model B is also functionally correct, but it adds unnecessary commentary and extra conditional logic, which is less aligned with the instruction to return only the corrected function. (Second judge pass, order swapped — scores are the average of both: Model A cleanly fixes the race by storing the in-flight promise before awaiting and deletes the cache entry on rejection so retries work, exactly matching the requirements. Model B is also largely correct, but it adds unnecessary complexity and comments, making it less direct for a prompt that asked to return only the corrected function.)

Strict JSON extraction

Extract every meeting from this text into a JSON array of {"title","day","start24","durationMinutes"}. Use 24-hour "HH:MM" for start24, integer minutes for duration, and title-case titles. Return ONLY valid JSON, no commentary. Text: "Standup is daily at 9am for a quarter hour. The design review runs thursday from 2:30-4pm. Payroll sync — first monday, 11:00 to noon."

DeepSeek-V4-Flash: ```json [ { "title": "Standup", "day": "daily", "start24": "09:00", "durationMinutes": 15 }, { "title": "Design Review", "day": "thursday", "start24": "14:30", "durationMinutes": 90 }, { "title": "Payroll Sync", "day": "first monday", "start24": "11:00", "durationMinutes": 60 } ] ```
Cohere-command-a-plus-05-2026: ```json [ {"title": "Standup", "day": "Daily", "start24": "09:00", "durationMinutes": 15}, {"title": "Design Review", "day": "Thursday", "start24": "14:30", "durationMinutes": 90}, {"title": "Payroll Sync", "day": "First Monday", "start24": "11:00", "durationMinutes": 60} ] ```

Model B is better because it preserves title case consistently for the textual fields, while Model A leaves the day values in lowercase despite the prompt’s normalization style. Both models correctly extract the meetings, times, and durations, but Model B adheres more cleanly to the requested formatting. (Second judge pass, order swapped — scores are the average of both: Model B is better because it correctly normalizes the day values into title case while preserving all meeting details accurately. Model A extracts the same meetings and times correctly, but its day fields remain lowercase, making it less compliant with the formatting implied by the task.)

classification-support-triage

Classify each incoming message into one category: Billing, Technical Issue, Sales Lead, or Account Access. Return exactly 5 lines, one per item, in this format: <number> | <category> | <brief justification of 4-8 words>. 1. "Your April invoice shows 14 seats, but we removed three in March. Please correct the charge." 2. "The export button spins forever in Firefox on our warehouse PCs, but works in Chrome." 3. "We're evaluating vendors for 120 field reps and need SSO plus offline mode pricing by Friday." 4. "I changed phones and no longer receive the verification codes to sign in." 5. "Can you explain why VAT appears on one subsidiary's renewal quote but not the others?"

DeepSeek-V4-Flash: 1 | Billing | Disputes invoice seat count and requests charge correction. 2 | Technical Issue | Reports export button malfunction specifically in Firefox browser. 3 | Sales Lead | Evaluates vendor for 120 reps, requests pricing. 4 | Account Access | Cannot receive verification codes after changing phones. 5 | Billing | Questions VAT discrepancy on subsidiary’s renewal quote.
Cohere-command-a-plus-05-2026: <|START_TEXT|>1 | Billing | Invoice discrepancy, request charge correction 2 | Technical Issue | Export button hangs in Firefox only 3 | Sales Lead | Vendor evaluation, request pricing, SSO 4 | Account Access | Phone change causes missing verification codes 5 | Billing | VAT inconsistency across subsidiary quotes<|END_TEXT|>

Model A and Model B both classify all five items correctly, but Model A better follows the required output format. Model B adds extraneous start/end tokens, which violates the instruction to return exactly five lines in the specified format. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both classify all five items correctly, but Model B better follows the required format by keeping each justification within 4-8 words. Model A violates the brevity constraint on multiple lines by using 9-word justifications and adds unnecessary punctuation.)

SQL window query

Given Postgres tables `orders(id, customer_id, created_at, total_cents)` and `customers(id, name)`, write ONE SQL query returning each customer's name and their SECOND-largest order total (in dollars, 2 decimals), excluding customers with fewer than two orders. Order by that value descending. Return ONLY the SQL.

DeepSeek-V4-Flash: ```sql SELECT c.name, ROUND(o.total_cents / 100.0, 2) AS second_largest_order FROM ( SELECT customer_id, total_cents, DENSE_RANK() OVER (PARTITION BY customer_id ORDER BY total_cents DESC) AS rnk FROM orders ) o JOIN customers c ON c.id = o.customer_id WHERE o.rnk = 2 ORDER BY second_largest_order DESC; ```
Cohere-command-a-plus-05-2026: ```sql WITH ranked_orders AS ( SELECT c.name, o.total_cents / 100.0 AS total_dollars, ROW_NUMBER() OVER (PARTITION BY c.id ORDER BY o.total_cents DESC) AS row_num FROM customers c JOIN orders o ON c.id = o.customer_id ), filtered_ranked_orders AS ( SELECT name, total_dollars FROM ranked_orders WHERE row_num = 2 ), customer_order_counts AS ( SELECT c.id, COUNT(o.id) AS order_count FROM customers c JOIN orders o ON c.id = o.customer_id GROUP BY c.id HAVING COUNT(o.id) >= 2 ) SELECT f.name, ROUND(f.total_dollars, 2) AS second_largest_order FROM filtered_ranked_orders f JOIN customer_order_counts c ON f.name = c.id::text ORDER BY second_largest_order DESC; ```

Model A is much closer: it returns the requested columns, excludes customers with fewer than two distinct order totals via the rank filter, and orders correctly, though using DENSE_RANK means ties can yield multiple rows per customer instead of exactly the second order amount. Model B has a serious join bug (`f.name = c.id::text`) that makes the query incorrect, and it is unnecessarily complex despite using ROW_NUMBER for the second order. (Second judge pass, order swapped — scores are the average of both: Model A is closer to the requested single-query solution and correctly returns customers with a second-ranked order total, though it uses DENSE_RANK so it may return multiple rows per customer on ties rather than the second order. Model B has a serious join bug by matching customer name to customer id text, which makes the query incorrect despite using ROW_NUMBER and explicitly filtering to customers with at least two orders.)

Find the contradiction

The following spec contains exactly one internal contradiction. Quote the two conflicting sentences verbatim and explain the conflict in one sentence. Do not fix it. Spec: "Free accounts may create up to three projects. Every account, regardless of tier, may archive unlimited projects. Archiving a project does not count against the project limit. Free accounts are limited to three projects total, including archived ones."

DeepSeek-V4-Flash: **Conflict:** "Free accounts may create up to three projects." "Free accounts are limited to three projects total, including archived ones." **Explanation:** The first sentence implies the limit applies only to created (active) projects, while the second sentence explicitly includes archived projects in the same limit, contradicting the earlier statement that archiving does not count against the limit.
Cohere-command-a-plus-05-2026: **Conflicting Sentences:** 1. "Archiving a project does not count against the project limit." 2. "Free accounts are limited to three projects total, including archived ones." **Explanation:** The first sentence states that archived projects do not count toward the project limit, while the second sentence states that archived projects do count toward the limit for free accounts, creating a contradiction.

Model B identifies the actual contradictory pair verbatim: one sentence says archived projects do not count against the limit, while the other says archived projects are included in the free-account total. Model A quotes a weaker pair and relies on an extra inference about active versus created projects rather than the direct contradiction in the spec. (Second judge pass, order swapped — scores are the average of both: Model B identifies the actual contradictory pair verbatim and explains the conflict directly. Model A quotes a non-contradictory pair and relies on an unquoted third sentence in its explanation, so it is less correct and less faithful to the task.)

instruction-following-bio

Write a speaker bio for Maren Ilyas, a procurement analytics lead speaking at an internal operations summit. Requirements: - Exactly 4 bullet points - Each bullet must start with "- " - Total word count across all bullets: 52 to 60 words - Include these facts exactly once each: works at Alder Quay Foods; led a 9% reduction in packaging waste; mentors two junior analysts; based in Rotterdam - Do not use the words "passion", "driven", or "expert" - Final bullet must end with the exact text: "Session: Buying smarter, wasting less." - No first-person voice

DeepSeek-V4-Flash: - Maren Ilyas is Procurement Analytics Lead at Alder Quay Foods, based in Rotterdam. - She led a 9% reduction in packaging waste through data-driven supplier insights. - Maren mentors two junior analysts, strengthening her team’s analytical capabilities. - Her talk will explore forecasting and spend optimization. Session: Buying smarter, wasting less.
Cohere-command-a-plus-05-2026: - Maren Ilyas, Procurement Analytics Lead at Alder Quay Foods, is based in Rotterdam and oversees data-driven initiatives to optimize supply chain efficiency. - She recently led a successful project that achieved a 9% reduction in packaging waste across the company’s distribution network. - Outside of her role, Maren mentors two junior analysts, fostering skill development and collaboration within her team. - Session: Buying smarter, wasting less.

Model A is better because it satisfies all required facts exactly once, uses exactly four bullets, avoids banned words, and ends the final bullet with the exact required text; its main issue is falling short of the 52–60 word total. Model B also misses the word-count requirement and has a weaker final bullet, since it consists only of the session line and provides less substantive bio content overall. (Second judge pass, order swapped — scores are the average of both: Model A better satisfies the required 52–60 total word count and includes all mandated facts exactly once while ending the final bullet with the exact session text. Model B exceeds the word-count limit and its final bullet contains only the session line, making it less effective as a speaker bio despite otherwise meeting most constraints.)

translation-es-mx

Translate into Mexican Spanish for a friendly but professional notice in a mobile banking app. Keep it to 2 sentences and under 28 words total. Source: "We couldn't verify your new device yet. For your security, transfers above $750 are temporarily paused until 6:00 PM tomorrow."

DeepSeek-V4-Flash: No pudimos verificar tu nuevo dispositivo aún. Por seguridad, transferencias mayores a $750 están pausadas hasta mañana a las 6:00 PM.
Cohere-command-a-plus-05-2026: <|START_TEXT|>Aún no hemos podido verificar tu nuevo dispositivo. Por tu seguridad, las transferencias mayores a $750 estarán pausadas hasta las 6:00 PM de mañana.<|END_TEXT|>

Model A is a clean, accurate Mexican Spanish translation that stays within 2 sentences and under 28 words. Model B is also accurate and natural, but it includes extraneous markup tokens, which hurts instruction following and overall quality. (Second judge pass, order swapped — scores are the average of both: Model A is slightly better because it stays natural and concise while preserving the meaning, and it fits the two-sentence, under-28-word constraint. Model B is also accurate, but "hasta las 6:00 PM de mañana" sounds less natural in Mexican Spanish and is a bit more awkward for an app notice.)

Constraint scheduling

Four talks (A, B, C, D) fill four 1-hour slots 9,10,11,12. Constraints: A is before D; C is not first; B is immediately after A; D is not at 12. Give the ONE valid schedule as 'slot: talk' lines, then a one-line justification. If impossible, say so and explain.

DeepSeek-V4-Flash: 9: A 10: B 11: C 12: D Justification: A must be before D, B immediately after A forces A at 9 and B at 10; C cannot be first (9) and D cannot be at 12, so C at 11 and D at 12 satisfies all constraints (D at 12 is allowed only if impossible elsewhere, but here D at 11 would conflict with the only remaining slot for C, and D cannot be at 12? Wait—checking: D is not at 12, so D at 12 violates that. Correction: Actually D cannot be at 12, so the only valid arrangement is A=9, B=10, C=12, D=11, but then C is last, not first, which is fine. Let me re-evaluate: A before D, B after A → AB block must be 9-10 or 10-11. If AB at 9-10, remaining slots 11 and 12: C cannot be first (fine), D cannot be at 12 → D must be at 11, C at 12. That gives 9A,10B,11D,12C. Thus valid schedule: 9: A 10: B 11: D 12: C Justification: A before D, B immediately after A forces A9 B10; D not at 12 so D11, C12 (C not first satisfied).
Cohere-command-a-plus-05-2026: <|START_TEXT|>9: A 10: B 11: D 12: C Justification: A precedes D (9 < 11), C is not first (slot 12), B follows A immediately (9→10), and D is not at 12 (slot 11).<|END_TEXT|>

Model B gives the correct unique schedule cleanly and matches the requested format with a concise justification. Model A eventually reaches the same correct schedule, but it includes an incorrect intermediate schedule, self-contradiction, and extra text that weakens instruction adherence and clarity. (Second judge pass, order swapped — scores are the average of both: Model B gives the unique valid schedule directly and its justification is fully correct and concise. Model A eventually reaches the same schedule, but it first presents an invalid one, includes contradictory self-corrections, and does not cleanly follow the requested output format.)

Matchup powered by OpenRouter.