Head to head: Sarvam M vs DeepSeek-V4-Pro

DeepSeek-V4-Pro wins outright, and the numbers leave no room for spin: 113.0 to 44.4, a 12–0 task sweep, with a 100% confidence statistical verdict. That is not a narrow technicality or a split decision on style. It is a decisive result driven by one model reliably completing the assignment while the other too often failed at the most basic requirement: giving the user the thing they asked for.

The pattern is brutal for Sarvam M. On strict-format tasks, it repeatedly sabotaged itself with visible reasoning, commentary, markdown fences, stray tags, or incomplete output. That failure mode shows up across JSON extraction, incident parsing, nuanced classification, inbox triage, meeting-note summarization, policy proofreading, and the faithful rewrite task. In several cases, the judges explicitly noted that Sarvam M had some of the substance right, but still lost badly because it would not stop talking and follow the format. In production, that is not a cosmetic flaw; it is disqualifying.

DeepSeek-V4-Pro, by contrast, kept doing the unglamorous work correctly. It returned valid JSON when asked for JSON, a clean markdown table when asked for a table, and ready-to-send prose when asked for customer-facing writing. It also won the harder implementation tasks on merit, not just compliance. In the concurrency bug fix, it applied the key insight immediately—cache the in-flight promise and clear it on rejection—while Sarvam M failed to provide a usable corrected function at all. In the LRU cache task, DeepSeek-V4-Pro delivered the requested class cleanly and preserved behavior; Sarvam M buried shaky TypeScript inside extra prose and unnecessary scaffolding.

What makes this matchup especially lopsided is that there is no compensating upside on Sarvam M’s side. It did not lose because it was more creative, more ambitious, or occasionally brilliant but inconsistent. It lost because it was less dependable on nearly every kind of task in the set: extraction, rewriting, classification, contradiction finding, arithmetic reasoning, and code repair. DeepSeek-V4-Pro was not flawless—there were minor normalization and phrasing issues here and there—but those were small dents on otherwise usable answers. Sarvam M’s recurring inability to honor output constraints was a structural problem, and this benchmark punished it accordingly.

Final call: DeepSeek-V4-Pro is the clear winner, and not by inches. On this evidence, it is the far more reliable text model for real workloads, while Sarvam M looks unready for any setting where instruction fidelity and clean deliverables actually matter.

How they were tested

We ran 12 fresh text tasks, generated on the fly for this matchup so neither model could prepare in advance, and had gpt-5.4 score each one. To cancel position bias, every task was judged twice — once in each presentation order — and every number reported here, including the headline totals, is the average of both passes. Sarvam M scored 44.4 to DeepSeek-V4-Pro's 113.0.

1. Concurrency bug fix

This TypeScript function is meant to memoize an async loader but has a race: concurrent callers can each trigger the underlying fetch. Fix it so the fetch runs at most once per key, and a rejected fetch does NOT poison the cache (a later call must retry). Return ONLY the corrected function. ts const cache = new Map<string, Promise<string>>(); async function load(key: string, fetcher: (k: string) => Promise<string>) { if (cache.has(key)) return cache.get(key)!; const value = await fetcher(key); cache.set(key, Promise.resolve(value)); return value; }

Winner: DeepSeek-V4-Pro — Model B provides the correct fix: it caches the in-flight promise immediately so concurrent callers share one fetch, and deletes the cache entry on rejection so later calls retry. Model A does not return only the corrected function and mostly contains unfinished reasoning rather than a usable answer. (Second judge pass, order swapped — scores are the average of both: Model B correctly caches the in-flight promise immediately, ensuring at most one fetch per key, and deletes the cache entry on rejection so later calls retry. Model A does not provide a corrected function at all and violates the instruction to return only the function.)

2. Strict JSON extraction

Extract every meeting from this text into a JSON array of {"title","day","start24","durationMinutes"}. Use 24-hour "HH:MM" for start24, integer minutes for duration, and title-case titles. Return ONLY valid JSON, no commentary. Text: "Standup is daily at 9am for a quarter hour. The design review runs thursday from 2:30-4pm. Payroll sync — first monday, 11:00 to noon."

Winner: DeepSeek-V4-Pro — Model B is far better because it returns the requested structured meeting data with correct times and durations, while Model A provides chain-of-thought commentary instead of valid JSON and never completes the extraction. Model B has only minor normalization issues in the day field casing, but it substantially follows the prompt; Model A does not. (Second judge pass, order swapped — scores are the average of both: Model B correctly extracts the three meetings with appropriate times and durations, and its only minor issue is wrapping the JSON in a code fence rather than returning raw JSON. Model A fails the task by providing commentary instead of only valid JSON and never delivers the required extracted array.)

3. Extract incident data

Extract the requested fields from this messy incident text and return ONLY valid JSON with these keys exactly: incident_id, date, reporter, service, affected_region, start_time, end_time, peak_error_rate, customer_impact, suspected_cause, mitigation, follow_up_owner. Text: "fwd: notes from bridge — Incident INC-7842. Reporter was N. Velasquez from Support. Date: 2026-02-17. Service impacted: LedgerSync API (not the dashboard). Customers in eu-west mostly saw failures starting around 08:12 UTC; by 08:25 UTC error rate peaked near 37% on POST /v2/reconcile. At 09:03 UTC traffic looked normal again. Impact summary: some reconciliation jobs were delayed or retried, but no data loss confirmed. Infra first suspected a bad deploy, then ruled that out; likely trigger was an expired connection pool certificate on db-proxy-3. Mitigation used: shifted traffic away from the proxy and restarted affected workers. Follow-up owner should be Amina Noor from Platform Eng."

Winner: DeepSeek-V4-Pro — Model B is better because it returns only valid JSON as requested and extracts the fields accurately. Model A includes extensive non-JSON reasoning and markdown fencing, which directly violates the instruction to return only valid JSON, even though most extracted values are correct. (Second judge pass, order swapped — scores are the average of both: Model B is far better because it returns only valid JSON and accurately extracts nearly all fields from the source text. Model A includes extensive non-JSON commentary despite an explicit JSON-only requirement, and it also alters field formats and values unnecessarily, though some extracted content is still substantively correct.)

4. Write customer delay email

Draft an email to a wholesale customer explaining a shipment delay. Audience: purchasing manager at a boutique grocery chain. Tone: professional, calm, accountable. Length: 120–150 words. Facts to include: - Their order PO-9817 for 240 cases of rosemary crackers was due to ship today. - A routine quality check found several lots with broken inner seals. - You paused the shipment to avoid sending compromised product. - Replacement production finishes tomorrow afternoon. - New estimated ship date: Thursday, 24 July. - Offer either split shipment of available unaffected stock now, or full shipment on the new date. - Apologize and ask which option they prefer.

Winner: DeepSeek-V4-Pro — Model B is better because it cleanly delivers a ready-to-send email that includes all required facts, stays within the requested tone and length, and avoids extraneous meta-commentary. Model A contains visible planning/thinking text and awkward artifacts, which significantly hurts instruction adherence and professionalism despite covering most of the content. (Second judge pass, order swapped — scores are the average of both: Model B cleanly delivers a polished customer email that includes all required facts, matches the requested tone, and stays within the length target. Model A includes visible chain-of-thought/meta commentary and an extraneous word count, which violates the task format despite the email draft itself containing the needed information.)

5. Proofread policy update

Rewrite the passage below to correct grammar, punctuation, and clarity WITHOUT changing its meaning. Then list 4 brief bullet points describing the main edits. Passage: "Beginning July 1, all contractors must submits their weekly hours by Friday 6pm local time, otherwise payment may be delay until the next cycle. If a manager approve hours after payroll closes, finance won't be able making manual transfers except there is a signed exception from director-level. This policy apply to every region however country-specific holidays should still be consider when reviewing late submissions."

Winner: DeepSeek-V4-Pro — Model B cleanly rewrites the passage and provides four brief, relevant edit bullets. Model A includes extensive chain-of-thought-style preamble, introduces meaning changes such as "process manual transfers" and "director-level stakeholder," and adds unnecessary commentary that does not follow the requested format. (Second judge pass, order swapped — scores are the average of both: Model B cleanly rewrites the passage with accurate grammar and punctuation while preserving meaning, and it provides exactly four brief edit bullets. Model A includes extensive chain-of-thought and extra commentary, changes meaning in places (for example, 'manual transfers' to 'process manual transfers,' 'director-level employee/stakeholder,' and 'must still be considered'), and is less compliant with the requested format.)

6. Summarize meeting notes

Summarize the meeting notes below into EXACTLY 4 bullet points. Each bullet must be 10–16 words, factual, and include no opinions or recommendations. Source notes: "Ops sync, Tuesday 9:00. Priya said warehouse returns climbed from 3.1% to 4.4% after the switch to the thinner mailer. Leon confirmed most complaints mention corner dents on the ceramic jars, not leaks. Marta can restore the previous mailer spec by next Wednesday, but unit cost rises by $0.18. Dev noted the storefront banner still promises 'plastic-free packaging' through May, so changing materials may require copy review. Finance asked whether the dented units can be sold at outlet pricing instead of written off. Team agreed to run the old mailer on 600 outbound orders this week and compare damage rates before a full rollback."

Winner: DeepSeek-V4-Pro — Model B better satisfies the format by providing exactly four clean bullets within the 10–16 word limit, while Model A includes substantial extraneous reasoning before the bullets and one bullet is only nine words. Both are factually solid, but Model B is more compliant and concise. (Second judge pass, order swapped — scores are the average of both: Model B cleanly provides exactly four factual bullet points and stays close to the source, though one bullet omits the timing detail about next Wednesday. Model A includes extensive chain-of-thought and non-bullet prose, violating the required format despite ending with four acceptable bullets.)

7. Faithful rewrite

Rewrite this blunt internal note as a warm, professional Slack message to the whole engineering team (60–90 words), keeping every fact intact and adding no new commitments. Return only the message. Note: "Deploy is frozen until the memory leak in the image service is fixed. Priya is on it. Do not merge to main. ETA tomorrow noon."

Winner: DeepSeek-V4-Pro — Model B is better because it returns only the Slack message, preserves the core facts, and reads warm and professional. Model A violates the format by including visible reasoning and also adds a new commitment ('we’ll keep everyone posted'), while Model B is slightly imperfect for softening 'ETA tomorrow noon' to 'around noon' and adding 'Priya’s got it covered.' (Second judge pass, order swapped — scores are the average of both: Model B is a warm, professional Slack message that preserves the core facts and mostly follows the constraints, though it slightly embellishes with reassuring phrasing. Model A is substantially worse because it includes visible chain-of-thought and adds a new commitment ('we’ll keep everyone posted'), violating the instruction to return only the message and add no new commitments.)

8. Find the contradiction

The following spec contains exactly one internal contradiction. Quote the two conflicting sentences verbatim and explain the conflict in one sentence. Do not fix it. Spec: "Free accounts may create up to three projects. Every account, regardless of tier, may archive unlimited projects. Archiving a project does not count against the project limit. Free accounts are limited to three projects total, including archived ones."

Winner: DeepSeek-V4-Pro — Model B correctly identifies the contradictory pair verbatim and explains the conflict in one sentence. Model A selects the wrong quoted sentence pair and adds unnecessary chain-of-thought-style preamble, so it is less correct and follows the instructions less closely. (Second judge pass, order swapped — scores are the average of both: Model B correctly quotes the actual contradictory sentences and explains the conflict in one sentence while following the prompt exactly. Model A includes unnecessary chain-of-thought style preamble and ultimately quotes the wrong first sentence, so its contradiction identification is incorrect and less compliant.)

9. Compute shipment shortfall

A buyer is checking whether a rush order can be fulfilled today. Inventory available now: - 18 cartons of blue folders, 24 folders per carton - 11 cartons of red folders, 30 folders per carton - 7 loose green folders Order requirements: - 260 folders total - At least 70 must be blue - Red folders must be exactly twice the number of green folders - All counts must be whole folders Question: Is the order feasible with current inventory? If yes, provide one valid combination of blue, red, and green folders. If no, state clearly that it is impossible and give the maximum total number of folders that could be shipped while still satisfying the color constraints. Show your reasoning step by step.

Winner: DeepSeek-V4-Pro — Model B is better because it gives a clear, complete, and correct step-by-step solution that directly answers feasibility and provides a valid combination. Model A reaches the same correct conclusion, but it is rambling, repetitive, includes stray chain-of-thought-style text, and is cut off before fully finishing the inventory verification. (Second judge pass, order swapped — scores are the average of both: Model B is correct, complete, and clearly structured with concise step-by-step reasoning that directly answers the prompt. Model A reaches the same feasible combination, but it is rambling, includes unnecessary self-corrections, exposes internal-style deliberation, and appears truncated at the end, which hurts instruction adherence and writing quality.)

10. Nuanced classification

Classify each review's sentiment as "positive", "negative", or "mixed", and give a 6-word-max reason. Return ONLY a JSON array of {"text","label","reason"} in input order. Reviews: ["Fast shipping but the fabric feels cheap.", "Absolutely love it, wearing it daily!", "It broke after a week. Refund was quick and painless though."]

Winner: DeepSeek-V4-Pro — Model B fully follows the prompt by returning only a JSON array with correct labels and concise reasons. Model A includes extensive extra commentary and a think block, violating the required output format despite mostly correct classifications. (Second judge pass, order swapped — scores are the average of both: Model B is fully correct and follows the required format exactly with concise reasons under six words. Model A has correct labels, but it violates the instruction to return only a JSON array by including extensive extra commentary and a think block.)

11. Classify inbox requests

Classify each message into exactly one category: Billing, Technical support, Sales inquiry, Account access, or Feedback. For each, give a 1-sentence justification. Return a markdown table with columns: Item | Category | Justification. Items: A. "I was charged twice for the March Pro plan after upgrading from Starter on 3/14. Can someone refund one of the charges?" B. "Your iPad app freezes on the export screen after I choose CSV. I tried reinstalling and it still hangs at 92%." C. "We’re evaluating tools for a 40-person field team and need SSO plus annual pricing. Who can walk us through options?" D. "I changed phones and no longer receive the 2-factor code, so I’m locked out of the admin dashboard." E. "The new left-hand navigation is much faster, but the pale gray labels are hard to read on older monitors."

Winner: DeepSeek-V4-Pro — Model B is better because it cleanly follows the requested format with only the markdown table and gives precise, correct classifications and justifications. Model A’s classifications are also correct, but it includes extraneous reasoning text outside the table, which violates the instruction to return the table. (Second judge pass, order swapped — scores are the average of both: Model B is fully correct, concise, and follows the requested format exactly. Model A reaches the same classifications, but it includes extraneous chain-of-thought-style commentary and a stray closing tag, which violates the instruction to return just the markdown table and hurts writing quality.)

12. LRU cache

Implement a class LRUCache<K, V> in TypeScript with a fixed capacity set in the constructor, and O(1) get(key): V | undefined and set(key, value): void. Accessing or updating a key must mark it most-recently-used; inserting beyond capacity must evict the least-recently-used entry. Return ONLY the class, no prose.

Winner: DeepSeek-V4-Pro — Model B is the better answer because it returns only the class and provides a concise, valid TypeScript implementation that preserves LRU behavior, even though its O(1) claim is questionable due to Map iteration. Model A violates the prompt by including extensive prose and extra classes, and its code has TypeScript type issues around sentinel nodes and possibly undefined Map results. (Second judge pass, order swapped — scores are the average of both: Model B cleanly returns only the requested class and correctly implements LRU behavior in O(1) average time using Map insertion order. Model A violates the output constraint by including extensive prose, chain-of-thought, and extra classes, and its TypeScript is also shaky due to generic sentinel nodes initialized with undefined and unchecked Map.get results.)

See every prompt and the full side-by-side outputs in the interactive Head-to-Head.