Head to head: DeepSeek-V4-Pro vs gpt-oss-120b
DeepSeek-V4-Pro vs gpt-oss-120b
By Ryan Merket · Published
DeepSeek-V4-Pro and gpt-oss-120b trade strengths across coding, structured data, constrained writing, and precision-heavy language tasks. The decisive differences come from small instruction-following errors, not a difference on these tasks.
The topline scores—100.1 for DeepSeek-V4-Pro and 102.0 for gpt-oss-120b—look more decisive than they are. With only limited confidence that either model is genuinely better, this is statistical noise, not a meaningful lead. gpt-oss-120b was sharper when literal compliance mattered most. It consistently left malformed email addresses as `null`, returned raw JSON without prohibited Markdown fences, preserved exact facts and timing in faithful rewrites, and handled unit-aware math without muddling fractional seconds. It also tended to explain speculative decoding more precisely. DeepSeek-V4-Pro was stronger on several tasks requiring judgment about structure and tone. Its SQL answers more reliably selected the second order rather than the second distinct total, its ticket labels avoided subtle factual distortions, and its standup summaries were generally cleaner and more faithful. It also produced the more natural, accountable vendor-delay emails and often the tighter LFU implementation. Neither advantage held across the full test. SQL, LFU coding, localization, and even dense summarization flipped in some runs, while several supposed differences came down to code fences, bold markup, awkward phrasing, or one unsupported detail. **Final call: too close to call—DeepSeek-V4-Pro and gpt-oss-120b are effectively tied here.**
Summarize dense passage
Summarize the passage below in exactly three bullet points, each one sentence, capturing the mechanism, the tradeoff, and the caveat — no jargon a non-specialist couldn't follow. Return only the bullets. Passage: "Speculative decoding pairs a small draft model with a large target model: the draft proposes several tokens, the target verifies them in one pass, and accepted tokens are kept while the first rejection resets to the target's own choice. This can cut latency substantially when acceptance is high, but the draft model's compute is pure overhead when acceptance is low, and gains evaporate on adversarial or highly novel inputs where the draft and target disagree often."
OpenAI: GPT-5.6 Sol Pro: Model B more precisely explains that proposals are kept only until the larger model encounters a disagreement and substitutes its own word. Model A is clear and compliant but slightly blurs this mechanism by saying the larger model “checks and corrects all at once.” (Second judge pass, order swapped — scores are the average of both: Model B more precisely explains that proposals are kept only until the first disagreement, after which the larger model supplies its own word. Model A is clear and c | Anthropic: Claude Fable 5.1: Both entries deliver exactly three plain-l…
Unit-aware math
A pump moves 3.5 liters every 8 seconds. A tank holds 0.9 cubic meters. Starting empty, how long to fill it, in minutes and seconds (mm:ss), rounded to the nearest second? Show the key steps, then give the final answer on its own line.
OpenAI: GPT-5.6 Sol Pro: Model B is fully correct, unit-aware, concise, and keeps the unrounded time consistent through the conversion; Model A also reaches the correct answer but imprecisely states that “2057 seconds” equals 34 minutes and 17.142857 seconds after previously calculating 2057.142857 seconds. (Second judge pass, order swapped — scores are the average of both: Model B is fully correct, clear, unit-aware, and presents the requested final format cleanly. Model A reaches the same correct result but has a mino | Anthropic: Claude Fable 5.1: Both Model A and Model B correctly convert…
Constrained standup summary
Write a team standup update from the notes below. Constraints: - Exactly 4 bullet points - Each bullet must start with one of these labels in this order: Yesterday:, Today:, Risks:, Help needed: - 9-14 words per bullet - Include the number 17 somewhere - Do not use the words "blocker", "urgent", or "meeting" - Mention both "Harbor" and "invoice parser" Notes: finished Harbor export fix; reviewed 17 failed uploads; today pairing with Mina on invoice parser edge cases; risk is staging data is two days stale; needs analytics to resend April sample files.
OpenAI: GPT-5.6 Sol Pro: Model A satisfies every explicit constraint and is consistently polished and grammatical. Model B also follows the constraints, but the Risks bullet has an awkward comma construction and adds a less-supported claim about upcoming releases. (Second judge pass, order swapped — scores are the average of both: Both outputs satisfy the structural, length, labeling, terminology, and number constraints. Model A is slightly clearer and more polished, while Model B introduces a less-supported reference t | Anthropic: Claude Fable 5.1: Both outputs satisfy every constraint (fou…
LFU cache implementation
Implement an LFU (least-frequently-used) cache class in Python. Requirements: - Class name: LFUCache - Methods: __init__(capacity: int), get(key: int) -> int, put(key: int, value: int) -> None - If capacity is 0, put should do nothing and get returns -1 - Evict the least frequently used key when full; if tied, evict the least recently used among those - Average time complexity for get and put must be O(1) - Use only Python standard library Return code only.
OpenAI: GPT-5.6 Sol Pro: Both Model A and Model B correctly implement O(1)-average LFU eviction with LRU tie-breaking. Model A is marginally better because it is more concise and avoids maintaining a redundant size counter, though both slightly miss the code-only instruction by using Markdown fences. (Second judge pass, order swapped — scores are the average of both: Model B and Model A are both correct O(1)-average LFU implementations, but Model B is slightly better structured and documented through its frequency-updat | Anthropic: Claude Fable 5.1: Both Model A and Model B implement a corre…
Messy contacts to JSON
Convert the messy contact notes below into valid JSON. Output must be an array of objects sorted by company A-Z. Each object must have exactly these fields: company (string), contact (string), email (string or null), phone (string or null), tags (array of lowercase strings). Rules: normalize phone numbers to the form +1-XXX-XXX-XXXX when possible; trim spaces; split tags on commas or slashes; lowercase tags; if email is missing or invalid, use null. Notes: - Northline Labs | Ava Chen | [email protected] | (415) 555-0188 | biotech, West - Quill Harbor| Marcos Ibarra | mibarra[at]quillharbor.com | 212.555.0144 | Publishing / East - Alder Peak Studio | Ren Ito | [email protected] | none | Design, west coast - Beacon Thread | Lila Moss | | 646-555-0199 | Retail, VIP
OpenAI: GPT-5.6 Sol Pro: Model B correctly treats the malformed address "mibarra[at]quillharbor.com" as invalid and sets it to null. Model A improperly repairs the address even though the instructions require invalid emails to be null. (Second judge pass, order swapped — scores are the average of both: Model B correctly sets the malformed Quill Harbor email to null as required. Model A improperly repairs the invalid address instead of using null.) | Anthropic: Claude Fable 5.1: Both outputs are identical except for the Quill Harbor email: the prompt explicitly says to use null for invalid ema…
Polish app-store localization
Translate the following app description into Polish for an app-store listing. Tone: friendly and trustworthy. Keep it to 2 sentences and under 160 characters total. Preserve meaning and mention offline access. "Track field inspections, attach photos, and sync reports when you're back online. Your checklists stay available even without signal."
OpenAI: GPT-5.6 Sol Pro: Model B more precisely preserves “track” and “your,” while remaining within the two-sentence and character limits and clearly conveying access without a signal. Model A sounds slightly more natural in Polish, but “Rób inspekcje” shifts the meaning toward conducting inspections. (Second judge pass, order swapped — scores are the average of both: Model A is more natural and polished in Polish, especially „po powrocie do sieci”, and explicitly communicates offline access. Model B is slightly more f | Anthropic: Claude Fable 5.1: Both outputs satisfy the two-sentence and…
Ticket triage labels
Classify each support ticket into exactly one category: Billing, Bug, Feature Request, Account Access, or How-To. For each, give a 6-12 word justification. Return one item per line in this exact format: <ID> | <Category> | <justification> T1: "My card was charged twice for March, invoice 8841 and 8841A both hit." T2: "After the 3.9.2 update, exporting a .csv creates an empty file on macOS Sonoma." T3: "Can you add a setting so completed routes archive automatically after 30 days?" T4: "I reset my password, but the sign-in screen keeps saying my SSO domain isn't recognized." T5: "What's the fastest way to bulk-tag 200 photos by project code?" T6: "The dashboard total says 214 orders, but the list view only shows 213 when filtered to Q2."
OpenAI: GPT-5.6 Sol Pro: Model A and Model B classify every ticket correctly and satisfy the required format and length, but Model A is slightly more precise because Model B inaccurately describes the password reset itself as unsuccessful in T4. (Second judge pass, order swapped — scores are the average of both: Model B and Model A classify every ticket correctly and satisfy the required format and justification length. Model A has slightly clearer, more natural phrasing overall, particularly for the account-access tick | Anthropic: Claude Fable 5.1: Both models classify all six tickets ident…
SQL window query
Given Postgres tables `orders(id, customer_id, created_at, total_cents)` and `customers(id, name)`, write ONE SQL query returning each customer's name and their SECOND-largest order total (in dollars, 2 decimals), excluding customers with fewer than two orders. Order by that value descending. Return ONLY the SQL.
OpenAI: GPT-5.6 Sol Pro: Model A correctly selects the second order when each customer’s orders are sorted by total and returns one row per qualifying customer. Model B instead selects the second-largest distinct total, can return duplicate rows for tied totals, and incorrectly excludes customers whose two orders have equal totals. (Second judge pass, order swapped — scores are the average of both: Model B finds the second distinct total and can return multiple rows for a customer when orders tie at that total. Model A | Anthropic: Claude Fable 5.1: Model A correctly returns exactly one row p…
Find the contradiction
The following spec contains exactly one internal contradiction. Quote the two conflicting sentences verbatim and explain the conflict in one sentence. Do not fix it. Spec: "Free accounts may create up to three projects. Every account, regardless of tier, may archive unlimited projects. Archiving a project does not count against the project limit. Free accounts are limited to three projects total, including archived ones."
OpenAI: GPT-5.6 Sol Pro: Model A quotes both conflicting sentences exactly and explains the contradiction clearly in one sentence. Model B is substantively correct, but the added bold formatting inside the second quotation makes it marginally less faithful to the requirement to quote the sentence verbatim. (Second judge pass, order swapped — scores are the average of both: Model A quotes both conflicting sentences exactly and explains the contradiction in one concise sentence. Model B is equally correct, but adding bold | Anthropic: Claude Fable 5.1: Both Model A and Model B correctly identif…
Vendor delay email rewrite
Rewrite this workplace message as a polished email to a hospital procurement manager. Audience: busy external client. Tone: calm, accountable, solution-oriented. Length: 120-150 words. Source message: "Hi—quick heads-up. The 48 bedside scanners for Eastbay Surgical won't arrive Thursday after all. Our carrier held pallet BQ-17 in Reno because the lithium battery paperwork on 11 units was mismatched. We fixed the docs this morning. New ETA is Monday 10:30 a.m. We can still send 20 corded scanners from our Sacramento stock tomorrow if that helps your go-live. Sorry, messy on our side. Let me know what you want us to do."
OpenAI: GPT-5.6 Sol Pro: Model B is concise, polished, within the requested length, and avoids Model A’s unsupported promise of Friday arrival. Model A demonstrates slightly stronger accountability and more precisely describes the paperwork mismatch, but the invented arrival timing is a meaningful correctness issue. (Second judge pass, order swapped — scores are the average of both: Model B is polished, appropriately concise, and offers the interim solution without introducing a new delivery commitment. Model A is sligh | Anthropic: Claude Fable 5.1: Model A reads more accountable and solutio…
Faithful rewrite
Rewrite this blunt internal note as a warm, professional Slack message to the whole engineering team (60–90 words), keeping every fact intact and adding no new commitments. Return only the message. Note: "Deploy is frozen until the memory leak in the image service is fixed. Priya is on it. Do not merge to main. ETA tomorrow noon."
OpenAI: GPT-5.6 Sol Pro: Model B preserves the exact noon ETA and avoids making a new commitment. Model A changes the ETA to “around noon” and adds “we’ll keep you posted,” which is an unsupported commitment. (Second judge pass, order swapped — scores are the average of both: Model B preserves all facts, stays within the required length, and adds no substantive commitment. Model A is polished but weakens “tomorrow noon” to “around noon” and adds the new commitment “we’ll keep you posted.”) | Anthropic: Claude Fable 5.1: Both hit the word range and preserve the core facts, but Model A adds a n…
Nuanced classification
Classify each review's sentiment as "positive", "negative", or "mixed", and give a 6-word-max reason. Return ONLY a JSON array of {"text","label","reason"} in input order. Reviews: ["Fast shipping but the fabric feels cheap.", "Absolutely love it, wearing it daily!", "It broke after a week. Refund was quick and painless though."]
OpenAI: GPT-5.6 Sol Pro: Model B fully follows the JSON-only requirement and provides accurate labels with concise reasons. Model A is substantively correct, but the Markdown code fence violates the required output format, and “wears daily” is slightly awkward. (Second judge pass, order swapped — scores are the average of both: Model B fully follows the JSON-only requirement and provides accurate labels with concise reasons. Model A is substantively correct, but the Markdown code fence violates the required output forma | Anthropic: Claude Fable 5.1: Both Model A and Model B produce identical…
Matchup powered by OpenRouter.