Head to head: DeepSeek-V4-Flash-0731 vs gpt-oss-120b
DeepSeek-V4-Flash-0731 vs gpt-oss-120b
This one is a genuine photo finish. Across the slate, DeepSeek-V4-Flash-0731 and gpt-oss-120b trade narrow wins, split the rest, and land in an aggregate dead heat that the stats simply do not separate.
The topline is simple: these models are effectively even here. The aggregate scores are 102.8 for DeepSeek-V4-Flash-0731 and 103.7 for gpt-oss-120b, and that gap is small enough that this remains **too close to call** — just 64% confidence that either model is truly better is nowhere near enough to pretend there’s a decisive leader. If you want the shape of the matchup, gpt-oss-120b was a bit sharper on instruction-following when structure and exact method mattered. It took **precise proofreading**, **SQL window query**, **messy contacts to JSON**, and **faithful rewrite**. Those wins were not about raw brilliance so much as cleaner compliance: using the requested window-function pattern, returning JSON without Markdown fences, and staying closer to the source text instead of "helpfully" adding new commitments. DeepSeek-V4-Flash-0731 answered with narrower but real advantages on tasks where correctness plus justification mattered. It won **constraint scheduling** by explaining why the solution was forced, beat gpt-oss-120b on the **TypeScript LRU cache** because the rival answer had a meaningful type inconsistency, and edged **constrained status note** by packing the required fields more precisely. That’s a credible counterweight, not a consolation prize. The bigger story, though, is how much ended level. They tied on the Python **LRU cache**, **support ticket labeling**, **clinic rota assignment**, **invoice facts extraction**, and **summarize dense passage**. In other words, on a large chunk of this benchmark, both models were simply doing the job to the same standard. **Final call: tie.** gpt-oss-120b had slightly more category wins, DeepSeek-V4-Flash-0731 had some cleaner task-specific hits, and the aggregate margin is too thin to support a real verdict. Call this matchup what it is: a statistical dead heat.
Precise proofreading
Correct ONLY the grammar, spelling, and punctuation errors in the sentence below without changing wording, tone, or meaning, then on a second line list each change as 'was -> now'. Sentence: "Its been a long time since we seen there new office, and me and him was suprised how quite it was."
Model A is better because it mostly limits changes to grammar, spelling, and punctuation and preserves the original wording, though it misses some grammatical errors and unnecessarily repeats the original sentence. Model B makes more corrections, but it violates the instruction not to change wording by rewriting phrases like "we seen" to "we saw" and "me and him" to "he and I." (Second judge pass, order swapped — scores are the average of both: Model B fixes all grammar, spelling, and punctuation errors and provides a complete change list, though it arguably changes wording more than necessary with "me and him -> he and I." Model A leaves major grammar errors uncorrected ("we seen," "me and him was") and also includes the original sentence before the corrected version, which does not follow the requested format.)
SQL window query
Given Postgres tables `orders(id, customer_id, created_at, total_cents)` and `customers(id, name)`, write ONE SQL query returning each customer's name and their SECOND-largest order total (in dollars, 2 decimals), excluding customers with fewer than two orders. Order by that value descending. Return ONLY the SQL.
Model B is better because it uses a single window-function query that directly returns the second-highest order per customer and naturally excludes customers with fewer than two orders. Model A can work, but it relies on repeated correlated subqueries instead of the requested window-query style and is less clean and efficient. (Second judge pass, order swapped — scores are the average of both: Model B is better because it uses a single window-query pattern that directly returns the second-largest order per customer and naturally excludes customers with fewer than two orders via `ROW_NUMBER() = 2`. Model A can also produce the requested result, but it relies on repeated correlated subqueries instead of the requested window approach, making it less aligned with the task and less efficient.)
LRU cache implementation
Write Python 3 code only. Implement an LRU cache class with this exact API: - LRUCache(capacity: int) - get(key: int) -> int returns value or -1 if missing - put(key: int, value: int) -> None Both operations must be O(1) time. Use your own doubly linked list + hash map; do not import OrderedDict or functools. Capacity is always >= 1. Include no explanations, no comments, no test code.
Model A and Model B both correctly implement an O(1) LRU cache using a hash map plus a custom doubly linked list, and both fully follow the prompt by providing Python 3 code only with the exact required API behavior. Model B is slightly more optimized stylistically with __slots__, while Model A is slightly more straightforward, but neither has a meaningful quality advantage for this task. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both correctly implement an O(1) LRU cache using a hash map plus a custom doubly linked list, and both fully follow the prompt by providing Python 3 code only with the required API and no comments or tests. Model B is slightly more compact, while Model A is equally correct and clear, so neither meaningfully outperforms the other.)
Support ticket labeling
Classify each ticket into exactly one category: billing, technical, account, shipping. Return one line per item in this exact format: ID | category | brief justification Items: T-881: "Charged twice for the March Pro plan after I switched cards. Please refund one payment." T-882: "Two-factor codes stopped arriving after I changed my phone number and now I can't sign in." T-883: "Order QN-54 shows delivered, but the replacement toner never reached our office." T-884: "Desktop app v5.2 crashes whenever I export a ledger to CSV." T-885: "Need the invoice updated to show our VAT ID and legal entity name."
Model A and Model B both assign the correct category to every ticket and follow the required line format with concise justifications. Model B is slightly more polished stylistically, but the difference is not meaningful enough to outweigh that both outputs are fully correct and compliant. (Second judge pass, order swapped — scores are the average of both: Model B and Model A classify all five tickets correctly and follow the required output format with one line per item. Model A is slightly more concise, while Model B is slightly more explanatory, but neither has a meaningful quality advantage for this task.)
Clinic rota assignment
Assign four clinicians to four half-day clinic slots. Return ONLY a JSON object mapping each slot to one clinician. Clinicians: Dr Ames, Dr Bello, Dr Chen, Dr Duarte. Slots: Mon-AM, Mon-PM, Tue-AM, Tue-PM. Constraints: 1) Dr Chen can work only in the morning. 2) Dr Bello cannot do Mon-PM. 3) Dr Ames must be scheduled later in the week than Dr Bello. 4) Dr Duarte is unavailable Tue-AM. 5) Mon-AM must be either Dr Bello or Dr Duarte. 6) Tue-PM cannot be Dr Chen. 7) Dr Ames cannot do Mon-AM. Use each clinician exactly once. There is exactly one valid assignment.
Model A and Model B provide the same valid assignment, which satisfies all scheduling constraints and uses each clinician exactly once. Both also follow the required JSON-only format, with the only difference being whitespace, which does not affect quality. (Second judge pass, order swapped — scores are the average of both: Model B and Model A provide the same valid JSON assignment, and it satisfies all scheduling constraints exactly. Both follow the instruction to return only a JSON object, with no meaningful difference in quality.)
Invoice facts extraction
Extract the requested fields from this messy inline invoice thread and return ONLY a JSON object with exactly these keys: vendor_name, invoice_id, invoice_date, due_date, currency, subtotal, tax, shipping, total, po_number, bill_to_company, line_items. line_items must be an array of objects with keys description, qty, unit_price, line_total. Text: "Fwd: AP copy — Northline BioSystems // INV NB-2048-A Hi Mara, please process this one. Bill to: Helio Quarry Labs, Attn AP, 18 Vardon St. Vendor footer says Northline BioSystems Ltd. Remit refs: PO# HQ-7719. Invoice date 2025-02-14; terms Net 21 (so due 2025-03-07). Currency: USD. Items: - Cryo labels, 6 packs @ $18.50 = $111.00 - Solvent trap, 2 @ $74 = $148 - Bench mat (ESD), 3 @ $29.99 = $89.97 Sub-total 348.97 / Sales tax 27.92 / Freight 14.00 Grand Total: USD 390.89 Ignore old balance mention below: 'previous invoice NB-1991 remains unpaid at 88.00' — not part of this invoice."
Model A and Model B are identical and correctly extract all requested invoice fields, including the due date, PO number, bill-to company, and line items, while excluding the unrelated old balance. Both also follow the instruction to return exactly the required JSON structure. (Second judge pass, order swapped — scores are the average of both: Model B and Model A extract the same invoice facts accurately, including the due date, totals, PO number, bill-to company, and all line items, while correctly ignoring the old balance note. Model B and Model A also fully follow the required JSON-only structure with exactly the requested keys, so neither is better.)
Messy contacts to JSON
Convert the messy contact notes below into valid JSON only. Output an array of objects sorted by last_name A-Z. Schema for each object: {"first_name":"","last_name":"","company":"","email":"","phone":"","tags":[]}. Normalize phones to digits and dashes in this form XXX-XXX-XXXX. Lowercase emails. tags must be lowercase strings, unique, sorted A-Z. Notes: 1) "Mina Park — Lattice Harbor; email: Mina.Park@LatticeHarbor.com ; cell 415.555.0188 ; tags: West, renewals, VIP" 2) "R. dos Santos, Oriole Freight, +1 (646) 555-0131, r.dossantos@oriolefreight.io, tags=logistics; east" 3) "Jules Berman / Novera Health / jberman@noverahealth.org / 3125550199 / tags: pilot, Midwest, pilot" 4) "Asha Iqbal at Cinder Peak - phone 206-555-0144 - email ASHA@cinderpeak.co - tags: partner"
Model B is slightly better because both outputs have the same correctly parsed, normalized, and sorted contact data, but Model A wraps the JSON in a Markdown code fence while the prompt required valid JSON only. Model B follows that requirement exactly. (Second judge pass, order swapped — scores are the average of both: Model B fully satisfies the prompt: it returns valid JSON only, with correctly normalized phones, lowercased emails, deduplicated/sorted lowercase tags, and entries sorted by last_name. Model A’s content is otherwise correct, but it violates the instruction to output JSON only by wrapping the array in Markdown code fences.)
Faithful rewrite
Rewrite this blunt internal note as a warm, professional Slack message to the whole engineering team (60–90 words), keeping every fact intact and adding no new commitments. Return only the message. Note: "Deploy is frozen until the memory leak in the image service is fixed. Priya is on it. Do not merge to main. ETA tomorrow noon."
Model B is more faithful to the original note while still sounding warm and professional. Model A adds new commitments and content not present in the source (such as keeping the team posted, inviting questions, and adding context), which violates the instruction to keep every fact intact and add no new commitments. (Second judge pass, order swapped — scores are the average of both: Model B is warmer and professional while preserving the key facts with only minor wording drift around the ETA. Model A adds multiple new commitments and details not in the note ("we’ll keep you posted," "let us know if you have any questions or need context") and changes Priya’s status from "is on it" to "actively investigating," making it less faithful.)
Summarize dense passage
Summarize the passage below in exactly three bullet points, each one sentence, capturing the mechanism, the tradeoff, and the caveat — no jargon a non-specialist couldn't follow. Return only the bullets. Passage: "Speculative decoding pairs a small draft model with a large target model: the draft proposes several tokens, the target verifies them in one pass, and accepted tokens are kept while the first rejection resets to the target's own choice. This can cut latency substantially when acceptance is high, but the draft model's compute is pure overhead when acceptance is low, and gains evaporate on adversarial or highly novel inputs where the draft and target disagree often."
Model A and Model B both fully satisfy the format and content requirements: each gives exactly three one-sentence bullets covering the mechanism, tradeoff, and caveat in clear non-technical language. Model A is slightly more precise about switching to the larger model’s own choice, while Model B is slightly smoother stylistically, so neither is meaningfully better overall. (Second judge pass, order swapped — scores are the average of both: Model A is slightly better because it preserves the reset behavior more clearly by saying the larger model switches to its own word on the first mistake, while still staying simple and concise. Model B is also strong, but "stopping at the first it rejects" is a bit less explicit about the replacement step described in the passage.)
Constraint scheduling
Four talks (A, B, C, D) fill four 1-hour slots 9,10,11,12. Constraints: A is before D; C is not first; B is immediately after A; D is not at 12. Give the ONE valid schedule as 'slot: talk' lines, then a one-line justification. If impossible, say so and explain.
Model A and Model B give the same valid schedule, but Model A is slightly better because its justification explains why the arrangement is uniquely forced, not just that it satisfies the constraints. Both follow the requested format closely, though Model A’s reasoning is more complete and useful. (Second judge pass, order swapped — scores are the average of both: Model A is better because it gives the same correct unique schedule while also briefly showing why it is forced, which better satisfies the request for a justification. Model B is also correct and well-formatted, but its justification is slightly less informative.)
LRU cache
Implement a class `LRUCache<K, V>` in TypeScript with a fixed capacity set in the constructor, and O(1) `get(key): V | undefined` and `set(key, value): void`. Accessing or updating a key must mark it most-recently-used; inserting beyond capacity must evict the least-recently-used entry. Return ONLY the class, no prose.
Model A is concise, valid TypeScript, and correctly implements LRU behavior in O(1) using Map insertion order. Model B attempts a classic linked-list design, but its map is typed as Map<K, ListNode> while ListNode is generic and instantiated with K,V, making the code internally inconsistent and likely invalid TypeScript; it also returns more than just the requested class. (Second judge pass, order swapped — scores are the average of both: Model A is better because it is concise, valid TypeScript, and correctly provides O(1) get/set with proper LRU behavior using Map insertion order. Model B has a significant type error: it declares the map and node references as a non-generic ListNode, which makes key/value types incompatible with K and V and undermines correctness in TypeScript.)
Constrained status note
Write a project status note that follows ALL rules exactly: - Exactly 4 bullet points - Each bullet starts with "- " - Total response length must be 220-260 characters including spaces and line breaks - Include these fields somewhere, exactly once each: Owner: Priya Nair ; Due: 18 Sep 2026 ; Risk: vendor delay - Mention budget is on track - Do NOT use the words "project", "update", "blocked", or "issue" - Do NOT use any emoji or hashtags
Model A better satisfies the required content: it has exactly four bullets, includes all required fields exactly once, mentions budget on track, and avoids banned words, though it appears to exceed the 220-260 character limit. Model B is shorter and clear, but it fails the requirement to include the exact field "Owner: Priya Nair" because it changes the phrasing, so Model A is stronger overall. (Second judge pass, order swapped — scores are the average of both: Model A better satisfies the constraints: it has exactly four bullets, includes each required field exactly once, mentions budget is on track, and avoids the banned words. Model B is solid, but it appears to exceed the maximum character limit and is slightly less precise in packing the required fields efficiently.)
Matchup powered by OpenRouter.