Head to head: DeepSeek-V4-Pro vs gpt-oss-120b

DeepSeek-V4-Pro vs gpt-oss-120b

By · Published

RuntimeWire Head-to-Head: Head to head: DeepSeek-V4-Pro vs gpt-oss-120b
RuntimeWire Head-to-Head matchup

DeepSeek-V4-Pro and gpt-oss-120b trade strengths across coding, structured data, constrained writing, and precision-heavy language tasks. The decisive differences come from small instruction-following errors, not a difference on these tasks.

The topline scores—100.1 for DeepSeek-V4-Pro and 102.0 for gpt-oss-120b—look more decisive than they are. With only limited confidence that either model is genuinely better, this is statistical noise, not a meaningful lead. gpt-oss-120b was sharper when literal compliance mattered most. It consistently left malformed email addresses as `null`, returned raw JSON without prohibited Markdown fences, preserved exact facts and timing in faithful rewrites, and handled unit-aware math without muddling fractional seconds. It also tended to explain speculative decoding more precisely. DeepSeek-V4-Pro was stronger on several tasks requiring judgment about structure and tone. Its SQL answers more reliably selected the second order rather than the second distinct total, its ticket labels avoided subtle factual distortions, and its standup summaries were generally cleaner and more faithful. It also produced the more natural, accountable vendor-delay emails and often the tighter LFU implementation. Neither advantage held across the full test. SQL, LFU coding, localization, and even dense summarization flipped in some runs, while several supposed differences came down to code fences, bold markup, awkward phrasing, or one unsupported detail. **Final call: too close to call—DeepSeek-V4-Pro and gpt-oss-120b are effectively tied here.**

Summarize dense passage

Summarize the passage below in exactly three bullet points, each one sentence, capturing the mechanism, the tradeoff, and the caveat — no jargon a non-specialist couldn't follow. Return only the bullets. Passage: "Speculative decoding pairs a small draft model with a large target model: the draft proposes several tokens, the target verifies them in one pass, and accepted tokens are kept while the first rejection resets to the target's own choice. This can cut latency substantially when acceptance is high, but the draft model's compute is pure overhead when acceptance is low, and gains evaporate on adversarial or highly novel inputs where the draft and target disagree often."

DeepSeek-V4-Pro: - The method works by having a smaller, faster model guess multiple words ahead, which a larger, more accurate model then checks and corrects all at once. - This speeds up generation when the guesses are mostly right, but the smaller model's work becomes wasted effort if its guesses keep getting rejected. - The speed boost disappears entirely on unpredictable or unfamiliar content where the two models frequently disagree.
gpt-oss-120b: - A small, fast model proposes the next words and a larger, more accurate model checks them, keeping the proposals until it finds one it disagrees with and then picks its own word. - This speeds things up when the fast model’s suggestions are usually correct, but when they’re often wrong the extra work just adds delay. - The benefit disappears for tricky or unfamiliar inputs where the two models frequently disagree.

OpenAI: GPT-5.6 Sol Pro: Model B more precisely explains that proposals are kept only until the larger model encounters a disagreement and substitutes its own word. Model A is clear and compliant but slightly blurs this mechanism by saying the larger model “checks and corrects all at once.” (Second judge pass, order swapped — scores are the average of both: Model B more precisely explains that proposals are kept only until the first disagreement, after which the larger model supplies its own word. Model A is clear and c | Anthropic: Claude Fable 5.1: Both entries deliver exactly three plain-l…

Unit-aware math

A pump moves 3.5 liters every 8 seconds. A tank holds 0.9 cubic meters. Starting empty, how long to fill it, in minutes and seconds (mm:ss), rounded to the nearest second? Show the key steps, then give the final answer on its own line.

DeepSeek-V4-Pro: Let’s break it down step-by-step. --- **Step 1: Convert the tank volume to liters** 1 cubic meter = 1000 liters. Tank volume = \(0.9 \times 1000 = 900\) liters. --- **Step 2: Find the pump rate** The pump moves 3.5 liters in 8 seconds. Rate = \( \frac{3.5}{8} = 0.4375\) liters per second. --- **Step 3: Calculate total time in seconds** \[ \text{Time} = \frac{900}{0.4375} = 2057.142857\ \text{seconds} \] --- **Step 4: Convert seconds to minutes and seconds** 2057 seconds = \( 34\) minutes and \(17.142857\) seconds. Rounded to the nearest second: **34 minutes, 17 seconds**. --- **Final answer:** ``` 34:17 ```
gpt-oss-120b: **Key steps** 1. Convert the tank volume to liters \[ 0.9\ \text{m}^3 = 0.9 \times 1000\ \text{L}= 900\ \text{L} \] 2. Determine the pump’s flow rate \[ \frac{3.5\ \text{L}}{8\ \text{s}} = 0.4375\ \text{L/s} \] 3. Compute the time needed to move 900 L \[ t = \frac{900\ \text{L}}{0.4375\ \text{L/s}} = \frac{900}{0.4375}\ \text{s} = \frac{900}{7/16}\ \text{s}= 900 \times \frac{16}{7}\ \text{s} = \frac{14400}{7}\ \text{s} \approx 2057.14\ \text{s} \] 4. Convert seconds to minutes and seconds \[ 2057.14\ \text{s}= 34\ \text{min} \;+\; 17.14\ \text{s} \] Rounded to the nearest second → 17 s. **Answer** 34:17

OpenAI: GPT-5.6 Sol Pro: Model B is fully correct, unit-aware, concise, and keeps the unrounded time consistent through the conversion; Model A also reaches the correct answer but imprecisely states that “2057 seconds” equals 34 minutes and 17.142857 seconds after previously calculating 2057.142857 seconds. (Second judge pass, order swapped — scores are the average of both: Model B is fully correct, clear, unit-aware, and presents the requested final format cleanly. Model A reaches the same correct result but has a mino | Anthropic: Claude Fable 5.1: Both Model A and Model B correctly convert…

Constrained standup summary

Write a team standup update from the notes below. Constraints: - Exactly 4 bullet points - Each bullet must start with one of these labels in this order: Yesterday:, Today:, Risks:, Help needed: - 9-14 words per bullet - Include the number 17 somewhere - Do not use the words "blocker", "urgent", or "meeting" - Mention both "Harbor" and "invoice parser" Notes: finished Harbor export fix; reviewed 17 failed uploads; today pairing with Mina on invoice parser edge cases; risk is staging data is two days stale; needs analytics to resend April sample files.

DeepSeek-V4-Pro: - Yesterday: Finalized Harbor export fix and reviewed 17 failed uploads for patterns. - Today: Pairing with Mina to resolve tricky invoice parser edge cases. - Risks: Staging environment data is now two days stale, slowing verification. - Help needed: Analytics team needs to resend the April sample files promptly.
gpt-oss-120b: - Yesterday: Fixed Harbor export bug and reviewed 17 failed uploads. - Today: Pairing with Mina on invoice parser edge cases and testing. - Risks: Staging data is two days stale, may affect upcoming releases. - Help needed: Analytics team to resend April sample files for validation.

OpenAI: GPT-5.6 Sol Pro: Model A satisfies every explicit constraint and is consistently polished and grammatical. Model B also follows the constraints, but the Risks bullet has an awkward comma construction and adds a less-supported claim about upcoming releases. (Second judge pass, order swapped — scores are the average of both: Both outputs satisfy the structural, length, labeling, terminology, and number constraints. Model A is slightly clearer and more polished, while Model B introduces a less-supported reference t | Anthropic: Claude Fable 5.1: Both outputs satisfy every constraint (fou…

LFU cache implementation

Implement an LFU (least-frequently-used) cache class in Python. Requirements: - Class name: LFUCache - Methods: __init__(capacity: int), get(key: int) -> int, put(key: int, value: int) -> None - If capacity is 0, put should do nothing and get returns -1 - Evict the least frequently used key when full; if tied, evict the least recently used among those - Average time complexity for get and put must be O(1) - Use only Python standard library Return code only.

DeepSeek-V4-Pro: ```python from collections import defaultdict, OrderedDict class LFUCache: def __init__(self, capacity: int): self.capacity = capacity self.min_freq = 0 self.key_to_val = {} self.key_to_freq = {} self.freq_to_keys = defaultdict(OrderedDict) def get(self, key: int) -> int: if key not in self.key_to_val: return -1 freq = self.key_to_freq[key] self.freq_to_keys[freq].pop(key) if not self.freq_to_keys[freq]: del self.freq_to_keys[freq] if self.min_freq == freq: self.min_freq += 1 self.key_to_freq[key] = freq + 1 self.freq_to_keys[freq + 1][key] = None return self.key_to_val[key] def put(self, key: int, value: int) -> None: if self.capacity == 0: return if key in self.key_to_val: self.key_to_val[key] = value self.get(key) return if len(self.key_to_val) >= self.capacity: evict_key, _ = self.freq_to_keys[self.min_freq].popitem(last=False) if not self.freq_to_keys[self.min_freq]: del self.freq_to_keys[self.min_freq] del self.key_to_val[evict_key] del self.key_to_freq[evict_key] self.key_to_val[key] = value self.key_to_freq[key] = 1 self.freq_to_keys[1][key] = None self.min_freq = 1 ```
gpt-oss-120b: ```python from collections import OrderedDict from typing import Dict, Tuple class LFUCache: """ Least‑Frequently‑Used (LFU) cache with O(1) average time for get and put. - If capacity is 0 the cache is inert: put does nothing, get always returns -1. - When the cache is full the entry with the smallest access frequency is evicted. If several entries share that frequency, the least‑recently used among them is removed. """ def __init__(self, capacity: int): self.capacity: int = capacity self.size: int = 0 # current number of stored keys self.min_freq: int = 0 # smallest frequency present self.key_to_val_freq: Dict[int, Tuple[int, int]] = {} # key -> (value, freq) self.freq_to_keys: Dict[int, OrderedDict[int, None]] = {} # freq -> ordered dict of keys def _update_freq(self, key: int) -> None: """Move *key* from its current frequency bucket to the next one.""" value, freq = self.key_to_val_freq[key] # Remove from the old frequency bucket old_bucket = self.freq_to_keys[freq] del old_bucket[key] if not old_bucket: # Clean up empty bucket del self.freq_to_keys[freq] if self.min_freq == freq: self.min_freq += 1 # Add to the new frequency bucket new_freq = freq + 1 new_bucket = self.freq_to_keys.setdefault(new_freq, OrderedDict()) new_bucket[key] = None # value not needed; we only care about order self.key_to_val_freq[key] = (value, new_freq) def get(self, key: int) -> int: if key not in self.key_to_val_freq: return -1 value, _ = self.key_to_val_freq[key] self._update_freq(key) return value def put(self, key: int, value: int) -> None: if self.capacity == 0: return if key in self.key_to_val_freq: # Update the existing entry's value and frequency _, freq = self.key_to_val_freq[key] self.key_to_val_freq[key] = (value, freq) # keep freq, update value self._update_freq(key) return # Need to insert a new key if self.size == self.capacity: # Evict the least-frequently-used key (and LRU among ties) evict_bucket = self.freq_to_keys[self.min_freq] evict_key, _ = evict_bucket.popitem(last=False) # LRU within min_freq if not evict_bucket: del self.freq_to_keys[self.min_freq] del self.key_to_val_freq[evict_key] self.size -= 1 # Insert the new key with frequency 1 self.key_to_val_freq[key] = (value, 1) bucket_one = self.freq_to_keys.setdefault(1, OrderedDict()) bucket_one[key] = None self.min_freq = 1 self.size += 1 ```

OpenAI: GPT-5.6 Sol Pro: Both Model A and Model B correctly implement O(1)-average LFU eviction with LRU tie-breaking. Model A is marginally better because it is more concise and avoids maintaining a redundant size counter, though both slightly miss the code-only instruction by using Markdown fences. (Second judge pass, order swapped — scores are the average of both: Model B and Model A are both correct O(1)-average LFU implementations, but Model B is slightly better structured and documented through its frequency-updat | Anthropic: Claude Fable 5.1: Both Model A and Model B implement a corre…

Messy contacts to JSON

Convert the messy contact notes below into valid JSON. Output must be an array of objects sorted by company A-Z. Each object must have exactly these fields: company (string), contact (string), email (string or null), phone (string or null), tags (array of lowercase strings). Rules: normalize phone numbers to the form +1-XXX-XXX-XXXX when possible; trim spaces; split tags on commas or slashes; lowercase tags; if email is missing or invalid, use null. Notes: - Northline Labs | Ava Chen | [email protected] | (415) 555-0188 | biotech, West - Quill Harbor| Marcos Ibarra | mibarra[at]quillharbor.com | 212.555.0144 | Publishing / East - Alder Peak Studio | Ren Ito | [email protected] | none | Design, west coast - Beacon Thread | Lila Moss | | 646-555-0199 | Retail, VIP

DeepSeek-V4-Pro: ```json [ { "company": "Alder Peak Studio", "contact": "Ren Ito", "email": "[email protected]", "phone": null, "tags": ["design", "west coast"] }, { "company": "Beacon Thread", "contact": "Lila Moss", "email": null, "phone": "+1-646-555-0199", "tags": ["retail", "vip"] }, { "company": "Northline Labs", "contact": "Ava Chen", "email": "[email protected]", "phone": "+1-415-555-0188", "tags": ["biotech", "west"] }, { "company": "Quill Harbor", "contact": "Marcos Ibarra", "email": "[email protected]", "phone": "+1-212-555-0144", "tags": ["publishing", "east"] } ] ```
gpt-oss-120b: ```json [ { "company": "Alder Peak Studio", "contact": "Ren Ito", "email": "[email protected]", "phone": null, "tags": ["design", "west coast"] }, { "company": "Beacon Thread", "contact": "Lila Moss", "email": null, "phone": "+1-646-555-0199", "tags": ["retail", "vip"] }, { "company": "Northline Labs", "contact": "Ava Chen", "email": "[email protected]", "phone": "+1-415-555-0188", "tags": ["biotech", "west"] }, { "company": "Quill Harbor", "contact": "Marcos Ibarra", "email": null, "phone": "+1-212-555-0144", "tags": ["publishing", "east"] } ] ```

OpenAI: GPT-5.6 Sol Pro: Model B correctly treats the malformed address "mibarra[at]quillharbor.com" as invalid and sets it to null. Model A improperly repairs the address even though the instructions require invalid emails to be null. (Second judge pass, order swapped — scores are the average of both: Model B correctly sets the malformed Quill Harbor email to null as required. Model A improperly repairs the invalid address instead of using null.) | Anthropic: Claude Fable 5.1: Both outputs are identical except for the Quill Harbor email: the prompt explicitly says to use null for invalid ema…

Polish app-store localization

Translate the following app description into Polish for an app-store listing. Tone: friendly and trustworthy. Keep it to 2 sentences and under 160 characters total. Preserve meaning and mention offline access. "Track field inspections, attach photos, and sync reports when you're back online. Your checklists stay available even without signal."

DeepSeek-V4-Pro: Rób inspekcje w terenie, dodawaj zdjęcia i synchronizuj raporty po powrocie do sieci. Listy kontrolne są dostępne nawet offline.
gpt-oss-120b: Śledź inspekcje terenowe, dodawaj zdjęcia i synchronizuj raporty po powrocie online. Twoje listy kontrolne są dostępne nawet bez sygnału.

OpenAI: GPT-5.6 Sol Pro: Model B more precisely preserves “track” and “your,” while remaining within the two-sentence and character limits and clearly conveying access without a signal. Model A sounds slightly more natural in Polish, but “Rób inspekcje” shifts the meaning toward conducting inspections. (Second judge pass, order swapped — scores are the average of both: Model A is more natural and polished in Polish, especially „po powrocie do sieci”, and explicitly communicates offline access. Model B is slightly more f | Anthropic: Claude Fable 5.1: Both outputs satisfy the two-sentence and…

Ticket triage labels

Classify each support ticket into exactly one category: Billing, Bug, Feature Request, Account Access, or How-To. For each, give a 6-12 word justification. Return one item per line in this exact format: <ID> | <Category> | <justification> T1: "My card was charged twice for March, invoice 8841 and 8841A both hit." T2: "After the 3.9.2 update, exporting a .csv creates an empty file on macOS Sonoma." T3: "Can you add a setting so completed routes archive automatically after 30 days?" T4: "I reset my password, but the sign-in screen keeps saying my SSO domain isn't recognized." T5: "What's the fastest way to bulk-tag 200 photos by project code?" T6: "The dashboard total says 214 orders, but the list view only shows 213 when filtered to Q2."

DeepSeek-V4-Pro: T1 | Billing | Customer was charged twice for the same invoice period. T2 | Bug | Software update caused CSV export to produce empty files. T3 | Feature Request | Asking for automatic route archiving after 30 days. T4 | Account Access | Cannot sign in due to unrecognized SSO domain error. T5 | How-To | Seeking fastest method to bulk-tag photos by project. T6 | Bug | Dashboard total and filtered list view show mismatched counts.
gpt-oss-120b: T1 | Billing | Duplicate charge on March invoices requires billing correction. T2 | Bug | Update introduced bug causing empty CSV export on macOS. T3 | Feature Request | User wants automatic route archiving after thirty days. T4 | Account Access | Password reset unsuccessful; SSO domain not recognized during login. T5 | How-To | User asks for quickest method to bulk‑tag photos. T6 | Bug | Dashboard total mismatches list view, indicating counting bug.

OpenAI: GPT-5.6 Sol Pro: Model A and Model B classify every ticket correctly and satisfy the required format and length, but Model A is slightly more precise because Model B inaccurately describes the password reset itself as unsuccessful in T4. (Second judge pass, order swapped — scores are the average of both: Model B and Model A classify every ticket correctly and satisfy the required format and justification length. Model A has slightly clearer, more natural phrasing overall, particularly for the account-access tick | Anthropic: Claude Fable 5.1: Both models classify all six tickets ident…

SQL window query

Given Postgres tables `orders(id, customer_id, created_at, total_cents)` and `customers(id, name)`, write ONE SQL query returning each customer's name and their SECOND-largest order total (in dollars, 2 decimals), excluding customers with fewer than two orders. Order by that value descending. Return ONLY the SQL.

DeepSeek-V4-Pro: ```sql SELECT c.name, ROUND(customer_orders.second_largest_cents / 100.0, 2) AS second_largest_total_dollars FROM ( SELECT customer_id, NTH_VALUE(total_cents, 2) OVER ( PARTITION BY customer_id ORDER BY total_cents DESC ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING ) AS second_largest_cents, ROW_NUMBER() OVER (PARTITION BY customer_id) AS rn FROM orders ) customer_orders JOIN customers c ON c.id = customer_orders.customer_id WHERE customer_orders.rn = 1 AND customer_orders.second_largest_cents IS NOT NULL ORDER BY second_largest_total_dollars DESC; ```
gpt-oss-120b: ```sql SELECT c.name, ROUND(o.total_cents / 100.0, 2) AS second_largest_order_total FROM ( SELECT customer_id, total_cents, DENSE_RANK() OVER (PARTITION BY customer_id ORDER BY total_cents DESC) AS rnk FROM orders ) o JOIN customers c ON c.id = o.customer_id WHERE o.rnk = 2 -- second‑largest distinct total per customer ORDER BY second_largest_order_total DESC; ```

OpenAI: GPT-5.6 Sol Pro: Model A correctly selects the second order when each customer’s orders are sorted by total and returns one row per qualifying customer. Model B instead selects the second-largest distinct total, can return duplicate rows for tied totals, and incorrectly excludes customers whose two orders have equal totals. (Second judge pass, order swapped — scores are the average of both: Model B finds the second distinct total and can return multiple rows for a customer when orders tie at that total. Model A | Anthropic: Claude Fable 5.1: Model A correctly returns exactly one row p…

Find the contradiction

The following spec contains exactly one internal contradiction. Quote the two conflicting sentences verbatim and explain the conflict in one sentence. Do not fix it. Spec: "Free accounts may create up to three projects. Every account, regardless of tier, may archive unlimited projects. Archiving a project does not count against the project limit. Free accounts are limited to three projects total, including archived ones."

DeepSeek-V4-Pro: The two conflicting sentences are: - "Archiving a project does not count against the project limit." - "Free accounts are limited to three projects total, including archived ones." The conflict is that the first sentence says archived projects are not counted toward the limit, but the second sentence says they are.
gpt-oss-120b: **Conflicting sentences** 1. “Archiving a project does not count against the project limit.” 2. “Free accounts are limited to three projects total, **including archived ones**.” **Explanation:** One sentence says archived projects are excluded from the limit, while the other says they are included, creating a direct contradiction.

OpenAI: GPT-5.6 Sol Pro: Model A quotes both conflicting sentences exactly and explains the contradiction clearly in one sentence. Model B is substantively correct, but the added bold formatting inside the second quotation makes it marginally less faithful to the requirement to quote the sentence verbatim. (Second judge pass, order swapped — scores are the average of both: Model A quotes both conflicting sentences exactly and explains the contradiction in one concise sentence. Model B is equally correct, but adding bold | Anthropic: Claude Fable 5.1: Both Model A and Model B correctly identif…

Vendor delay email rewrite

Rewrite this workplace message as a polished email to a hospital procurement manager. Audience: busy external client. Tone: calm, accountable, solution-oriented. Length: 120-150 words. Source message: "Hi—quick heads-up. The 48 bedside scanners for Eastbay Surgical won't arrive Thursday after all. Our carrier held pallet BQ-17 in Reno because the lithium battery paperwork on 11 units was mismatched. We fixed the docs this morning. New ETA is Monday 10:30 a.m. We can still send 20 corded scanners from our Sacramento stock tomorrow if that helps your go-live. Sorry, messy on our side. Let me know what you want us to do."

DeepSeek-V4-Pro: Subject: Updated Delivery Timeline – Eastbay Surgical Bedside Scanners Dear [Name], I’m writing to let you know about a delay affecting the 48 bedside scanners for Eastbay Surgical. Our carrier held pallet BQ-17 in Reno due to mismatched lithium battery documentation for 11 units. We resolved the paperwork this morning, and the revised delivery estimate is Monday at 10:30 a.m. I recognize this may impact your plans. To help bridge the gap, we can ship 20 corded scanners from our Sacramento stock tomorrow for Friday arrival, which may support your go-live readiness. Please let me know if that would be useful, or if you prefer to wait for the full shipment on Monday. I apologize for the disruption—this should have been caught earlier. We’re committed to keeping you fully informed as the shipment progresses. Best regards, [Your Full Name] [Your Title] [Your Contact Information]
gpt-oss-120b: **Subject:** Updated Delivery Schedule for Eastbay Surgical Scanners Dear [Procurement Manager’s Name], I wanted to inform you that the shipment of the 48 bedside scanners scheduled for Thursday will be delayed. Our carrier retained pallet BQ‑17 in Reno because the lithium‑battery documentation for 11 units was incomplete. We corrected the paperwork this morning and have secured a new estimated arrival of **Monday, 10:30 a.m.** To support your go‑live timeline, we can dispatch 20 corded scanners from our Sacramento inventory tomorrow. Please let us know if this interim solution meets your needs or if you prefer an alternative arrangement. We apologize for the inconvenience and appreciate your understanding. I am monitoring the situation closely and will keep you updated on any further developments. Thank you for your partnership. Best regards, [Your Name] [Title] [Company]

OpenAI: GPT-5.6 Sol Pro: Model B is concise, polished, within the requested length, and avoids Model A’s unsupported promise of Friday arrival. Model A demonstrates slightly stronger accountability and more precisely describes the paperwork mismatch, but the invented arrival timing is a meaningful correctness issue. (Second judge pass, order swapped — scores are the average of both: Model B is polished, appropriately concise, and offers the interim solution without introducing a new delivery commitment. Model A is sligh | Anthropic: Claude Fable 5.1: Model A reads more accountable and solutio…

Faithful rewrite

Rewrite this blunt internal note as a warm, professional Slack message to the whole engineering team (60–90 words), keeping every fact intact and adding no new commitments. Return only the message. Note: "Deploy is frozen until the memory leak in the image service is fixed. Priya is on it. Do not merge to main. ETA tomorrow noon."

DeepSeek-V4-Pro: Hey everyone! Just a quick heads-up: we’ve temporarily frozen deployments while we sort out a memory leak in the image service. Priya is actively working on a fix, and the current ETA is tomorrow around noon. In the meantime, please hold off on merging anything to main. Thanks for your patience and cooperation—we’ll keep you posted as things move along!
gpt-oss-120b: Hey team, just a quick heads‑up: we’ve placed a temporary freeze on any new deployments while Priya works on fixing the memory leak in the image service. Please hold off on merging anything into the main branch until the issue is resolved. We’re aiming to have the fix in place by tomorrow at noon. Thanks for your patience and cooperation—let’s keep the momentum going!

OpenAI: GPT-5.6 Sol Pro: Model B preserves the exact noon ETA and avoids making a new commitment. Model A changes the ETA to “around noon” and adds “we’ll keep you posted,” which is an unsupported commitment. (Second judge pass, order swapped — scores are the average of both: Model B preserves all facts, stays within the required length, and adds no substantive commitment. Model A is polished but weakens “tomorrow noon” to “around noon” and adds the new commitment “we’ll keep you posted.”) | Anthropic: Claude Fable 5.1: Both hit the word range and preserve the core facts, but Model A adds a n…

Nuanced classification

Classify each review's sentiment as "positive", "negative", or "mixed", and give a 6-word-max reason. Return ONLY a JSON array of {"text","label","reason"} in input order. Reviews: ["Fast shipping but the fabric feels cheap.", "Absolutely love it, wearing it daily!", "It broke after a week. Refund was quick and painless though."]

DeepSeek-V4-Pro: ```json [ {"text":"Fast shipping but the fabric feels cheap.", "label":"mixed", "reason":"Fast shipping, cheap fabric."}, {"text":"Absolutely love it, wearing it daily!", "label":"positive", "reason":"Love it, wears daily."}, {"text":"It broke after a week. Refund was quick and painless though.", "label":"mixed", "reason":"Broke fast, but easy refund."} ] ```
gpt-oss-120b: [ {"text":"Fast shipping but the fabric feels cheap.", "label":"mixed", "reason":"Fast shipping, but cheap fabric."}, {"text":"Absolutely love it, wearing it daily!", "label":"positive", "reason":"Loved it, uses daily."}, {"text":"It broke after a week. Refund was quick and painless though.", "label":"mixed", "reason":"Broke quickly, refund easy."} ]

OpenAI: GPT-5.6 Sol Pro: Model B fully follows the JSON-only requirement and provides accurate labels with concise reasons. Model A is substantively correct, but the Markdown code fence violates the required output format, and “wears daily” is slightly awkward. (Second judge pass, order swapped — scores are the average of both: Model B fully follows the JSON-only requirement and provides accurate labels with concise reasons. Model A is substantively correct, but the Markdown code fence violates the required output forma | Anthropic: Claude Fable 5.1: Both Model A and Model B produce identical…

Matchup powered by OpenRouter.