Head to head: gpt-oss-120b vs cohere-command-a
gpt-oss-120b vs cohere-command-a
This wasn’t a stylistic squeaker; it was a broad, repeatable win. gpt-oss-120b beat cohere-command-a across instruction-heavy, format-sensitive, and reasoning tasks, with a decisive 99% confidence edge.
gpt-oss-120b takes this matchup cleanly: **102.5 to 80.1, with a 99% confidence verdict**, and a **10–2 task win split**. That margin matters because it wasn’t built on one flashy outlier. Model A kept winning in the unglamorous places that separate reliable production behavior from demo-friendly competence: strict JSON output, exact proofreading diffs, contradiction finding, localization tone control, and bug-fix tasks where permissive parsing or sloppy formatting gets you in trouble fast. The pattern is simple: **gpt-oss-120b followed directions better and made fewer costly mistakes**. It returned plain JSON when asked instead of hiding valid content inside code fences. It handled the JavaScript currency parser more correctly by rejecting malformed inputs that `parseFloat` would wrongly accept. It wrote the vendor-delay update with all required facts, including the vendor name, and it produced tighter, more natural microcopy in the Spanish localization task. Even where both models were substantively correct—support ticket triage, unit-aware math—Model A was usually cleaner and more precise in format and presentation. Cohere-command-a’s losses weren’t cosmetic. In the clinic scheduling task it violated key constraints; in contradiction detection it identified the wrong pair; in SQL it buried an at-least-plausible answer under commentary despite being told to return one query only. That’s the recurring issue in this head-to-head: Model B often looked capable, but too often it **failed the assignment as given**. For users who care about exact output contracts, that is not a minor defect—it is the product. To be fair, cohere-command-a did take two categories. It was the better fit on the Python log-redaction helper once instruction-following penalties were counted against Model A’s fenced/extraneous output, and it also won the Polish localization task by staying closer to the requested length and source meaning. But those are isolated wins, not signs of a balanced contest. **Final call: gpt-oss-120b is the clear winner.** Not because it is more verbose or more polished in the abstract, but because across this suite it was the model you could trust to do the job requested, in the format requested, without introducing avoidable errors.
Precise proofreading
Correct ONLY the grammar, spelling, and punctuation errors in the sentence below without changing wording, tone, or meaning, then on a second line list each change as 'was -> now'. Sentence: "Its been a long time since we seen there new office, and me and him was suprised how quite it was."
Model A is better because it cleanly corrects the sentence and lists only actual changes, closely matching the requested format. Model B makes the same core corrections but adds quotation marks not present in the original and includes many unchanged tokens, which does not follow the instruction to list each change as 'was -> now'. (Second judge pass, order swapped — scores are the average of both: Model A is better because it provides the corrected sentence and a concise list of actual changes, while Model B adds many unnecessary 'no change' items that do not match the requested format. Both models make the same substantive corrections, but Model A follows the instruction to list each change more cleanly and precisely.)
Clinic on-call schedule
Assign exactly one person to each shift and use each person exactly once. People: Nora, Imran, Celia, Mateo Shifts: Mon AM, Mon PM, Tue AM, Tue PM Constraints: - Nora cannot do any PM shift. - Imran cannot do Mon AM. - Celia must work immediately before Mateo in time order. - Tue AM must be Nora or Imran. - Mateo cannot do Tue PM. Return only the final assignment as four lines in this format: "Mon AM: Name" etc.
Model A satisfies all constraints: each person is used exactly once, Nora is on a non-PM shift, Imran is not on Mon AM, Celia is immediately before Mateo, Tue AM is Nora, and Mateo is not on Tue PM. Model B violates multiple constraints, including assigning Mateo to Tue PM and failing to place Celia immediately before Mateo in time order. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both violate key scheduling constraints, so neither is better overall. Model B assigns Mateo to Tue PM despite that being forbidden, while Model A fails the requirement that Celia must work immediately before Mateo in time order.)
Support ticket triage labels
Classify each incoming support line into exactly one category: Billing, Bug, Feature Request, Account Access, or Shipping. For each, give a 4-8 word justification. Return a JSON array of objects with keys id, category, justification. Items: 1) "I was charged twice for order QN-1842 after the page froze." 2) "Please add a dark-mode toggle to the kiosk dashboard." 3) "Reset link says expired the moment I click it." 4) "Package for invoice 7719 still shows 'label created' after 9 days." 5) "Exporting March leads from AtlasCRM gives a blank CSV in Firefox 127." 6) "Can you switch our plan from annual to monthly starting next cycle?"
Model A and Model B both classify all six items correctly, but Model A follows the requested output format more cleanly by returning plain JSON rather than a fenced code block. Model A’s justifications are also slightly more specific and directly tied to each ticket while staying within the 4–8 word limit. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both classify all six items correctly and provide valid JSON with 4–8 word justifications. Model A is slightly better because its justifications are more specific and directly grounded in each support line, while Model B is a bit more generic in places.)
Strict JSON extraction
Extract every meeting from this text into a JSON array of {"title","day","start24","durationMinutes"}. Use 24-hour "HH:MM" for start24, integer minutes for duration, and title-case titles. Return ONLY valid JSON, no commentary. Text: "Standup is daily at 9am for a quarter hour. The design review runs thursday from 2:30-4pm. Payroll sync — first monday, 11:00 to noon."
Model A better follows the requirement to return only valid JSON, while Model B incorrectly wraps the JSON in a code fence. Model B is slightly more faithful on the day field by preserving "First Monday," but the formatting violation makes Model A the better overall answer. (Second judge pass, order swapped — scores are the average of both: Model A is better because it returns only valid JSON and correctly extracts the meetings and times; Model B includes Markdown code fences, which violates the instruction to return only JSON. Model B is slightly more faithful in preserving "First Monday," but the formatting error makes Model A the stronger overall response.)
Unit-aware math
A pump moves 3.5 liters every 8 seconds. A tank holds 0.9 cubic meters. Starting empty, how long to fill it, in minutes and seconds (mm:ss), rounded to the nearest second? Show the key steps, then give the final answer on its own line.
Model A and Model B both compute the conversion and fill time correctly and both give the final answer as 34:17. Model A is slightly more concise while Model B is slightly more explicit, but both follow the instructions well and are equally strong overall. (Second judge pass, order swapped — scores are the average of both: Model A is slightly better because it is fully correct, concise, and follows the requested format more cleanly by showing key steps and then giving the final answer on its own line. Model B is also correct, but it is a bit more verbose and its final answer is embedded in explanatory text rather than presented as cleanly.)
Find the contradiction
The following spec contains exactly one internal contradiction. Quote the two conflicting sentences verbatim and explain the conflict in one sentence. Do not fix it. Spec: "Free accounts may create up to three projects. Every account, regardless of tier, may archive unlimited projects. Archiving a project does not count against the project limit. Free accounts are limited to three projects total, including archived ones."
Model A identifies the actual contradictory pair verbatim and explains the conflict accurately in one sentence. Model B quotes a non-contradictory pair and relies on an unquoted third sentence in its explanation, so it is less correct and follows the task less precisely. (Second judge pass, order swapped — scores are the average of both: Model A identifies the actual contradictory pair verbatim and explains the conflict directly. Model B quotes a non-contradictory sentence as one half of the pair and relies on an unquoted third sentence in its explanation, so it is less correct and less faithful to the task.)
Vendor delay status update
Draft a Slack status update to the operations team. Audience: internal coworkers. Tone: calm, direct, accountable. Length: 80-110 words. Facts to include: - Vendor: Pine Metric - Issue: overnight inventory sync failed for 14 store locations - Cause: expired API certificate discovered at 06:40 - Current workaround: manual CSV imports started at 07:15 - Risk: dashboards may show stale counts until about noon - Next update promised by 10:30 - Do not blame the vendor or sound alarmist.
Model A is better because it includes all required facts, names Pine Metric explicitly, and maintains a calm, accountable internal status-update tone within the requested length. Model B is clear and well written, but it omits the vendor name and adds slightly awkward phrasing about operations proceeding with manual imports, which weakens correctness and fit. (Second judge pass, order swapped — scores are the average of both: Model A is better because it includes all required facts, explicitly names Pine Metric, and reads more naturally as a Slack update while keeping a calm, accountable tone. Model B is solid but omits the vendor name and adds slightly awkward or unnecessary phrasing like "No need to escalate further at this time" and "Let me know if you have questions.")
Localization with tone
Translate this app onboarding line into natural, friendly European Spanish suitable for a mobile toast (keep it under 60 characters, no exclamation marks): "You're all set — your first backup starts tonight." Return only the translation, then the character count in parentheses.
Model A is shorter, natural for a mobile toast, and stays comfortably under the 60-character limit while preserving the meaning. Model B is also accurate and compliant, but it sounds more verbose and less toast-like for onboarding microcopy. (Second judge pass, order swapped — scores are the average of both: Model A is more natural and concise for a friendly mobile toast in European Spanish, and it stays comfortably under the limit with an accurate character count. Model B is understandable but sounds more literal and slightly less toast-like, and its stated character count appears incorrect.)
JavaScript currency parser fix
Review this JavaScript function. It should convert human-entered prices like "$1,299.50", "19.99", or "€08" into numbers, and return null for invalid inputs. It currently has a subtle bug. Find the bug, fix it, and return only the corrected code. ```js function parsePrice(input) { if (!input) return null; const cleaned = input.replace(/[$€,\s]/g, ''); const value = parseInt(cleaned, 10); return Number.isNaN(value) ? null : value; } ```
Model A better fixes the bug by using Number so the entire cleaned string must be a valid numeric value, which correctly returns null for malformed inputs like "12abc"; it also guards against an empty cleaned string. Model B improves decimal handling but parseFloat still accepts invalid trailing characters, so it is less correct for the requirement to return null for invalid inputs. (Second judge pass, order swapped — scores are the average of both: Model A is better because it fixes the decimal-parsing bug while also correctly rejecting inputs that become empty after cleaning and avoiding parseFloat's permissive partial parsing of malformed strings. Model B addresses the main bug but can still accept invalid inputs like an empty cleaned string or partially numeric text, so it is less correct overall.)
SQL window query
Given Postgres tables `orders(id, customer_id, created_at, total_cents)` and `customers(id, name)`, write ONE SQL query returning each customer's name and their SECOND-largest order total (in dollars, 2 decimals), excluding customers with fewer than two orders. Order by that value descending. Return ONLY the SQL.
Model A is concise and returns only SQL, but it uses DENSE_RANK so it finds the second-largest distinct total rather than necessarily the second order, which may not match the prompt. Model B badly violates the instruction to return only one SQL query by including extensive commentary and multiple alternative queries, even though one of its embedded queries is closer to correct. (Second judge pass, order swapped — scores are the average of both: Model A is better because it provides a single SQL query and is concise, whereas Model B violates the instruction to return only one SQL query by including extensive commentary and multiple alternatives. However, both have correctness issues: Model A uses DENSE_RANK(), which returns the second-distinct total rather than the second order total and can include customers with only one distinct value across multiple orders, while Model B’s final query has the same semantic issue despite also filtering for at least two orders.)
Python log redaction helper
Write Python 3 code only. Create a function `redact_log(line: str) -> str` that replaces any email address with `[EMAIL]` and any IPv4 address with `[IP]`, while leaving the rest of the line unchanged. Handle multiple occurrences in one line. Example: `"login from 192.168.1.9 by maya@northbay.dev"` -> `"login from [IP] by [EMAIL]"`. Use only the standard library.
Model A is better because it provides valid Python 3 code only, handles multiple occurrences with a single compiled regex, and uses a stricter IPv4 pattern that avoids matching invalid addresses like 999.999.999.999. Model B is simpler and mostly works, but its IPv4 regex is overly permissive and its email character class `[A-Z|a-z]` is slightly sloppy, reducing correctness and polish. (Second judge pass, order swapped — scores are the average of both: Model B is the better fit because it provides valid Python code only and correctly redacts multiple email and IPv4-like matches using the standard library. Model A has a somewhat stronger IPv4 regex, but it violates the prompt by wrapping the answer in Markdown code fences and adding extra example/CLI code, so its instruction-following is worse.)
Polish product update localization
Translate this app update note into Polish for customer-facing release notes. Tone: friendly and reassuring. Length: 55-75 words. Keep the meaning and nuance intact; do not add features. Source: "We fixed an issue that could cause saved delivery routes to disappear after editing a stop with a weak signal. Your existing routes are safe. We also made barcode scanning faster on older phones and clarified the warning shown before you delete a route."
Model B is closer to the requested length and keeps the main meaning in a friendly, reassuring tone, despite minor phrasing issues. Model A is fluent, but it exceeds the length limit and adds content not present in the source, such as thanks for patience and reports. (Second judge pass, order swapped — scores are the average of both: Model B stays closer to the source and covers all requested points in a friendly, reassuring tone, though it adds a mildly promotional closing sentence. Model A reads naturally, but it adds multiple unsupported details such as thanks for reports and extra reassurance, and it exceeds the requested length range.)
Matchup powered by OpenRouter.