Head to head: Kimi-K3 vs gpt-oss-120b
Kimi-K3 vs gpt-oss-120b
By Ryan Merket · Published
This one is a genuine photo finish. Kimi-K3 and gpt-oss-120b trade blows across coding, reasoning, editing, and extraction tasks so evenly that the aggregate gap is noise, not a verdict.
The scoreboard says 108.0 for Kimi-K3 and 103.7 for gpt-oss-120b, but the only number that really matters is the confidence: **50%**. In plain English, that means this matchup is a statistical dead heat. There is no honest winner to declare here; these models were effectively even. The shape of the tie is interesting, though. **Kimi-K3 tended to win on instruction discipline and exactness**: the concurrency bug fix was a recurring bright spot, with Kimi more reliably caching the in-flight promise correctly and returning only the requested function. It also had strong showings in SQL and several formatting-sensitive proofreading tasks, where cleaner adherence to the prompt mattered as much as raw correctness. **gpt-oss-120b, meanwhile, often looked better when robustness and implementation hygiene were the differentiator.** It had the edge in some debugging and LRU-cache evaluations thanks to safer handling of edge cases like invalid capacity or malformed input, and it frequently read as the more systematic explainer in reasoning-heavy prompts. On localization, it also often sounded more natural and product-ready, especially for short UI copy. What keeps this from tilting either way is that nearly every apparent advantage got canceled somewhere else. Several tasks were outright ties, and even among the split decisions, the second-pass judging repeatedly softened first-pass wins into negligible differences. In other words: one model might be slightly cleaner on one coding prompt, the other slightly sharper on the next, but **there is no stable, publication-worthy separation between them in this run**. **Final call: too close to call. Kimi-K3 and gpt-oss-120b are effectively tied in this head-to-head, and claiming a real winner would overread the data.**
Classification & labeling
Classify each customer-support message into exactly one category: - Billing - Technical Issue - Account Access - Feature Request - Shipping/Delivery Return one line per item in this exact format: `<number>. <category> - <brief justification>` Items: 1. "I was charged twice for order 7714 even though the checkout page froze." 2. "The reset link says it's expired the moment I click it." 3. "Can you add dark mode to the analyst dashboard?" 4. "My replacement filters were marked delivered yesterday, but nothing is at the front desk." 5. "The Android app closes whenever I tap Export CSV." 6. "Where can I download invoices for March?"
Model A and Model B both classify all six items correctly and follow the required one-line format with brief justifications. Model A is slightly more explanatory, while Model B is slightly more concise, but neither has a meaningful quality advantage for this task. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both classify all six items correctly and follow the required one-line format with concise justifications. Model A is slightly more polished in phrasing, but the difference is negligible for this task.)
Step-by-step reasoning
A clinic buys nitrile gloves in 3 box sizes: - small box: 40 pairs for $11.20 - medium box: 75 pairs for $19.50 - bulk box: 120 pairs for $30.00 They need at least 430 pairs for a vaccination drive and have storage space for at most 6 boxes total. They want the lowest possible total cost. If multiple options tie on cost, choose the one with fewer boxes; if still tied, choose the one with more bulk boxes. What combination of boxes should they buy, and what is the total cost? Show your reasoning.
Model B is better because it gives a clearer, more systematic optimization setup and explicitly frames the constraints and objective before checking feasible combinations. Both models reach the correct answer, but Model B’s reasoning is more complete and polished, while Model A’s table is slightly less rigorous in how it summarizes the search. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both reach the correct minimum-cost combination of 3 bulk boxes and 1 medium box for $109.50, and both show valid reasoning. Model B is more exhaustive and explicit, while Model A is more concise and still sufficiently justified, so neither is clearly better overall.)
LRU cache
Implement a class `LRUCache<K, V>` in TypeScript with a fixed capacity set in the constructor, and O(1) `get(key): V | undefined` and `set(key, value): void`. Accessing or updating a key must mark it most-recently-used; inserting beyond capacity must evict the least-recently-used entry. Return ONLY the class, no prose.
Model B is better because it correctly handles zero or negative capacity by making set a no-op, while Model A can behave incorrectly by attempting to evict from an empty map and then inserting anyway. Both follow the requested API and achieve O(1) operations using Map insertion order, but Model B is slightly cleaner and more robust. (Second judge pass, order swapped — scores are the average of both: Model B correctly implements LRU behavior with O(1) operations using Map insertion order and safely handles nonpositive capacity by making set a no-op. Model A is mostly correct, but it fails for capacity 0 by evicting nothing before reinserting, allowing the cache to grow when it should hold no items.)
Concurrency bug fix
This TypeScript function is meant to memoize an async loader but has a race: concurrent callers can each trigger the underlying fetch. Fix it so the fetch runs at most once per key, and a rejected fetch does NOT poison the cache (a later call must retry). Return ONLY the corrected function. ```ts const cache = new Map<string, Promise<string>>(); async function load(key: string, fetcher: (k: string) => Promise<string>) { if (cache.has(key)) return cache.get(key)!; const value = await fetcher(key); cache.set(key, Promise.resolve(value)); return value; } ```
Model A correctly stores the in-flight promise before awaiting, ensuring only one fetch per key, and deletes the cache entry on rejection so later calls retry. Model B intends to do the same, but it returns a different promise than the one stored in the cache during the initial call and includes extra code fencing and declarations instead of returning only the corrected function. (Second judge pass, order swapped — scores are the average of both: Model A correctly stores the in-flight promise immediately and deletes it on rejection, so concurrent callers share one fetch and failures do not poison the cache. Model B has a subtle bug: it stores the raw in-flight promise in the cache but returns a different wrapped promise, so concurrent callers can receive a promise that remains cached even after rejection, violating the retry requirement; it also includes extra code outside the requested corrected function.)
Editing & proofreading
Rewrite the passage to fix grammar, punctuation, and clarity without changing meaning. Then provide 3 brief bullet points noting the main edits. Passage: "Our field team were on site Tuesday, however the generator inspection didn't start until 11:15 because security had not been told we was coming. The lead tech say the coolant line look stable, but there is vibrations near the south mount that needs a closer check. If parts is required, procurement can order them next week, this may delay restart by 2 days. Please advise whether operations want us to proceed or to pause repairs."
Model A better preserves the original meaning while fixing grammar and punctuation cleanly; its edits are accurate and its bullet points directly reflect the main changes. Model B is strong, but it makes more substantive wording changes such as "had not been informed," "lead technician says," and "would like us to proceed," which slightly shifts tone and tense from the source. (Second judge pass, order swapped — scores are the average of both: Model B is better because it cleanly fixes the grammar, punctuation, and clarity while preserving meaning and providing exactly three brief edit notes. Model A is also strong, but it introduces a few less consistent choices, such as retaining quotation marks, capitalizing "Operations," and mixing tense choices in a way that is slightly less polished.)
SQL window query
Given Postgres tables `orders(id, customer_id, created_at, total_cents)` and `customers(id, name)`, write ONE SQL query returning each customer's name and their SECOND-largest order total (in dollars, 2 decimals), excluding customers with fewer than two orders. Order by that value descending. Return ONLY the SQL.
Model A correctly returns the second-highest order per customer using ROW_NUMBER, excludes customers with fewer than two orders, and orders the result properly. Model B uses DENSE_RANK, which returns the second-largest distinct total rather than the second order total, so it can incorrectly include customers with only one distinct order amount repeated and does not match the prompt as well. (Second judge pass, order swapped — scores are the average of both: Model A better matches the likely intent of "second-largest order total" by returning the second order amount per customer via ROW_NUMBER(), while excluding customers with fewer than two orders. Model B uses DENSE_RANK(), which instead returns the second-largest distinct total and can include customers who have multiple orders tied at the top value, so it is less correct for this prompt.)
Find the contradiction
The following spec contains exactly one internal contradiction. Quote the two conflicting sentences verbatim and explain the conflict in one sentence. Do not fix it. Spec: "Free accounts may create up to three projects. Every account, regardless of tier, may archive unlimited projects. Archiving a project does not count against the project limit. Free accounts are limited to three projects total, including archived ones."
Model A and Model B both identify the same two conflicting sentences verbatim and explain the contradiction accurately in one sentence without attempting to fix it. Model A is slightly more explicit, while Model B is slightly more concise, but neither is meaningfully better for this task. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both identify the same two conflicting sentences verbatim and explain the contradiction accurately in one sentence without attempting to fix it. Model A is slightly more polished stylistically, but the task performance is effectively identical.)
Localization with tone
Translate this app onboarding line into natural, friendly European Spanish suitable for a mobile toast (keep it under 60 characters, no exclamation marks): "You're all set — your first backup starts tonight." Return only the translation, then the character count in parentheses.
Model B is more natural and concise for a friendly mobile toast in European Spanish, and it stays comfortably under the limit. Model A is accurate but sounds more formal and technical with "copia de seguridad," making it less toast-like. (Second judge pass, order swapped — scores are the average of both: Model B is more natural and concise for a friendly mobile toast in European Spanish while staying well under the limit. Model A is also correct, but "copia de seguridad" feels more cumbersome for this UI context and the punctuation makes it slightly less smooth.)
Information extraction
Extract the shipment records below into a JSON array. For each record, return: `tracking_id`, `customer`, `ship_date` (YYYY-MM-DD), `city`, `items` (array of item names), `fragile` (true/false), and `delivery_window` (exact text like `9am-1pm` or null). Be accurate and include all records. Messy notes: - 04/17/2026 — TRK-8831A :: Customer=Meridian Dental Lab, drop to Boise. Contents: "chair arm kit" x2; "sterile tray". Mark FRAGILE. Window 9am-1pm. - Apr 18 2026 | tracking TRK-8832A | for Nila Cortez | city: Reno | items: desk lamp, shade | no frag | leave at reception. - 2026-04-18 / id=TRK-8833A / customer "Pine & Dock LLC" / destination Spokane / contents: 4x ceramic mug / fragile=yes / delivery after 3pm. - 4-19-2026 ; TRK-8834A ; customer: Orson Biotech ; to Eugene ; items = coolant pack, sensor cable, sensor cable ; fragile: no ; window: 8:30am-10am.
Model A is better because it correctly treats the one-sided constraint "after 3pm" as not matching the requested delivery-window format examples and sets it to null, while Model B inserts a value that does not fit the specified pattern. Both extract all records accurately, but Model A shows slightly better adherence to the schema despite adding extra explanatory text outside the JSON. (Second judge pass, order swapped — scores are the average of both: Model B is better because it includes all four records in valid JSON and captures the explicit delivery constraint for TRK-8833A as the exact text "after 3pm," which aligns with the schema asking for exact delivery-window text or null. Model A is also strong, but it drops that available field value and adds extra explanatory notes outside the requested JSON-only output.)
Debugging & code review
Language: JavaScript The function should group invoice lines by SKU and sum qty*unitPrice into cents, returning an object like {"A12": 1234, ...}. It fails on some realistic inputs. Find the bug(s), fix them, and return the corrected code only. ```js function totalsBySku(lines) { const out = {}; for (const line in lines) { const sku = line.sku.trim(); const qty = parseInt(line.qty, 10) || 0; const priceCents = Math.round(parseFloat(line.unitPrice) * 100) || 0; if (!out[sku]) out[sku] = 0; out[sku] += qty + priceCents; } return out; } const lines = [ { sku: ' A12 ', qty: '2', unitPrice: '3.50' }, { sku: 'B07', qty: '1', unitPrice: '10.00' }, { sku: 'A12', qty: '3', unitPrice: '3.50' } ]; console.log(totalsBySku(lines)); ```
Model B is better because it fixes the core bugs and also handles realistic bad inputs more safely by guarding missing or blank SKUs. Model A corrects the iteration and multiplication issues, but it can still throw if a line lacks a valid sku property. (Second judge pass, order swapped — scores are the average of both: Model B fixes both core bugs: iterating array elements correctly and summing qty * unitPrice in cents rather than qty + priceCents; it also adds reasonable guards for missing SKU and invalid price input. Model A fixes the iteration and multiplication bug but is less robust on realistic inputs because it still assumes line.sku is always present and valid.)
Algorithm & data structures
Implement in Python a class `RollingMedian` for a stream of integers with methods: - `add(x: int) -> None` - `median() -> float` Requirements: - `add` must be O(log n) - `median` must be O(1) - If there are an even number of items, return the average of the two middle values as a float. - Handle negatives and duplicates. - Return code only, no explanation. Example behavior: add(5), add(1), median() -> 3.0 add(9), median() -> 5.0 add(1), median() -> 3.0
Model A correctly maintains the invariant that all lower-half elements are <= all upper-half elements and keeps the heaps balanced, so median() is correct in O(1) after O(log n) insertion. Model B can return incorrect medians because its rebalancing allows the upper heap to have more elements without ensuring the lower heap holds the extra item, violating the ordering/size invariant needed for correct median computation. (Second judge pass, order swapped — scores are the average of both: Model B cleanly maintains the two-heap invariant and correctly handles odd and even counts with O(log n) insertion and O(1) median lookup. Model A is close, but its insertion logic can leave an out-of-order partition when the upper heap is empty and a larger value is added after a smaller one, producing an incorrect median.)
Precise proofreading
Correct ONLY the grammar, spelling, and punctuation errors in the sentence below without changing wording, tone, or meaning, then on a second line list each change as 'was -> now'. Sentence: "Its been a long time since we seen there new office, and me and him was suprised how quite it was."
Model A better follows the requested format by putting the corrected sentence on one line and listing each change on separate subsequent lines exactly as 'was -> now'. Model B’s correction is also accurate, but it compresses all changes onto one line, uses a different arrow symbol, and splits the phrase-level change into token-level edits rather than preserving the requested listing style as closely. (Second judge pass, order swapped — scores are the average of both: Model A is better because it follows the required second-line change listing format more closely, using separate 'was -> now' entries and correctly grouping the linked grammar change. Model B’s correction is accurate, but its change list is compressed onto one line and splits the coordinated grammar fix less cleanly.)
Matchup powered by OpenRouter.