Head to head: Kimi-K2.7-Code vs DeepSeek-V4-Flash
Kimi-K2.7-Code vs DeepSeek-V4-Flash
This wasn’t a blowout, but Kimi-K2.7-Code did enough across the right tasks to finish ahead. It won more often, posted the higher aggregate score, and looked slightly steadier on instruction-following and exactness, even if DeepSeek-V4-Flash stayed competitive throughout.
Kimi-K2.7-Code takes this matchup on points, not by demolition. The aggregate score favors A **112.4 to 99.8**, the task tally goes **5–3 with 4 ties**, and the statistical read is a **75% confidence lean**—which is exactly the right framing here. This is a real edge, not a rout. Where Kimi earned the verdict was on tasks that punish sloppiness. It was better on **nuanced classification**, **constraint scheduling**, **find the contradiction**, and **summarizing a dense passage**—all areas where exact compliance and precise reading matter more than polish. In several of those, DeepSeek didn’t just lose on style; it lost on correctness or format discipline, from wrapping JSON incorrectly to producing a schedule that violated the stated constraint to missing the actual contradictory pair. DeepSeek-V4-Flash absolutely had its moments. It was the stronger pick on **localization with tone**, **es-mx support translation**, and **meeting-notes summarization**, where its phrasing was often a touch more natural or more faithful to the source notes. And the four ties matter: on **inventory math**, **messy-orders JSON normalization**, **python bug-fixing**, and effectively the straightforward calculation work, neither model created meaningful separation. The interesting wrinkle is that several order-swapped judge passes flipped individual task calls, which tells you this matchup is closer than the raw win count alone suggests. But even after accounting for that noise, Kimi still comes out ahead because its wins cluster around higher-value failure modes: instruction-following, logical consistency, and extracting the exact requested detail. **Final call: Kimi-K2.7-Code wins, but this is a lean verdict—not a landslide.**
Nuanced classification
Classify each review's sentiment as "positive", "negative", or "mixed", and give a 6-word-max reason. Return ONLY a JSON array of {"text","label","reason"} in input order. Reviews: ["Fast shipping but the fabric feels cheap.", "Absolutely love it, wearing it daily!", "It broke after a week. Refund was quick and painless though."]
Both models classify the sentiments correctly, but Model A follows the output-format instruction exactly. Model B violates the requirement to return only a JSON array by wrapping the answer in a Markdown code block, despite having slightly more polished reasons. (Order-swapped judge pass: Both models classify all three reviews correctly, but B follows the 6-word-max reason constraint exactly and returns plain JSON. A violates the word-limit on every reason and includes Markdown code fences, making it less compliant.)
Constraint scheduling
Four talks (A, B, C, D) fill four 1-hour slots 9,10,11,12. Constraints: A is before D; C is not first; B is immediately after A; D is not at 12. Give the ONE valid schedule as 'slot: talk' lines, then a one-line justification. If impossible, say so and explain.
Model A gives the unique valid schedule and a concise justification. Model B's schedule violates the constraint that B is immediately after A, and its explanation contradicts its own output. (Order-swapped judge pass: Model B gives the unique valid schedule satisfying all constraints. Model A reverses the immediate-order constraint between A and B and therefore produces an invalid schedule despite a clear explanation.)
Find the contradiction
The following spec contains exactly one internal contradiction. Quote the two conflicting sentences verbatim and explain the conflict in one sentence. Do not fix it. Spec: "Free accounts may create up to three projects. Every account, regardless of tier, may archive unlimited projects. Archiving a project does not count against the project limit. Free accounts are limited to three projects total, including archived ones."
Model A identifies the actual contradictory pair verbatim and explains the conflict accurately in one sentence. Model B quotes one non-conflicting sentence and adds an explanation that is not supported by that sentence. (Order-swapped judge pass: Model B identifies the actual contradictory pair verbatim and explains the conflict directly. Model A quotes a non-contradictory sentence pair and adds an interpretation instead of isolating the exact contradiction.)
Unit-aware math
A pump moves 3.5 liters every 8 seconds. A tank holds 0.9 cubic meters. Starting empty, how long to fill it, in minutes and seconds (mm:ss), rounded to the nearest second? Show the key steps, then give the final answer on its own line.
Both outputs are correct, show the key steps clearly, and provide the final answer as 34:17. Model A is slightly more concise, while Model B is slightly more explanatory, but neither is meaningfully better for this task. (Order-swapped judge pass: Both outputs are correct and arrive at 34:17, but Model B follows the requested format more cleanly by explicitly labeling key steps and placing the final answer alone on its own line without extra text.)
Localization with tone
Translate this app onboarding line into natural, friendly European Spanish suitable for a mobile toast (keep it under 60 characters, no exclamation marks): "You're all set — your first backup starts tonight." Return only the translation, then the character count in parentheses.
Both are natural and within the limit, but B is more concise and toast-friendly while preserving the meaning. A is slightly more formal/heavy with "copia de seguridad" and uses a character count that appears off by one. (Order-swapped judge pass: A is more concise and toast-friendly while staying natural and under the character limit. B is accurate and still within limit, but it is wordier and less suitable for a short mobile toast.)
inventory-reorder-math
A café tracks beans for three drinks. Each latte uses 18 g, each cappuccino 16 g, each flat white 20 g. Yesterday they sold 74 lattes, 51 cappuccinos, and 39 flat whites. They start today with 4.8 kg of beans. A shipment of 1.25 kg arrives at noon, but 0.35 kg must be discarded due to a torn bag. They want to end the day with at least 0.9 kg in reserve. Assuming today’s sales match yesterday exactly, how many additional grams of beans can they use for staff training samples today without dropping below the reserve? Show your calculation.
Both outputs are fully correct, show the calculation clearly, and follow the instruction exactly. Model B is slightly more streamlined, while Model A is equally clear and well-structured, so neither is meaningfully better overall. (Order-swapped judge pass: Both outputs are fully correct, show the calculation clearly, and arrive at the same correct result of 1,872 grams. Model B is slightly more concise, while Model A is slightly more step-by-step, but neither is meaningfully better overall.)
es-mx-support-translation
Translate this customer-support message into Spanish (Mexico). Keep the meaning, sound calm and professional, and use 28–34 words total. Do not mention regions other than Mexico. Text: “We’ve confirmed your refund, but your bank may take 3 to 5 business days to post it. If the charge still appears after that, reply with the last four digits of the card.”
Both translations are accurate, calm, professional, and within the 28–34 word limit. Model B reads slightly more naturally and precisely in Spanish by using “después de ese plazo,” which better matches the original timing condition. (Order-swapped judge pass: Both are accurate and professional, but A better preserves the original meaning by explicitly conveying 'after that period' and stays within the 28–34 word limit. B is also strong, but 'después' is slightly less precise than 'after that' and the phrasing is a bit less complete.)
messy-orders-to-json
Clean this inline order log into VALID JSON only. Output an array of objects using this exact schema and key order: {"order_id": string, "customer": string, "email": string|null, "ship_date": "YYYY-MM-DD"|null, "items": [{"sku": string, "qty": integer}], "priority": boolean}. Rules: trim spaces, uppercase SKUs, combine duplicate SKUs within an order by summing qty, normalize dates, and convert missing email/ship date to null. Data: [Order A-104 | customer: Nila Ortega | email: nila.ortega@brimco.io | ship: 4/9/25 | items: qz-11 x2; qz-11 x 1; lm-3 x4 | priority: yes] [Order B-017 | customer: Rowan Pike | email: none | ship: pending | items: aa-9 x 10 | priority: no] [Order C-882 | customer: Petra Senn | email: petra@north-glen.net | ship: 2025-04-11 | items: tk-7 x1; LM-3 x 2; tk-7 x 3 | priority: true]
Both outputs are valid JSON and correctly normalize dates, trim fields, uppercase and combine duplicate SKUs, and convert missing values to null while preserving the required schema and key order. The only difference is formatting style, which does not affect quality for this task. (Order-swapped judge pass: Both outputs are valid JSON arrays matching the exact schema and key order, with correctly normalized dates, trimmed fields, uppercased and merged SKUs, and proper null/boolean conversions. The only differences are insignificant formatting choices.)
meeting-notes-3-bullets
Summarize these meeting notes into EXACTLY 3 bullet points. Each bullet must be one sentence, start with a bolded topic tag like **Timeline:**, and contain only information supported by the notes. Notes: “Monday sync, Harbor Rail redesign. Mina said the vendor images won’t be licensed until May 14, so the homepage mock can’t be finalized this week. Dev lead Ivo confirmed the mobile nav fix is ready for QA today, and desktop search speed improved from 1.8s to 0.9s after caching changes. Priya wants pricing copy reviewed by legal before launch; she will send the draft by Wednesday noon. The team moved the public beta target from May 20 to May 27. No decision yet on whether testimonials stay on the landing page.”
Model B is more faithful to the notes and avoids unsupported causation, while still meeting the exact formatting requirements. Model A incorrectly implies the beta delay was caused by the image licensing issue and slightly distorts Priya's action by saying she will send the draft to legal rather than simply send the draft by Wednesday noon. (Order-swapped judge pass: Model A is fully supported by the notes and cleanly follows the exact 3-bullet, one-sentence format. Model B is mostly strong but introduces an unsupported causal link between the beta delay and the image licensing issue, reducing correctness.)
python-dedupe-fix
The Python function below should return a list with duplicates removed while preserving the first occurrence order. It mostly works but has a subtle bug for some inputs. Find the bug and return ONLY the corrected code. ```python def unique_ids(values, seen=set()): result = [] for v in values: if v not in seen: seen.add(v) result.append(v) return result ```
Both outputs provide the same correct fix for the mutable default argument bug and follow the instruction to return only corrected code.
Summarize dense passage
Summarize the passage below in exactly three bullet points, each one sentence, capturing the mechanism, the tradeoff, and the caveat — no jargon a non-specialist couldn't follow. Return only the bullets. Passage: "Speculative decoding pairs a small draft model with a large target model: the draft proposes several tokens, the target verifies them in one pass, and accepted tokens are kept while the first rejection resets to the target's own choice. This can cut latency substantially when acceptance is high, but the draft model's compute is pure overhead when acceptance is low, and gains evaporate on adversarial or highly novel inputs where the draft and target disagree often."
Model A better captures all three requested elements, especially the mechanism and caveat, including the fallback behavior after a rejected guess. Model B is clear and concise but omits the key detail that the first rejection resets to the large model's own choice. (Order-swapped judge pass: Both follow the format and explain the idea clearly, but B is slightly better because it preserves the key caveat in the mechanism that the first rejected guess triggers a fallback to the larger model's own choice. A is very good but omits that reset behavior and is a bit less precise overall.)
ticket-labeling
Classify each support ticket into exactly one category: billing, bug, feature_request, account_access. Return a JSON array; each object must have keys in this order: id, category, justification. Keep each justification to 8 words or fewer. Tickets: T1: “After yesterday’s update, exporting to CSV creates a blank file on both Chrome and Edge.” T2: “Can you add an approval step before invoices are sent to clients?” T3: “I was charged twice for order 7714 and need one payment reversed.” T4: “The password reset link says expired immediately, even when I open it right away.” T5: “Please let us hide archived projects from the left sidebar without deleting them.”
Model A correctly classifies all five tickets and follows the required JSON structure with concise justifications. Model B misclassifies T4 as a bug instead of account_access, though its formatting and brevity are otherwise strong. (Order-swapped judge pass: Model A correctly classifies all tickets and follows the required plain JSON format. Model B misclassifies T4 as account_access instead of bug and wraps the JSON in code fences, which is less compliant.)
Matchup powered by OpenRouter.