Head to head: Llama-4-Scout-17B-16E-Instruct vs Cohere-command-a-plus-05-2026

On the numbers, Cohere-command-a-plus-05-2026 takes it: 95.5 to 84.5 in aggregate, with a 6-4 task edge and 73% confidence. That is enough to call a winner, but not enough to pretend there is daylight everywhere. This matchup was competitive, and the confidence level says exactly that: lean, not landslide.

Where Cohere earns the verdict is consistency on the boring but decisive stuff. It was better at following output constraints in ticket-routing JSON and Python log redaction, more precise on timezone-sensitive debugging in the JavaScript date fix, and simply more correct on the scheduling puzzle, where Llama produced an invalid schedule. It also edged the contradiction task and the unit-aware math problem by being cleaner and tighter. That profile matters: these are the kinds of failures that turn a plausible answer into one you cannot safely ship.

Llama was not outclassed. It won the SQL window query cleanly, with the more direct and properly formatted solution, and it also took localization with tone, where Cohere drifted on the requested format. It beat Cohere on the Go LRU cache as well, though that result comes with a wrinkle: the two judge passes disagreed sharply on which implementation was actually sound. Llama also picked up the proofreading task in the averaged result, another sign that this was closer than the topline score suggests.

The pattern is straightforward. Cohere is the steadier editor's choice: better at staying inside the lines, less likely to fumble a critical edge case, and more dependable when correctness and format compliance have to coexist. Llama is the more uneven but still credible rival, with real strengths in database querying and some structured generation tasks, but too many misses here came from exactly the kinds of avoidable errors that decide head-to-head evaluations.

Final call: Cohere-command-a-plus-05-2026 wins on reliability, not dominance. If you want the model that more often gives you the answer in the right shape and with fewer hidden correctness traps, Cohere gets the nod; Llama stays in the conversation, but it does not take this matchup.

How they were tested

We ran 12 fresh text tasks, generated on the fly for this matchup so neither model could prepare in advance, and had gpt-5.4 score each one. To cancel position bias, every task was judged twice — once in each presentation order — and every number reported here, including the headline totals, is the average of both passes. Llama-4-Scout-17B-16E-Instruct scored 84.5 to Cohere-command-a-plus-05-2026's 95.5.

1. Find the contradiction

The following spec contains exactly one internal contradiction. Quote the two conflicting sentences verbatim and explain the conflict in one sentence. Do not fix it. Spec: "Free accounts may create up to three projects. Every account, regardless of tier, may archive unlimited projects. Archiving a project does not count against the project limit. Free accounts are limited to three projects total, including archived ones."

Winner: Cohere-command-a-plus-05-2026 — Model B is slightly better because it quotes the two conflicting sentences verbatim and gives a concise one-sentence explanation exactly as requested. Model A is also correct, but its explanation is a bit more verbose than necessary for the task. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both correctly identify the same two contradictory sentences and explain the conflict accurately. Model B is slightly more concise, while Model A is slightly cleaner in presentation; both follow the task well overall.)

2. Localization with tone

Translate this app onboarding line into natural, friendly European Spanish suitable for a mobile toast (keep it under 60 characters, no exclamation marks): "You're all set — your first backup starts tonight." Return only the translation, then the character count in parentheses.

Winner: Llama-4-Scout-17B-16E-Instruct — Model A is cleaner and closer to the requested output format, while Model B includes extraneous markup and omits the required parentheses around the character count. Both have character-count issues and are somewhat long for a mobile toast, but Model A is more polished and compliant overall. (Second judge pass, order swapped — scores are the average of both: Model B is more natural and concise for a mobile toast, and its character count is accurate. Model A sounds less idiomatic in European Spanish, exceeds the 60-character limit, includes quotation marks, and gives an incorrect count.)

3. SQL window query

Given Postgres tables orders(id, customer_id, created_at, total_cents) and customers(id, name), write ONE SQL query returning each customer's name and their SECOND-largest order total (in dollars, 2 decimals), excluding customers with fewer than two orders. Order by that value descending. Return ONLY the SQL.

Winner: Llama-4-Scout-17B-16E-Instruct — Model A more directly and correctly returns each customer's second-largest order total with proper 2-decimal formatting and excludes customers with fewer than two orders via the row-number filter. Model B is mostly correct, but it does not explicitly format to 2 decimals and uses an unnecessary aggregation step that could be problematic if customer names are not unique. (Second judge pass, order swapped — scores are the average of both: Model A more directly satisfies the requirement by returning each customer's second-largest order total formatted to two decimals in a single query and naturally excluding customers with fewer than two orders via ROW_NUMBER() = 2. Model B is mostly correct, but it does not explicitly round to two decimals and uses an unnecessary aggregation step.)

4. Classification & labeling — support ticket routing

Classify each ticket into exactly one category: billing, bug, feature_request, or account_access. Return a JSON array where each element has keys id, category, and why (max 12 words). Tickets: 1. T-881: "I was charged twice for the August Pro plan renewal." 2. T-882: "After the latest update, exporting to CSV downloads an empty file." 3. T-883: "Please add dark mode scheduling by sunset." 4. T-884: "The password reset link says expired immediately after I click it." 5. T-885: "Can you invoice our team annually instead of monthly?" 6. T-886: "Mobile app crashes when I attach a HEIC photo to a report."

Winner: Cohere-command-a-plus-05-2026 — Model B is better because it returns only the requested JSON array content without extra prose, while Model A adds surrounding text and a follow-up sentence. Both models classify the tickets correctly, but Model B adheres more strictly to the output-format instruction. (Second judge pass, order swapped — scores are the average of both: Model B and Model A classify all tickets correctly, but Model B adheres more closely to the requested output format by providing just the JSON array content without extra surrounding commentary. Model A adds explanatory text before and after the JSON, which violates the instruction to return a JSON array.)

5. Debugging & code review — JavaScript date grouping fix

Language: JavaScript. The function should group events by YYYY-MM-DD in UTC and count them, but it gives wrong keys around time zones. Find and fix the bug. Return the corrected code only. javascript function countByDay(events) { const out = {}; for (const e of events) { const d = new Date(e.timestamp); const key = `${d.getFullYear()}-${String(d.getMonth()+1).padStart(2,'0')}-${String(d.getDate()).padStart(2,'0')}`; out[key] = (out[key] || 0) + 1; } return out; }

Winner: Cohere-command-a-plus-05-2026 — Model B is better because it builds the key using UTC getters, which addresses the core bug of local-time date extraction; Model A still uses local getters after mutating the date, so it can produce incorrect keys in non-UTC environments. Both mostly follow the instruction to return corrected code only, though Model A also adds an inline comment and Model B uses a needlessly roundabout UTC conversion. (Second judge pass, order swapped — scores are the average of both: Model B is better because it produces the key using UTC-based getters, which addresses the timezone bug, though the extra conversion via toUTCString is unnecessary. Model A is incorrect because it mutates the date to UTC midnight but still formats it with local-time getters, so the key can still be wrong around time zones.)

6. Constraint scheduling

Four talks (A, B, C, D) fill four 1-hour slots 9,10,11,12. Constraints: A is before D; C is not first; B is immediately after A; D is not at 12. Give the ONE valid schedule as 'slot: talk' lines, then a one-line justification. If impossible, say so and explain.

Winner: Cohere-command-a-plus-05-2026 — Model B is better because it ultimately reaches the correct conclusion that the constraints are impossible to satisfy, while Model A presents an invalid schedule that directly violates the requirement that D is not at 12. Model B is somewhat messy and includes contradictory intermediate text, but its final answer is correct; Model A is concise but incorrect. (Second judge pass, order swapped — scores are the average of both: Model B ultimately reaches the correct conclusion that the schedule is impossible, while Model A presents an invalid schedule and incorrectly claims it satisfies the constraints. Model B is weakened by initially giving a wrong schedule and extra formatting, but it is still substantially better on correctness and overall task adherence.)

7. Algorithm & data structures — Go LRU cache

Language: Go. Implement an LRUCache with methods Constructor(capacity int) LRUCache, Get(key int) int, and Put(key int, value int). All operations must be O(1) average time using a hash map plus a doubly linked list. Get returns -1 if missing. Evict the least recently used item when capacity is exceeded. Return code only.

Winner: Llama-4-Scout-17B-16E-Instruct — Model A correctly uses a hash map plus doubly linked list with O(1) average operations and is mostly complete, though it can panic when capacity is 0 and includes an unnecessary main function. Model B has a linked-list bug in add that fails to set the old tail predecessor's Prev pointer, which breaks eviction/removal behavior after multiple inserts, so its implementation is not correct despite otherwise matching the requested design. (Second judge pass, order swapped — scores are the average of both: Model B correctly implements the required O(1) average-time LRU cache with a hash map plus an explicit doubly linked list and matches the requested API closely. Model A is mostly functional but does not follow the requested data-structure approach as stated, adds an unnecessary main function, and can panic when capacity is 0 because it removes from an empty list.)

8. Data wrangling — messy orders to JSON

Convert the messy inline order notes below into valid JSON: an array of objects with exactly these keys in this order: order_id (string), customer (string), items (array of strings), total_usd (number), paid (boolean). Normalize paid so yes/paid/true => true and no/unpaid/false => false. Trim spaces. Preserve item order. Notes: - Order A-104 | customer: Nia Vale | items: cable; usb hub ; laptop stand | total: $58.70 | status: paid - Order A-105 | customer: Omar Chen | items: notebook | total: 12.00 USD | status: unpaid - Order A-106 | customer: Priya Doss | items: water bottle; trail mix; socks | total: $27 | status: yes - Order A-107 | customer: Leo Maren | items: monitor arm ; HDMI cable | total: USD 44.95 | status: false Return only the JSON.

Winner: Tie — Model A and Model B both produce valid JSON with the required keys in the correct order, correctly normalized boolean values, trimmed item strings, and preserved item order. The only difference is numeric formatting for totals, which does not affect JSON validity or correctness for this task. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both produce valid JSON arrays with the required keys in the correct order, correctly normalized paid values, trimmed items, and preserved item order. The only differences are numeric formatting choices like 58.70 vs 58.7, which are equivalent in JSON numbers and do not affect correctness or instruction adherence.)

9. Practical coding — Python log redaction

Language: Python 3. Write a function sanitize_log(line: str) -> str for an API gateway. Replace any IPv4 address with [IP], any email address with [EMAIL], and any bearer token value in patterns like Authorization: Bearer abc123... with Authorization: Bearer [TOKEN]. Preserve all other text exactly. Assume input is a single log line. Return code only.

Winner: Cohere-command-a-plus-05-2026 — Model B is slightly better because it provides the same core functionality with cleaner, more concise code and avoids the extra example text that violates the "Return code only" instruction. Both Model A and Model B are somewhat imperfect because their IPv4 regexes also match invalid addresses like 999.999.999.999, but Model B adheres more closely to the requested output format. (Second judge pass, order swapped — scores are the average of both: Model B and Model A are very similar in core behavior, but Model B is slightly better because it returns only the requested function code, while Model A violates the prompt by adding example usage outside the function. Both correctly handle the requested redactions in a straightforward way, though neither validates IPv4 ranges strictly and both use a somewhat loose email regex.)

10. Step-by-step reasoning — warehouse pick timing

A picker starts at Packing Desk P and must collect all four bins, then return to P. Walking times in minutes are symmetric: - P-A 4, P-B 6, P-C 5, P-D 7 - A-B 3, A-C 6, A-D 4 - B-C 2, B-D 5 - C-D 3 Picking time at each bin is 1 minute. What is the minimum total time in minutes to start at P, visit A, B, C, D exactly once each in any order, include picking time at each visited bin, and return to P? Give the minimum time and one optimal route.

Winner: Tie — Model A and Model B both arrive at the correct minimum total time of 23 minutes and provide a valid optimal route. Neither fully proves optimality by checking all permutations despite claiming step-by-step reasoning, but both are clear and adequately follow the prompt. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both give the correct minimum total time of 23 minutes and provide a valid optimal route. Model B is slightly cleaner, while Model A is a bit more verbose and mentions a heuristic unnecessarily, but neither makes a substantive error or violates the instructions.)

11. Unit-aware math

A pump moves 3.5 liters every 8 seconds. A tank holds 0.9 cubic meters. Starting empty, how long to fill it, in minutes and seconds (mm:ss), rounded to the nearest second? Show the key steps, then give the final answer on its own line.

Winner: Cohere-command-a-plus-05-2026 — Model B is slightly better because it is equally correct but presents the unit-aware steps more cleanly and explicitly, with a clearer final-answer section. Model A also gets the right result, but its conversion to minutes and seconds is a bit less precise in presentation. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both compute the conversion and fill time correctly and both end with the correct standalone final answer, 34:17. Model B is slightly more polished in formatting, but Model A also follows the instructions well and is equally correct overall.)

12. Precise proofreading

Correct ONLY the grammar, spelling, and punctuation errors in the sentence below without changing wording, tone, or meaning, then on a second line list each change as 'was -> now'. Sentence: "Its been a long time since we seen there new office, and me and him was suprised how quite it was."

Winner: Llama-4-Scout-17B-16E-Instruct — Model A is better because it makes fewer unnecessary changes and is cleaner, though it still fails to correct "we seen" to "we saw" and adds extra formatting not requested. Model B correctly fixes "seen" to "saw," but it includes many no-change items, extra commentary, and contradictory reasoning, which violates the instruction to provide only the corrected sentence and a concise change list. (Second judge pass, order swapped — scores are the average of both: Model B is better because it fully corrects the grammar and spelling errors, including changing "we seen" to "we saw," while Model A leaves that error uncorrected. Both models violate the instruction to provide only the corrected sentence and a second line of changes, but Model B is still more correct overall despite its extra commentary and overly long change list.)

See every prompt and the full side-by-side outputs in the interactive Head-to-Head.