This matchup wasn’t especially close: one model consistently turned in cleaner, more reliable answers across coding, reasoning, and editing tasks. The loser had a few real bright spots, but not enough to dent a statistically clear result.
Cohere-command-a-plus-05-2026 takes this head-to-head decisively. The topline says it all: **96.4 to 80.5**, with a **95% confidence** verdict and a **22-9 task win edge**. That is not a vibes-based win; it is a broad, repeatable performance gap.
Where Cohere separated itself was reliability under instructions. It repeatedly beat Llama on tasks where small implementation details matter: URL query redaction, proofreading, scheduling, TTL/LRU cache semantics in Go, tone-sensitive localization in Spanish, and unit-aware math presentation. The pattern is consistent in the judge notes: Model B was usually the one that preserved order, handled edge cases, fixed the actual bug, or simply followed the requested format more cleanly. Llama, by contrast, kept bleeding points through avoidable mistakes — extra wrapper text, example output when the prompt said code only, brittle parsing, and partial correctness on spec-heavy coding tasks.
Llama was not outclassed everywhere. It was better in **French customer-facing localization**, where its phrasing was often more natural and constraint-aware, and it had some legitimate wins in **SQL** and parts of the **LRU cache** and **Python bugfix** set when instruction-following mattered more than embellishment. On the strongest reading of its case, Llama is the model you’d rather have for a narrow slice of concise, polished copy tasks and some minimal-code responses.
But that case is too narrow to change the verdict. Across the full spread, Cohere was the sturdier model: better at getting the details right, better at staying on spec, and better at avoiding the kind of sloppy failure modes that turn a plausible answer into a wrong one. **Final call: Cohere-command-a-plus-05-2026 wins clearly.**
js-query-redact
JavaScript (Node.js 18). Write a function `sanitizeUrl(url)` that removes tracking query parameters from a URL string while preserving everything else, including hash fragments and parameter order. Remove any parameter whose name, case-insensitively, is one of: `utm_source`, `utm_medium`, `utm_campaign`, `gclid`, `fbclid`, `mc_eid`. If the same removable key appears multiple times, remove all of them. Keep blank values and duplicate non-tracking parameters unchanged. Examples:
- `https://nova.example/shop?sku=41&utm_source=newsletter&ref=hp#top` -> `https://nova.example/shop?sku=41&ref=hp#top`
- `https://a.co/?fbclid=Z&x=1&X=2` -> `https://a.co/?x=1&X=2`
Return code only.
Llama-4-Scout-17B-16E-Instruct: ```javascript
/**
* Removes tracking query parameters from a URL string.
*
* @param {string} url - The URL string to sanitize.
* @returns {string} The sanitized URL string.
*/
function sanitizeUrl(url) {
// Define the tracking query parameter names to remove (case-insensitive).
const trackingParams = ['utm_source', 'utm_medium', 'utm_campaign', 'gclid', 'fbclid', 'mc_eid'];
// Split the URL into its components.
const [baseUrl, queryStringWithHash] = url.split('?');
const [queryString, hash] = queryStringWithHash ? queryStringWithHash.split('#') : [queryStringWithHash, ''];
// If there's no query string, return the original URL.
if (!queryString) return url;
// Split the query string into individual parameters.
const params = queryString.split('&');
// Filter out the tracking parameters.
const filteredParams = params.filter(param => {
const [key] = param.split('=');
return !trackingParams.includes(key.toLowerCase());
});
// Join the filtered parameters back into a query string.
const sanitizedQueryString = filteredParams.join('&');
// Reconstruct the sanitized URL.
const sanitizedUrl = baseUrl + (sanitizedQueryString ? '?' + sanitizedQueryString : '') + (hash ? '#' + hash : '');
return sanitizedUrl;
}
// Example usage:
console.log(sanitizeUrl('https://nova.example/shop?sku=41&utm_source=newsletter&ref=hp#top'));
// Output: https://nova.example/shop?sku=41&ref=hp#top
console.log(sanitizeUrl('https://a.co/?fbclid=Z&x=1&X=2'));
// Output: https://a.co/?x=1&X=2
```
Cohere-command-a-plus-05-2026: <|START_TEXT|>```js
/**
* Removes specified tracking query parameters from a URL string.
* Preserves everything else, including hash fragments and parameter order.
*
* @param {string} url - The URL to sanitize.
* @returns {string} The sanitized URL.
*/
function sanitizeUrl(url) {
const tracking = new Set([
'utm_source',
'utm_medium',
'utm_campaign',
'gclid',
'fbclid',
'mc_eid'
]);
try {
const parsed = new URL(url);
const cleanedParams = new URLSearchParams();
// Iterate over existing search parameters in order
for (const [key, value] of parsed.searchParams) {
if (!tracking.has(key.toLowerCase())) {
cleanedParams.append(key, value);
}
}
// Update the search part (omitted if empty)
parsed.search = cleanedParams.toString();
return parsed.toString();
} catch (_) {
// If the URL is malformed, return it unchanged
return url;
}
}
```<|END_TEXT|>
Model B is better because it correctly removes matching parameters case-insensitively while preserving duplicate non-tracking parameters and order via URLSearchParams. Model A has parsing flaws with URLs containing multiple '?' or '#' characters and includes extra example output despite the prompt saying return code only, while Model B is cleaner though it may normalize URL formatting through URL/toString(). (Second judge pass, order swapped — scores are the average of both: Model B is more correct and robust: it removes the specified keys case-insensitively, preserves duplicate non-tracking parameters and hash fragments, and avoids the reconstruction bugs present in Model A. Model A violates the "code only" instruction by including example usage, and its manual splitting can break URLs containing multiple '?' or '#' characters in the query or fragment.)
fr-localize-status-update
Translate this product update into French for customers in France. Tone: warm and professional. Keep it to 2 sentences, under 42 words total, preserving the meaning and apology.
"We found a billing error that may have charged some BrightRoute users twice on March 4. We're reversing the duplicate charge automatically within 3 business days, and we're sorry for the hassle."
Llama-4-Scout-17B-16E-Instruct: Nous avons détecté une erreur de facturation ayant pu doubler les frais pour certains utilisateurs BrightRoute le 4 mars. Nous annulons automatiquement le double débit sous 3 jours ouvrés, et nous nous excusons pour la gêne.
Cohere-command-a-plus-05-2026: <|START_TEXT|>Nous avons détecté une erreur de facturation qui a pu entraîner un double prélèvement pour certains utilisateurs de BrightRoute le 4 mars. Nous annulons automatiquement ce double paiement sous 3 jours ouvrables et nous nous excusons pour ce désagrément.<|END_TEXT|>
Model A is more natural and customer-facing in French for France, while preserving the apology and staying within the length and sentence constraints. Model B is also accurate, but the added markup tokens violate the instruction and make it less polished despite otherwise solid wording. (Second judge pass, order swapped — scores are the average of both: Model B is more accurate and natural for customers in France: it clearly conveys a possible double charge, keeps the apology, and uses customer-friendly phrasing. Model A is also strong, but "doubler les frais" is less precise than a duplicate charge and "la gêne" sounds slightly less polished than "ce désagrément.")
Unit-aware math
A pump moves 3.5 liters every 8 seconds. A tank holds 0.9 cubic meters. Starting empty, how long to fill it, in minutes and seconds (mm:ss), rounded to the nearest second? Show the key steps, then give the final answer on its own line.
Llama-4-Scout-17B-16E-Instruct: To solve this problem, let's break it down into steps:
1. **Convert the tank's volume to liters**: Since 1 cubic meter = 1000 liters,
- 0.9 cubic meters = 0.9 * 1000 = 900 liters.
2. **Determine the pump's rate in liters per second**:
- The pump moves 3.5 liters every 8 seconds. So, its rate is 3.5 / 8 = 0.4375 liters per second.
3. **Calculate the time to fill the tank in seconds**:
- The tank holds 900 liters. At a rate of 0.4375 liters per second, the time to fill the tank is 900 / 0.4375 = 2057.14 seconds.
4. **Convert the time to minutes and seconds and round to the nearest second**:
- 2057.14 seconds ≈ 2057 seconds.
- To convert seconds into minutes and seconds, divide by 60: 2057 / 60 = 34.2833 minutes.
- The whole minutes are 34, and the remaining seconds are 0.2833 * 60 ≈ 17 seconds.
5. **Format the answer as mm:ss**:
- 34 minutes and 17 seconds = 34:17.
**34:17**
Cohere-command-a-plus-05-2026: To determine how long it takes to fill the tank, follow these steps:
1. **Convert the tank volume to liters:**
\[
0.9 \text{ cubic meters} = 0.9 \times 1000 \text{ liters} = 900 \text{ liters}
\]
2. **Calculate the pump rate in liters per second:**
\[
\text{Pump rate} = \frac{3.5 \text{ liters}}{8 \text{ seconds}} = 0.4375 \text{ liters/second}
\]
3. **Determine the total time in seconds to fill the tank:**
\[
\text{Time (seconds)} = \frac{900 \text{ liters}}{0.4375 \text{ liters/second}} = 2057.142857 \text{ seconds}
\]
4. **Convert the total time to minutes and seconds:**
\[
\text{Minutes} = \left\lfloor \frac{2057.142857}{60} \right\rfloor = 34 \text{ minutes}
\]
\[
\text{Remaining seconds} = 2057.142857 - (34 \times 60) = 2057.142857 - 2040 = 17.142857 \text{ seconds}
\]
5. **Round the remaining seconds to the nearest second:**
\[
17.142857 \approx 17 \text{ seconds}
\]
Thus, the time to fill the tank is:
\[
\boxed{34:17}
\]
Model A and Model B both compute the unit conversions and fill time correctly, arriving at 34:17. Model B is slightly more precise and polished in presentation, while Model A is also clear and fully follows the requested format, so neither has a meaningful quality advantage overall. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both compute the correct result, 34:17, and both show the key steps. Model B is slightly better because its unit handling and minute/second conversion are a bit more precise and clearly presented, while Model A rounds to whole seconds earlier than necessary before converting.)
Localization with tone
Translate this app onboarding line into natural, friendly European Spanish suitable for a mobile toast (keep it under 60 characters, no exclamation marks): "You're all set — your first backup starts tonight." Return only the translation, then the character count in parentheses.
Llama-4-Scout-17B-16E-Instruct: "Todo configurado, tu primera copia de seguridad comienza esta noche." (57)
Cohere-command-a-plus-05-2026: Todo listo, tu primera copia de seguridad comienza esta noche. (57)
Model B is better because it provides a natural, friendly European Spanish toast and correctly avoids quotation marks, while Model A adds quotes and its stated character count appears inaccurate. Both exceed the 60-character limit, but Model B is otherwise cleaner and more idiomatic. (Second judge pass, order swapped — scores are the average of both: Model B is more natural and friendly for a mobile toast in European Spanish and better matches the requested concise onboarding tone. Model A adds quotation marks, exceeds the 60-character limit, and sounds slightly less idiomatic for this context.)
Precise proofreading
Correct ONLY the grammar, spelling, and punctuation errors in the sentence below without changing wording, tone, or meaning, then on a second line list each change as 'was -> now'. Sentence: "Its been a long time since we seen there new office, and me and him was suprised how quite it was."
Llama-4-Scout-17B-16E-Instruct: Here is the corrected sentence:
"It's been a long time since we seen their new office, and he and I were surprised how quiet it was."
Here are the changes:
* Its -> It's
* seen there -> seen their
* me and him -> he and I
* was -> were
* suprised -> surprised
* quite -> quiet
Cohere-command-a-plus-05-2026: "It's been a long time since we saw their new office, and he and I were surprised how quiet it was."
- Its -> It's
- been -> been (no change)
- seen -> saw
- there -> their
- new -> new (no change)
- office -> office (no change)
- and -> and (no change)
- me -> I
- and -> and (no change)
- him -> he
- was -> were
- suprised -> surprised
- quite -> quiet
- it -> it (no change)
- was -> was (no change)
- ending punctuation added: .
Model B is better because it fully corrects the grammar error "we seen" to "we saw," while Model A leaves that error in place. Both models add unnecessary extra wording and Model B includes some needless no-change items, but Model B is still more correct overall. (Second judge pass, order swapped — scores are the average of both: Model B is better because it fully corrects the sentence, including changing "we seen" to "we saw," while Model A leaves that grammatical error uncorrected. Model B also more closely follows the requested second-line change listing format, whereas Model A adds extra framing text and misses a required change.)
LRU cache
Implement a class `LRUCache<K, V>` in TypeScript with a fixed capacity set in the constructor, and O(1) `get(key): V | undefined` and `set(key, value): void`. Accessing or updating a key must mark it most-recently-used; inserting beyond capacity must evict the least-recently-used entry. Return ONLY the class, no prose.
Llama-4-Scout-17B-16E-Instruct: ```typescript
class LRUCache<K, V> {
private capacity: number;
private cache: Map<K, [V, number]>;
private counter: number;
constructor(capacity: number) {
this.capacity = capacity;
this.cache = new Map();
this.counter = 0;
}
get(key: K): V | undefined {
if (!this.cache.has(key)) {
return undefined;
}
const [value, _] = this.cache.get(key)!;
this.cache.delete(key);
this.cache.set(key, [value, ++this.counter]);
return value;
}
set(key: K, value: V): void {
if (this.cache.has(key)) {
this.cache.delete(key);
} else if (this.cache.size === this.capacity) {
const lruKey = this.findLRUKey();
this.cache.delete(lruKey);
}
this.cache.set(key, [value, ++this.counter]);
}
private findLRUKey(): K {
let lruKey: K | null = null;
let minCounter = Infinity;
for (const [key, [, counter]] of this.cache) {
if (counter < minCounter) {
minCounter = counter;
lruKey = key;
}
}
return lruKey as K;
}
}
```
Model A is valid TypeScript and mostly implements LRU behavior, but it violates the O(1) requirement because eviction scans the map. Model B aims for the correct O(1) design, but its TypeScript is invalid due to the nested class declaration and related type usage, so it is not a workable implementation as written. (Second judge pass, order swapped — scores are the average of both: Model A is better because it is valid TypeScript syntax and returns only a class, while Model B includes invalid TypeScript constructs such as a nested class declaration and extra wrapper text/code fences. Model A does miss the required O(1) eviction by scanning to find the least-recently-used key, so neither fully satisfies the prompt.)
SQL window query
Given Postgres tables `orders(id, customer_id, created_at, total_cents)` and `customers(id, name)`, write ONE SQL query returning each customer's name and their SECOND-largest order total (in dollars, 2 decimals), excluding customers with fewer than two orders. Order by that value descending. Return ONLY the SQL.
Llama-4-Scout-17B-16E-Instruct: ```sql
SELECT
c.name,
ROUND(o.total_cents::numeric / 100, 2) AS second_largest_order
FROM
customers c
JOIN
(
SELECT
customer_id,
total_cents,
ROW_NUMBER() OVER (PARTITION BY customer_id ORDER BY total_cents DESC) AS row_num
FROM
orders
) o ON c.id = o.customer_id
WHERE
o.row_num = 2
ORDER BY
o.total_cents DESC;
```
Cohere-command-a-plus-05-2026: ```sql
WITH ranked_orders AS (
SELECT
c.name,
o.total_cents / 100.0 AS total_dollars,
ROW_NUMBER() OVER (PARTITION BY c.id ORDER BY o.total_cents DESC) AS row_num
FROM customers c
JOIN orders o ON c.id = o.customer_id
),
filtered_orders AS (
SELECT
name,
total_dollars
FROM ranked_orders
WHERE row_num = 2
)
SELECT
name,
ROUND(total_dollars, 2) AS second_largest_order
FROM filtered_orders
ORDER BY second_largest_order DESC;
```
Model A and Model B both correctly use a window function to select the second-highest order per customer, exclude customers with fewer than two orders, format dollars to two decimals, and sort descending. Model A is slightly more direct, while Model B is a bit more verbose with CTEs, but neither has a substantive correctness or instruction-following advantage. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both correctly use a window function to select each customer's second-highest order and exclude customers with fewer than two orders. Model A is slightly more concise, while Model B orders by the aliased output column as requested; both satisfy the prompt well.)
Constraint scheduling
Four talks (A, B, C, D) fill four 1-hour slots 9,10,11,12. Constraints: A is before D; C is not first; B is immediately after A; D is not at 12. Give the ONE valid schedule as 'slot: talk' lines, then a one-line justification. If impossible, say so and explain.
Llama-4-Scout-17B-16E-Instruct: Here is the valid schedule:
9: A
10: B
11: C
12: D
Justification: This order satisfies all constraints: A is before D, C is not first, B is immediately after A, and D is not at 12 is also not satisfied as per the schedule D is at 12, reevaluating the constraints and the schedule.
Reevaluation:
9: A
10: B
11: D
does not work as C is missing, and
C is not first can be satisfied when C is at 11 or 12; D cannot be at 12; A before D; B after A.
Let's try
9: C
does not work as C is first.
9: A
10: B
C can be at 11 or 12
A before D and D not at 12
so
9: A
10: B
11: C
12: D does not work
A at 9
B at 10
D at 11
C at 12
9: A
10: B
11: D
12: C
Justification: This order satisfies all constraints: A is before D, C is not first, B is immediately after A, and D is not at 12.
Cohere-command-a-plus-05-2026: **Valid Schedule:**
9: A
10: B
11: C
12: D
**Justification:**
- A is before D (A at 9, D at 12).
- C is not first (C is at 11).
- B is immediately after A (B at 10, following A at 9).
- D is not at 12 (D is at 12, but this is the only valid slot given the constraints).
However, upon re-evaluation, the constraint "D is not at 12" is violated in the above schedule. Let's correct this:
**Corrected Valid Schedule:**
9: A
10: B
11: D
12: C
**Justification:**
- A is before D (A at 9, D at 11).
- C is not first (C is at 12).
- B is immediately after A (B at 10, following A at 9).
- D is not at 12 (D is at 11).
This schedule satisfies all constraints.
**Final Answer:**
9: A
10: B
11: D
12: C
Justification: All constraints are satisfied: A before D, C not first, B immediately after A, and D not at 12.
Model B ultimately gives the correct unique schedule and ends with a clear final answer, while Model A includes more confused intermediate reasoning and a malformed partial attempt before arriving at the same schedule. Both violate the instruction to provide only the schedule plus a one-line justification, but Model B is cleaner and more coherent overall. (Second judge pass, order swapped — scores are the average of both: Model B ultimately gives the unique valid schedule correctly and ends with a clear final answer, though it includes an unnecessary false start. Model A also reaches the correct schedule, but it is more rambling, less cleanly formatted, and includes more distracting intermediate reasoning that does not follow the requested concise output format as well.)
jsonl-to-clean-json
Convert the messy inline records below into a valid JSON array. Output only JSON. Schema for each object, in this exact key order: `ticket_id` (string), `customer` (string), `priority` ("low"|"medium"|"high"), `hours_open` (number), `tags` (array of lowercase strings), `resolved` (boolean). Rules: trim spaces, normalize priority aliases (`L`->low, `med`->medium, `HIGH`->high), parse hours as numbers, split tags on `|` or `,`, drop empty tags, lowercase tags, and sort objects by `hours_open` descending.
Records:
`id= Q-104 ; customer=Helio Print ; priority= med ; hours_open= 17.5 ; tags=invoice|AP ; resolved=no`
`customer=Northbay Bikes;resolved=yes;tags= ;hours_open=2;priority=L;id=Q-099`
`priority=HIGH; id=Q-121; hours_open= 41 ; customer= Mira Labs ; tags= outage,SEV-2,Backend ; resolved= false`
Model A and Model B produce identical valid JSON arrays that correctly normalize fields, parse values, lowercase and split tags, preserve the required key order, and sort by hours_open descending. Neither has any errors or instruction-following issues, so they are equally strong. (Second judge pass, order swapped — scores are the average of both: Model B and Model A produce identical valid JSON arrays that correctly normalize fields, lowercase and split tags, convert resolved values to booleans, preserve the required key order, and sort records by hours_open descending. There are no observable differences in correctness, instruction following, or output quality between Model B and Model A.)
ttl-lru-cache-go
Language: Go. Implement a TTL-enabled LRU cache for string keys and int values. API:
- type Cache struct
- func NewCache(capacity int) *Cache
- func (c *Cache) Put(key string, value int, ttlSeconds int)
- func (c *Cache) Get(key string, now int64) (int, bool)
Rules: expiration time is set at Put using Unix-seconds offset from now=0, so expiresAt = int64(ttlSeconds); Get treats an item as expired when now >= expiresAt and removes it. Capacity counts only unexpired items. On Put, if key exists, update value+TTL and mark most recently used. If insertion would exceed capacity, evict the least recently used unexpired item. Required complexity: O(1) average for Get and Put, excluding garbage collection. Use only the Go standard library. Return code only.
Llama-4-Scout-17B-16E-Instruct: ```go
package main
import (
"container/list"
"sync"
"time"
)
// Cache is a TTL-enabled LRU cache for string keys and int values.
type Cache struct {
mu sync.RWMutex
capacity int
cache map[string]*list.Element
lruList *list.List
}
// cacheItem represents an item in the cache.
type cacheItem struct {
key string
value int
expiresAt int64
}
// NewCache returns a new Cache instance with the given capacity.
func NewCache(capacity int) *Cache {
return &Cache{
capacity: capacity,
cache: make(map[string]*list.Element),
lruList: list.New(),
}
}
// Put inserts or updates a key-value pair with a TTL.
func (c *Cache) Put(key string, value int, ttlSeconds int) {
c.mu.Lock()
defer c.mu.Unlock()
expiresAt := time.Now().Unix() + int64(ttlSeconds)
if elem, ok := c.cache[key]; ok {
// Update existing item
item := elem.Value.(*cacheItem)
item.value = value
item.expiresAt = expiresAt
c.lruList.MoveToFront(elem)
return
}
// Add new item
item := &cacheItem{key, value, expiresAt}
elem := c.lruList.PushFront(item)
c.cache[key] = elem
// Evict LRU item if capacity exceeded
for c.lruList.Len() > c.capacity {
backElem := c.lruList.Back()
c.lruList.Remove(backElem)
item := backElem.Value.(*cacheItem)
delete(c.cache, item.key)
}
}
// Get retrieves a value by key and checks its TTL.
func (c *Cache) Get(key string, now int64) (int, bool) {
c.mu.Lock()
defer c.mu.Unlock()
elem, ok := c.cache[key]
if !ok {
return 0, false
}
item := elem.Value.(*cacheItem)
if now >= item.expiresAt {
// Remove expired item
c.lruList.Remove(elem)
delete(c.cache, key)
return 0, false
}
// Move accessed item to front
c.lruList.MoveToFront(elem)
return item.value, true
}
func main() {}
```
Cohere-command-a-plus-05-2026: ```go
package lru
import (
"container/list"
"sync"
)
type entry struct {
key string
value int
expiresAt int64
}
type Cache struct {
capacity int
mu sync.RWMutex
items map[string]*list.Element
lruList *list.List
}
func NewCache(capacity int) *Cache {
return &Cache{
capacity: capacity,
items: make(map[string]*list.Element),
lruList: list.New(),
}
}
func (c *Cache) Put(key string, value int, ttlSeconds int) {
c.mu.Lock()
defer c.mu.Unlock()
now := int64(0) // Assume now=0 for expiration calculation
expiresAt := now + int64(ttlSeconds)
if elem, exists := c.items[key]; exists {
c.lruList.MoveToFront(elem)
elem.Value.(*entry).value = value
elem.Value.(*entry).expiresAt = expiresAt
return
}
newEntry := &entry{key: key, value: value, expiresAt: expiresAt}
elem := c.lruList.PushFront(newEntry)
c.items[key] = elem
for c.lruList.Len() > c.capacity {
c.evict()
}
}
func (c *Cache) Get(key string, now int64) (int, bool) {
c.mu.Lock()
defer c.mu.Unlock()
if elem, exists := c.items[key]; exists {
entry := elem.Value.(*entry)
if now < entry.expiresAt {
c.lruList.MoveToFront(elem)
return entry.value, true
}
c.removeElement(elem)
}
return 0, false
}
func (c *Cache) evict() {
for c.lruList.Len() > 0 {
back := c.lruList.Back()
entry := back.Value.(*entry)
if entry.expiresAt > 0 { // Always evictable since we check expiration
c.removeElement(back)
break
}
c.lruList.Remove(back) // Remove expired items from the list
}
}
func (c *Cache) removeElement(e *list.Element) {
c.lruList.Remove(e)
delete(c.items, e.Value.(*entry).key)
}
```
Model B is better because it correctly sets expiration relative to now=0 and removes expired items on Get, whereas Model A incorrectly uses the real current time and adds an unnecessary main function. Both miss the rule that capacity counts only unexpired items and may evict expired entries incorrectly or fail to purge them before capacity checks, but Model B is closer to the requested semantics overall. (Second judge pass, order swapped — scores are the average of both: Model B at least uses the required TTL basis of now=0 and removes expired items on Get, though its eviction logic is flawed and it does not properly ensure capacity counts only unexpired items. Model A is substantially less correct because it computes expiration from time.Now(), adds an unnecessary main function, uses the wrong package shape, and also ignores the requirement that capacity count only unexpired items.)
python-window-bugfix
Python 3. The function below is supposed to return the length of the longest substring of s containing at most k distinct characters, but it fails on cases like s='abaccc', k=2. Find the bug and return corrected code only.
```python
def longest_k_distinct(s, k):
left = 0
counts = {}
best = 0
for right, ch in enumerate(s):
counts[ch] = counts.get(ch, 0) + 1
while len(counts) >= k:
old = s[left]
counts[old] -= 1
if counts[old] == 0:
del counts[old]
left += 1
best = max(best, right - left + 1)
return best
```
Llama-4-Scout-17B-16E-Instruct: ```python
def longest_k_distinct(s, k):
left = 0
counts = {}
best = 0
for right, ch in enumerate(s):
counts[ch] = counts.get(ch, 0) + 1
while len(counts) > k:
old = s[left]
counts[old] -= 1
if counts[old] == 0:
del counts[old]
left += 1
best = max(best, right - left + 1)
return best
```
Cohere-command-a-plus-05-2026: <|START_TEXT|>```python
def longest_k_distinct(s: str, k: int) -> int:
"""
Returns the length of the longest substring of s that contains
at most k distinct characters.
"""
# Edge case: no characters allowed -> only empty substring
if k == 0:
return 0
left = 0
counts = {}
best = 0
for right, ch in enumerate(s):
counts[ch] = counts.get(ch, 0) + 1
# Shrink window only while we have *more* than k distinct chars
while len(counts) > k:
old = s[left]
counts[old] -= 1
if counts[old] == 0:
del counts[old]
left += 1
# Window now satisfies the condition (<= k distinct chars)
best = max(best, right - left + 1)
return best
```<|END_TEXT|>
Model B is better because it fixes the core bug and also correctly handles the k=0 edge case, while Model A would raise an error or behave incorrectly there. Both follow the request to return corrected code only, but Model B is slightly more robust and polished. (Second judge pass, order swapped — scores are the average of both: Model A provides the minimal corrected code exactly as requested. Model B fixes the bug correctly too, but it adds type hints, a docstring, comments, and wrapper markers, which goes beyond the instruction to return corrected code only.)
proofread-incident-note
Edit the passage for grammar, clarity, and professional tone without changing meaning. Then, on a new line, briefly note the main changes in one sentence.
"Yesterday our team were notified about a issue with the overnight export job, which had failed silently and wasn't caught until clients emailed support this morning. We have since identify the root cause, a rotated credential that wasn't updated in one service, and put in a temporary fix while a permanent patch are being reviewed."
Llama-4-Scout-17B-16E-Instruct: Here is the revised passage:
"Yesterday, our team was notified about an issue with the overnight export job, which had failed silently and wasn't caught until clients emailed support this morning. We have since identified the root cause, a rotated credential that wasn't updated in one service, and put in a temporary fix while a permanent patch is being reviewed."
The main changes include correcting grammatical errors such as subject-verb agreement, verb tense consistency, and punctuation, as well as rephrasing some sentences for improved clarity and professionalism.
Cohere-command-a-plus-05-2026: **Edited Passage:**
Yesterday, our team was notified of an issue with the overnight export job, which failed silently and was not detected until clients contacted support this morning. We have since identified the root cause—a rotated credential that was not updated in one service—and implemented a temporary fix while a permanent patch is being reviewed.
**Main Changes:**
Corrected subject-verb agreement ("team was notified"), improved clarity and punctuation, and standardized verb tenses for consistency.
Model B is better because it produces a cleaner, more professional edit while preserving meaning, with stronger phrasing such as "was not detected" and "implemented a temporary fix." Model A is solid but adds unnecessary framing text and is slightly less polished in clarity and tone. (Second judge pass, order swapped — scores are the average of both: Model B fully edits the passage for grammar, clarity, and professional tone while preserving meaning, and it provides the required one-sentence summary of changes on a new line. Model A fixes some errors but leaves weaker phrasing such as "notified about," retains contractions and less professional wording, and adds extra framing text rather than cleanly following the requested format.)
