Head to head: SpaceXAI: Grok 4.6 vs Anthropic: Claude Opus 4.8
SpaceXAI: Grok 4.6 vs Anthropic: Claude Opus 4.8
This matchup tests whether polished prose can outweigh exact instruction-following, factual fidelity, and reliable code. Across 12 tasks, the gap emerged less from raw capability than from disciplined execution.
Grok 4.6 wins decisively: **113.2 to 99.8**, with **96% confidence**. The task ledger—six wins for Grok, one for Claude Opus, and five ties—shows a broad advantage rather than a result driven by one outsized benchmark. The most consequential separation came in the Go worker-pool task. Grok produced a correct deadlock-free implementation, while Claude introduced an unsafe `WaitGroup.Add`/`Wait` race that could close the results channel before jobs were registered. Grok was also stricter where exactness mattered: it returned bare valid JSON, preserved release-note qualifiers such as the 2,000-project threshold and previous 100-ID limit, and adhered more closely to source wording in proofreading and rewriting. Claude’s writing was often smoother, but polish repeatedly came with small fidelity costs: “around noon” replaced a firm noon ETA, “I’ll keep you posted” added a commitment, and its dense-passage summary blurred model agreement into objectively “correct” guesses. Its sole win came in Japanese support translation, where its refund wording was more natural and unambiguous. The five ties—including LRU code, payroll reasoning, PostgreSQL, promise caching, and contradiction detection—confirm that Claude remained competitive on core capability. **Final call: Grok 4.6 is the clear winner. Its advantage is not stylistic flash but dependable precision, stronger constraint compliance, and—on the worker-pool task—materially safer code.**
summarize-release-notes
Summarize the following changelog into EXACTLY 4 bullet points. Each bullet must be 9–13 words, start with a strong verb, and mention only facts stated below. Changelog: - Web dashboard now loads project lists incrementally, reducing initial render time on accounts with more than 2,000 projects. - Fixed a bug where CSV exports silently omitted rows containing embedded line breaks in comments. - Added an approval step for invoice edits above $5,000; workspace owners can disable it. - API: POST /v2/tasks/bulk-close now accepts up to 500 task IDs per request, previously 100. - Mobile app version 3.8.1 fixes a crash triggered by opening push notifications after logging out. - Renamed the setting “Guests can upload files” to “External collaborators can upload files”; behavior is unchanged. - Audit logs now retain IP addresses for 180 days instead of 90.
Model A is more precise and preserves key qualifiers, including the project threshold and comments context. Model B reads more smoothly but broadens or weakens some details through less specific wording. (Second judge pass, order swapped — scores are the average of both: Model A preserves more stated specifics, including comments, the 2,000-project threshold, and the previous 100-ID limit. Model B follows the format but substitutes the vaguer phrase “large accounts” and omits more relevant details.)
debug-go-worker-pool
The following Go code is intended to process all jobs concurrently and return one result per input in the same order. It sometimes hangs or returns too few results. Find the bug(s), fix them, and return the corrected code only. ```go package main import "sync" func DoubleAll(nums []int) []int { jobs := make(chan int) results := make(chan struct { idx int val int }) var wg sync.WaitGroup worker := func() { for j := range jobs { results <- struct { idx int val int }{idx: j, val: nums[j] * 2} wg.Done() } } for i := 0; i < 4; i++ { go worker() } out := make([]int, len(nums)) for i := range nums { wg.Add(1) jobs <- i } close(jobs) go func() { wg.Wait() close(results) }() for r := range results { out[r.idx] = r.val } return out } ```
Model A correctly avoids the deadlock and uses the WaitGroup to track worker completion, with all Add calls occurring before Wait. Model B introduces an invalid Add/Wait race that can close results before jobs are registered, and it also violates the request to return corrected code only. (Second judge pass, order swapped — scores are the average of both: Model B introduces unsafe concurrent use of WaitGroup.Add and Wait, allowing Wait to observe zero and close results before jobs are registered, and it also violates the corrected-code-only requirement. Model A correctly tracks worker lifetimes, submits jobs concurrently with result collection, closes results only after all workers exit, and returns only code.)
Strict JSON extraction
Extract every meeting from this text into a JSON array of {"title","day","start24","durationMinutes"}. Use 24-hour "HH:MM" for start24, integer minutes for duration, and title-case titles. Return ONLY valid JSON, no commentary. Text: "Standup is daily at 9am for a quarter hour. The design review runs thursday from 2:30-4pm. Payroll sync — first monday, 11:00 to noon."
Model A provides the correct extraction as valid JSON with no extra text. Model B extracts the meetings correctly, but the Markdown code fence violates the requirement to return only valid JSON. (Second judge pass, order swapped — scores are the average of both: Model A returns valid JSON exactly as requested, with all meetings and normalized fields correct. Model B extracts the same correct data but wraps the array in Markdown code fences, violating the requirement to return only valid JSON.)
Faithful rewrite
Rewrite this blunt internal note as a warm, professional Slack message to the whole engineering team (60–90 words), keeping every fact intact and adding no new commitments. Return only the message. Note: "Deploy is frozen until the memory leak in the image service is fixed. Priya is on it. Do not merge to main. ETA tomorrow noon."
Model A preserves all facts, meets the 60–90-word limit, and adds no new commitment, though some phrasing is overly effusive. Model B is more natural for Slack, but it softens “tomorrow noon” to “around noon” and adds the new commitment “I’ll keep you posted.” (Second judge pass, order swapped — scores are the average of both: Model A preserves the freeze, memory-leak issue, Priya’s ownership, no-merge instruction, and noon ETA without adding a commitment. Model B is more natural and polished, but “I’ll keep you posted” adds a new commitment and “around noon” slightly alters the stated ETA.)
algo-lru-cache
Implement an LRU cache in Python 3. Requirements: - Class name: LRUCache - Methods: - __init__(self, capacity: int) - get(self, key: int) -> int # return -1 if missing - put(self, key: int, value: int) -> None - Both get and put must be O(1) average time. - When capacity is exceeded, evict the least recently used key. - A successful get makes the key most recently used. - Updating an existing key must also make it most recently used. - Use only the Python standard library. Return code only.
Model A and Model B are functionally identical, correctly use OrderedDict for average O(1) operations, and satisfy all behavioral requirements; both only slightly miss the strict “code only” instruction by including Markdown fences. (Second judge pass, order swapped — scores are the average of both: Model B and Model A are functionally identical, correctly implement all required LRU behavior in average O(1) time, and use only the standard library; both slightly miss the code-only instruction by including Markdown fences.)
Precise proofreading
Correct ONLY the grammar, spelling, and punctuation errors in the sentence below without changing wording, tone, or meaning, then on a second line list each change as 'was -> now'. Sentence: "Its been a long time since we seen there new office, and me and him was suprised how quite it was."
Model A is better because it limits changes more closely to the requested grammar, spelling, and punctuation corrections while preserving the original wording better. Model B introduces an unnecessary wording change from "me and him" to "he and I," which violates the instruction not to change wording beyond needed corrections. (Second judge pass, order swapped — scores are the average of both: Model A is better because it corrects the grammatical errors while preserving the original wording more closely, whereas Model B unnecessarily changes "me and him" to "he and I," violating the instruction not to change wording beyond grammar, spelling, and punctuation fixes. Model A also follows the required change-list format more closely overall, though both models incorrectly introduce "we've seen" instead of the more minimal correction "we saw.")
translate-ja-support-reply
Translate this customer-support reply into natural Japanese. Use a polite, reassuring tone suitable for business email. Keep it to 2 sentences and under 80 Japanese characters total. Source: "Thanks for flagging this. I found the duplicate charge, voided it, and your bank should reflect the reversal within 3–5 business days."
Model B is clearer and more natural because it specifies that the refund will be reflected, whereas Model A’s 「銀行へ反映されます」 is slightly awkward and ambiguous. Both satisfy the two-sentence and character-limit requirements, though Model B’s 「返金」 is a minor simplification of the original reversal. (Second judge pass, order swapped — scores are the average of both: Model B is more natural and clearly conveys that the duplicate charge was confirmed, canceled, and will be reversed within the stated period. Model A follows the format and tone requirements, but「銀行へ反映されます」is less idiomatic and slightly unclear.)
reasoning-shift-payroll
A clinic is checking one nurse's pay for a single day. Rules: - Base rate: $32/hour. - Overtime: any time above 8.0 worked hours that day is paid at 1.5x base. - Night premium: +$4/hour for any time worked between 22:00 and 06:00. - Unpaid meal break: subtract exactly 30 minutes, taken from the middle of the longest continuous work segment. - Times are local and on the same date span unless crossing midnight. Shifts worked: 1) 17:40–21:10 2) 21:35–02:25 3) 03:05–07:20 Question: What is the nurse's total pay for the day, in dollars, rounded to the nearest cent? Show the calculation clearly.
Both Model A and Model B are correct and follow the instructions, arriving at $479.33 with the meal break, overtime, and night premium handled properly. Model B is slightly better because its step-by-step breakdown makes the midnight crossing, centered break, and pay components easier to verify. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both reach the correct total and clearly account for the unpaid break, overtime, and night premium. Model A is slightly better because it uses exact minute-based arithmetic and presents the calculation more concisely without intermediate decimal approximations.)
sql-refund-rate-by-week
Write ONE PostgreSQL query only. Schema: customers(id INT PRIMARY KEY, region TEXT) orders(id INT PRIMARY KEY, customer_id INT, placed_at TIMESTAMP, channel TEXT) refunds(id INT PRIMARY KEY, order_id INT, refund_amount NUMERIC(10,2), created_at TIMESTAMP) Question: For each region, find the 2024 ISO week with the highest refund rate, where refund rate = refunded orders / total orders in that region-week. Count an order as refunded if it has at least one row in refunds within 30 days after the order's placed_at. Only include region-weeks with at least 8 total orders. Return: region, iso_week (format IYYY-IW), total_orders, refunded_orders, refund_rate. Break ties by choosing the earlier week. Order results by region.
Model A and Model B both correctly use ISO year 2024, count each order at most once as refunded, enforce the eight-order minimum, select the highest rate with the earlier week as the tie-breaker, and order the final rows by region. Their DISTINCT ON and ROW_NUMBER approaches are equivalently valid in PostgreSQL. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both correctly filter by ISO year 2024, count refunds with an existence check within the 30-day window, enforce the eight-order minimum, choose the highest rate with the earlier ISO week as the tie-breaker, and order by region. Model B uses ROW_NUMBER while Model A uses DISTINCT ON, but both approaches are valid PostgreSQL solutions.)
Concurrency bug fix
This TypeScript function is meant to memoize an async loader but has a race: concurrent callers can each trigger the underlying fetch. Fix it so the fetch runs at most once per key, and a rejected fetch does NOT poison the cache (a later call must retry). Return ONLY the corrected function. ```ts const cache = new Map<string, Promise<string>>(); async function load(key: string, fetcher: (k: string) => Promise<string>) { if (cache.has(key)) return cache.get(key)!; const value = await fetcher(key); cache.set(key, Promise.resolve(value)); return value; } ```
Model B is slightly better because it returns the cached promise directly without Model A’s unnecessary async wrapper; Model A is still functionally correct. Model A and Model B each slightly miss the instruction to return only the function by also repeating the cache declaration. (Second judge pass, order swapped — scores are the average of both: Both models cache the in-flight promise before other callers can initiate another fetch and delete the entry on rejection. Model A is marginally better because it preserves the original async function semantics, including converting a synchronous fetcher throw into a rejected promise, while Model B could throw synchronously.)
Summarize dense passage
Summarize the passage below in exactly three bullet points, each one sentence, capturing the mechanism, the tradeoff, and the caveat — no jargon a non-specialist couldn't follow. Return only the bullets. Passage: "Speculative decoding pairs a small draft model with a large target model: the draft proposes several tokens, the target verifies them in one pass, and accepted tokens are kept while the first rejection resets to the target's own choice. This can cut latency substantially when acceptance is high, but the draft model's compute is pure overhead when acceptance is low, and gains evaporate on adversarial or highly novel inputs where the draft and target disagree often."
Model A describes the first mismatch more precisely and cleanly separates the mechanism, tradeoff, and caveat. Model B is equally accessible and well structured, but “correct guesses” and “large, accurate model” slightly oversimplify the relationship between the models. (Second judge pass, order swapped — scores are the average of both: Model A more precisely describes acceptance as agreement with the large model and explicitly states that the large model supplies its own word at the first mismatch, while Model B slightly blurs this by calling guesses objectively “correct” or “wrong.”)
Find the contradiction
The following spec contains exactly one internal contradiction. Quote the two conflicting sentences verbatim and explain the conflict in one sentence. Do not fix it. Spec: "Free accounts may create up to three projects. Every account, regardless of tier, may archive unlimited projects. Archiving a project does not count against the project limit. Free accounts are limited to three projects total, including archived ones."
Model A and Model B both quote the exact conflicting sentences and accurately explain the contradiction in one sentence without attempting to fix it. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both identify the exact conflicting sentences and explain the contradiction correctly in one sentence. Model B is slightly better because its numbered presentation and explicit reference to the free tier make the response marginally clearer.)
Matchup powered by OpenRouter.