Head to head: GLM 5.2 vs OpenAI: GPT-5.6 Sol

The scoreboard says it plainly: OpenAI: GPT-5.6 Sol wins 113.0 to 93.5, with 99% confidence. It took 9 task wins to GLM 5.2’s 1, with 2 ties. That’s not a vibes-based edge or a judge’s whim; it’s a decisive result spread across coding, extraction, SQL, classification, and reasoning.

What separates Sol here is not flashy creativity but discipline. It repeatedly did the boring, important things right: returning exactly one JSON object when asked, avoiding Markdown fences in code-only tasks, keeping justifications within strict word limits, and producing cleaner normalized outputs. That showed up in lease extraction, strict JSON extraction, support-ticket classification, nuanced classification, the LRU cache task, and the Python log redactor. GLM was often close on substance, but too often lost on compliance details that matter in production.

The biggest quality gap came on tasks where correctness and query logic actually bite. In the SQL window query, GLM simply answered the wrong problem by computing a max-minus-min difference instead of the second-highest order. In reasoning-shift-coverage, its analysis was mostly sound but the final answer was truncated, which is still a miss. Sol, by contrast, was consistently complete and structurally correct. Even where both models were strong, Sol tended to be the one that was tighter, cleaner, and less error-prone.

GLM’s lone outright win — the European Spanish localization toast — is real and deserved. Its phrasing was more natural, more concise, and better suited to the mobile UI constraint. And the two ties show GLM is not outclassed everywhere: on the weekly refunds SQL and ops-email proofreading, it was competitive enough that judge preference split. But those are islands in a match otherwise controlled by Sol.

Final call: GPT-5.6 Sol is the clear winner. GLM 5.2 looks capable, but Sol was markedly better at turning correct intent into correct, usable outputs — and in this test, that difference was decisive.

How they were tested

We ran 12 fresh text tasks, generated on the fly for this matchup so neither model could prepare in advance, and had gpt-5.4 score each one. To cancel position bias, every task was judged twice — once in each presentation order — and every number reported here, including the headline totals, is the average of both passes. GLM 5.2 scored 93.5 to OpenAI: GPT-5.6 Sol's 113.0.

1. extract-lease-details

Extract the requested fields from the messy text below. Return exactly one JSON object with keys: tenant_name, property_address, lease_start, lease_end, monthly_rent_usd, security_deposit_usd, parking_spaces, pets_allowed, notice_days, utilities_included. Messy text: "Draft notes from call with Juniper Ridge Properties re: new lease. Tenant will be Malik Rowan (spell checked). Unit is 4C at 1187 Waverly Avenue, Spokane, WA 99204. Term discussed: starts 2025-09-15 and runs through 2026-09-14. Base rent is $1,845/mo; deposit equals $1,200. They said one reserved parking stall is included behind the building. Pets: cats okay, no dogs — mark as pets allowed yes. Tenant must give 45 days notice before moving out. Utilities covered by landlord are water, trash, and sewer; tenant handles electric + internet. Ignore old quote for 1189 Waverly and the earlier rent figure of $1,795 from April."

Winner: OpenAI: GPT-5.6 Sol — Model B is slightly better because it cleanly returns exactly one JSON object and formats the property address more naturally as a standard address string. Model A is also highly accurate, but its address phrasing ("Unit 4C at ...") is a bit less clean for extraction purposes. (Second judge pass, order swapped — scores are the average of both: Model B is fully correct and follows the instruction to return exactly one JSON object. Model A extracts the same details correctly, but it wraps the JSON in a Markdown code block, which violates the formatting requirement.)

2. python-log-redactor

In Python, write a function redact_log(line: str) -> str for application logs. Requirements: - Replace every email address with [EMAIL]. - Replace every IPv4 address with [IP]. - Replace every occurrence of token= followed by one or more non-space characters with token=[REDACTED]. - Preserve all other text exactly. - Use only the Python standard library. - Return code only, no explanation. Example: Input: "login failed for nora.ives@elmbyte.io from 203.44.18.9 token=abC99_x at 09:14" Output: "login failed for [EMAIL] from [IP] token=[REDACTED] at 09:14"

Winner: OpenAI: GPT-5.6 Sol — Model B is better because it returns valid Python code only and uses more precise regexes, especially for IPv4 addresses, avoiding false matches like invalid octets that Model A would redact. Model A is still solid and follows the task, but its IP pattern is overly permissive compared with Model B's stricter implementation. (Second judge pass, order swapped — scores are the average of both: Model B better satisfies the prompt with code-only output and more careful regexes, especially for valid IPv4 matching and cleaner email boundaries while preserving other text. Model A includes Markdown fences and a comment despite the code-only instruction, and its IPv4 pattern overmatches invalid addresses like 999.999.999.999.)

3. LRU cache

Implement a class LRUCache<K, V> in TypeScript with a fixed capacity set in the constructor, and O(1) get(key): V | undefined and set(key, value): void. Accessing or updating a key must mark it most-recently-used; inserting beyond capacity must evict the least-recently-used entry. Return ONLY the class, no prose.

Winner: OpenAI: GPT-5.6 Sol — Model B is slightly better because it returns only the class as requested and handles non-integer or negative capacities more defensively while preserving the required O(1) behavior. Model A is also largely correct, but it wraps the class in a code fence despite the prompt explicitly asking for only the class. (Second judge pass, order swapped — scores are the average of both: Model B fully satisfies the prompt with a correct O(1) Map-based LRU implementation and returns only the class as requested. Model A is also functionally solid, but it violates the output constraint by including Markdown code fences and is slightly less robust in its typing and constructor handling.)

4. sql-weekly-refunds

You are given a retail analytics schema. Table orders( order_id INT, customer_id INT, store_id INT, ordered_at TIMESTAMP, status TEXT, total_amount NUMERIC ) Table refunds( refund_id INT, order_id INT, refunded_at TIMESTAMP, refund_amount NUMERIC, reason TEXT ) Write ONE SQL query that returns, for each store_id and calendar week (starting Monday) in Q1 2025, these columns: - week_start - store_id - completed_orders - refunded_orders - refund_rate_pct - avg_days_to_refund Rules: - Count only orders with status = 'completed'. - A refunded order is a completed order that has at least one refund record within 30 days after ordered_at. - If an order has multiple refund rows, count it once as refunded and use the earliest refunded_at to compute days-to-refund. - Only include weeks where the store had at least 20 completed orders. - Q1 2025 means ordered_at from 2025-01-01 inclusive to 2025-04-01 exclusive. - refund_rate_pct = 100.0 * refunded_orders / completed_orders, rounded to 2 decimals. - avg_days_to_refund should average only refunded orders, rounded to 2 decimals. Return a single correct SQL query.

Winner: Tie — Model A is slightly better because it cleanly isolates Q1 completed orders, deduplicates refunds via the earliest qualifying refund, and matches the 30-days-after requirement more precisely. Model B is also strong, but it uses a <= 30 days boundary where Model A uses a stricter within-30-days interpretation, making Model A the safer choice for correctness. (Second judge pass, order swapped — scores are the average of both: Model B is slightly better because it matches the 30-day window rule more faithfully by including refunds exactly 30 days after ordered_at, while Model A uses a strict less-than cutoff and could exclude boundary-case refunds. Both otherwise correctly count completed orders by store and Monday-based calendar week in Q1 2025, deduplicate multiple refunds via the earliest refunded_at, and apply the 20-order threshold.)

5. Faithful rewrite

Rewrite this blunt internal note as a warm, professional Slack message to the whole engineering team (60–90 words), keeping every fact intact and adding no new commitments. Return only the message. Note: "Deploy is frozen until the memory leak in the image service is fixed. Priya is on it. Do not merge to main. ETA tomorrow noon."

Winner: OpenAI: GPT-5.6 Sol — Model B is more faithful to the original note while still sounding warm and professional, and it avoids adding an unnecessary invitation for questions. Model A is clear, but it introduces extra language that edges closer to a new commitment and is slightly less precise. (Second judge pass, order swapped — scores are the average of both: Model B is better because it preserves all facts without adding a new invitation or implied commitment, while still sounding warm and professional. Model A is also strong, but "Let me know if you have any questions!" adds content not present in the original note, which slightly reduces faithfulness to the prompt.)

6. proofread-ops-email

Edit the passage for grammar, punctuation, clarity, and concise professional tone without changing meaning. Then provide 3 brief bullet points noting the main changes. Passage: "Hi team, yesterdays maintenance window went mostly as planned however there was two unexpected failovers in the Dublin cluster. Because the cache wasn't warmed, API latency spiked to 1.8 seconds for around 14 minutes, which is higher then our SLO. We have already identify the misconfigured health-check and a patch is being tested right now. Customer support has been notified, but there isn't any reports of data loss and no action is needed from clients at this time. We'll send another update by 16:30 UTC, unless testing uncover additional issues."

Winner: Tie — Model A better follows the requested format by clearly labeling the edited passage and providing three specific, substantive bullet points about the main changes. Model B’s edit is strong, but its bullets are more generic, and capitalizing "Customer Support" is a less neutral stylistic choice than Model A’s phrasing. (Second judge pass, order swapped — scores are the average of both: Model B delivers a cleaner, more concise professional edit while preserving meaning and providing the requested three brief bullet points. Model A is also strong, but its formatting is less concise, and its change notes are more verbose and include a slightly less faithful shift from "there isn't any reports" to "There are no reports," which subtly changes the time framing.)

7. Strict JSON extraction

Extract every meeting from this text into a JSON array of {"title","day","start24","durationMinutes"}. Use 24-hour "HH:MM" for start24, integer minutes for duration, and title-case titles. Return ONLY valid JSON, no commentary. Text: "Standup is daily at 9am for a quarter hour. The design review runs thursday from 2:30-4pm. Payroll sync — first monday, 11:00 to noon."

Winner: OpenAI: GPT-5.6 Sol — Model B is better because it correctly normalizes all three meeting titles into title case without adding an extra article. Model A is otherwise accurate, but "The Design Review" does not faithfully extract the title from the source text and is less compliant with the formatting instruction. (Second judge pass, order swapped — scores are the average of both: Model B is better because it correctly extracts all three meetings and uses clean title-case titles that match the requested normalization. Model A is also mostly correct, but "The Design Review" is less well normalized than Model B's "Design Review" for this extraction task.)

8. classify-support-tickets

Classify each support ticket into exactly one category: Billing, Shipping, Account Access, Product Defect, or Feature Request. Give a brief justification (max 8 words). Return exactly one JSON array of objects with keys id, category, justification. Items: - id: T01 | "My card was charged twice after I upgraded to the Pine tier." - id: T02 | "The reset-password link expires instantly on both Chrome and Safari." - id: T03 | "Package says delivered to Dock 3, but our office only has Dock B and nothing arrived." - id: T04 | "The blender starts, then emits a burning smell and stops after 20 seconds." - id: T05 | "Please add a way to export audit logs to CSV every Friday." - id: T06 | "I can't sign in because the app keeps asking for a 6-digit code I never receive." - id: T07 | "The invoice still shows 18 seats even though we removed 4 last month."

Winner: OpenAI: GPT-5.6 Sol — Model A and Model B both classify all tickets correctly and return valid JSON, but Model B better follows the justification limit because every justification stays within 8 words while Model A exceeds it on multiple entries. Model B’s phrasing is also slightly tighter and clearer. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both classify all tickets correctly and return valid JSON, but Model B’s justifications are consistently more precise to the ticket details. Model A is slightly weaker because one justification exceeds the 8-word limit and a few are a bit more generic.)

9. SQL window query

Given Postgres tables orders(id, customer_id, created_at, total_cents) and customers(id, name), write ONE SQL query returning each customer's name and their SECOND-largest order total (in dollars, 2 decimals), excluding customers with fewer than two orders. Order by that value descending. Return ONLY the SQL.

Winner: OpenAI: GPT-5.6 Sol — Model B correctly uses a window function to select each customer's second-highest order and formats it in dollars, while Model A incorrectly computes the difference between the maximum and minimum order totals rather than the second-largest order. Model B also better matches the requested query shape and ordering, though it may treat tied top totals as distinct second rows depending on interpretation. (Second judge pass, order swapped — scores are the average of both: Model B correctly uses a window function to rank each customer's orders and returns the row with rank 2, excluding customers with fewer than two orders implicitly and ordering by that value descending. Model A is incorrect because MAX(total_cents) - MIN(total_cents) is not the second-largest order total, and it also violates the prompt's output constraint by wrapping the SQL in a code fence.)

10. reasoning-shift-coverage

A clinic manager is checking whether tomorrow's front-desk coverage meets policy. Policy: - Front desk must be staffed by at least 2 people from 08:00-12:00, at least 3 people from 12:00-16:00, and at least 2 people from 16:00-18:00. - Count only staff marked "front-desk trained". - Unpaid lunch breaks do NOT count toward coverage. Schedule for tomorrow: - Alma — front-desk trained — 08:00-16:00, lunch 12:00-12:30 - Ben — front-desk trained — 08:00-12:00, no lunch - Cora — not front-desk trained — 09:00-17:00, lunch 13:00-13:30 - Dev — front-desk trained — 11:30-18:00, lunch 15:00-15:30 - Esme — front-desk trained — 12:00-18:00, lunch 14:00-14:30 - Farid — front-desk trained — 10:00-14:00, no lunch Question: Does the schedule satisfy policy for the full day? If not, list every exact time interval that is understaffed and by how many people. Show your reasoning.

Winner: OpenAI: GPT-5.6 Sol — Model B is fully correct, clearly identifies both exact understaffed intervals and shortages, and directly answers the question with concise reasoning. Model A’s analysis is mostly correct, but its final answer is visibly truncated, so it does not fully complete the requested response. (Second judge pass, order swapped — scores are the average of both: Model B is fully correct, identifies both exact understaffed intervals and shortages, and clearly shows the reasoning. Model A’s analysis table is correct, but its final answer is cut off and therefore does not fully satisfy the instruction to list every exact interval and explain the result.)

11. Nuanced classification

Classify each review's sentiment as "positive", "negative", or "mixed", and give a 6-word-max reason. Return ONLY a JSON array of {"text","label","reason"} in input order. Reviews: ["Fast shipping but the fabric feels cheap.", "Absolutely love it, wearing it daily!", "It broke after a week. Refund was quick and painless though."]

Winner: OpenAI: GPT-5.6 Sol — Model A and Model B both classify all three reviews correctly, but Model B follows the 6-word-max reason constraint exactly on every item. Model A exceeds the limit on its second reason, so Model B is better overall. (Second judge pass, order swapped — scores are the average of both: Model B and Model A both classify all three reviews correctly and return valid JSON in order. Model B is slightly better because all its reasons stay within the 6-word limit, while Model A exceeds that limit in the first and second entries.)

12. Localization with tone

Translate this app onboarding line into natural, friendly European Spanish suitable for a mobile toast (keep it under 60 characters, no exclamation marks): "You're all set — your first backup starts tonight." Return only the translation, then the character count in parentheses.

Winner: GLM 5.2 — Model A is more natural and concise for a mobile toast, stays well under 60 characters, and accurately conveys that the first backup begins tonight. Model B is understandable but less natural in European Spanish, more verbose, and shifts the meaning slightly to "will be tonight" rather than "starts tonight." (Second judge pass, order swapped — scores are the average of both: Model A is more natural and concise for a mobile toast in European Spanish, and it stays comfortably under the limit with an accurate character count. Model B is understandable, but "copia de seguridad" feels heavier and less toast-like, and its character count appears incorrect.)

See every prompt and the full side-by-side outputs in the interactive Head-to-Head.