# Audit notes — population run 2026-08-17

Scope of human audit (per spec + capped deviation documented in methodology): ALL 22 A losses, ALL ties +
not-same-story pairs, 10-pair seeded random win sample (seed 424242), top-10 gaps per direction,
all 22 VentureBeat pairs, B-loss skim (headers + type distribution + 3 full blocks: company_announcement,
named_competitor, social). Verdicts overturned: **0**. Judging quality high: rationales track the actual
judged texts; per-pass A/B position swaps do not flip explanations (spot-checked side mapping on
pA-8de3d8a72611, pA-1ad9ed5cacef, pA-d8d338c17348 — all consistent); adjudication triggers fired where
expected; ~7 transient judge JSON-parse failures all retried clean.

## A losses (22) — all genuine

- **VentureBeat (17 losses, spread across 17 RW articles / 12 clusters — not concentrated):** VB's
  enterprise long-form carries more granular tech detail (benchmark/pricing tables, architecture sections,
  exec interview quotes) than RW's tighter same-story pieces, chiefly on model/product-release stories.
  Judges frequently still praise RW's distinct strengths (corrective framing, synthesis, sourcing) while
  scoring VB higher on depth. VB text was capped @8000ch — uncapped it would be longer; cap favors RW,
  so the losses are conservative (i.e., extra-valid). RW rw_chars median 6053 vs VB 8000 (capped).
- **WIRED (4 losses):** legit scoops — exclusive founder interviews (Discovery Loop ×2, ChatTJB) and an
  original GPS-jamming investigation that RW itself credited.
- **TechCrunch (1 loss, Apple spyware, −0.333, adjudicated):** plausible either way; TC text polluted but
  had real Apple-direct facts (110 countries).

## A wins sample (10, seed 424242) + top gaps

- Legit clean wins: Verge/Telegram (Verge mostly republished Durov's post; RW added verification),
  Metabase-vs-TC breach, GPT-5.6-Cyber-vs-TC, Archer/Wisk (+1.25, close and fair).
- **Bloomberg wins are text-availability artifacts.** Sampled Bloomberg "articles" are paywall stubs
  (pA-68f8a47040a1 Astra 2662ch ≈2 paragraphs; pA-f03e6430633c Qwen 2525ch) or video-clip pages that are
  mostly player chrome with 1–2 sentences of prose (pA-c2e56109ebd0 SpaceX 'Identity Crisis';
  pA-af2c4d05510c Decart; top-gap pA-8de3d8a72611 Muse Glimmer +7.375 is a 3-sentence video blurb wrapped
  in ~3.4k chars of sidebar links). Judges correctly describe them as "short summary/video item" — the
  comparison is honest about the text it saw, but it is NOT a like-for-like article comparison.
  → findings.md must report Bloomberg 66-0-0 (+5.40 mean gap) as a feed-truncation artifact, excluded from
  headline quality claims. Only 8/66 Bloomberg pairs ≥5000ch; comp median 3894ch.
- **Validity-floor weakness (disclose):** the ≥2500ch pair-validity floor was measured on RAW extracted
  chars; boilerplate-heavy pages (video chrome, consent banners, related-links rails) pass the floor with
  <1000ch of prose. Quantified in diag_prose_density.csv; future runs should gate on prose-density.
- **Nav-cruft budget tax:** TC/WIRED bodies carry Cloudflare "Checking your Browser…" stubs + share/nav
  cruft; pollution eats the competitor's 8000–12000ch judge budget → biases toward RW. Competitors still
  won all 22 losses despite the tax (strengthens loss validity); disclosed as a pro-RW bias in wins.

## A ties / not-same-story

- 1 genuine tie (GPT-5.6-Cyber vs VB — complementary pieces).
- 3 judged NOT_SAME_STORY → invalid (2 Bloomberg + 1 Axios, all short texts): cluster-QC catches, correct.

## B losses (52 of 512 graded) — expected pattern, no verdict problems

Type distribution: social 18, company_announcement 16, named_competitor 10, other_news 7, research 1.
Scoop attribution in losses: OTHER 42, unclear 9, RW 1. Pattern is coherent: **RW loses mostly to rich
primary documents** — vendor announcements (e.g., Cloudflare Radar Researcher blog: architecture, model
names, Iran outage metrics), first-person researcher disclosures (tl;dv Firestore exposure: subdomain
names, temporal stats — judged text starts with garbage glyph columns from the blog's ASCII art, real
content follows), and genuine competitor exclusives (Axios Astra "Exclusive" with on-the-record Michael
Dalton quotes; its stored title "Just a moment..." is a scrape-time Cloudflare-interstitial artifact,
judged text contains the real story after consent boilerplate). Judges consistently credit RW for
sourcing/context/restraint while scoring the primary doc higher on originality/depth. That is the honest
result for a synthesis wire and is a finding, not a flaw.

## VentureBeat wins (4) + tie (1)

Where RW beats VB it is explainable: e.g., pA-d8d338c17348 (+2.0 clean) — RW added corrective framing
(Opus 4.6 vs 4.8 caveat, Opus 5 leaderboard reversal), cost economics, and researcher quotes on top of
the paper; VB wrote the explainer. Three other RW-verdicts landed via adjudication on near-zero gaps
(0, +0.166, +0.667) — VB matchup is genuinely close outside its 17 wins.

## Disclosure items for findings.md

1. Bloomberg 66-0-0 = feed/paywall truncation artifact (8/66 full-length; video-page stubs pass char floor).
2. Text pollution (TC/WIRED Cloudflare stubs, Bloomberg player chrome, consent banners) biases judge
   budget toward RW in wins; losses survived the bias.
3. Pair-validity floor on raw chars, not prose chars — quantified in diag_prose_density.csv; future-run fix.
4. VB is the only losing matchup and the losses are genuine long-form depth, spread across 12 clusters.
5. B gradeable pool = 512 of 3,252 cited links (matched in DB + ≥800ch) — coverage, not a census.
