Cloudflare says smaller hash rings reclaimed more than 100TB of RAM

Why it matters

Cloudflare's claimed 100TB gain shows how mature infrastructure companies can create meaningful capacity by revisiting old defaults instead of buying more hardware.

Cloudflare engineers Kevin Guthrie, Mariia Iurchenko, Zaidoon Abd Al Hadi and Ivan Babrou say they reclaimed more than 100TB of RAM across Cloudflare's network by shrinking a consistent-hashing data structure and generating about 90% fewer virtual hashes per server.

The four engineers detailed the work in a September 18th engineering post. Babrou started the investigation after finding that Pingora Backend Router, an internal load-balancing service known as PBR, was consuming considerably more memory than expected. Some processes used as much as 6GB for the affected structures, according to Cloudflare.

The fix follows a technical instinct that runs back to Cloudflare's founding team. Matthew Prince and Lee Holloway began with Project Honey Pot, which tracked how spammers harvested email addresses, before joining Michelle Zatlyn to build Cloudflare in 2009. The founders became preoccupied with removing latency after early investors warned that putting Cloudflare between a website and its visitors could slow the web down. Holloway designed the early platform and recruited its first engineering group, establishing an architecture that Cloudflare's founders later described as fundamental to the business.

Nearly sixteen years after Cloudflare's public launch, that discipline has become a search for waste measured across thousands of servers. A few bytes in one structure can become a fleet-wide capacity problem.

The hash ring had become a memory warehouse

PBR uses consistent hashing to route cacheable requests by URL to the server holding the relevant file. Stable routing lets Cloudflare keep one copy of a cached file in each data center and locate it again without moving large portions of the cache whenever servers enter or leave the pool.

Each server is represented by points on a hash ring. More points generally produce a more even distribution of requests. Pingora inherited a baseline of 160 points per server, then multiplied that number according to factors such as available storage. Cloudflare also maintains separate rings for combinations of compliance requirements and caching features because every server cannot handle every request.

Cloudflare says those feature combinations produced dozens of separate rings containing an enormous number of hashes. In the worked example Cloudflare used to explain the problem, a server weight of 625 took the server from the 160-point baseline to 100,000 hashes. The post describes thousands of servers and dozens of rings without providing exact fleet-wide counts of servers, rings or entries, so the more-than-100TB result remains an internal Cloudflare measurement rather than a reproducible fleet-wide benchmark.

Abd Al Hadi found the first reduction in the representation of each point. The existing Rust structure stored a 32-bit hash and a 32-bit index identifying the server. PBR did not need an index capable of addressing roughly 4.3 billion servers. A 16-bit index, which can represent about 65,000 values, was enough for Cloudflare's stated requirements.

Rust's alignment rules meant that changing the index field alone still left an 8-byte structure. The engineers instead stored the hash and index in a raw 6-byte array with accessor methods. That cut memory use for the consistent-hashing entries by 25%.

Guthrie's math made fewer hashes defensible

The larger gain came from questioning how many points each server actually needed.

Guthrie derived an exact expression for the coefficient of variation across a consistent-hashing ring with multiple points per server. Cloudflare's analysis showed diminishing returns as the point count rose. In Cloudflare's example, the final 90,000 hashes between 10,000 and 100,000 produced a 0.7% reduction in expected distribution error.

High point counts also create more collisions in the 32-bit hash space. Cloudflare's simulations showed error beginning to rise between 10,000 and 100,000 hashes per server in a modeled data center with 2,048 servers. Based on that analysis, the engineers reduced the number of generated hashes per server by about 90%, which Cloudflare says produced no appreciable loss in load distribution.

The work fits Guthrie's recent focus inside Cloudflare. In a 2024 performance post, Guthrie wrote that pingora-origin consumed the equivalent of 40,000 saturated CPU cores and that an optimized function's share of sampled CPU usage fell from 1.71% to 0.34%, implying savings of about 548 equivalent saturated cores. The Pingora repository's README says the Rust framework has served more than 40 million internet requests per second for more than a few years, where modest per-request or per-process savings quickly become material capacity.

A careful rollout avoided an origin traffic spike

Replacing a hash ring changes which server receives a cacheable request. Cloudflare said an immediate network-wide switch would have invalidated cached content and sent a surge of requests to customers' origin servers.

PBR temporarily carried the old and new rings in memory together. Cloudflare's migration framework made a stable, request-by-request choice between them, allowing engineers to reverse course without redeploying PBR. The group began in small validation locations and expanded through progressively larger sets of data centers.

The rollout separated the percentage of traffic using the new ring from the locations where cache entries were allowed to move. Cloudflare says its data-center-scoped controls kept the rollout's blast radius small rather than spreading cache churn across all locations at once. Engineers monitored backend-selection traces, process memory, startup time, connection errors, cache behavior and origin traffic before removing the old ring.

A chart in Cloudflare's engineering post compares PBR memory during the week of the change with data from a few weeks before. Cloudflare says the difference reached 100TB after the large rings were decommissioned. Cloudflare does not translate that reclaimed capacity into lower hardware spending. The memory could instead support larger caches, new services or traffic growth on the same machines.

Cloudflare has been adding work to those machines at a steady pace. RuntimeWire reported this month on its automatic post-quantum key exchange across 45 billion daily origin connections and its OpenAI-powered vulnerability defense tied to production traffic. Each additional network service competes for finite CPU and memory on the same global footprint, making internal efficiency part of Cloudflare's product economics.

The second 100TB claim in a month

The PBR reduction follows a separate DNS cache optimization published on August 27th. That team cut the benchmarked per-entry footprint of Cloudflare's 1.1.1.1 cache from 953 bytes to 420 bytes and reported approximately 100TB less working-set memory after a rollout completed across all services on July 6th, 2026.

Together, the two teams claim roughly 200TB of reclaimed memory, although they measured different systems and neither figure has been independently audited. The DNS group said it planned to reinvest its capacity in a larger cache. Cloudflare has not specified how PBR's reclaimed memory will be allocated.

Developers can inspect and test the latest work because Cloudflare placed it behind a v2 feature in the open-source pingora-ketama crate. Cloudflare's engineering post says the new implementation includes the compact 6-byte storage format, a faster sorting method and controls for scaling the base number of hashes per node. The original ring remains available, allowing both versions to run together during migrations.

The result came from Babrou spotting an oversized service, Abd Al Hadi questioning four bytes in a Rust structure, and Guthrie and Iurchenko helping turn the observation into an algorithmic and operational change. Cloudflare's 100TB estimate attracts the headline. The repeatable part is smaller: inspect assumptions that became defaults, calculate what they still buy, and give engineers a rollback path before changing them across the fleet.