Naive byte-range caching can result in the wastage of gigabytes of origin bandwidth due to small requests. To address this, the architecture within Pingora was rebuilt to automatically coordinate partial fetches, which consequently halved the p99 latency for R2.
Range requests are the mechanism by which substantial files are delivered. Caching these naively frequently causes wasted disk operations and network bandwidth when operating a Content Delivery Network (CDN) at scale.
The previous method relied on static zone configuration for range-request caching. This involved mapping each client request to fixed-size ranges of the file, with the range size and file extensions configured per zone. This created a compromise: small ranges led to numerous small origin requests and memory pressure on Cloudflare from coalescing them in-memory, while large ranges caused Cloudflare to download substantially more data than the client actually needed.
This design also introduced operational difficulties. Concurrent requests falling within the same range could block each other, regardless of the distance between the clients' requested ranges. Furthermore, the nginx-based cache could not integrate range caching with concurrent streaming, meaning each range had to be fully downloaded before any client could read it.
The transition to a Pingora-based caching server provided an opportunity to resolve these implementation issues that previously necessitated a difficult-to-use setting for customers. Pingora inherently stores every cached file on disk in fixed-size chunks, which eliminates the need for a configurable range size.
With this new approach, Cloudflare checks which bytes are already cached for a range request. It serves those directly, and for missing bytes, it fetches origin ranges aligned to the cache's fixed-sized-chunk boundaries. The downstream response still only contains the bytes the client requested, even if the fetch required retrieving an entire aligned chunk.
A significant advancement involves coordinating cached and in-flight data. The edge component now acts as a coordinator, reading what it can from disk or memory, fetching missing coverage from the origin using aligned ranges, writing the new bytes, and combining all pieces for the client response. If an existing download covers another request's bytes, the second request can attach to that download, sharing work during traffic bursts for popular large files.
Before rolling out Origin Range Requests broadly, existing users of the older feature were migrated to the new implementation, beginning with the workloads causing the most strain. Large range sizes previously meant a request that would not fill a floppy disk could trigger the download of "a few DVDs' worth of data from the origin and move it across several data centers, sometimes after the client had already given up and gone elsewhere."
Following the migration of R2 to the new system, p99 time to first byte decreased by approximately half across R2's storage regions for both cache hits and misses. Additionally, a recent enhancement allowing the coordinator to combine disk-resident bytes with upstream fetched bytes in a single response reduced tiered cache traffic for R2 and similar workloads by about 6% week over week, equating to "petabytes less data moving between our data centers every day."
This feature, now available as a self-service option on all plans, removes the need for customers to tune a range size or contact support. Customers enable it using a Cache Rule, allowing Cloudflare to manage the chunking and cache coordination automatically for traffic where clients naturally request parts of a large cacheable object, such as "Progressive and seekable video such as MP4" or "Resumable large downloads."
This development reflects a broader industry trend toward abstracting complex internal infrastructure limitations away from the end-user, enabling efficient serving of large assets without requiring deep knowledge of the underlying cache topology.
