# Cache Transcoding: Compressing Storage Once to Save Bandwidth and Capacity Forever
## The Rising Cost of Memory
Memory costs have surged dramatically in recent years. Both RAM and hard disk drive prices have climbed sharply, putting pressure on every organization that manages large-scale distributed infrastructure. Companies running massive storage products—from content delivery networks to cloud-based caching systems—must find ever more clever ways to stretch their existing hardware into doing more work.
The fundamental challenge is this: if every byte stored on disk and transferred across a network costs more than it used to, how do you keep delivering fast, reliable service without exploding your budget? The answer many are now exploring is smarter compression—not just at the edge where data arrives, but deep inside the storage pipeline itself.
## What Is Cache Transcoding?
Cache Transcoding is an architectural approach that intentionally compresses eligible cache objects before they are written to persistent storage. Rather than preserving the exact byte representation received from an origin server, the system evaluates each response and, when appropriate, encodes it using a high-performance compression algorithm before the data ever touches the disk.
The core idea is a trade: spend a small amount of additional CPU processing at the moment an asset enters the cache, in exchange for dramatically reduced storage footprint and dramatically lower inter-data-center bandwidth consumption over the entire lifetime of that asset. Because cache entries that are “hot”—frequently requested—are stored once and served thousands or millions of times, the encoding cost is amortized across every subsequent access.
The system works hand-in-hand with tiered cache architectures, where an upper cache tier feeds a lower cache tier. Because the compressed representation is preserved as data moves between tiers, the bandwidth savings multiply across the entire infrastructure, not just at a single node.
## Understanding Zstandard (zstd)
At the heart of this approach is a compression algorithm called Zstandard, or zstd for short. Zstandard was developed by Yann Collet and open-sourced in 2016. It is a lossless compression method, which means that when compressed data is decoded, every single byte returned to the consumer is identical to the original—nothing is lost, nothing is approximated.
What makes Zstandard particularly attractive for caching systems is its deliberate design philosophy: it balances compression ratio with encoding and decoding speed. Early comparative testing showed that Zstandard compresses data roughly 42% faster than Brotli while producing nearly identical file sizes, and generates output approximately 11.3% smaller than gzip at a comparable speed. For a system that must process enormous volumes of traffic continuously, this balance between compression density and processing cost is critical.
The specific implementation described here uses Zstandard at compression level 3. This level captures the majority of the available storage savings without converting cache fills into a CPU bottleneck, keeping the system’s overall throughput within acceptable bounds.
## Why Not Everything Benefits From Compression
One of the most important design decisions in any transcoding system is knowing what not to compress—and that decision is as important as knowing what to compress.
Images, video files, and font files are almost always already compressed by their creators or by upstream services. In a representative traffic sample, this media category accounted for 21.4% of requests but consumed 63.3% of total bytes. Attempting to compress already-compressed data would waste CPU cycles for zero storage benefit and could even make files slightly larger.
The real opportunity lies in compressible text formats. HTML documents, JSON payloads, CSS stylesheets, and JavaScript source code accounted for 67.3% of requests but only 22.3% of bytes in the same sample. Within that text slice, approximately 71% arrived at the cache without any content encoding applied and compressed remarkably well. In controlled testing, these eligible assets shrunk to roughly one-third of their original on-disk size.
## Eligibility Criteria and the 4 KiB Threshold
To ensure the system makes intelligent decisions about what to compress, a series of eligibility checks filters incoming responses. The transcoding system applies the following conditions before encoding a response:
– **HTTP Status Code:** The response must be a 200 OK. Error responses and redirects are left untouched.
– **Content-Encoding Header:** Must be unset, indicating the origin has not already compressed the body. This prevents double-compression.
– **Content-Type:** Must be a text-based MIME type considered compressible.
– **Known Content-Length:** The response body length must be known and must be at least 4 kibibytes (KiB).
The 4 KiB minimum threshold is a deliberate choice, not an arbitrary one. Smaller responses contribute relatively little to total storage footprint, but they still incur per-object overhead during encoding and decoding. Keeping the threshold at 4 KiB eliminates a large volume of tiny requests while excluding only about 1% of the bytes that would otherwise be eligible for compression. This accelerates the decision-making process without meaningfully sacrificing compression savings.
Both the compression level and the size threshold are designed as tunable parameters rather than fixed constraints. The initial values serve as a conservative starting point, and further tuning can explore whether more aggressive settings yield better overall efficiency.
## The Performance Numbers
The compression performance was measured in a controlled testing environment that processed over one million requests across multiple cache servers.
| Metric | Value |
|—|—|
| Compression Ratio | Approximately 2.8x |
| Encoding Cost | Roughly 4.31 nanoseconds per byte (~232 MB/s, paid once per cache fill) |
| Decoding Cost | Roughly 1.56 nanoseconds per byte (~641 MB/s, paid on every cache hit) |
A critical observation emerges from these numbers: encoding is roughly 2.7 times slower per byte than decoding, but since every cached object is written once and read many times, the total system cost remains overwhelmingly favorable. The bulk of the CPU investment happens a single time, while the storage and bandwidth benefits compound with every subsequent request.
## How the System Preserves Correctness
Cache Transcoding handles several distinct paths through the caching pipeline, and correctness must be maintained regardless of which path a given request follows.
**On a Cache Miss:** When a response arrives at the proxy and is hers to cache, the body is encoded using Zstandard, written to disk in its compressed form, then decoded back to its original representation before being returned to the client. This means the client receives bytes identical to what the origin server sent—transparency is preserved end-to-end.
**On a Cache Hit:** The stored Zstandard bytes are pulled from disk, decoded in memory, and served to the client as the original content. The client never knows that compression happened behind the scenes.
**With Tiered Cache Transfers:** When a compressed object moves between cache tiers, it travels in its compressed form both across the network and while stored on the receiving tier’s disk. Decoding only happens at the final tier—the one directly facing the client. This is a key efficiency: the bandwidth saved between tiers multiplies the value of storing objects in compressed form.
The system uses a storage encoding marker within the cache metadata to track which objects are stored in compressed form. If a cache layer receives an object from another tier that is already compressed, it recognizes that state and preserves it rather than attempting to re-encode or decode unnecessarily.
## The Test Campaign
The prototype was validated through an extensive correctness campaign covering every major request path: cache misses, cache hits, single-tier fills, tiered cache fills, and numerous edge cases. Test traffic was engineered to follow specific paths through the architecture by varying cache keys, and distributed tracing was used to confirm precisely where encoding and decoding operations occurred.
A separate performance campaign sent over one million requests across ten cache servers, alternating between configurations with tiered caching enabled and disabled. This division allowed the engineering team to isolate the storage benefits of local caching from the network bandwidth benefits of compressed cross-tier transfers.
It is worth noting that the test corpus was deliberately chosen to be highly compressible. Two test assets of approximately 195 KiB and 272 KiB were used, both shrinking by roughly 2.8 times. While this provided a clear, mathematically clean signal for validating the architecture, it does not represent the full diversity of content found in a real-world fleet. Broader testing with a wider range of content types and object sizes is needed before applying the measured compression ratio as a universal constant across all deployments.
## Frequently Asked Questions
**Q: What is the difference between Cache Transcoding and traditional content encoding (like gzip)?**
A: Traditional content encoding is performed at the origin server or an edge proxy before the response reaches the cache. Cache Transcoding happens inside the cache layer itself, compressing an already-received response before writing it to disk. This allows compression to be applied even when the origin did not send encoded content.
**Q: Does the client receive a compressed response?**
A: No. Clients always receive the original, uncompressed content. Compression happens inside the cache infrastructure and is transparent to all downstream consumers.
**Q: What happens if a compressed cache object needs to be served to a client that does not understand Zstandard?**
A: The decoding happens before the response leaves the cache layer. The client always receives decompressed data with the original content encoding.
**Q: Why was Zstandard chosen over other algorithms like Brotli or gzip?**
A: Zstandard was selected because it achieves a compression ratio close to Brotli while encoding significantly faster. Its decode speed is also excellent, which matters since decoding happens on every cache hit rather than just once per cache fill.
**Q: What is the impact on cache eviction behavior?**
A: By reducing the on-disk size of cached objects, Cache Transcoding effectively increases cache density. This means more objects can be retained before eviction is necessary, reducing the chance that useful content is removed simply because an uncompressed representation consumed too much space.
**Q: Can this approach be applied to already-compressed assets like images?**
A: The system deliberately skips pre-compressed content types. Images, video, and fonts are typically already compressed by their origin, and attempting to re-compress them would waste CPU without meaningful storage savings—and could even increase file size.
**Q: How does the CPU cost compare to the storage savings?**
A: The CPU overhead is kept to a few percent of total processing cost under the tested traffic and reuse patterns. In return, storage capacity effectively triples and inter-data-center bandwidth consumption drops significantly. The trade is considered highly favorable.
**Q: What parameters can be tuned if deploying this system?**
A: Both the compression level and the minimum response size threshold are configurable parameters. Additionally, eligibility criteria such as the minimum Content-Length, accepted MIME types, and minimum HTTP status codes can all be adjusted based on workload characteristics.
## Conclusion
Cache Transcoding represents a compelling architectural choice for anyone operating large-scale distributed caching systems under growing memory pressure. By accepting a modest increase in CPU usage at the point of cache fill, organizations can unlock dramatic improvements in effective storage capacity and network efficiency across their entire infrastructure.
The approach is built on well-understood principles—lossless compression, intelligent content filtering, and careful architectural integration with tiered cache topologies. The testing performed validates that the trade-off between processing cost and storage savings is not only favorable but substantial.
As memory and storage costs continue their upward trajectory, innovations like this will become increasingly essential. The ability to fit more customer data into existing hardware—not by buying more machines, but by making smarter use of what is already there—is exactly the kind of engineering advantage that allows infrastructure to scale gracefully under real-world demand.
Future work in this space can explore more aggressive compression settings, support for additional content types, handling of range requests, and integration with downstream components that may already understand compressed representations natively.
Thank you for reading
