# How a DNS Platform Slashed Cache Memory Usage by Over 50%
At massive scale, even the tiniest inefficiencies compound into enormous costs. A global DNS resolution platform that stores over 250 billion cache entries at any given time discovered that wasting a single byte per entry translated to more than 250 gigabytes of memory consumed across its entire server fleet. That kind of waste is unsustainable when you are processing billions of queries daily.
The engineering team behind this platform embarked on a series of five successive memory optimizations. The results were dramatic: the per-entry memory footprint was cut by more than half, freeing approximately 100 terabytes of RAM across the fleet. To put that in perspective, the reclaimed memory is equivalent to the total RAM found in over 130 modern servers. Surprisingly, the platform did not sacrifice speed for space — insert throughput actually rose by 43%, and lookup latency dropped by 19%.
This article walks through the specific problems identified and the solutions applied at each layer of the cache implementation.
—
## Understanding the Cache Structure
When the system starts up, the cache begins empty. As DNS queries arrive, entries are stored until the cache reaches its maximum capacity, at which point older or less popular items are evicted to make room for new ones.
Each cached item is a key-value pair. The key captures what was queried — the domain name, the record type (such as `A`, `AAAA`, or `TXT`), whether the response was authenticated, and any associated tags used for routing decisions like EDNS Client Subnet (ECS).
The value holds the actual DNS response: the answer records, authority records, additional records, and metadata such as creation timestamp, hit count, and Time-to-Live (TTL). Both the key and value structures, however, carried significant overhead that was only necessary during the construction phase, not during long-term storage.
—
## Measuring the Impact of Each Change
Before making any modifications, the team built a benchmark that populated the cache with synthetic entries mimicking real-world traffic patterns. The distribution was roughly 56% `A` records, 25% `AAAA` records, and 19% `TXT` records, with each entry containing between one and four individual records. Variable-length record types like `TXT` were randomized between 64 and 224 bytes to approximate the average response size seen in production.
A custom allocator wrapped around the system allocator tracked every allocation made per cache entry, counting both the number and the size of each allocation. The team also measured insert throughput and lookup latency across the entire cache flow to ensure that memory savings did not come at a performance cost.
These benchmarks approximated production conditions. Actual process memory usage also depends on traffic mix, cache occupancy, allocator behavior, and memory consumed by components outside the cache itself. To validate the findings, the team monitored resident memory on production instances throughout each rollout phase.
—
## Optimization One: Eliminating Unused Capacity
The first target was the `Vec` type, a common dynamic array used throughout the cache entry structures. A `Vec` stores three pieces of information: a pointer to heap-allocated data, the current number of elements, and the total capacity it has reserved. When an element is appended, the `Vec` checks whether the length has exceeded the capacity, and if so, it reallocates with extra room for future growth.
However, once a DNS response is stored in the cache, it is never modified again. The capacity field becomes dead weight — 8 bytes per `Vec` that serve no purpose. Worse, the over-allocated heap space is wasted entirely. A `Vec` with capacity for eight items but only five stored leaves three slots unused on the heap.
The fix was straightforward: replace `Vec
Each cache entry contained eight `Vec` and `String` fields. Switching all of them to `Box<[T]>` and `Box
—
## Optimization Two: Fewer Lists, Fewer Pointers
The next issue was structural. The cache entry stored the answer, authority, and additional record sections in three separate lists. Each list was a `Box<[Record]>`, requiring an 8-byte pointer and an 8-byte length field — 16 bytes per list, or 48 bytes for all three.
In DNS, the number of records in each section fits comfortably within a 16-bit unsigned integer (`u16`). By storing all records in a single contiguous list and using two `u16` offsets to mark the boundaries of each section, the team replaced three separate lists with two 2-byte offsets. This eliminated two lists, each carrying an 8-byte pointer and 8-byte length, saving 28 bytes per cache entry.
These savings did not always map one-to-one with the bytes removed from individual fields. The Rust compiler inserts padding between fields to satisfy alignment requirements, and it rounds a struct’s total size up to a multiple of its alignment. Removing a small field can therefore eliminate additional padding beyond the field’s own size. The team also packed several boolean fields into a single bitflag, which reduced surrounding padding and caused the enclosing struct to shrink by more than the individual booleans would suggest.
—
## Optimization Three: Dropping the Redundant Owner
Every DNS record has an owner — the domain name to which the record belongs. In many common cases, that owner is identical to the domain being queried. For instance, a query for `example.com A` returns two `A` records, both with the owner `example.com`.
The DNS wire format handles repeated owners efficiently using name compression, as defined in RFC 1035. Instead of encoding the same domain twice, subsequent occurrences store a 2-byte pointer back to the first occurrence. This works well on the wire, but in the cache the team had been storing the full owner name alongside every single record. Following compression pointers during cache lookups would have been prohibitively expensive on the hot path, so they traded memory for speed at that time.
The insight was that most records do have an owner identical to the queried domain. For those cases, the owner could be dropped entirely and reconstructed at read time from the cache key — which is already available during every lookup. This avoided a heap allocation for the majority of records. Only when the owner genuinely differed, such as the `A` records behind a `CNAME` chain, was the full name stored on the heap. In practice, the vast majority of cached records fell into the first category, making this a highly effective optimization.
—
## Optimization Four: Right-Sizing Enums with Boxing
Rust enums are sum types — each variant can carry different data, but the enum always occupies the size of its largest variant. For the record data field, the team had initially modeled each DNS record type as an enum variant: `A` holding an IPv4 address (4 bytes), `AAAA` holding an IPv6 address (16 bytes), and so on up to `NAPTR`, which holds three variable-length text fields, a domain name, and two integers, totaling 136 bytes.
The problem was stark. The entire enum, including the variant tag and padding, ballooned to 144 bytes. An `A` record, which makes up the majority of DNS traffic, only needed 4 bytes but was forced to occupy 144 bytes — wasting over 120 bytes per record on padding alone. Since a single cache entry could contain multiple records, this waste multiplied quickly.
The solution was to box the larger enum variants. Small and common variants like `A` and `AAAA` remained stored inline within the enum, while larger variants like `TXT`, `NAPTR`, and `SVCB` were moved to the heap, with the enum storing an 8-byte pointer instead. For `A` and `AAAA` records, this saved 120 bytes per record. Smaller types like `TXT` and `CNAME` still occupied the 24-byte enum, but their heap allocation was sized to their actual data rather than padded to 144 bytes. The largest variant, `NAPTR`, did pay slightly more due to the added heap pointer and allocation overhead, but `NAPTR` records are rare in practice, making the tradeoff well worthwhile.
However, boxing introduced its own costs that needed to be addressed.
—
## The Hidden Costs of Boxing
Boxing has two distinct downsides. The first is allocator overhead. Each boxed variant becomes a separate heap allocation, and allocators round up to the nearest size class to manage memory efficiently. A `TXT` record requesting 32 bytes might fit exactly into a 32-byte size bin, but an `MX` record requesting 40 bytes might round up to 48 bytes, wasting 8 bytes. At the scale of billions of allocations, this internal fragmentation adds up.
The second cost is poor memory locality. Without boxing, all record enum values for a cache entry sit in a single contiguous block of memory. With boxing, each variant’s data lives in a separate heap region scattered across memory. Reading a boxed record requires following a pointer, and when that pointer lands far from the rest of the entry data, the CPU must fetch an entirely new cache line. Across millions of cache entries, this pointer chasing degrades performance significantly.
—
## Optimization Five: Storing Records in Wire Format
To eliminate both the allocator overhead and the locality problems of boxing, the team took a different approach: storing record data as raw bytes in a compact wire format.
Instead of a list of parsed enum variants, the records are stored as a single `Box<[u8]>` buffer. Each record is encoded as a 2-byte length prefix followed by its raw bytes, packed contiguously in memory. This eliminates the per-variant enum overhead entirely and avoids the individual heap allocations that boxing required. The data is packed tightly together, dramatically improving CPU cache locality.
The tradeoff is that records can no longer be randomly indexed — the buffer must be read sequentially. This adds some complexity for features like round-robin rotation of `A` and `AAAA` records, but since record counts per entry are small, the cost is negligible.
There is also a practical benefit during response construction. When building a DNS response from cached records, most record types can be copied directly from the buffer into the outgoing message without any parsing. Previously, each parsed record had to be serialized field by field back into DNS wire format. The new layout skips that work entirely for `A`, `AAAA`, `TXT`, and all DNSSEC record types by copying their encoded bytes directly. Only records containing domain names — such as `CNAME`, `NS`, `MX`, and `SOA` — still require parsing so that DNS name compression can be applied. Since records that support direct copying make up the vast majority of traffic, this reduced work on the lookup path. Combined with the improved memory locality, this change alone reduced cache lookup latency by 5%.
Building the record data buffer was also optimized. The team introduced a reusable scratchspace buffer that persists across cache insertions. Since previous writes have already grown the buffer to a sufficient size, it rarely needs to be reallocated. Once the records are serialized into the scratchspace buffer, a single `Box<[u8]>` is allocated and the entire buffer is copied into it in one `memcpy` call. This replaces the separate allocation for each boxed record with one allocation for all record data. It also avoids the waste from shrinking a `Vec
—
## The Results
The production measurements confirmed that the per-entry savings observed in benchmarks translated directly to real-world memory reductions across the fleet.
At the p99 percentile, per-instance memory usage dropped from 9.3 GB to 5.3 GB — a 43% reduction in resident memory. At the p90 percentile, memory dropped from 6.5 GB to 3.8 GB — a 42% reduction. Instances with fuller caches saw the largest absolute savings, which makes sense since they were carrying the greatest number of entries and benefiting from every optimization applied to each one.
Across all five optimizations combined, the per-entry memory footprint was reduced from 953 bytes to 420 bytes, a 56% cut. Per-entry allocations dropped from 1.1 KB to 461 bytes, a 58% reduction. The production reductions were somewhat smaller than the benchmark numbers because resident memory includes the cache alongside all other process data, but after the rollouts stabilized, aggregate working-set memory across the fleet was roughly 100 terabytes lower.
Performance metrics moved in the opposite direction of what one might expect from a memory reduction. Cache insert throughput increased by 43%, and lookup latency dropped by 19%. Fewer allocations, better memory locality, and less wasted space all contributed to a faster system, not a slower one.
| Metric | Before | After | Change |
|—|—|—|—|
| Per-entry net footprint | 953 bytes | 420 bytes | -56% |
| Per-entry allocations | 1.1 KB | 461 bytes | -58% |
| Cache insert throughput | 625,000 entries/s | 893,000 entries/s | +43% |
| Cache lookup latency | 828 ns | 670 ns | -19% |
—
## Frequently Asked Questions
**Why does the cache need to store multiple versions of the same query?**
When EDNS Client Subnet information is in use, authoritative servers return different answers depending on the client’s network. This means a single domain query can produce multiple distinct cached responses, increasing both the number of entries and the memory each one consumes.
**How was memory usage measured in production?**
The team used a custom allocator wrapping the system allocator to track allocations during benchmark runs, and then measured resident memory across actual production instances during the rollout of each optimization to confirm the real-world impact.
**What percentage of cache records have an owner identical to the queried domain?**
The vast majority do. Most DNS responses contain records where the owner matches the queried name, which is why dropping the owner field for those cases was such an effective optimization. Only records behind `CNAME` chains or other alias scenarios require storing the full owner name separately.
**Does boxing enum variants always save memory?**
No. Boxing is most effective for variants that are significantly larger than the pointer size, particularly those with variable-length data. For small variants that fit neatly within the enum’s natural size, boxing can actually increase memory usage due to allocator overhead and the added pointer. The team boxed only the larger variants and kept small ones inline.
**Why store records in wire format instead of keeping them parsed?**
Storing records in wire format eliminates per-variant enum overhead and removes individual heap allocations for each record, which reduces memory fragmentation and improves locality. The tradeoff is that records must be read sequentially rather than randomly accessed, but since most record operations involve iterating through all records in an entry rather than jumping to specific ones, the sequential access pattern is well-suited to this layout.
**What happens to the freed memory?**
The team plans to reinvest reclaimed memory into increasing overall cache capacity without raising the memory footprint, which should improve cache hit rates and reduce upstream query volume to authoritative servers.
—
## Conclusion
Optimizing memory usage at the scale of 250 billion cache entries requires looking beyond algorithmic complexity and examining the precise layout of data in memory. Each of the five changes described here targeted a specific form of structural waste: unused capacity in dynamic arrays, redundant list metadata, duplicated owner names, enum padding, and scattered heap allocations. Together, they delivered a 56% reduction in per-entry memory footprint while simultaneously improving throughput and latency.
The broader lesson is that memory efficiency and performance are not opposing goals. By reducing allocations and improving data locality, it is possible to build systems that are both smaller and faster. These principles apply well beyond DNS caching to any large-scale system where memory is a constrained and costly resource.
Thank you for reading
