# How Cloudflare Makes DNS Changes Reach the Edge Faster
Every time someone updates a DNS record or changes a domain setting, a complex process kicks in to make sure the change is reflected across the entire global network. Cloudflare handles millions of these updates every day, and making sure they all work quickly and correctly is a massive engineering challenge. Let’s break down how it works and what changed recently to make it faster.
## The DNS Pipeline in Simple Terms
When you change a DNS record — say, pointing a domain to a new server — that change doesn’t just teleport to every server worldwide instantly. Instead, it travels through a series of stages, like an assembly line.
The pipeline works in roughly six steps:
1. A change is made, either through an API, a dashboard, or another Cloudflare service.
2. The change gets saved to a database.
3. An event is generated and sent through a message queue called Kafka.
4. A special component collects and organizes these events, then passes them along to the Zone Builder.
5. The Zone Builder reads the relevant zone data and converts it into a format that the edge servers can understand and use.
6. The Zone Builder compares the new data against what is already stored in Quicksilver, Cloudflare’s distributed key-value store, and writes only the differences.
Once Quicksilver has the updated data, it propagates those changes across Cloudflare’s network of data centers so that every DNS query gets the latest information.
## Why “Full Rebuilds” Became a Bottleneck
In the early days, whenever any part of a DNS zone changed, the system would perform what is called a “full build.” This meant reading every single DNS record in the zone, converting them all, and writing them back out. It was simple and reliable, but it had a serious flaw: the work grew proportionally with the size of the zone.
Some zones now hold tens of millions of records. Performing a full rebuild on a zone that large is incredibly expensive in terms of memory and processing power. The zone simply might not fit in memory at all, let alone be processed quickly.
At the same time, changes were happening far more frequently — not just to individual records, but also to zone-wide settings like Universal SSL, which affects certificate behavior and related DNS records at the domain’s root.
## The Solution: Smarter, Smaller Builds
The breakthrough was to stop rebuilding entire zones and instead figure out the smallest amount of work that would still produce a correct result. This is called a “targeted build.”
### Making Events More Specific
The first step was making events carry more information about what actually changed. Instead of just saying “something in this zone changed,” an event could describe the specific setting that was updated, what its previous value was, and what its new value is. This lets the system choose the right level of work.
### Five Types of Builds
With more detailed events, the build planner can now choose between five different build types:
– **Resources only:** Rebuild the DNS records at specific domain names, without touching zone-wide metadata. This is the path for most routine record changes like A records, CNAMEs, or TXT records.
– **Resources and metadata:** Rebuild specific domain names along with shared metadata. This is needed for changes like wildcard records or nameserver updates.
– **Apex and metadata:** Rebuild the root domain and its metadata. This handles zone-wide settings like Universal SSL that can change records at the domain’s bare name.
– **Purge:** Remove a zone entirely if it should no longer be published.
– **Full zone:** Rebuild everything, as a fallback when the system cannot prove that a smaller build would be correct.
The golden rule is that every piece of data a targeted build produces must be identical to what a full build would produce. If there is any doubt, the system falls back to a full build.
### Combining Work Together
When multiple changes come in for the same zone at once, a unified scheduler combines them into a single build. For example, if someone changes an A record for one subdomain and a wildcard record for another, both changes are processed in one build pass rather than two separate ones. This avoids duplicate work and keeps the pipeline efficient.
### Dealing With Shared Data
One tricky part of DNS is that multiple zones can share the same domain name. For instance, two different Cloudflare customers might both have a zone for the same domain. There is a shared metadata key that acts like a directory, telling edge servers which internal zones are active.
When a change happens, the system must figure out whether it affects just one zone or the shared metadata for all zones with that name. A record change for a specific subdomain only affects that zone. But a nameserver change affects the shared directory and must update both the zone records and the metadata.
### Safety Checks Along the Way
Even after planning a build, the system does not rush ahead blindly. Just before executing, it reads a fresh snapshot of the current zone state to make sure nothing has changed. This handles cases where:
– A targeted build now needs to become a full build because a new feature was enabled.
– A zone should no longer be published on a particular network.
– A zone no longer belongs in the metadata directory.
– The shared metadata key needs to be rebuilt with the current set of active zones.
## The Results
The impact of these changes was significant. After the rollout:
– Over 92% of builds no longer required rebuilding the entire zone.
– Peak memory usage dropped by about 85%, falling from tens of gigabytes down to a much more manageable level.
– The worst-case delays shrank dramatically, with the slowest 1% of builds dropping from over 14 minutes to just over a minute.
– Median build completion time was halved, from about 7.7 seconds down to roughly 3.3 seconds.
These improvements meant the system could handle growth more gracefully — zones with millions of records no longer caused the same level of strain, and changes flowed through the pipeline more steadily instead of in unpredictable bursts.
## Quality Assurance
Before any of these changes went live to all customers, they were thoroughly tested:
– Equivalence tests compared targeted builds against full builds to ensure they produced identical output.
– A mirror system was set up that consumed the same production events and built zones in a separate environment, comparing the results continuously.
– A dashboard tracked any differences between the two environments, and the team verified that no new types of mismatches appeared before the rollout.
## FAQ
**What is a DNS zone?**
A DNS zone is the complete collection of DNS records and settings associated with a domain name, such as example.com.
**What is Quicksilver?**
Quicksilver is Cloudflare’s distributed key-value store that holds DNS data and makes it available across the global network of data centers.
**What is Kafka in this context?**
Kafka is a message queue system that receives events from various sources — database changes, API calls, and internal service updates — and delivers them to the Zone Builder for processing.
**Why do some builds still do a “full zone” rebuild?**
Full builds are still necessary in certain situations: when a zone is being published for the first time, when a change could affect records anywhere in the zone, or when an event does not provide enough detail to safely target a smaller build. The system is designed to avoid guesswork — if it cannot prove a smaller build is safe, it does the full one.
**What does it mean that builds are “targeted”?**
Targeted builds only rebuild the specific records and metadata that a change actually affects, rather than rebuilding every single record in the zone. This makes them much faster and less memory-intensive.
**What is the “rebuild interval”?**
The rebuild interval is a short waiting period that prevents one very busy zone from constantly triggering builds and consuming resources that other zones need. Changes that arrive during this interval are merged into the next build.
**How does the system handle multiple zones with the same domain name?**
A shared metadata key acts as a directory, listing which internal zone IDs are currently active for a given domain name. Changes are carefully routed so that zone-specific updates only modify that zone’s records, while shared changes update both the records and the directory.
## Conclusion
Optimizing a DNS propagation pipeline for millions of zones — some containing tens of millions of records — is a formidable engineering problem. By replacing blunt full rebuilds with targeted builds, combining related work intelligently, and adding rigorous safety checks, Cloudflare significantly reduced the resource demands and improved the speed of the entire system. The result is a pipeline that scales better, uses less memory, and delivers DNS changes to the edge faster than ever before.
Thank you for reading
