## **Securing Cloudflare Workers Against Advanced Remote Spectre Attacks**
In 2021, we assessed **remote Spectre attacks** against Cloudflare Workers and deployed a production defense called **Dynamic Process Isolation (DyPrIs)**, which identifies maliciously looking scripts and isolates them into separate processes. Since then, newer techniques in the area of stabilizing Spectre attacks have been discovered. To understand if these techniques posed a threat to our Workers production environment, we decided to internally reassess the remote Spectre attack. Building an updated proof-of-concept on the production environment allowed us to empirically assess the risk of Spectre attacks under production workloads.
—
### **Cloudflare Workers Security Model**
Cloudflare Workers runs untrusted JavaScript on the edge. Leveraging language-level isolation, in the form of V8 isolates, tens of thousands of tenants can share the same operating-system process. Each Worker has its own separate JavaScript heap. This design keeps startup latency low and lets us run many tenants very efficiently compared to full process isolation. Around the runtime we have multiple layers of defense such as automated V8 patch pipelines, a two-layered sandbox consisting of Linux namespaces and seccomp filters, Cap’n Proto RPC, and the possibility to schedule certain scripts in separate process sandboxes. Still, a single arbitrary read vulnerability within a Worker process can lead to cross-tenant leakage. One vulnerability that is very hard to mitigate exploits the nature of speculative execution, namely in-process **Spectre**.
—
### **Spectre Overview**
You can think of speculative execution in terms of hiking. At some point, you arrive at a branch and have to predict where to go. If the prediction was correct, you saved some time and could enjoy the sun and a refreshing drink at a mountain hut. However, if you speculate in the wrong direction, you have to turn back. The trail looks untouched, but your footsteps remain in the mud.
Speculative execution in CPUs works similarly. The branch prediction performs an educated guess about a branch’s outcome ahead of time and the CPU speculatively executes it. If the prediction was correct, speculative execution saved some time. However, if the prediction is incorrect, the CPU has to discard the results, roll back and execute the other branch. Because these speculatively executed instructions only exist temporarily in the CPU pipeline and are never permanently retired or committed, the literature refers to them as transient instructions and generalizes the concept as transient execution.
However, due to the transient execution, there are still some traces left in the microarchitectural state—for instance, in CPU caches. Thus, an attacker can use Spectre to transiently access memory out of bounds, encode a single bit of information into the cache state, and exploit the latency of reaccessing data to infer whether the bit was set or not.
—
### **Attack Primitives and Challenges**
To mount a successful side-channel attack in production, an external attacker has to overcome additional obstacles such as activity on shared hardware resources, interrupts, context switches, and coarse-grained timers.
– **Spectre Gadget:** A carefully crafted gadget allows transient access to out-of-bounds memory and encodes a single bit into the cache.
– **Signal Amplification:** Using the PLRU cache-replacement policy, attackers amplify a single cache event to overcome noisy remote timers.
– **Remote Timer:** A WebSocket connection to an external server provides a reliable timing source, even over larger distances.
– **Isolate Co-location:** Durable Objects and WebSocket keep-alive messages ensure attacker and victim run in the same process.
—
### **Defeating Isolate Resource Limits**
The Cloudflare Workers runtime enforces limits on all isolates to protect the platform and prevent abuse. For the purposes of conducting this attack, the relevant limits were 30 seconds of CPU time and 1,000 subrequests per invocation.
– **Durable Objects** keep a single isolate alive for hours, allowing long-lived attacks.
– **Interrupts and CPU throttling** are mitigated by splitting work into bursts and yielding regularly.
– **In-process isolation** via Memory Protection Keys (MPK) blocks direct cross-isolate memory reads.
—
### **Putting Everything Together**
By combining tree-based PLRU amplification with measurement loops, the attack successfully leaks memory from a co-located Worker. The final leak rate reached **12 bit/s with over 99% accuracy**.
—
### **Why Was This Not Detected?**
Two key factors allowed the attack to bypass existing defenses:
1. **DyPrIs isolation occurs after invocation**, but the attack uses long-lived WebSocket keep-alives to complete before isolation kicks in.
2. **I/O-heavy workloads inflate branch misprediction normalization**, lowering the detection threshold.
—
### **What We Changed**
We focused on three areas of continued V8 hardening, providing stronger in-process isolation, and improving detection:
#### **1. V8 Sandbox**
– The V8 memory sandbox removes raw 64-bit pointers from the JavaScript heap.
– This makes many memory-corruption primitives harder to exploit.
#### **2. Hardware-Assisted In-Process Isolation**
– Memory Protection Keys (MPK) divide memory into protection domains.
– Hardware-enforced boundaries block straightforward cross-isolate heap reads.
#### **3. Improved DyPrIs**
– Long-lived executions and I/O-heavy workloads are now treated as first-class security cases.
– We are investigating remote timing behavior as an additional behavioral signal.
—
### **FAQ**
**Q: What is Spectre?**
A: Spectre is a vulnerability that exploits speculative execution in CPUs to leak sensitive data via cache timing side channels.
**Q: How does DyPrIs work?**
A: DyPrIs monitors hardware performance counters and isolates suspicious scripts into separate processes to prevent cross-tenant attacks.
**Q: Can Spectre be fully mitigated?**
A: While no single mitigation is perfect, combining V8 sandboxing, hardware-assisted isolation, and behavioral detection significantly reduces risk.
**Q: Was any data actually stolen in the test?**
A: Yes, we successfully leaked a JWT token from a co-located Worker with high accuracy.
**Q: Are Cloudflare Workers still secure?**
A: Yes. The findings have led to improved defenses, and no evidence of prior exploitation was found.
—
### **Conclusion**
This research demonstrated a remote Spectre attack capable of leaking sensitive data from Cloudflare Workers under production conditions. By pushing the boundaries of speculative execution attacks, we gained valuable insights into the limitations of current defenses. The improvements made to DyPrIs, V8 sandboxing, and in-process isolation ensure a stronger security posture for Cloudflare Workers moving forward. Continuous innovation and proactive research remain essential in keeping edge compute platforms resilient against evolving threats.
