## **What is Verifiers v1?**
Verifiers is Prime Intellect’s environment stack for agentic reinforcement learning and evaluations. The original (v0) bundled data, agent logic, and infrastructure together. **Verifiers v1 unbundles** that stack into three composable, interoperable pieces:
– **Taskset**: Defines *what* the agent works on — the data, tools, and scoring functions.
– **Harness**: Solves the task (e.g., a ReAct loop, a CLI agent, or a custom agent).
– **Runtime**: Where the work happens — either locally or inside a containerized sandbox.
Because these pieces are decoupled, any taskset can run under any compatible harness, enabling flexibility and reuse.
—
## **How the Architecture Works**
At the center of v1 is a **verifiers-managed interception server**. It sits between the agent’s runtime and the inference server, proxying requests and responses while recording the full trace. Key properties include:
– **Trace recording on the fly**: Every request and response is captured without requiring replay.
– **Sampling and rewriting**: The server can set sampling parameters and rewrite tool responses to mitigate reward hacks during training.
– **Elastic scaling**: Each server multiplexes a fixed number of rollouts (default 32), and a pool scales elastically with observed concurrency.
– **Dialect support**: Verifiers natively support three wire-format dialects — OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages — normalized into a canonical type system (`vf.types`).
—
## **v0 vs v1: A Quick Comparison**
| Aspect | Verifiers v0 | Verifiers v1 |
|——–|————–|————–|
| Environment model | Bundled data, logic, infra | Split into taskset, harness, runtime |
| Trace growth | Quadratic in turns (repeated prompt–completion pairs) | Linear in turns (unique nodes) |
| Non-linear rollouts | Assumed linear | Native compaction and subagents via branches |
| Runtime handling | Builder manages lifecycle | Framework-managed `run` / `read` / `write` |
| Harness coupling | Tightly coupled to environment | Any compatible harness (Codex, Terminus 2, etc.) |
| Training data | Recomputed for prime-rl | Consumed directly from trace |
—
## **Use Cases with Examples**
– **Run large models on benchmarks**: For example, you can run **Nemotron 3 Ultra** on **Terminal-Bench 2** using the **Codex** harness.
– **Reuse datasets without rewriting reward logic**: Teams can reuse **Harbor** datasets directly. Prime Intellect ported Terminal Bench 2 into v1 with only a small class change and matched Harbor’s performance.
– **Seamless integration with prime-rl**: The same environments plug directly into **prime-rl** for training. A length-penalty ablation trained **GLM-4.5-Air** on **ScaleSWE** across six H200 nodes for two days, evaluated on **SWE-Bench-Verified** with stable results.
—
## **A Minimal Taskset and Launch**
Every run starts with a **taskset** that defines data and scoring, independent of the harness.
Example taskset (Python-like syntax):
“`python
import verifiers.v1 as vf
class AdditionData(vf.TaskData):
answer: int
class AdditionTask(vf.Task[AdditionData]):
@vf.reward
async def exact_match(self, trace: vf.Trace) -> float:
return float(trace.last_reply == str(self.data.answer))
class AdditionTaskset(vf.Taskset[AdditionTask, vf.TasksetConfig]):
def load(self) -> list[AdditionTask]:
return [
AdditionTask(
AdditionData(idx=i, prompt=f”What is {i} + {i}?”, answer=2 * i),
self.config.task,
)
for i in range(100)
]
“`
Launch it with a harness via TOML and CLI:
“`toml
model = “nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B”
[taskset]
id = “primeintellect/terminal-bench-2”
[harness]
id = “codex”
version = “0.116.0”
“`
“`bash
uv run eval @ path/to/config.toml
“`
—
## **Key Takeaways**
– Verifiers v1 splits an environment into a **taskset** (what), a **harness** (how), and a **runtime** (where).
– A verifiers-managed **interception server** proxies harness–inference requests and records traces on the fly.
– The message-graph trace is **linear in turns**, replacing v0’s quadratic prompt–completion pairs and enabling long-horizon training.
– It ships with full **prime-rl** training support; the legacy code path is now frozen.
– **Harbor** datasets and harnesses like **Codex** and **Terminus 2** work out of the box.
—
## **Frequently Asked Questions (FAQ)**
**Q: What problem does verifiers v1 solve compared to v0?**
A: v1 unbundles environment logic from data and infrastructure, enabling composability, reuse, and scalability. It also replaces quadratic trace growth with a linear message graph, supports non-linear rollouts, and integrates cleanly with prime-rl.
**Q: What is an interception server?**
A: It is a proxy between the harness and the inference server. It records every request and response, allows runtime rewrites, and manages sampling parameters — all critical for reliable RL training and evaluation.
**Q: Can I reuse my existing Harbor datasets with verifiers v1?**
A: Yes. Harbor datasets can be used directly, often with minimal or no changes to reward logic.
**Q: Which harnesses are supported today?**
A: As of now, verifiers supports OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages. More can be added via dialect adapters.
**Q: Is verifiers v1 ready for production training?**
A: Yes. The architecture is designed for production-scale agentic reinforcement learning, with elastic scaling, trace recording, and first-class prime-rl integration.
—
## **Conclusion**
Verifiers v1 represents a fundamental shift in how agentic evaluation and training environments are built. By decoupling tasks, harnesses, and runtime, it enables flexibility, reuse, and scalability while introducing a linear message graph that supports long-horizon agentic learning. With out-of-the-box support for major LLM APIs, integration with prime-rl, and compatibility with datasets like Harbor, verifiers v1 provides a robust foundation for next-generation agentic workflows. Whether you are benchmarking large language models or training agentic policies at scale, verifiers v1 offers a modern and extensible stack built for the demands of agentic AI.
