# Harness²: A New Open-Source Framework That Lets Your AI Agent Improve Itself — Without Upgrading the Model
## Introduction
What if your AI agent could get significantly better at its job without you ever retraining or replacing the underlying model? That is exactly the promise of Harness², a new open-source framework developed by researchers at Google Cloud AI Research and Northwestern University.
Instead of training a larger or smarter model, Harness² focuses on everything *around* the model — the prompts, tools, memory systems, and workflows that guide how an agent behaves. By refining this surrounding setup at test time, the framework can produce meaningful performance gains using the exact same model you already have.
Think of it like tuning the engine settings and driving strategy of a car rather than swapping in a bigger motor.
—
## What Exactly Is Harness²?
Harness² is a plug-and-play framework designed to improve an AI agent’s behavior by rewriting its operational setup during deployment. The key idea is simple yet powerful: the model’s internal weights never change, but the system that wraps around the model — known as the harness — gets refined and optimized through a structured, recursive process.
The framework is built for open-ended environments where tasks arrive unexpectedly and there is no clear right-or-wrong answer for every situation. Most self-improvement methods for AI agents rely on ground-truth labels or test cases to know what a “good” answer looks like. Harness² was specifically designed to work without those. It relies on behavioral contrasts instead — comparing how different setups perform on the same task and keeping only the changes that lead to better outcomes.
—
## The 8 Editable Components
At the heart of Harness² are eight distinct parts of an agent’s harness, each stored as a separate file that can be edited and rewritten. Together, these components control how the agent receives information, uses tools, delegates tasks, and behaves during execution.
Here is what each component does:
1. **System Prompt** — Defines the agent’s persona and the top-level procedure it follows on every call. This is the foundation of how the agent interprets its role.
2. **Guardrails** — Process rules that constrain the agent’s behavior, such as verifying outputs before finishing a task or preventing certain types of actions.
3. **Memory** — Persistent context that is made available to the agent across interactions, allowing it to remember past steps and decisions.
4. **Tool Guidance** — Instructions on how to use specific tools correctly and handle their outputs with precision, reducing errors caused by misuse.
5. **Subagents** — Delegated roles with their own specialized instructions, enabling the main agent to break complex work into parallel or sequential sub-tasks.
6. **Skills** — Reusable domain-specific procedures, like a reconciliation skill for financial tasks or a code-debugging skill for software engineering.
7. **Scripts** — Executable helper programs, such as scripts that reopen files, check formats, or validate intermediate results.
8. **Plugins (Middleware)** — Software layers that sit inside the tool loop and shape the agent’s behavior in real time, controlling how tools are called and how responses are processed.
This structure extends earlier work on agent harness engineering by adding one additional component compared to previous seven-file layouts, giving developers finer control over every aspect of agent behavior.
—
## How the Recursive Improvement Process Works
Harness² uses a two-level recursion system that operates both *within* a single task and *across* multiple tasks.
### Task-Level Recursion
When the agent tackles a new task, the framework follows a four-step cycle:
**Step 1 — Run the base harness.** The frozen model executes the task once using the current, unmodified setup. This provides a reference trajectory showing how the agent behaves out of the box.
**Step 2 — Propose two candidates.** An editor generates two different modifications to the harness. One candidate focuses on general agent weaknesses like planning or tool use (the invariant candidate). The other targets domain-specific knowledge and procedures relevant to the current task (the domain candidate).
**Step 3 — Probe both candidates.** Each candidate runs on the same task using the same model. By comparing their trajectories against the reference, the framework can see which edits actually help — without needing a ground-truth label.
**Step 4 — Compose the best edits.** A composer analyzes all the evidence and diffs, then merges the useful changes into the harness. It may adopt one candidate, blend elements from both, or decide to keep the original setup if neither candidate improves things.
### Harnessing-Level Recursion
After each task is completed, the framework reflects on what happened. A reflector reviews the full record and stores the adopted harness along with a lessons file called `EXPERIENCE.md`. This library of accumulated insights guides future proposals, meaning the system gets smarter over time.
Each new task still starts from the base harness, but the lessons library informs what kinds of edits are proposed and how they are composed. This means improvements from one task can transfer to future, unrelated tasks.
The search budget is controlled by a parameter called **k**. In parallel mode, the framework probes **k** invariant candidates and **k** domain candidates simultaneously, then creates **k** compositions to choose from. In sequential mode, it chains **k** refinement steps one after another.
—
## How Well Does It Perform?
The research team evaluated Harness² across three widely used professional benchmarks: **WorkBuddy-Bench**, **JobBench**, and **Harvey’s LAB**. The tests used two different model executors — **Gemini 3.5 Flash** and **GLM-5** — each running under two different harness backends called OpenCode and Codex. The default editor used for proposing harness changes was Claude Opus 4.8, and all reported results are averages over three runs.
### Benchmark Results (Single Refinement Step, k=1)
| Setup | WorkBuddy Average | JobBench | LAB Partial |
|—|—|—|—|
| Gemini 3.5 Flash, OpenCode | 72.4 → 77.5 | 31.5 → 36.0 | 79.9 → 81.9 |
| Gemini 3.5 Flash, Codex | 74.5 → 77.5 | 33.9 → 35.3 | 71.6 → 75.6 |
| GLM-5, OpenCode | 64.2 → 75.7 | 20.6 → 24.2 | 59.6 → 71.4 |
| GLM-5, Codex | 64.4 → 75.3 | 23.6 → 24.8 | 58.1 → 65.9 |
Every single setup improved across all three benchmarks after just one refinement step. The most dramatic gain came from GLM-5 running under OpenCode on WorkBuddy-Bench, where the score jumped from 64.2 to 75.7 — an **11.5-point increase** — surpassing the base Gemini 3.5 Flash setup.
Weaker executors generally saw larger gains, suggesting the framework is particularly effective at closing the gap between less capable models and stronger ones. When the search budget was increased to k=5, Gemini under OpenCode reached **80.8** on WorkBuddy-Bench. Additionally, three out of four setups used fewer steps per task after refinement, and Gemini under OpenCode reduced its average steps from 25.4 to 21.9.
—
## How It Compares to Other Methods
Harness² was compared against three other harness optimization approaches in the research community.
| Feature | Harness² | Meta-Harness | ReasoningBank | Harness Scaling |
|—|—|—|—|—|
| Developers | Google Cloud AI Research, Northwestern | Stanford IRIS Lab | Google Research | Wang et al. |
| What Changes | Full 8-component harness, per task | Harness code, searched offline | Retrieved memory; harness fixed | Harness, revised per instance |
| Needs Training Split | No | Yes | No | No |
| Learns Across Tasks | Yes (library + lessons) | Deploys 1 best harness | Yes (memory) | No |
| Code License | Apache 2.0 | MIT | Not disclosed | Not disclosed |
| WorkBuddy Avg (Gemini/OpenCode) | 77.5 (80.8 at k=5) | 76.4 | 73.4 | 75.3 |
| JobBench (Gemini/OpenCode) | 36.0 | 35.8 | 30.8 | 32.9 |
| LAB Partial (Gemini/Codex) | 75.6 | 76.5 | 74.8 | 67.9 |
Harness² stands out because it does not require a separate training split, it learns improvements across tasks through its experience library, and it modifies the entire harness rather than just one part. On WorkBuddy-Bench, it outperforms Meta-Harness and ReasoningBank at k=1, and at k=5 it reaches the highest score reported across all methods. It also uses an Apache 2.0 license, making it fully open and commercially usable.
—
## Getting Started
The framework runs on Linux with Python 3.12 or 3.13 and the `uv` package manager. A local demo mode lets you explore both parallel and sequential recursion without needing any model credentials or API keys.
To set up a local demo:
“`
uv venv –python 3.12 .venv
source .venv/bin/activate
uv pip install -e ‘.[render]’
harness2-demo –output runs/demo –k 2
“`
For model-backed runs, you will need Node.js 20 or later, the Google Cloud CLI, and a Vertex AI project. Additional dependencies include Pandoc and Poppler for LAB, and Docker for WorkBuddy-Bench. By default, executors run under OpenCode, but you can switch to Codex by adding the `–substrate cx` flag.
—
## Frequently Asked Questions
### Q: Does Harness² require retraining the model?
A: No. The model weights are completely frozen. Harness² only modifies the surrounding system — prompts, tools, memory, and orchestration — leaving the model itself untouched.
### Q: Do I need a training dataset to use this?
A: No. Unlike many other methods, Harness² does not require a separate training split or ground-truth labels. It works entirely through behavioral contrasts during task execution.
### Q: How is this different from just writing better prompts?
A: Prompt engineering is a manual, one-off activity. Harness² automates a structured, recursive process that proposes, tests, and composes harness changes based on actual observed behavior. It also builds a reusable lessons library that improves over time across multiple tasks.
### Q: Will using Harness² slow down my agent?
A: Yes, somewhat. Each refinement step adds extra runs per task (at minimum, one reference run plus two probe runs). The tradeoff is a more capable agent that may also complete tasks in fewer overall steps.
### Q: Can I use Harness² with any AI model?
A: The framework is model-agnostic in design. It supports any model that can be called through Vertex AI or a compatible interface. The research team tested it with Gemini 3.5 Flash and GLM-5, but the approach should work with other models too.
### Q: Is Harness² safe to use in production?
A: The framework is open-source under the Apache 2.0 license, which allows both research and commercial use. However, the researchers note that users should sandbox their deployments and carefully evaluate harness changes before trusting them in critical workflows.
### Q: What is the EXPERIENCE.md file?
A: It is a persistent lessons log stored alongside the harness. After each task, the framework records which edits helped and why. This file becomes a growing knowledge base that guides future improvement proposals, enabling the system to learn across tasks without retraining.
—
## Conclusion
Harness² represents a compelling new direction in AI agent development — one that shifts focus from making models bigger to making the systems around them smarter. By treating the harness as an editable, improvable artifact, the framework gives developers a practical way to squeeze more performance out of their existing models.
The combination of recursive task-level improvement, cross-task learning through a lessons library, and ground-truth-free behavioral evaluation makes Harness² both flexible and scalable. Whether you are working with a strong model and want to squeeze out every last percentage point, or with a weaker model and want to close the gap, the framework offers a structured, open, and transparent path forward.
As AI agents become more central to software engineering, knowledge work, and automation, approaches like Harness² remind us that the real bottleneck is not always the model itself — it is how we set it up to succeed.
Thank you for reading
