# Hermit: A New Open-Source Framework for Training AI Agent Adapters
An open-source framework called **Hermit** has been released by a research lab called Infini-AI-Lab. It is a lightweight Python tool designed to help developers customize AI agents — like Codex or Claude Code — for new tasks without the need to retrain the underlying model or modify the core agent harness. The framework hit its first release, version 0.1.0, in late October 2026.
## What Makes Hermit Different?
Most approaches to improving AI agents involve either fine-tuning the model itself or building a separate agent from scratch. Hermit takes a third path: it trains an **external harness adapter** — essentially a smart plugin — while keeping both the model and the original harness completely unchanged.
This adapter can contain instructions, reusable skills, memory structures, extra tools, executable code, and even multi-agent coordination logic. Because the adapter is separate from the frozen model and harness, it can be moved between compatible systems and updated independently.
The team behind Hermit highlights three main advantages of this approach. First, many popular harnesses are closed-source, and this design works with them anyway. Second, editing an existing harness risks breaking carefully balanced interactions between components. Third, a well-trained adapter transfers across models and harnesses, saving significant redevelopment effort.
## How the Training Loop Works
Hermit uses an iterative, feedback-driven training process. Here is how a single training cycle operates:
1. **Run tasks** — The agent executes a batch of tasks using the current adapter.
2. **Score outputs** — A judge evaluates the agent’s results and generates feedback.
3. **Propose updates** — An “Evolver” component reads the trajectories and feedback, then suggests an improved adapter.
4. **Validate** — The proposal is checked against a predefined adapter template to ensure it is well-formed.
5. **Commit** — The validated adapter version is saved into a git-based version tree.
Every version of the adapter is stored in a git repository, making it easy to inspect previous iterations, roll back to earlier versions, or deploy a specific release. Some learner programs can run in sandboxed environments using Linux with bubblewrap.
## Five Built-In Evolution Methods
Hermit ships with five evolution strategies right out of the box:
– **GEPA** — Reflective evolution with Pareto selection
– **SkillOpt** — Optimizes reusable agent skills
– **MetaWorld Model (MWM)** — Builds a world model for stateful environments
– **Harness²** — Learns while streaming through test data
– **Continual Harness** — Adapts continuously during task execution
Two additional methods — ACE and ReHarness — are listed as coming soon. Users can also build their own custom Evolver if needed.
## Installation and Deployment
Getting started is straightforward:
“`bash
pip install hermit-harness
hermit train –config run.yaml
hermit eval –config run.yaml –adapter final
hermit deploy –adapter DIR –workspace WS –host codex
hermit convert –adapter DIR –output OUT –target claude-code
“`
The offline Quick Start requires no GPU, model account, or dataset. For real workloads, Hermit supports any API backend or a local server running vLLM, SGLang, or Ollama. Reference backends include Fireworks, OpenAI, Anthropic, and DeepSeek.
Trained adapters can be deployed directly into workspaces for Codex, Claude Code, or Pi. They can also be converted into native plugins for these platforms or for Hermes. A demonstration of the framework successfully coordinated five Codex agents working together to prove a theorem using a finished Lean collaboration adapter.
## Benchmark Performance
All results reported so far use GLM 5.3 Flash as the acting model and are self-published by the team.
**OfficeQA (Recipe Benchmark):** Tests 172 questions about U.S. Treasury Bulletins. The best adapter, Harness², reaches 0.651 accuracy compared to 0.541 for a no-adapter baseline. The prompt-only approach scores 0.576, while the initial skill scores 0.576. GEPA and SkillOpt are trained on 50 questions and tested while frozen.
**CL-Bench (Database and Poker):** On database exploration, the MWM adapter achieves 0.795 accuracy versus 0.580 for in-context learning. On exploitable poker, MWM wins 3.08 big blinds per hand compared to 2.51 for standard in-context learning.
**ServeLearnBench (Retail Support):** Streams 624 requests under a hidden and shifting policy. MWM scores 45.3 on hidden-policy tests, significantly ahead of the best learning method on the leaderboard (Continual Harness at 28.7). An oracle with full knowledge of the policy scores 89.2.
## How Hermit Compares to Similar Tools
| Feature | Hermit | GEPA | ACE | Agent Lightning |
|—|—|—|—|—|
| Trains | External harness adapter | Any text parameter | Context playbook | Prompts and model weights |
| Changes model weights? | No | No | Not disclosed | Yes |
| Built-in methods | 5 | Reflective evolution | Generator, Reflector, Curator | RL, prompt optimization, SFT |
| Deploy targets | Codex, Claude Code, Pi; plugins | DSPy | Not disclosed | LangChain, AutoGen, CrewAI and more |
| License | Apache-2.0 | MIT | Apache-2.0 | MIT |
Hermit stands out for its explicit focus on keeping the model frozen while still achieving strong gains through adapter training. Its deploy-to-plugin workflow also makes it practical for teams that want to integrate trained adapters into existing agent infrastructure quickly.
## FAQ
**Q: What exactly is a harness adapter?**
A: It is a trainable external component that wraps around an existing AI agent. Think of it as a smart plugin that can hold instructions, skills, memory, tools, code, and coordination logic. It gets updated through training while the underlying model and harness stay completely untouched.
**Q: Do I need a GPU to use Hermit?**
A: No. The offline Quick Start works without a GPU, model account, or dataset. You only need compute resources when running actual training or evaluation tasks against real models.
**Q: Can I use Hermit with closed-source harnesses?**
A: Yes. Because Hermit trains an external adapter rather than editing the harness itself, it works seamlessly with closed-source systems like Codex and Claude Code.
**Q: How many evolution methods are available?**
A: Five methods ship with the initial release — GEPA, SkillOpt, MWM, Harness², and Continual Harness. ACE and ReHarness are planned for future updates, and you can also write your own custom Evolver.
**Q: Are the benchmark results independently verified?**
A: The reported results are self-published by the Infini-AI-Lab team and are mostly based on single runs. They should be treated as indicative rather than definitive until independent replication occurs.
**Q: What license does Hermit use?**
A: Hermit is released under the Apache 2.0 license, which permits both commercial and non-commercial use.
## Conclusion
Hermit offers a practical and flexible approach to agent customization. By separating the trainable adapter from the frozen model and harness, it enables rapid iteration without the overhead of retraining large models or maintaining custom forks of existing agent systems. The framework’s plugin-based deployment and multi-platform support make it accessible to a wide range of developers. While the benchmark results are self-reported and mostly based on single runs, the initial performance gains — particularly with the MWM adapter on CL-Bench — are compelling. As additional evolution methods are released and independent evaluations emerge, Hermit has the potential to become a valuable tool in the AI agent development ecosystem.
Thank you for reading
