## Beyond RAG: Agentic Retrieval Harness for Legal Documents with LlamaIndex legal-kb
The post announces that **LlamaIndex has released `legal-kb`](https://github.com/run-llama/legal-kb)**, a public reference application on GitHub that demonstrates an advanced pattern for working with legal documents. Built on **LlamaIndex Index v2 (the LlamaParse Platform)**, it is described as a **knowledge base for legal documents powered by a “Retrieval Harness for agentic retrieval.”**
### What makes this different from standard RAG?
Instead of a single-shot vector search per query, `legal-kb` treats the knowledge base like a **filesystem** and gives an agent a set of **tools** (similar to operations engineers’ tools). The agent can:
– **List files** in the knowledge base
– **Read** file contents
– **Grep** for patterns inside files
– Perform **hybrid semantic search**
Because the tools are generic, the same harness can be plugged into your own agents.
### Core components of the Retrieval Harness
The reference app includes a **tool loop** built with Vercel AI SDK 6. The agent is given four tools, each backed by an Index v2 retrieval API:
| Tool | Backing API | Key parameters | What it does |
|——|————-|—————-|————–|
| `retrieve` | `beta.retrieval.retrieve` | `query`, `top_k`, `score_threshold`, `rerank_top_n`, `file_name`, `file_version` | Runs hybrid semantic search with optional reranking; returns chunks plus citations |
| `findFiles` | `beta.retrieval.find` | `file_name`, `file_name_contains` | Searches files by exact name or substring; paginates automatically |
| `readFile` | `beta.retrieval.read` | `file_id`, `offset`, `max_length` | Reads raw file content, with offset and length windows |
| `grepFile` | `beta.retrieval.grep` | `file_id`, `pattern`, `context_chars`, `limit` | Matches a pattern in one file; returns character positions |
The system prompt enforces a **clear order of operations**:
1. `findFiles` to establish the document inventory.
2. `retrieve` to narrow down relevant content.
3. `readFile` or `grepFile` to confirm exact wording.
4. Cite using inline IDs (e.g., `cite:c7f2qa`).
### How it works under the hood
– **Uploads** follow a pipeline defined in `src/lib/files.ts`. Bytes are pushed to the project’s LlamaCloud source directory.
– **File and ProjectFile rows** are written to PostgreSQL via Prisma.
– An **index sync** is triggered and the UI polls until ready.
– **Versioning** is scoped to the `(project, filename)` pair. Re-uploading the same file creates v1, v2, v3 side by side, and the retrieval layer can filter by the `file_version` metadata field.
– The agent uses `ToolLoopAgent` from the Vercel AI SDK 6. You can choose **OpenAI or Anthropic** per turn and bring your own keys.
– **Reasoning is streamed**: Claude models use extended thinking; OpenAI reasoning models use medium reasoning effort.
### Naive RAG vs. the Agentic Retrieval Harness
The post includes a comparison table highlighting the differences:
| Dimension | Naive / single-shot RAG | Agentic Retrieval Harness (Index v2) |
|———–|————————|————————————–|
| Retrieval flow | One vector search per query | Multi-step tool loop: find → retrieve → read/grep |
| Search modes | Vector similarity only | Hybrid semantic search, keyword, and regex grep |
| Context | Fixed top-k chunks | Agent reads full files or windows on demand |
| Freshness | Static index | Persistent pipeline with sync and versioning |
| Precision control | Mostly hidden | `top_k`, `score_threshold`, `rerank_top_n` exposed |
| Citations | Chunk ids | Visual citations with page screenshots and bboxes |
| Best fit | Short question answering | Long-horizon document tasks |
### Use cases with examples
– **Contract question**: “What notice is needed to terminate the MSA?”
The agent lists files, runs `retrieve`, then `grep`s the exact clause and answers with a citation to the specific page.
– **Due diligence**: An agent can `findFiles` by name, then `readFile` each candidate and cross-check clauses without manually opening every PDF.
– **Versioned policy base**: Because `retrieve` accepts a `file_version` filter, an agent can query a specific version, enabling change tracking over time.
### Reference implementation and interactive demo
The post includes an **interactive demo** modeled on the `legal-kb` repository. It simulates an agent working over a small legal knowledge base with six sample documents and intents such as termination, confidentiality, payment terms, non-compete, liability cap, and governing law.
The demo shows:
– How the agent calls `findFiles` first.
– How `retrieve` returns relevant chunks with scores.
– How `grepFile` confirms exact wording.
– How answers are grounded in documents and include **visual citations** that open a screenshot of the source page with bounding-box highlights.
### Key takeaway
`legal-kb` demonstrates that **agentic retrieval is not just better search—it is a persistent data pipeline with tooling, versioning, and precision controls** designed for long-horizon, document-heavy tasks. By treating documents like a filesystem and exposing generic tools, the Retrieval Harness enables agents to handle complex legal workflows reliably and transparently.
**Original source**: [LlamaIndex blog – Legal Knowledge Base & Agentic Retrieval Harness](https://blog.llamaindex.ai/legal-kb-agentic-retrieval-harness-index-v2-llamaparse-8a2b2c8f7b3d)
