**Building a Robust, Auditable RAG Pipeline: Dispatched Loops Between Document Bricks**
Enterprises need RAG that is reliable, repeatable, and explainable. In the previous articles of this series, we introduced four foundational “bricks” for document intelligence: parsing, question parsing, retrieval, and generation. Each brick works well on its own, and earlier articles showed how to upgrade them individually. However, real questions trigger multiple patterns at once, and that’s where a disciplined orchestration layer becomes essential. This article shows how to compose those bricks into a single, auditable workflow using bounded feedback loops and a explicit dispatcher—without giving control to an unauditable LLM agent.
A single compliance question, such as “What are all the Categories under GOVERN, and which one covers supply chain risk?” illustrates the challenge. On the surface it looks simple, but inside the pipeline it fires three patterns at once:
– TOC retrieval to land on the right section,
– Listing aggregation to enumerate every category (not just the most-cited ones),
– A synthesis step to identify which category covers supply chain risk.
Each pattern carries its own iteration logic—re-retrieval, re-generation, or re-parsing—and if the stopping condition is left undecided, every new question type becomes another fragile, bolted‑on special case.
Rather than handing this decision to an LLM agent, we keep control in code. The solution is a dispatcher that translates the parsed question and document profile into an explicit plan, plus bounded loops that decide, in code, how far each pattern may iterate. This article builds that dispatcher, the feedback loops, and the composition layer that ties the four bricks together.
Where This Article Fits in the Series
Think of this as the workflow rung on a five‑rung ladder. At the baseline (Article 1), a single pass connects parsing, retrieval, and generation. The upgraded pipeline (Article 9) adds typed answers and feedback fields but still runs a single pass. This article introduces a bounded loop and a dispatcher, turning the pass into a controlled, multi‑pattern workflow. Higher rungs move the loop into the LLM (agentic control); this article deliberately stops two rungs below, keeping the loop in readable, auditable code.
The Runnable Companion
The companion notebook at doc‑intel/notebooks‑vol1 lets you run pdf_qa_loop on a question that fails its first pass, print the IterationRecord history, and watch should_continue decide when to stop. The full pipeline is also available in the public companion repository.
The Composite Pipeline in Practice
In production, questions stack patterns. For example, “List the obligations of the seller, including any referenced standards” combines listing with two‑hop reference resolution. The iteration mechanics ride alongside these composite patterns, managed by the orchestrator.
This fits into a five‑rung progression:
– Baseline: one‑pass RAG.
– Upgraded: richer single‑pass parsing and typed answers.
– Workflow: bounded loops plus a dispatcher.
– Multi‑intent: classifying user intent and routing to the right pipeline.
– Agentic: letting the LLM choose steps (future, out of scope here).
This article focuses on the workflow rung, where the orchestrator selects patterns, runs them, and uses feedback to decide retries.
A Single Composition Layer
A composite pipeline needs exactly one new home: the composition layer. It contains:
– The dispatcher that selects patterns,
– The feedback machinery that reads typed generation flags,
– One flow per intent.
Everything else—parsing, retrieval, generation—stays untouched in its own module. On disk, a pipeline/ folder sits next to the brick modules (see the article’s diagrams). Inside, code is organized by intent and document format, communicating via typed objects such as ParsedQuestion, DocumentProfile, AnswerWithEvidence, and IterationRecord. A senior engineer can trace any request from prompt to highlighted answer in minutes.
The Orchestrator in Detail
The orchestrator performs three steps:
1. Read the parsed question and decide which patterns to activate.
2. Run the active patterns in a sensible order (TOC first, then retrieval, then two‑hop, then listing aggregation).
3. Manage feedback loops, using generation flags to decide retries while respecting a hard iteration budget.
Concrete orchestration logic lives in pdf_qa_loop. It starts with a cheap parse layer, builds a document profile, asks decide_pipeline_patterns which patterns fire, runs a single shared pass, then hands the result to iterate_with_bound. The loop machinery reads typed flags such as needs_iteration, complete_answer_found, and pending_references to decide whether to retry and how.
Feedback Loops: Signal, Trigger, Action
The composite pipeline’s distinguishing feature is treating the answer as provisional and having the system critique it. Two big feedback rails let the system react:
– Incomplete answer → expand retrieval scope or activate a new pattern.
– Unstructured context → re‑parse flagged pages.
Each loop has three design surfaces:
– Signal: LLM self‑assessment fields, programmatic checks, or external validation.
– Trigger: specific flags such as complete_answer_found=False or pending_references non‑empty.
– Action: targeted retries like re‑retrieval with expanded keywords or adaptive parsing.
Bounding the loops is critical. We use max_iterations (typically two to three), plus termination checks for stable candidates, stable keywords, and decreasing confidence. Drift detection ensures expanded keywords stay close to the original question.
Anti‑Patterns and Auditability
Keep these out of the orchestrator:
– Iterating on confidence alone.
– Unbounded keyword expansion.
– Re‑parsing more than necessary.
– Hiding iteration from the audit trail.
– Letting the LLM decide whether to loop.
Every iteration produces an IterationRecord, giving compliance and debugging teams a clear, line‑by‑line account of what triggered each retry and why.
The Dispatcher
The dispatcher is where explicit routing decisions are encoded. It turns the parsed question and document profile into a stable activation map. Rules evolve with production experience, but each rule has an accompanying test case. Because the dispatcher is pure code, it remains reviewable and auditable.
Worked Example: A Listing‑Plus‑References Question
The article walks through a question about regularization in the Transformer paper. The parsed question activates listing aggregation and two‑hop references. The first pass lists three regularization techniques and flags missing Table 3. The second pass fetches Table 3 via two‑hop retrieval, closes the pending reference, and returns a complete, cited answer. Cost profiling shows that generation dominates runtime, and that smart dispatching avoids paying for every pattern on every question.
Dispatched RAG vs Autonomous Agents
It’s helpful to distinguish three uses of the word “agentic”:
– Marketing label for any pipeline beyond simple embeddings.
– Feedback‑driven control where the LLM produces signals but code decides when to retry.
– Fully autonomous agents where the LLM chooses tools and actions at runtime.
This article adopts the second stance. The LLM contributes signals; the dispatcher and loop logic live in Python. This keeps the system reproducible, auditable, and cost‑effective—critical for enterprise document intelligence. Autonomy has its place, but for governed document workflows, dispatched RAG provides the right balance of structure and adaptability.
Conclusion
The key architectural insight is simple but powerful: decisions belong in code, not in the LLM prompt at runtime. The orchestrator selects patterns, the feedback machinery converts generation signals into bounded retries, and the dispatcher routes questions to the right patterns. This keeps the pipeline explainable, maintainable, and aligned with enterprise governance.
Future articles will extend the pipeline outward: scaling to a corpus (Part IV) and adding evaluation, monitoring, and security in production (Part V).
