TL;DR: Reliability techniques (approaches that improve an LLM’s accuracy by using additional inference, such as feedback-driven retries, ensembling, generator/critic refinement loops, verification passes, and difficulty-aware routing) are spread across academic papers, each locked inside its own custom codebase. We consolidated 28 reliability techniques (21 communication-theoretic methods spanning 6 families, plus 7 established baselines: Self-Consistency, Self-Refine, CoVe, BoN, Weighted BoN, CISC, MoA), all benchmarked against a plain single-pass baseline, under one unified API, with 3 adaptive routers (SemKNN plus two local ACM routers) layered on top. We then demonstrated that dynamically routing the technique per prompt lets you navigate a quality-vs-cost tradeoff curve. In our paper’s benchmark using one specific model lineup — Nemotron and Devstral as the two generators and GLM-5.1 as the judge — the adaptive router achieved roughly 56% cost savings at equivalent quality, or about a 7% quality improvement at equivalent cost, compared to the best fixed method we tested in that same lineup. A single parameter (λ) controls the tradeoff. The general finding (adaptive routing beats any single fixed method) should hold broadly, but the exact numbers depend on the model lineup, and we haven’t yet run the full sweep across other model combinations.

Getting started is as simple as change one import:

python - from openai import OpenAI + from agentcodec.openai import OpenAI

Specify reliability="harq_ir" (or any of the 28 available techniques), and your existing client.chat.completions.create(...) calls retain their native OpenAI response format. Equivalent drop-in shims are available for Anthropic and Ollama.

• GitHub:
• Working paper:

After spending considerable time studying reliability methods from the research literature, we kept running into the same frustration: every paper comes with its own bespoke codebase, its own prompt format, its own scoring rubric, its own model wrapper. Trying to answer a question like "should I use self-refine or best-of-N for this task?" turned into a week of integration work per comparison.

The communication-theory perspective is what made everything click: an LLM acts as a noisy channel Y = A(X) + N, and every reliability technique from wireless communications has a direct counterpart in the agent world:

We packaged everything into a single library: 28 reliability techniques (the 7 established baselines are included within that count of 28, not in addition to it), plus the uncoded single-pass baseline they’re all measured against, plus 3 adaptive routers (SemKNN and two local ACM routers) that pick the best technique for each prompt. The full breakdown is in the README.

The minimal version

python from agentcodec import ReliabilityModule

mod = ReliabilityModule.from_dict({ "models": [ # Spatial diversity: two different model families = uncorrelated errors {"model": "qwen3:8b", "base_url": " "api_key": "ollama"}, {"model": "llama3.1:8b", "base_url": " "api_key": "ollama"}, ], "judge": {"model": "gemma3:12b", "base_url": " "api_key": "ollama"}, "critic": {"same": True}, "strategy": {"type": "fixed", "technique": "harq_ir", "params": {"max_rounds": 4}}, })

result = mod.run("Prove the sum of the first n odd integers is n^2.", category="reasoning") print(result.text, result.cost_usd, result.cost_source, result.technique_used)

Swap "harq_ir" for "diversity_mrc", "turbo", "fountain", or any other technique. Same API, same ReliabilityResult structure, same cost-source tier on every output. For production use, switch strategy to routed and the library automatically selects the right technique per prompt (cheap baseline for easy prompts, diversity_mrc for hard ones).

Three things worth highlighting

Beyond the technique catalog, three aspects of the implementation required significant effort:

1. Native async streaming for all but 2 techniques (acm_soft, acm_learned), with role-tagged events. mod.astream() drives AsyncOpenAI / AsyncAnthropic / httpx.AsyncClient end-to-end (no worker-thread bridge) and emits TokenEvents tagged with a role: "answer", "thinking", "draft", "critique", "verification", "candidate", "synthesis". So when you stream a HARQ-IR run, you can render the round-by-round drafts and critiques live, not just the final answer:

python async for ev in mod.astream("Explain QUIC vs TCP."): if isinstance(ev, TokenEvent): if ev.role == "answer": print(ev.text, end="", flush=True) elif ev.role == "draft": print(f"n[draft] {ev.text}") elif ev.role == "critique": print(f"n[CRITIC] {ev.text}") elif ev.role == "thinking": pass # captured to result.thinking_text elif isinstance(ev, FinalEvent): print(f"ndone — {ev.result.technique_used}, " f"thinking_cost=${ev.result.thinking_cost_usd:.4f}")

Parallel-branch techniques fan out concurrently via asyncio.gather. diversity_mrc with two models actually runs them in parallel, and you see per-branch ProgressEvents as each one completes.

2. Thinking-text capture across all backends. Anthropic ThinkingBlock, OpenAI reasoning_content (+ exact reasoning_tokens from usage.completion_tokens_details), Ollama msg.thinking, and inline <think>...</think> tag stripping (DeepSeek-R1, Qwen3, GLM-4.5+, Nemotron) all populate result.thinking_text and split result.cost_usd into thinking_cost_usd + answer_cost_usd. So you can finally see what the o-series / Claude / DeepSeek is actually charging you for.

3. Drop-in compat shims with expose_reliability_stream=True. Default: the shim looks identical to the native SDK, delta.content for the answer, delta.reasoning_content