## Retrieving Answers Hidden Inside Tables
In enterprise document intelligence, the difference between a useful answer and a hallucinated paragraph often comes down to respecting the structure the document already contains. Many documents store their most important facts in tables—insurance schedules, model comparisons, or compliance matrices. Treating those tables as plain paragraphs forces retrieval systems to do filtering the retriever should have done, leading to bloated context and unreliable answers. This article shows how to retrieve at the row level so that a question like “what is the cap for vehicle theft?” returns exactly the row that matters, not the entire table.
### When Retrieval Meets Table Structure
A table on a page is a rectangular unit: a header, body rows, and a footer. Most retrieval systems treat that whole rectangle as one chunk. That works for questions asking about the shape of the table—such as “which events are covered?”—but it fails for targeted questions that expect a single row. A targeted query asks about one specific fact, yet the system returns every row, forcing the language model to pick the right line from dozens of unrelated entries.
This mismatch stems from a conflict between two units of information. The document’s unit is the full table; the user’s unit is often a single row. Retrieval should therefore offer both scales: the whole table for synthesis tasks and individual rows for targeted lookups. The solution described here adds a row-level index built on top of the existing paragraph- and table-level indices, with an intelligent dispatcher that chooses the right scale based on the question.
### Building the Row-Level Index
Row-level retrieval starts from the output of the parsing brick, which already emits tables in a stable format: consecutive pipe-delimited lines (`| … |`) representing the table structure. The key insight is that each body row can be serialized into its own retrievable chunk without changing the parser’s contract.
The pipeline uses two stages:
1. *Grouping*: Identify contiguous pipe-delimited lines that belong to the same table, including handling page breaks.
2. *Serialization*: Extract the separator row to identify headers, handle multi-row headers (a common edge case), and emit one row-level record per body line.
Each serialized row follows the format:
`col: val | col: val | …`
This format mirrors how paragraphs are already embedded and keyword-matched, so it works with existing retrievers without modification. Each row also carries its original location—`page_num` and `line_num`—so answers can be cited precisely, just like paragraph-level evidence.
### Retrieval at Two Scales
With both table-level and row-level indices available, the system can route queries intelligently:
– **Targeted lookup** (e.g., “cap for vehicle theft?”) routes to the row-level index and returns exactly one row with its column headers.
– **Synthesis** (e.g., “which layer types are compared?”) matches multiple rows from the same table. The dispatcher detects this and widens the search to the full table, avoiding under-constrained answers.
– **Mixed queries** (e.g., “compare the vehicle-theft cap with the fire cap”) retrieve two or more rows from the same table and let the generator stitch them together.
A simple widening rule prevents over-truncation: when a significant portion of a table (for example, more than 60% of its body rows) matches the same query, the system treats the match as table-level. This keeps behavior consistent for genuine summaries while avoiding hallucination on partial data.
### Worked Examples
#### Insurance Guarantees Table
Consider a 40-row insurance guarantees table where each row is an event with a cap, deductible, and eligibility condition. A targeted query for “vehicle theft” keyword-matches a single serialized row of about 120 characters, compared to a full slice of nearly 1,000 characters. On a 40-row contract, this represents a roughly 40× reduction in context for the same answer quality.
#### Attention Is All You Need (Table 1)
On the Attention paper’s Table 1, which lists four layer types, a targeted query such as “What is the complexity of self-attention per layer?” retrieves exactly one row (about 120 characters) instead of the entire table (about 500 characters). A synthesis query that asks for all layer types retrieves all four rows and widens to the full table, preserving the correct behavior.
### Handling Edge Cases: Multi-Row Headers
Not all tables are perfectly structured. Some tables use multi-row headers where a logical header spans two visual lines. The serializer detects this pattern by spotting empty cells in the first header line and forward-filling labels. It then checks whether the next line is a sub-header (labels only) followed by data rows. When this pattern is confirmed, the labels are merged into a single header row, the sub-header line is dropped, and the data rows are serialized with the corrected labels. This fix runs only when necessary, so ordinary single-line headers are never altered.
### Integration with the Existing Pipeline
The row-level index complements rather than replaces the paragraph- and table-level indices:
– The serialized row frame is cached alongside the parsed line frame, preserving compatibility with existing retrievers.
– A flag in the retriever enables row-level search only when needed, keeping token usage efficient.
– The question parser activates this flag based on query shape, ensuring the right scale is chosen automatically.
No changes are required elsewhere in the system. Paragraph-level retrieval continues to work as before, and citations still point to exact page and line locations.
### Frequently Asked Questions
**Q: Does this change how paragraph-level retrieval works?**
A: No. Paragraph-level retrieval is untouched and continues to use the same `line_df` contract.
**Q: What happens if a targeted query matches multiple rows from different tables?**
A: The system treats each table independently. If multiple tables match, the generator receives all corresponding rows and must disambiguate using table IDs and context, just as it would with paragraph-level evidence.
**Q: Is row-level retrieval always better than keyword search on paragraphs?**
A: Not always. For questions that span multiple rows or require summarization, table-level retrieval remains more efficient and coherent. Row-level retrieval shines for precise, single-row facts.
**Q: How does the system avoid hallucinating partial tables?**
A: The widening rule only promotes row-level matches to table-level when a substantial fraction of the table matches. Otherwise, the row-level answer is returned as-is, preventing overclaiming.
### Conclusion
Structured data should be retrieved as structure, not flattened into noise. By adding a row-level retrieval scale and a lightweight dispatcher, this approach delivers precise answers from tables while keeping context compact and citations faithful. It complements existing document-intelligence pipelines, requires no changes to parsers or downstream consumers, and significantly improves accuracy for targeted questions on contracts, comparison tables, and reports.
