Constructing Price-Environment friendly Agentic RAG on Lengthy-Textual content Paperwork in SQL Tables

a dependable, low-latency, cost-efficient RAG system on a SQL desk that shops giant paperwork in long-text fields — with out altering the prevailing schema?

This isn’t a theoretical drawback.

In most enterprises, vital enterprise information already lives inside conventional relational databases. Proposals, stories, contracts, articles — all saved in TEXT or LONGTEXT columns — designed for key phrase matching and aggregations, not semantic retrieval.

With the arrival of LLMs, enterprise calls for have developed to structured computation, deep semantic understanding and contextual insights in a pure, conversational method.

For instance:

• What number of initiatives over $1M have been permitted from 2023 to 2025?
• Summarize the most important developments seen in expertise over the past 6 months
• What has been the differentiators of successful proposals in 2025?

They require a retrieval technique that may resolve when to compute, when to semantically search, and when to mix each. On this article, I’ll reveal an Agentic RAG structure that operates instantly on prime of a conventional SQL database — with out schema adjustments — and talk about the design ideas required to make it dependable in manufacturing.

System setup

For this illustration, I’ve used a subset of the Social Animal 10K Articles with NLP dataset, which has a lot of information articles and weblog posts together with metadata. The SQL database created has the next columns — url, title, authors, published_date, article_category, word_count and the full_content.

The title will be thought-about to be a novel identifier (main key) for the content material. The article classes are expertise, enterprise, sports activities, journey, well being, leisure, politics and vogue. The articles are distributed roughly evenly throughout the classes. The LLM used is gemini-2.5-flash and FAISS to index and retailer the vector embeddings. The design is relevant for any selection of LLM or vector database.

Structure

Apart from embedding the uncooked textual content, we mirrored the vector retailer metadata with the identical fields current in SQL (besides the total content material). This permits for Filtering, as we’ll see within the outcomes. For lengthy paperwork, a sliding window chunking and embedding technique will be adopted with the metadata hooked up to every embedding.

for idx, row in df_sql.iterrows():
    content material = str(row['full_content']).strip()
    if not content material:
        proceed
        
    metadata =  IRIS', 'May this foam-filled £7,250 monitor bike rival the Group GB Hope HB.T?', 'Aave, Chainlink, and Binance are concentrating on enhancing the Shapella, whereas Huge Eyes Coin is aiming for a big closing push'], 'question': 'crypto'
    
    doc = Doc(page_content=content material, metadata=metadata)
    paperwork.append(doc)

We constructed two specialised, clever instruments that the ReAct agent can invoke utilizing the next structure. The ReAct (router) agent orchestrates your entire question pipeline by intelligently deciding which software to invoke primarily based on the character of the question. It makes use of the metadata and question context to find out whether or not the SQL software, the vector software, or a hybrid method is most applicable. The next determine depicts the question resolution stream:

The instruments are as follows:

1. search_database (SQL software): Handles questions that require computation, aggregation or advanced logic. It executes SQL queries
2. search_articles (Vector software): Handles questions on content material, subject or particular entities. Accepts a pure language question, and optionally, metadata filters to execute a world semantic search (eg: “articles about children“) or search a subset of the data (eg: “filter_authors=’XYZ’, “query”=”articles”).

As will be seen within the above determine, a question can take the next pathways:

1. For computations (eg; what number of articles..), inequalities/vary (eg: Articles revealed between Jan and Apr 2023) or aggregations (eg; What’s the common phrase depend..), use the SQL software solely.
2. Semantic search, with or with out filters, use the Vector software as defined above.
3. Hybrid question: Hybrid queries are important once you want each structured knowledge (e.g., filtering by date) and unstructured content material (e.g., semantic search on articles). The question has a metadata filter standards (often class or a date vary), for which the SQL software is used to fetch articles. Then the checklist of titles is handed to the Vector software to carry out a semantic search on solely these articles. An instance can be “between march and will 2023 is there any article about mom’s day in vogue“

Outcomes

Listed below are the outcomes of some queries of every kind. It will be fascinating to notice the software pathway adopted in every case.

1. “what are the articles by year and category?” — It is a advanced computation question, so solely the SQL software can be used.

QUERY: what are the articles by 12 months and class ?
TOOL SELECTION:
  - search_database:  echojobs.io', 'Driving knowledge to ship web zero with life cycle assessments', 'The Way forward for Work: Tendencies and Predictions for the Subsequent Decade', 'Dangers of Outsourcing Software program Improvement to Uruguay', 'Kochie Arrest Information Addressed, Affected By Rip-off', 'Listening to worker retention 
DOCUMENTS SELECTED:
  - Software Output: [('1970', 'business', 1), ('2001', 'politics', 1), ('2013', 'politics', 1), ('2018', 'entertainment', 1), ('2020', 'entertainment', 1), ('2020', 'sports', 1), ('2022', 'fashion', 1), ('2023', 'business', 19), ('2023', 'entertainment', 16), ('2023', 'fashion', 16), ('2023', 'health', 18), ('2023', 'politics', 16), ('2023', 'sports', 16), ('2023', 'technology', 18), ('2023', 'travel', 18)]
RESULT:
Listed below are the variety of articles by 12 months and class:
* 1970: enterprise (1)
* 2001: politics (1)
* 2013: politics (1)
* 2018: leisure (1)
* 2020: leisure (1), sports activities (1)
* 2022: vogue (1)
* 2023: enterprise (19), leisure (16), vogue (16), well being (18), politics (16), sports activities (16), expertise (18), journey (18)

2. “what articles about children do you have?” — This doesn’t match any of metadata classes that the schema consists of, so the agent decides to make use of the Vector software with a world semantic search.

QUERY: what articles about youngsters do you've got
TOOL SELECTION:
  - search_articles: 'filter_titles': ['NPR quits Twitter, says Musk-led platform is “undermining our credibility”', 'Crypto.com arena considers rebrand after FTX fallout reignites investor anger', "World's Biggest EV Battery Maker Unveils S", 'What Checks Should You Perform Before Buying a Used Car? - Free Car Mag', 'Preeclampsia Diagnostics Market 2023-2030 Overall Analysis Report 
DOCUMENTS SELECTED:
  - Tool Output: --- Document 1 ---
Source: 
Title: 10 Films That Children Should Watch
Category: entertainment
Date: 2020-11-14 02:30:36
Content: On this children's day, here's a list of stories designed to help young ones understand the world around them, gain life lessons and be dazzled by colourful imagination. It's a great time to be a...
  - 
  - 
  - 
RESULT:
Here are some articles about children:

1.  10 Films That Children Should Watch (entertainment)
2.  Save the Children and THINKMD Expand Partnership to Improve the Lives of Children Globally (health)
3.  St. Lucie County School Board decides to keep challenged books in school libraries (health)

3. “what are the trends in fashion?” — The agent finds the category = fashion, and runs the semantic match using the Vector tool with this filter criteria.

QUERY: what are the trends in fashion
TOOL SELECTION:
  - search_articles: 'filter_titles': ['NPR quits Twitter, says Musk-led platform is “undermining our credibility”', 'Crypto.com arena considers rebrand after FTX fallout reignites investor anger', "World's Biggest EV Battery Maker Unveils S", 'What Checks Should You Perform Before Buying a Used Car? - Free Car Mag', 'Preeclampsia Diagnostics Market 2023-2030 Overall Analysis Report 
DOCUMENTS SELECTED:
  - Tool Output: --- Document 1 ---
Source: 
Title: The Best Thing We Saw in Milan Today: India Mahdavi for Gebrüder Thonet Vienna - Sight Unseen
Category: fashion
Date: 2023-04-18 12:00:00
Content: How to Live With Objects

The Best Thing We Saw in Milan Today: India Mahdavi for Gebrüder Thonet Vienna

Sight Unseen is on the ground at the Milan Furniture Fair right now and we’ll be bringing you load...
  - 
  - 
  - 
RESULT:
The search results mention the following trends in fashion: India Mahdavi for Gebrüder Thonet Vienna, clean make up & skincare, and 80's-inspired jewelry.

4. “tell me technology articles about crypto in 2023” — This is a hybrid query where the SQL tool will be used to get the titles in 2023 for category = technology, then the Vector tool will be invoked with the query = crypto and the title list. The result will be found within that subset.

QUERY: tell me technology articles about crypto in 2023
TOOL SELECTION:
  - search_database: {'query': "SELECT title FROM articles WHERE article_category = 'technology' AND published_date LIKE '2023%'"}
  - search_articles:  IRIS', 'Could this foam-filled £7,250 track bike rival the Team GB Hope HB.T?', 'Aave, Chainlink, and Binance are concentrating on enhancing the Shapella, while Big Eyes Coin is aiming for a significant final push'], 'question': 'crypto'
DOCUMENTS SELECTED:
  - Software Output: [('NPR quits Twitter, says Musk-led platform is “undermining our credibility”',), ('Crypto.com arena considers rebrand after FTX fallout reignites investor anger',), ("World's Biggest EV Battery Maker Unveils S",), ('What Checks Should You Perform Before Buying a Used Car? - Free Car Mag',), ('Preeclampsia Diagnostics Market 2023-2030 Overall Analysis Report | Fortune Business Insights',), ('Spanish startup on ‘mission to save planet’s beer’ from climate crisis',), ('Best third-party apps for Re...
  - Tool Output: --- Document 1 ---
Source: 
Title: Crypto.com arena considers rebrand after FTX fallout reignites investor anger
Category: technology
Date: 2023-04-19 01:15:52
Content: The Crypto.com Arena, formerly known as the Staples Center, is reportedly considering a rebrand after the recent controversy surrounding FTX, a cryptocurrency exchange. The FTX fallout reignited anger among investors, wh...
  - 
  - 
  - 
RESULT:
Here are some technology articles from 2023 that mention crypto:

1.  **Crypto.com arena considers rebrand after FTX fallout reignites investor anger**: This article discusses the possibility of the Crypto.com Arena rebranding due to the controversy surrounding the FTX cryptocurrency exchange.
2.  **Aave, Chainlink, and Binance are concentrating on enhancing the Shapella, while Big Eyes Coin is aiming for a significant final push**: This article mentions the Shapella upgrade and its implications for cryptocurrencies on the Ethereum network, including Binance, Aave, and Chainlink.
3. **Software Engineer, Service Platform at Stripe | echojobs.io**: This article mentions Stripe, a financial infrastructure platform that allows businesses to accept payments.

Key Considerations

As with any architecture, there are design principles to be considered for a robust application. Here are a few of them:

• Tool docstrings vs system prompt: These are two types of instructions that guide the agent behavior in different ways. It is important to use them for the intended purposes without any overlap or conflict for a reliable agent performance. Tool docstring, located inside the @tool decorator, describes what the tool does and how to use it. Besides the tool name, it defines the parameters, types and descriptions.

@tool
def search_articles(query: str, filter_category: Optional[str] = None, ...):
    """Helpful for locating details about particular matters, summaries, or particulars inside articles.
    
    You may filter by metadata for precision:
    - `filter_category`: 'well being', 'tech', and many others.
    - `filter_titles`: Listing of actual titles to retrieve (BATCH MODE).
    - `filter_date`: Printed date (YYYY-MM-DD) for EXACT or PARTIAL match solely.
    ...
    """

• Alternatively, the system immediate intelligently guides the routing technique for the agent, enabling it to resolve when to make use of the SQL software, Vector software or a mix. It’s also probably the most advanced and fragile part of the appliance. It defines how instruments are mixed in hybrid workflows, offers examples of appropriate software utilization, and specifies obligatory guidelines and constraints. To adequately design the system immediate, It’s essential to start with a take a look at case repository of anticipated person queries, present examples within the system immediate, and proceed enriching it for deviations that come up for edge instances throughout operations.

system_prompt = (
    "1. **LISTING/BROWSING QUERIES** (e.g., 'what articles are in politics'):n"
    "   - **ALWAYS use [search_database] to list titlesn"
    "   - DO NOT use [search_articles] without a semantic queryn"
    ...
    "### MANDATORY RULESn"
    "1. **DATE RANGES & INEQUALITIES**: Use SQL first, then pass titles to vector tooln"
    ...
)

• Pre and Submit filtering vector databases: It is a delicate level that may have unintended and hard-to-explain outcomes for particular queries. Think about the next two queries the place the one distinction is the mis-spelt title: “summarize articles about Doo ley in politics on 17th apr 2023” and “summarize articles about Dooley in politics on seventeenth apr 2023“. Each the queries comply with similar paths, whereby the SQL software efficiently selects the titles for this class and date (there may be just one article mentioning Choose Dooley), then the Vector software known as on this title checklist with the question. Surprisingly, for the primary question, the Vector software returns "Tool Output: No documents found matching the criteria." for this minor spelling error even when the checklist has just one article to pick from, whereas for the second question it returns the right article.

QUERY: QUERY: summarize articles about Doo ley in politics on seventeenth apr 2023
TOOL SELECTION:
  - search_database: {'question': "SELECT title FROM articles WHERE published_date LIKE '2023-04-17%' AND article_category = 'politics'"}
  - search_articles: {'question': 'Doo ley', 'filter_category': 'politics', 'filter_titles': ['Judge Dooley Ends Hartford Police Consent Decree Despite Concerns']}
DOCUMENTS SELECTED:
  - Software Output: [('Judge Dooley Ends Hartford Police Consent Decree Despite Concerns',)]
  - Software Output: No paperwork discovered matching the factors.

QUERY: summarize articles about Dooley in politics on seventeenth apr 2023
TOOL SELECTION:
  - search_database: {'question': "SELECT title FROM articles WHERE published_date LIKE '2023-04-17%' AND article_category = 'politics'"}
  - search_articles: {'question': 'Dooley', 'filter_category': 'politics', 'filter_titles': ['Judge Dooley Ends Hartford Police Consent Decree Despite Concerns']}
DOCUMENTS SELECTED:
  - Software Output: [('Judge Dooley Ends Hartford Police Consent Decree Despite Concerns',)]
  - Software Output: --- Doc 1 ---
Supply: 
Title: Choose Dooley Ends Hartford Police Consent Decree Regardless of Considerations
Class: politics
Date: 2023-04-17 05:36:24
Content material: Choose Dooley has ended the almost 50 years of federal oversight of police in Hartford, regardless of continued issues the division nonetheless has not employed sufficient minority officers to mirror the town's giant Black and Hispanic populations.

And the reason being not only a weaker embedding because of incorrect spelling. It’s as a result of FAISS (and Chroma and many others) carry out post-filtering — first do a world seek for the question, after which filter the outcomes for the metadata (= the title checklist). On this case, the right article doesn’t function within the top_k = 3 articles after semantic search. A pre-filtering database, alternatively, would have carried out the semantic search solely on the articles within the title checklist and located the right article even with the inaccurate spelling.

• Can all metadata filters be faraway from the Vector Software?: Sure, it’s attainable, however its a increased value choice, as easy semantic queries with a metadata filter (comparable to class or creator), will turn out to be a hybrid question, requiring two software calls, including to token utilization and latency. A realistic center floor can be to maintain dates (and probably different numeric metadata comparable to phrase counts on this case) within the SQL solely, and mirror all textual content and categorical metadata within the vector database.

Conclusion

Constructing RAG on prime of SQL is just not about including embeddings. It’s about designing the precise retrieval technique.

When structured metadata and long-form content material reside in the identical desk, the actual problem is orchestration — deciding when to compute with SQL, when to semantically search, and when to mix each. Refined particulars like metadata filtering and gear routing could make the distinction between a dependable system and one which silently fails.

With a well-designed Agentic RAG layer, legacy SQL databases can energy semantic functions with out schema adjustments, expensive migrations, or efficiency trade-offs.

Join with me and share your feedback at www.linkedin.com/in/partha-sarkar-lets-talk-AI

Reference

Social Animal 10K Articles with NLP — Dataset by Alex P (Proprietor) (CC BY-SA 4.0)

_{Pictures used on this article are generated utilizing Google Gemini. Dataset used underneath CC-BY-SA 4.0 license. Figures and underlying code created by me.}