Essentially the most helpful reply a RAG system can provide is usually “that’s not on this doc.” However when requested a query, a mannequin tends to reply anyway, so the sincere “not discovered” needs to be inbuilt, not hoped for. Getting the system to say no, and to be proper when it does, is tougher than getting it to say sure.
This text is a bonus in Enterprise Doc Intelligence, a sequence that builds an enterprise RAG system from 4 bricks. It justifies the “I don’t know”: a assured unsuitable reply is a bug, a naked no-answer is sort of as dangerous, and every of the 4 bricks has one piece of proof to indicate.
🧭 New to the sequence? Each article on this sequence sits on our two In direction of Knowledge Science creator pages, Angela Shi and Kezhan Shi. That’s the shortest solution to see what is roofed and the place this one sits.
📓 Runnable companion notebooks are on GitHub: doc-intel/notebooks-vol1.
Ask the company chatbot “How a lot electrical energy does AI eat globally?” and it returns “I can’t discover a solution on this doc.” Rephrase and ask once more: similar reply. At that time most customers quit and go to Google.
The system did the suitable factor. The doc it runs on is the World Financial institution’s Commodity Markets Outlook (April 2025 concern; CC BY 3.0 IGO, as declared on the World Financial institution Open Data Repository phrases of use which states the default license for pre-2023 publications), a 63-page quarterly report on oil, agriculture, and metals costs. AI electrical energy consumption will not be in there. There isn’t any reply to seek out. The system was not mendacity.
However the consumer didn’t be taught something helpful. They can not inform whether or not the pipeline seemed in every single place it ought to have, or whether or not it gave up after one shallow embedding search. “I can’t discover a solution” is a verdict that should include a protection. In any other case it reads as failure, not as a truth.
The sequence argued that structured output (Article 8) makes “sure” solutions verifiable, by forcing the mannequin to quote its proof: a quote, a web page quantity, a confidence. The identical logic applies to “no” solutions, and the identical 4 bricks contribute, one piece of proof every. This text walks the 4 bricks as soon as extra, on a query whose proper output is no, and exhibits what every brick has to supply so the no-answer is defensible.
1. The 4 bricks, 4 sorts of proof
The argument compresses to 1 desk.
The remainder of the article walks every row so as, then ties them collectively on the CMO / AI electricity-consumption case.
2. Parsing: the relational tables are the proof
The price of a lacking parse is uneven. A false constructive (extracting noise) makes retrieval messier however is recoverable downstream. A false unfavorable (failing to extract an actual token) is silent: the no-answer verdict will look right, however it’s unsuitable. The reply was there; the pipeline simply by no means noticed it.
The excellent news is that the parsing brick (Article 5B, the relational knowledge mannequin) already produces precisely the information the absence declare wants. The output of parse_pdf will not be a wall of textual content; it’s a small relational set of DataFrames: one per type of factor the doc comprises. line_df is the central desk: one row per textual content line throughout the entire doc, with its web page quantity, bounding field on the web page, and the detected line sort (prose, heading, desk row, determine caption). The opposite tables hold off it.
For a no-answer verdict, we don’t write a brand new parsing operate. We mixture over the tables that exist already:
The three locations an absence declare nonetheless must verify past line_df:
-
Photos that carry textual content: Charts typically maintain their axis labels and legends as image-embedded textual content fairly than typographic spans. The
image_dfregistry lists each such picture; an OCR cross provides anocr_textcolumn to that desk with out touchingline_df. The parsing brick exposes each earlier than and after states; the protection report names what number of photographs had been OCR’d. -
Tables: A quantity in a cell parses as a line whose neighbours are the opposite cells of the identical row, not the phrases above it on the web page. If the reply is “AI consumed 460 TWh in 2024” and the determine sits in a desk with out the phrase AI in any close by cell, a sweep on AI misses it.
object_registryflags which pages host tables so the retrieval brick can do a column-aware sweep there (Article 5 part on desk parsing). -
Cross-references that didn’t resolve.
cross_ref_dflists each “see annex B”, “determine 3.D”, “desk 11.E” point out within the physique. Any row whose goal will not be inobject_registryis a parsing hole: the physique guarantees an object the extraction didn’t discover. Every hole is a candidate spot the place the reply may very well be hiding.
The parsing brick’s piece of proof is due to this fact a small abstract derived from the tables it already produces, not a brand new part. The no-answer verdict can state: “63 pages parsed, 63 with textual content, 0 unresolved cross-references.” That’s sufficient for the consumer to belief the parsing layer did its work.
3. Query parsing: enumerate the vocabulary
The retrieval step is barely pretty much as good because the key phrases it sweeps on. If the professional lists AI however forgets synthetic intelligence, the retrieval misses each line that spells the idea out. The work of the question-parsing brick is to push the key phrase set to exhaustion, with the professional because the supply of reality and an algorithmic security internet beneath.
3.1 The professional is aware of the vocabulary
In enterprise RAG, the professional nearly all the time is aware of the key phrases of their area. That is the easy case and it covers most questions. They record the ideas that the reply would point out, and for every idea they enumerate the variants: synonyms, acronyms, abbreviations, English / French / German types if the corpus is multilingual.
For the CMO / AI electrical energy instance, the professional supplies this desk:
concept_keywords_df schema from Article 6 utilized to a ‘no reply’ query – Picture by creatorThat is precisely the concept_keywords_df schema from Article 6. The identical knowledge construction that drives regular retrieval additionally drives the absence declare. The signature is similar; solely the interpretation of the output modifications.
3.2 Idea clustering as a security internet
The uncommon case is the professional who isn’t positive they listed each variant. The security internet is algorithmic: cluster the corpus tokens by embedding similarity, then expose the clusters that the professional’s key phrases land in, and ask whether or not any cluster member ought to be added.
The output is an inventory, not a call. The professional decides what so as to add. The candidates are a suggestion to think about, not a substitute for the professional’s judgment. The sequence’ editorial place holds: amplify the professional, don’t substitute them. The clustering reveals what the professional could not have considered; the professional validates what belongs within the key phrase set.
The key phrase set, signed off by the professional, is the question-parsing brick’s piece of proof.
4. Retrieval: sweep, not top-k
For a standard query (Article 7), retrieval returns top-k: the few pages or traces almost certainly to comprise the reply. For a no-answer declare, that framing fails. The system can’t show the reply is absent by trying on the top-10 pages. It has to take a look at each web page that mentions any of the ideas.
The form of the retrieval name due to this fact modifications:
The return worth is a DataFrame, not a ranked record. Every row is an EVIDENCE entry: “on web page P, line L, the variant V of idea C was discovered within the snippet S”. The cardinality of the result’s what the no-answer verdict activates:
-
Zero rows for some idea (no variant seems anyplace) → robust proof the doc doesn’t cowl that idea in any respect.
-
A number of rows however by no means co-located (ideas seem on completely different pages, by no means collectively) → medium proof: the ideas exist however no passage talks about their intersection.
-
Co-located rows (a number of ideas in the identical web page or paragraph) → weak proof of absence. Era should have a look at the snippets and determine whether or not they reply the query or solely contact it.
On the CMO April 2025 case, the result’s unambiguous. The total sweep on the key phrase set above returns:
The hit-list is the retrieval brick’s piece of proof. On this query and this doc, the decision writes itself.
5. Era: structured “no reply” with justification
The final brick takes the parse protection report, the validated key phrase set, and the hit-list, and turns them into the precise response the consumer sees. The schema mirrors AnswerWithEvidence from Article 8, however the lively fields are completely different.
The technology step doesn’t invent any of the fields. The schema’s first three fields come straight from the earlier bricks (parse protection, the key phrase set, the hit-list). The mannequin’s job is narrower: choose the rationale, select which mentions to flag as closest, write the one-sentence “why this isn’t a solution” for every, and optionally recommend a reformulation.
On the CMO / AI electrical energy case, the mannequin’s output seems like this:
A consumer studying this output learns three issues directly. The system did search for AI underneath 9 completely different names and located zero hits anyplace in 63 pages. There’s one electricity-related passage, nevertheless it issues India’s coal-driven energy demand, not AI. And if they need the reply, the CMO is the unsuitable corpus to ask. The no-answer is now a helpful response.
6. The place this stops
Three edge instances the framework above doesn’t cowl cleanly, so as of significance.
Partial solutions: A query could have a part of its reply within the corpus and half outdoors it. “How does the EU regulate AI underneath the AI Act, and the way does that evaluate to U.S. coverage?” on an EU-only authorized corpus. The retrieval brick will return hits for the EU aspect and nothing for the U.S. aspect. The appropriate response is neither a assured sure nor a clear no; it’s a structured partial reply that exhibits what was discovered and is express about what was not. The schema for that may be a third sibling of AnswerWithEvidence and AbsenceJustification, with each answer_partial: str and missing_concepts: record[ConceptSearch] fields.
Ambiguous questions: “What about protection?” requested with out context. The system can’t sweep as a result of there is no such thing as a outlined idea set. The appropriate response is a clarification request, not a no-answer. The sign that distinguishes the 2 is the parsing of the query (Article 6): if the query parser can’t extract ideas, the query is the issue, not the corpus.
Hostile or out-of-scope questions: “What’s the which means of life?” on an insurance-policy corpus. The system can appropriately say no-answer, however spending effort on the parse-coverage report and the closest-mentions is wasted. The pipeline ought to have an upstream “is that this query in scope” verify that short-circuits the total sweep when the query parser flags the ideas as unrelated to any corpus tag. Pipeline value issues when this sort of question is frequent.
These instances share a construction: the no-answer schema above is the suitable form for absence claims, however it’s not the suitable form for each “I can’t assist” output. Treating it as one in all three siblings (sure / partial / no) fairly than a single fallback handles the variation cleanly.
7. Conclusion
A defensible “no reply” will not be a single sentence; it’s a chain of proof the 4 bricks produce collectively. Parsing stories protection, query parsing stories the professional key phrase set, retrieval stories the sweep, technology stories the closest point out and why it doesn’t reply. The consumer sees the work and may dispute the parse, the key phrase set, the closest point out.
The symmetry with the remainder of the sequence is the larger level. Sure solutions are verifiable when the schema forces the mannequin to quote its proof (Article 8); no solutions are verifiable when the schema forces the pipeline to reveal its search. It’s the similar self-discipline on the identical bricks, and each outputs land structured.
8. Sources and additional studying
The canonical benchmark establishing “no reply” as a first-class output is Rajpurkar et al. (SQuAD 2.0, ACL 2018). The framing the place many questions are unanswerable contained in the given passage is Choi et al. (QuAC, EMNLP 2018). The model-side reflection-token analogue of the pipeline-side sweep used right here is Asai et al. (Self-RAG, ICLR 2024). The article’s framing: the three-sibling reply schema (sure / partial / no) with a defensible proof chain on the no-answer department, parse-coverage report, expert-keyword sweep, closest-mentions, each step auditable.
Earlier within the sequence:
-
Doc Intelligence: sequence intro. What the sequence builds, brick by brick, and in what order.
What works, what breaks
-
Baseline Enterprise RAG, from PDF to highlighted reply. The four-brick pipeline finish to finish: PDF in, highlighted reply out.
-
Embeddings Aren’t Magic: The Predictable Failure Modes of RAG Retrieval. The place embedding similarity wins (synonyms, typos, paraphrase), the place it predictably breaks (unknown phrases, negation, term-vs-answer relevance), and find out how to use it anyway.
-
RAG will not be machine studying, and the ML toolkit solves the unsuitable drawback. Why chunk-size sweeps and finetuning optimize the unsuitable factor; route by query sort as an alternative.
-
From regex to imaginative and prescient fashions: which RAG method matches which drawback. Two axes, doc complexity and query management, that choose the method for every case.
-
10 widespread RAG errors we maintain seeing in manufacturing. Ten manufacturing errors, organized brick by brick, with the repair for every.
-
Doc parsing
-
Constructing Doc Construction with Loop Engineering: Recovering a PDF’s Define from Physique Typography for RAG. Rebuilding the define from physique typography when the PDF ships no contents web page in any respect: six alerts, one bounded loop.
-
Earlier than Full Agentic RAG: Know How You Determine, and the Parsing Strategies You Decide From. The parsing strategies as a list, and the choice of which to run, earlier than handing the loop to an agent.
Era
-
Most RAG Hallucinations Are Extraction Errors: Seven Patterns for a Typed Era Contract. Seven recurring methods a mannequin will get the extraction unsuitable, and the typed contract that catches each.
-
Loop engineering for RAG technology: an LLM cascade from an affordable native mannequin as much as a hosted flagship. Beginning on an affordable native mannequin and escalating solely when the reply doesn’t maintain up, measured.
One-document pipelines
-
Lower an Enterprise RAG Pipeline’s Latency and Price by Calling the LLM Much less, Not by Shopping for a Quicker Mannequin. Reducing a pipeline’s latency and price by calling the mannequin much less typically and cheaper, not by shopping for a sooner one.
-
RAG workflow and loop engineering: the dispatcher that decides when to loop and when to cease. Suggestions loops, bounded iteration, and the dispatcher, composed into one workflow.
-
Loop engineering for RAG: the small loops inside every step, the massive loops throughout the pipeline. The 2 scales of loop: small bounded loops inside every brick, massive generation-triggered loops throughout them.
-
One doc to a corpus
-
Three Sorts of RAG Corpus, and What It Prices to Construct for the Incorrect One. Three questions that let you know which form a doc assortment has, and the value of constructing for the unsuitable one.
