Earlier than Full Agentic RAG: Know How You Determine, and the Parsing Strategies You Decide From

Each RAG vendor now labels their document-parsing loop agentic. Open the code and it’s nearly at all times the identical factor: a rule-based dispatcher that reads just a few file indicators, picks an ordered plan of strategies, runs every in sequence, and folds the outputs. The LLMs stay inside particular person leaves (a heading validation loop, a imaginative and prescient reader on figures, an OCR post-processor). No LLM on the dispatch layer decides what to run subsequent. No suggestions loop the place an agent watches an output and re-plans.

That’s precisely what the dispatcher on this article does. So calling it agentic is a stretch, and calling it agentic with out quotes could be promoting the identical buzzword-inflation the remainder of the market sells. The quotes keep.

The trustworthy image, perform by perform:

• detect_document_nature(pdf_path): six deterministic flags learn from line_df / span_df. is_scanned, has_native_outline, has_sommaire, is_composite, has_rich_figures, has_tables_signal. Docstring says it plainly: “deterministic; no LLM”.
• plan_parsing_methods(nature): pure Python if / elif on the character label. Each department returns a hard-coded ordered checklist of MethodStep. No LLM.
• parse_pdf_agentic(pdf_path, llm_parse=…): loops over the plan and calls one adapter per methodology. The llm_parse kwarg is forwarded to strategies that use one (the body-structure loop, the longer term imaginative and prescient reader). The dispatcher itself makes zero LLM calls.
• synthesize_parsing_outputs(step_outputs): DataFrame merges + a _pick_richer heuristic. No LLM.

What true agentic parsing would add: an LLM that reads every methodology’s output, decides whether or not the corpus is carried out or a technique is value re-running with completely different parameters, and provides or drops strategies from the plan on the fly. Software use within the ReAct sense. Suggestions loops. That belongs to Quantity 3 (Agentic Bricks) of the collection, the place every brick will get an agent wrapper that observes, plans, and acts. This text stops one step quick.

So this text builds the trustworthy model of the sample that everybody calls agentic immediately: rule-based routing plus LLM leaves. It is sufficient to shut the parsing brick with a single parse_pdf_agentic(path) name that returns an enriched corpus. Quantity 3 will add the true agent on high.

Consider the paperwork the place a single parsing methodology isn’t sufficient: a 200-page contract with price tables, a quarterly report stuffed with charts and footnotes, a grant utility, a patent, a dense NIPS paper with equations and outcomes tables. Each defeats a special parser. Fitz will get the textual content however misses the desk cells. Docling will get the tables however the define is simply two ranges deep. Azure Format is powerful on each however has no font dimension. Mistral OCR returns markdown totally free however solely whenever you run it in opposition to a scanned web page. The crew wants every of those instruments on completely different paperwork, typically on completely different pages of the identical doc.

The reflex the collection has been constructing for eight articles is one methodology per drawback. That reflex is right on the particular person degree and it doesn’t scale to the doc. A manufacturing caller doesn’t wish to write a change assertion over parsers. They wish to name one perform and get again a corpus dict stuffed to the extent the doc deserves. That’s what this text closes with.

Two regimes coexist and it’s value naming them earlier than the code lands. The primary is what this text builds: the “agentic” one above. Learn the doc as soon as, decide a plan, execute each step, fold the outputs. The doc will get every thing it deserves in a single go, even when that go prices a number of LLM calls and one OCR run. The second is adaptive parsing (a later article within the collection): the caller doesn’t enrich something up entrance; as a substitute, the retrieval brick asks for the pages it wants and parsing runs on demand. The primary is ex ante, the second is lazy. Each belong within the pipeline. This text is simply in regards to the first.

The loop has 4 phases. Every is deterministic, low-cost, and inspectable; the selection of what to run is not an LLM choice. The LLMs enter every particular person methodology on the degree of its personal contract (fastened schema, injected callable, cached JSON), by no means on the dispatcher layer.

In: a PDF path. Optionally a pre-computed nature or plan. Optionally an llm_parse callable for strategies that use one. Out: an enriched corpus dict with line_df, span_df, toc_df, image_df, reference_df, table_df, plus the nature and plan used to supply it, so the run is totally reproducible.

The primary go reads the file’s nature: a small Pydantic mannequin with categorical flags. Low cost indicators solely. File probe plus a single line_df and span_df construct. No LLM.

The six flags are learn from the next indicators:

plan_parsing_methods reads the character and returns an inventory of MethodStep values. Every step names one parsing methodology, offers a one-line rationale, and carries an non-obligatory flag saying whether or not the dispatcher might skip it on error.

Two guidelines that preserve the plan trustworthy:

• Each plan begins with fitz_native. Line and span frames feed each downstream methodology; there isn’t a case the place skipping them saves time.
• Elective flags are used sparingly. image_pipeline and vision_llm_figures are opt-out as a result of they name out to a heavy software; the TOC and structure strategies are required as a result of they’re the load-bearing outputs.

The open-source parsing panorama is large and it retains rising, so deal with this as a catalog, not a hard and fast checklist. Right here is one id card per methodology: its household, its licence, an at-a-glance strip (the place it runs, its velocity, whether or not it calls an LLM, whether or not it exposes typography), the enter it takes, the output it returns, and the place it shines or breaks. Learn them as a set, not a sequence. Each card shares the identical template, so you may evaluate throughout them and decide the proper one for a given doc. Each methodology right here is open supply, and the licence sits on every card. The household is colour-coded: blue for native textual content parsers, teal for structure and desk fashions, amber for OCR readers, violet for construction and TOC restoration.

Native textual content parsers (blue) learn the textual content layer immediately, no mannequin. fitz is a budget baseline, PyMuPDF4LLM turns the identical learn into Markdown for RAG, pdfplumber offers you precise coordinates and dominated tables, and pdfminer.six is the basic low-level extractor beneath.

Format and desk fashions (teal) run a deep-learning structure go and hand again construction. They differ on Markdown versus cell-level tables, and on how a lot GPU they need.

OCR readers (amber) learn pixels when there isn’t a textual content layer. They differ on accuracy and on whether or not they get well tables.

Construction and TOC restoration (violet) rebuild the define. Learn the native define first; get well it from a printed contents web page or from physique headings when the file has neither; or partition the entire doc into typed components with Unstructured.

Methods to learn a card in a single look. The vitals strip is the comparability shortcut. Fashion tells you whether or not the body-typography indicators from Article 5octies (TOC reconstruction from physique typography) can have something to latch onto whenever you chain the body-structure loop after this methodology. LLM tells you if the tactic opens a socket. Runs and Pace set the price range. The diagram and rows beneath fill within the precise enter and output.

Every card can also be saved as its personal PNG underneath book_1/_figures/05_9_agentic_parsing_synthesis/en/identity_cards/, so a single methodology drops right into a slide deck, an analysis report, or a LinkedIn carousel with out cropping a grid.

The dispatcher runs every step by means of a small _run_step(step) shim that adapts to the tactic’s personal signature. Each parsing module (fitz, azure_layout, docling, easyocr, mistral_ocr, toc, toc.body_structure, imaginative and prescient, photos) is already prepared for this name; the dispatcher owns the glue, not the logic.

Two issues value spelling out in regards to the execution stage. First, every methodology carries its personal error dealing with. When an non-obligatory step raises, the dispatcher captures the exception into _error on the step output and retains going. A required step failure aborts the run so the caller sees the true error instantly. Second, each methodology’s uncooked output results in step_outputs alongside the merged corpus. An audit doesn’t must guess what every methodology returned; it may learn the hint finish to finish.

The final stage folds the per-step outputs into one dict with six frames: line_df, span_df, toc_df, image_df, reference_df, table_df, plus a sources checklist saying which methodology contributed every body.

The merging rule is easy: protect each native body verbatim, and when two strategies produced the identical key, preserve the strictly-more-informative one (larger row depend with a suitable column set). The rest concatenates. The total column-level reconciliation between, say, Docling desk cells and Azure Format cells lives within the particular person modules; the dispatcher doesn’t re-implement it.

Here’s what the dispatcher returns on information/paper/1706.03762v7.pdf, a 15-page NIPS paper with a local define. The character comes again as native-with-outline. The plan lists 4 steps: fitz_native (necessary, low-cost), fitz_native_toc (necessary, free from doc.get_toc()), toc_body_structure (advisory, catches the level-3 subsections the define missed) and image_pipeline (non-obligatory). The merged corpus dict carries a 15-row toc_df from the native define, a 1048-row line_df from fitz, and a 3480-row span_df from build_span_df. The picture and reference frames are empty on this run as a result of the paper has no charts and the reference-extraction methodology has not been wired into the plan but.

That run price one line_df construct, one span_df construct, one native TOC learn and one body-structure loop. No imaginative and prescient LLM, no OCR, no Docling. The plan matched the doc.

Agentic parsing is just not free. On a 15-page paper the four-step plan prices just a few hundred milliseconds. On a 300-page contract with tables, figures and no native define the plan grows to seven or eight strategies, a few of which name an LLM; a full run can take a minute and price actual cents. That is effective for paperwork you propose to make use of to their full extent. It’s wasteful for corpus-scale ingestion the place most pages are by no means queried.

The rule I observe: run agentic parsing on chosen paperwork, the contracts a reviewer will really stroll finish to finish, the papers a search outcome surfaced and a reader clicked. For every thing else, the adaptive path (a later article) parses on demand, pushed by the questions the pipeline really receives. The 2 regimes coexist; the caller picks which one the doc goes by means of.

This text closes brick 1. A caller who has a doc and desires every thing it presents calls parse_pdf_agentic(pdf_path) and will get again an enriched corpus dict with line_df, span_df, toc_df, image_df, reference_df and table_df, every stuffed to the extent the doc helps. Each methodology the sooner articles launched now has a spot in a single dispatcher and one enriched corpus. The selection of strategies is deterministic; each LLM lives inside its personal module with its personal contract, not on the dispatcher layer, so the plan is inspectable and the execution auditable. The synthesized dict is what Half III (retrieval) and Half IV (era) learn subsequent.

Two follow-ups. First, the module comes with stubs for azure_layout, docling_local, easyocr_scan, mistral_ocr, vision_llm_figures and image_pipeline. They name the prevailing methodology modules however don’t but carry a full end-to-end integration check on actual paperwork; these exams are on the identical ticket. Second, the adaptive parsing article (a later Vol.1 or Vol.2 piece) will describe the counterpart regime the place parsing runs lazily, pushed by retrieval demand.

Case 4 of Article 5octies (TOC reconstruction from physique typography), the body-structure loop, is without doubt one of the strategies the dispatcher picks from. The parser matrix from that article (fitz / Docling / Azure DI / Mistral OCR / EasyOCR) is precisely the matrix this text’s plan reads. The 2 articles kind the closing pair of the brick.

Earlier within the parsing brick:

Exterior sources (labored examples and prior artwork):