Tables in PDFs for RAG: Don’t Flatten the Grid

The quantity you want sits in a desk, on the intersection of a row and a column. Flatten the PDF to textual content and that intersection is gone: the label lands in a single place, the worth in one other, and the mannequin is left guessing which quantity belongs to which row. Tables are the place naive parsing quietly loses the reply.

This text is a bonus in Enterprise Doc Intelligence, a collection that builds an enterprise RAG system from 4 bricks. Tables in PDFs: a diagnostic and 5 composable operations that hold the grid, as a substitute of a choice tree.

🧭 New to the collection? Each article on this collection sits on our two In direction of Knowledge Science writer pages, Angela Shi and Kezhan Shi. That’s the shortest option to see what is roofed and the place this one sits.

📓 Runnable companion notebooks are on GitHub: doc-intel/notebooks-vol1.

The usual RAG pipeline reads a PDF, chunks it into textual content, embeds the chunks, and retrieves the closest match. The pipeline handles bullet factors and paragraphs advantageous. The second the reply to a query lives inside a desk cell, the usual pipeline begins hallucinating numbers, and sometimes no one notices till an auditor opens the supply doc.

This text is about why tables break enterprise RAG, and what to do as a substitute. It’s positioned as a bonus as a result of tables contact all 4 bricks (doc parsing, query parsing, retrieval, technology) and not one of the essential articles owns the subject alone. Article 5 mentions fitz.find_tables() with no technique. Article 10 escalates adaptive parsing with no table-specific cascade. Article 15 (getting ready the corpus) extracts fields from tables with out saying how you can retrieve from one. This bonus consolidates the fragments into one coherent therapy.

1. Why tables break the pipeline

A desk in a PDF will not be a desk within the knowledge sense. It’s a set of rectangles drawn on a web page, with textual content positioned in cells, usually with out specific row or column markers. The parser has to reconstruct the grid from spatial geometry. Typically it succeeds and also you get a clear DataFrame. Typically it doesn’t.

When it doesn’t, three issues go mistaken without delay. First, the row and column construction is misplaced, so the LLM downstream sees a stream of values with no relational which means. Second, the header is commonly solely on the primary web page of a multi-page desk, so pages two onward turn into numerical noise. Third, the line-level quotation self-discipline of Article 8 breaks down as a result of there isn’t a option to level at “row 47” when row 47 was by no means reconstructed as a row.

All three failures share a root. A desk is knowledge that somebody put right into a structure format as a result of the distribution format required it. The skilled who created the schedule knew it was knowledge. The parser that flattens the desk into textual content destroys what the producer of the doc already had. The correct transfer is to not deal with tables higher as textual content. It’s to revive them to their native structured type as early as doable and deal with them as knowledge from then on.

2. 4 methods to signify a desk

The identical desk can dwell within the pipeline at 4 completely different ranges of construction. Selecting the correct stage for every desk is the primary design choice, earlier than any operation runs. The 4 ranges should not alternate options to argue between within the summary. Every is the appropriate reply for a particular mixture of desk dimension, schema stability, and query form.

A. Row-as-line in line_df is the default. It’s what the parser produces naturally, and it’s sufficient for any query whose reply reads from the desk the identical means it reads from prose. Every row of the desk turns into one row of line_df with _type="desk" and the textual content rendered as a Markdown pipe row (| col1 | col2 | col3 |). The road retains its bounding field on the web page, so highlighting and quotation work the identical as for prose. Downstream bricks see “traces that occur to be Markdown-shaped” and by no means department on table-ness. That is what the Azure Doc Intelligence parser emits right this moment. A query like “what’s the deductible for property protection?” is answered from the Markdown row instantly, retrieved like some other line, cited like some other line. Most tables in mixed-content paperwork cease right here, as a result of no operation downstream wants to deal with the columns by identify.

B. Separate table_df. When you want to function on the 2D form, you carry the desk out of line_df into its personal DataFrame, with column headers preserved as DataFrame columns and rows as DataFrame rows. The line_df retains a placeholder line pointing on the desk by id. Three operations require this: concatenating a desk that continues throughout 5 pages with the header solely on the primary, projecting to 2 columns out of fourteen as a result of the query asks about one 12 months, filtering to the rows whose area matches the scope. None of those are doable at A. As soon as the desk is flattened into Markdown rows, the columns are textually seen however not addressable.

C. Columnar extraction with named, typed columns. Some tables recur throughout paperwork in a secure form: an insurance coverage contract’s premium desk, a monetary assertion’s revenue abstract, a regulatory schedule with fastened fields. These should not “tables in a PDF” anymore. They’re knowledge that the producer occurred to offer as PDF as a result of PDF was the distribution format, and the doc we learn is a structure of knowledge that already had columns and kinds earlier than it grew to become a PDF. The correct transfer is to revive that authentic type. Raise the tables right into a columnar retailer at ingestion time, listed by doc id and desk id. The selection of storage engine is orthogonal: Parquet on disk, DuckDB for in-process queries, Postgres if the corpus warrants an actual database. What issues is that columns are named (policy_number, start_date, premium_amount) and typed (date, decimal, varchar). As soon as that holds, the desk is knowledge. You question it with SQL, you be a part of throughout paperwork, you combination. C is what unlocks corpus-level questions: “what are the entire premiums throughout all my insurance coverage contracts?” requires that each contract’s premium column maps to the identical identify and sort, which is the assure C makes.

D. Columnar however heterogeneous. Typically you need corpus-level addressing however the tables resist a standard schema: completely different distributors, completely different variations, completely different layouts of “the identical” data. The content material lands in a single textual content column subsequent to its metadata (doc_id, web page, table_id). You retain document-level retrieval and full-text search throughout the corpus, however the 2D construction is gone. D is the trustworthy fallback when C’s preconditions should not met. It’s not often chosen proactively. It exhibits up when the workforce tried for C and couldn’t normalize the schemas within the time out there.

The dispatcher picks one stage per desk. Most keep at A. A couple of escalate to B when a continuation or a projection is required. Those that recur throughout paperwork in a identified form get promoted to C at ingestion time. The orphan recurring tables fall to D.

The scale that drive the selection should not mutually unique, which is why a linear choice tree fails. A local, well-parsed desk will also be very lengthy. A multi-page continuation may dwell in a doc the place 80% of the quantity is tabular. A desk the parser failed on may want column projection. Every actual desk sits on the intersection of three or 4 circumstances, and the appropriate reply is a small diagnostic per desk plus a handful of idempotent operations that transfer tables between ranges. That’s what the subsequent two sections describe.

3. The diagnostic: table_df_meta

For each desk detected in a doc, the diagnostic data 5 orthogonal properties. The result’s a small DataFrame (one row per desk) that the dispatcher reads to select the illustration stage (A, B, C, or D) and, when the extent is B, which operations to use.

Parse high quality: Three ranges. Good: the parser returned a clear grid with constant row and column counts (fitz.find_tables() on a local PDF with specific desk borders is the standard case). Partial: the parser returned cells however the grid is irregular (rows of various widths, lacking cells, merged cells misinterpreted), and the phrases have identified bounding containers on the web page. Failed: the parser discovered rectangles however no usable grid, the web page is scanned, or OCR returned textual content with out structural cues.

Dimension: The pair (n_rows, n_cols). The related threshold is “matches within the LLM context with the encircling query and immediate overhead”. Above the funds, projection (O3) turns into necessary; beneath, it’s non-obligatory. The precise cell rely is determined by the mannequin’s context window and the way a lot of it the prose and the system immediate already eat; no fastened threshold travels effectively throughout deployments.

Header standing: Three values. Current: the primary row is detected as header (both by font weight, by border, or as a result of its cells comprise principally quick textual content whereas later rows comprise numbers). Absent: no row qualifies, actually because the producer relied on the first-page header to cowl the subsequent pages too. Continuation: the desk is a continuation of a earlier one and its actual header lives on an earlier web page.

Multi-page continuity: Three values. Autonomous: the desk begins and ends on the identical web page. Continued-from-N: identical column rely, identical column x-positions, no header on this web page, suggesting it is a continuation of the desk on web page N. Continues-to-M: the desk on this web page ends on the backside and the subsequent web page begins with a desk of the identical column construction with no header. The detection rule is geometric (column positions match inside 5 pixels) plus a header examine.

Doc-level context: The ratio of whole desk space to whole textual content space within the doc. The three instance paperwork in part 5 sit at 6%, 13%, and 26% respectively. A doc that crosses roughly half its physique space in tables is one the place the appropriate structure stops being RAG-on-text-with-tables-as-a-special-case and turns into SQL-on-extracted-tables-with-text-as-annotation. The precise crossover is qualitative, not numeric; the diagnostic studies the ratio and the dispatcher reads it alongside the per-table fields.

These 5 columns of table_df_meta are impartial. A local, partial-quality, giant, headerless, multi-page desk in a table-dominant doc is a sound row. The dispatcher reads all 5 fields and composes the response.

4. 5 composable operations

Every operation takes a table_df (or a set of them) as enter and returns a reworked table_df. Most keep inside stage B and produce a cleaner B; O4 is the one promotion from B to C (or D). They’re idempotent: making use of an operation that doesn’t match its precondition is a no-op, so the composition is secure.

O1. Structural reconstruction from positions. Applies when parse high quality is partial and phrase bounding containers can be found. The parser delivered cells however the grid is damaged. The operation rebuilds the grid by clustering phrase positions into column bands (x-coordinate histogram peaks) and into row bands (y-coordinate gaps wider than line top). The cells then snap to the (column, row) grid. This may get better a clear table_df that the native parser missed (the Commodity column in part 5’s CMO instance and the over-split overview within the NIST instance are each O1 candidates). Price: one geometric cross per web page, negligible. Failure mode: irregular tables with merged cells throughout rows defeat the straightforward grid mannequin and set off O5.

O2. Multi-page concatenation with header propagation. Applies when multi-page continuity is continued-from-N or continues-to-M. The operation walks the consecutive tables, detects the run, copies the header from the primary desk of the run to the rows of the continuation tables, and emits a single concatenated table_df with a brand new column source_page to protect provenance. The geometric continuity examine (identical column rely, identical x-positions inside tolerance, headerless continuation) is the precondition. With out this operation, a 200-row schedule cut up throughout 8 pages produces 8 separate table_df objects of which 7 are semantically orphan.

O3. Query-driven projection. Applies when dimension exceeds the context threshold. Reads the query’s scope_filters and concept_keywords (Article 6) and initiatives the desk to the columns whose headers match the query ideas, then filters the rows whose values match the scope filters. A 200-row, 20-column schedule requested “what’s the premium for property protection in California” will get projected to columns [coverage_type, state, premium] and filtered to rows the place state = 'CA', returning possibly 4 rows to ship to the LLM. That is the “filter earlier than retrieval” of Article 17 (querying the corpus), utilized on the desk stage inside one doc.

O4. Columnar extraction (B → C, or B → D). Applies when the desk recurs throughout paperwork in a identified form, or when the doc is dominated by tables that share a schema. The operation lifts the desk(s) into the columnar retailer described in part 2 (stage C), listed by doc id and desk id. Subsequent questions are dispatched to a SQL agent (the sample of Article 17, querying the corpus) as a substitute of the retrieve-and-generate pipeline: the LLM writes SQL, the engine executes, the LLM interprets the consequence. The promotion succeeds at stage C when a standard schema could be recognized throughout the matching tables; it lands at D when the schemas resist normalization.

O5. Imaginative and prescient-LLM fallback. Applies when O1 has failed (or when parse high quality is failed from the beginning). The operation renders the web page area across the desk as a picture and sends it to a vision-capable LLM with a structured immediate asking for a JSON illustration of the desk. Price: at the very least an order of magnitude costlier than O1, which is a free native geometric cross. The precise price is determined by the mannequin and the picture dimension, however the hole is all the time extensive sufficient that O5 should keep a fallback, not a default, or the price compounds shortly throughout lengthy paperwork.

The operations compose. A desk that’s partial + continued-from-3 + giant + query requires filtering runs O1 then O2 then O3. A doc that’s table-dominant with continued tables runs O2 throughout all continuations, then O4 on the consolidated tables. The dispatcher picks the composition.

5. The dispatcher

The dispatcher reads table_df_meta and emits a sequence of operations per desk (or per doc, for O4). It’s deterministic and sufficiently small to slot in one Python file, in keeping with the determine.py sample of Article 13.

Earlier than the three examples, one preliminary level that sits beneath all of them: the parser alternative issues at the very least as a lot because the operation alternative. The identical web page parsed by two completely different instruments can land in very completely different diagnostic buckets, so the dispatcher’s first job is to determine which parser to invoke. Solely then does the per-table operation composition are available in.

Take Desk 3 of Consideration Is All You Want (Vaswani et al. 2017; arXiv non-exclusive distribution license, declared on the arXiv summary web page; knowledge/paper/1706.03762v7.pdf, web page 9). The desk is a hyperparameter-ablation grid with round twenty rows (header, base config, the (A)-(E) variation blocks, the large config) and 13 columns (the variation label, the structure hyperparameters N / d_model / d_ff / h / d_k / d_v, the regularization hyperparameters P_drop / ε_ls, prepare steps, and the metrics PPL / BLEU / params). It’s precisely the type of consequence desk the article ought to reward the reader with. The article additionally parses World Financial institution CMO tables (CC BY 3.0 IGO) and makes use of Azure Doc Intelligence (proprietary, Microsoft’s On-line Providers Phrases) as one of many parsers.

Azure Doc Intelligence on the identical web page recovers all 13 columns cleanly, together with the sparse cells:

The diagnostic reads radically completely different on the identical web page. With Fitz: failed parse high quality (Desk 3 unrecoverable with out O1+O5). With Azure DI: good. The Fitz pipeline has to do actual work to salvage something; the Azure pipeline will get clear table_df without spending a dime and skips O1-O5 completely. Selecting the correct parser up entrance absorbs work the operations in any other case should redo, and Azure’s per-page price is small in comparison with a mistaken reply on a hyperparameter-ablation query.

That is the spirit of Article 10’s adaptive escalation, restricted to the desk case. Begin with a budget parser. When the diagnostic flags a tough desk, escalate that desk (or that web page) to a stronger parser. The article 10 cascade and the B04 operation composition meet right here: the parser cascade decides what stage of table_df you begin from; the operations determine what to do with it after.

Three examples on actual public-domain paperwork. The scan outcomes beneath come from a Fitz find_tables() sweep over each web page; the parsed-table snapshots are produced by quick chunks on this article’s supply; each are reproducible.

Instance 1: NIST Cybersecurity Framework v1.1 (knowledge/nist/NIST.CSWP.04162018.pdf, 55 pages, 28 tables discovered by Fitz, table-area ratio 26%). The Framework Core lives in Appendix A. Pages 30 to 32 maintain three abstract tables of reducing width: the 26 by 8 overview proven above the place Fitz inflates 4 logical columns into 8, then a 17 by 6 refinement on web page 31, then an 8 by 4 cut up on web page 32. From web page 33 onward the principle Core desk runs throughout roughly 20 pages with a secure 4-column form (Perform, Class, Subcategory, Informative References) and 6 to 9 rows per web page. Every continuation web page carries a header row that Fitz detects, however the precise Perform and Class values are clean on most rows as a result of the producer solely writes them as soon as on the high of every block.

The diagnostic reads: partial parse high quality on pages 30-32 (over-split grid), good on pages 33 onward, continued throughout the run, medium whole dimension, doc not table-dominant. Composition: [O1 on pages 30-32 to fold extra columns back to 4, O2 across pages 33-51 to forward-fill the missing Function and Category values and concatenate]. The result’s one table_df with the total Framework Core (simply over 100 subcategories) joined on (page_num, table_id). With out O1 + O2, a query like “what’s subcategory PR.AC-3 about?” hits a continuation row the place Perform and Class are clean, and the LLM has no option to know the subcategory belongs to Shield → Identification Administration and Entry Management.

Instance 2: World Financial institution Commodity Markets Outlook, October 2025 (knowledge/cmo/CMO-October-2025.pdf, 66 pages, 38 tables discovered by Fitz, table-area ratio 6%, two-column doc structure). The Worth Forecasts desk sits on web page 17: 42 rows by 14 columns, one row per commodity, columns for historic years 2023-2024 and forecast years 2025-2027 throughout a number of revisions. Fitz captures the grid for the numeric block cleanly and utterly drops the Commodity column; each row’s first cell comes again empty. The diagnostic reads: partial parse high quality (label column lacking), autonomous (single web page), giant dimension, doc not table-dominant. Composition for a query like “what’s the wheat worth forecast for 2027?”: [O1 to recover the Commodity labels by clustering the left-margin word positions into a column band, then O3 to project to the (commodity=wheat, year=2027) cell]. With out O1, the LLM sees 42 rows of decimal numbers beneath “2027f” with no thought which one is wheat; O3 alone on the damaged parse initiatives to the appropriate 12 months however can’t choose the appropriate row.

The compositions are readable. An audit asking “why did this query return this row” walks the diagnostic, the operations utilized so as, and the ultimate table_df the LLM noticed. Each step is logged. There isn’t a hidden conduct.

6. The query kind modulates the response

As soon as the appropriate table_df is in hand (after diagnostic and composition), the query form decides how the reply is produced. Three patterns.

Cell lookup: “What’s the premium for property protection in California?” The reply is one cell. The retrieval has filtered to 1 row; technology reads the cell and returns it with line-level quotation again to the supply web page and row. The annotated PDF (Article 1’s highlighting) is reused: the cited cell will get the rectangle.

Vary or column: “What are the deductibles for all protection sorts?” The reply is a column slice. The retrieval has projected to the related columns and should return the entire desk or a filter; technology returns the structured slice as a small markdown desk embedded within the reply schema. Quotation is on the desk stage somewhat than the cell.

Mixture: “What’s the whole premium throughout all states?” The reply is a computation. The retrieval shouldn’t have occurred on this department in any respect. The dispatcher routes the query to the SQL agent (Article 17, querying the corpus), which writes SELECT SUM(premium) FROM schedule WHERE ..., executes, and the LLM interprets the scalar consequence. Quotation is the SQL question plus the consequence, not a passage within the supply doc.

The query kind is the modulator that determines the reply’s form. It doesn’t change the diagnostic or the operations; it solely adjustments what will get returned as soon as table_df is prepared.

7. What stays out of this text

A number of adjoining matters are actual, necessary, and deliberately deferred to follow-up work to maintain this bonus centered.

Cross-document desk becoming a member of: “Examine the premium tables throughout all my insurance coverage contracts.” This requires schema alignment throughout paperwork (the contract-A column premium_amount and contract-B column prime_annuelle have to map). It’s corpus-level work, associated to the sector extraction of Article 15 (getting ready the corpus) however utilized to complete tables somewhat than scalar fields. Future collection territory.

Purely visible tables: Bar charts introduced as tables, color-coded matrices, infographic tables the place the cell worth is encoded by hue or dimension. These want vision-LLM therapy that goes past O5’s “reconstruct the grid” sample. They want semantic interpretation of visible encodings. Observe-up work.

Advanced OCR on tables: Scanned tables with hand-written annotations, tables in non-Latin scripts, tables in paperwork the place the OCR layer is itself badly aligned with the visible layer. These want OCR-specific upstream work that’s its personal subject.

Lengthy structured varieties: Insurance coverage software varieties, tax returns, regulatory disclosure varieties: these are form-shaped, not table-shaped. The excellence issues. A type has named fields with values; a desk has rows of homogeneous construction. Kinds get discipline extraction (Article 15, getting ready the corpus). Tables get the therapy on this article. A doc mixing each wants each therapies routed by the diagnostic.

8. Conclusion

The diagnostic-plus-composition sample is the appropriate response at any time when a parsing drawback is multi-dimensional and the scale should not mutually unique: a linear choice tree drops dimensions that didn’t make the minimize, a diagnostic plus composable operations provides each artefact the therapy its particular properties name for. The skilled who constructed the desk knew it was knowledge; the system’s job is to revive the desk to its native structured type so the skilled’s question lands on it. Dealing with tables as textual content, nonetheless rigorously, drops the row-and-column construction that the skilled’s query is determined by.

9. Sources and additional studying

The vision-based table-structure mannequin behind most trendy desk extractors is Smock et al. (PubTables-1M / Desk Transformer, CVPR 2022). The top-to-end PDF pipeline with specific TableFormer module is Auer et al. (Docling Technical Report, 2024). The layout-detection benchmark behind desk detectors is Pfitzmann et al. (DocLayNet, KDD 2022). The article’s framing: the diagnostic-plus-operations sample, table_df_meta measures desk properties (form, header construction, merged cells, density), and 5 composable operations dispatch on these properties to do actual work on the grid.

Earlier within the collection:

• Doc Intelligence: collection intro. What the collection builds, brick by brick, and in what order.

What works, what breaks

• Baseline Enterprise RAG, from PDF to highlighted reply. The four-brick pipeline finish to finish: PDF in, highlighted reply out.

• Embeddings Aren’t Magic: The Predictable Failure Modes of RAG Retrieval. The place embedding similarity wins (synonyms, typos, paraphrase), the place it predictably breaks (unknown phrases, negation, term-vs-answer relevance), and how you can use it anyway.

• RAG will not be machine studying, and the ML toolkit solves the mistaken drawback. Why chunk-size sweeps and finetuning optimize the mistaken factor; route by query kind as a substitute.

• From regex to imaginative and prescient fashions: which RAG method matches which drawback. Two axes, doc complexity and query management, that choose the method for every case.

  • 10 widespread RAG errors we hold seeing in manufacturing. Ten manufacturing errors, organized brick by brick, with the repair for every.

Doc parsing

• Constructing Doc Construction with Loop Engineering: Recovering a PDF’s Define from Physique Typography for RAG. Rebuilding the define from physique typography when the PDF ships no contents web page in any respect: six alerts, one bounded loop.

• Earlier than Full Agentic RAG: Know How You Resolve, and the Parsing Strategies You Choose From. The parsing strategies as a listing, and the choice of which to run, earlier than handing the loop to an agent.

Technology

• Most RAG Hallucinations Are Extraction Errors: Seven Patterns for a Typed Technology Contract. Seven recurring methods a mannequin will get the extraction mistaken, and the typed contract that catches every one.

• Loop engineering for RAG technology: an LLM cascade from an inexpensive native mannequin as much as a hosted flagship. Beginning on an inexpensive native mannequin and escalating solely when the reply doesn’t maintain up, measured.

One-document pipelines

• Immediate Engineering Isn’t Sufficient: How 4 Bricks of Context Engineering Cease RAG Hallucinations. Why a greater immediate doesn’t repair a mistaken web page, and what every of the 4 bricks contributes to the context as a substitute.

• Lower an Enterprise RAG Pipeline’s Latency and Price by Calling the LLM Much less, Not by Shopping for a Quicker Mannequin. Slicing a pipeline’s latency and value by calling the mannequin much less usually and cheaper, not by shopping for a sooner one.

• Loop engineering for cross-references: when RAG solutions ‘see Part 7.2’ as a substitute of the particular reply. When the reply says “see Part X”, the pipeline loops again and fetches it.

• RAG workflow and loop engineering: the dispatcher that decides when to loop and when to cease. Suggestions loops, bounded iteration, and the dispatcher, composed into one workflow.

  • Loop engineering for RAG: the small loops inside every step, the massive loops throughout the pipeline. The 2 scales of loop: small bounded loops inside every brick, large generation-triggered loops throughout them.