Assemble Every RAG Era Immediate from a Base Immediate Plus the Guidelines Every Query Wants

One immediate per query form, composed at name time. That’s the dispatcher. The choice is the mega-prompt each RAG codebase drifts into: one system immediate dealing with quantities, dates, lists, tables, and free textual content directly. It grows a brand new conditional clause on each name (“if the reply is a date, use ISO 8601; if an quantity, ISO 4217; if an inventory, one merchandise per component…”), the mannequin reads all of it each time, and two months in no person remembers which clause was added for which case.

The dispatcher we’ll construct replaces that mess. Contract: a ParsedQuestion is available in; three issues come out: the schema (picked from ANSWER_REGISTRY by expected_answer_shape), the system immediate (a hard and fast BASE plus the fragments the transient requests), and the consumer immediate (query + key phrases + labeled passage strains). It calls the mannequin, persists the total uncooked response on the hint, returns a typed end result. Including a brand new form provides one fragment; including a brand new constraint provides one fragment; nothing combinatorial.

The options we dominated out: one mega-prompt with each clause at all times current (wastes tokens, exhausting to debug when a table-formatting clause bleeds into an quantity reply), or N impartial per-shape prompts (cleaner per-shape, however duplicates the cross-cutting constraints (format, distinguish, must-disambiguate) throughout each template, and forces re-syncing on each change).

The dispatcher reads a ParsedQuestion produced by the query parsing brick. The complete schema is relational (nested Pydantic objects, not a flat transient): key phrases (typed objects, not strings), expected_answer_shape, decomposition, scope_filters, an execution plan, parsing notes, plus the 2 preparations for the following bricks (retrieval: RetrievalQuery and era: GenerationBrief).

This text reads three elements of this object: expected_answer_shape (picks the schema and the form fragment), era (a quick carrying format constraints, disambiguation between shut candidates, and values to differentiate), and key phrases (echoed within the consumer immediate so the mannequin can flag which of them appeared within the retrieved passages). The boundary is sharp: form detection lives in query parsing; template meeting lives right here.

We construct ParsedQuestion inline under; the bundle’s present src/docintel/query/parse_question.py ships solely the minimal model. The schema and the dispatcher get promoted to the bundle as soon as the query parsing brick lands its full implementation.

class Key phrase(BaseModel):
    textual content: str
    weight: float = 1.0
    supply: Literal['direct','llm_expansion','expert_dictionary'] = 'direct'
    is_regex: bool = False

class RetrievalQuery(BaseModel):
    main_query: str
    rewrites: checklist[str] = []
    anchor_keywords: checklist[str] = []
    section_hint: str | None = None
    scope: str = 'default'

class GenerationBrief(BaseModel):
    original_question: str
    format_constraint: dict[str, str] = {}
    disambiguation: str | None = None
    must_distinguish: checklist[Distinction] = []

class ExecutionPlan(BaseModel):
    use_toc_navigation: bool = True
    use_keyword_retrieval: bool = True
    use_embeddings: bool = False
    iterate_on_feedback: bool = True
    max_iterations: int = 3

class ParsedQuestion(BaseModel):
    original_question: str
    key phrases: checklist[Keyword]
    expected_answer_shape: Literal['text','amount','date','boolean','list','table']
    decomposed_subquestions: checklist[str] = []
    activations: ExecutionPlan = ExecutionPlan()
    parsing_notes: checklist[str] = []
    suggested_clarification: str | None = None
    retrieval:  RetrievalQuery
    era: GenerationBrief

The consumer’s query phrasing scopes retrieval, with none new flag on the pipeline. When the query says “on web page 1”, “pages 5 to 7”, or “on the Pricing sheet”, that pointer rides on ParsedQuestion.structural_hints. Retrieval reads it and filters the search area. No chunk_strategy="passthrough", no bypass argument, no particular short-doc path. The operator controls scope by writing it contained in the query.

The sector carries one checklist per format. This text is PDF-only, so pages_hint is the one which issues right here; the sibling fields for the opposite codecs the collection reaches later keep out of scope:

class StructuralHints(BaseModel):
    pages_hint: checklist[int] | None = None  # PDF; a sheet/slide equal follows for later codecs

Single, vary, and checklist phrasings collapse to the identical flat checklist at parse time: “web page 1” turns into [1], “pages 5 to 7” turns into [5, 6, 7], “web page 2 and web page 7” turns into [2, 7]. Retrieval then filters with one expression, page_df[page_df.page_num.isin(pages_hint)], and doesn’t department on the form of the trace. Hinted pages are stored even when no key phrase matches them: the consumer pinned them explicitly, that’s the reply floor.

The short-doc case is the place this earns its maintain. When the supply is a CV, a one-page bill, or a 1-2 web page memo, the entire doc matches within the mannequin’s context window. The operator writes “Extract these fields from web page 1 of this CV”, “Learn web page 1 and return the bill line gadgets”, “On web page 1, checklist each named celebration”. Query parsing extracts pages_hint=[1]; retrieval filters line_df to strains on web page 1 (which on a one-page doc is each line); era reads the entire doc and runs the requested schema. The pipeline form is an identical to a 1000-page corpus question that occurs to land on a single web page: identical code, identical audit chain, identical contract.

The identical mechanism extends to the opposite codecs the collection reaches later: a sheet title or a slide quantity scopes retrieval the best way a web page quantity does right here. These codecs are out of scope; the purpose that carries over is that the scope pointer rides on the query, not on a pipeline flag.

The failure mode to keep away from is the other transfer: including key phrase filtering or embedding similarity on a one-page doc that doesn’t want it, and watching the filter drop fields the LLM would have caught. The schema does the field-by-field work on the LLM’s consideration; the corpus to filter is just too small for any retrieval sign so as to add worth. The structural trace within the query is the one sign the pipeline wants.

The BASE is shape-agnostic: it encodes the contract that holds for each name: cite, kind, fail actually. The fragments are concern-specific: one per form (quantity, date, checklist, desk, boolean, textual content), one per cross-cutting constraint (format, distinguish, disambiguation, decomposition). The dispatcher composes solely what the transient asks for; nothing else.

Article 8A framed why this issues: the mannequin predicts a believable continuation, it doesn’t look something up, so we anchor each declare to a supply line quantity the pipeline can verify moderately than to prose the mannequin can clean over. That anchor is just nearly as good because the quantity. The GLOBAL_LINE rule in BASE isn’t boilerplate. The primary time we ran this on the Consideration paper, the mannequin returned line_start=33 for a quote that lived at world line 267 (web page 6, line_in_page=33). The mannequin had picked the third column of the user-prompt passage rows as an alternative of the primary as a result of the columns had been unlabeled. Spelling it out within the system immediate and labeling the columns within the consumer immediate (part 1.3) made the bug go away. When a passage row carries a number of integer columns, the mannequin will choose whichever it appears like except you say which column holds the road quantity.

BASE = """You reply questions strictly from the offered doc passages.

Guidelines:
- Use solely data from the passages.
- Each merchandise in `gadgets` should carry at the least one Span citing supply line numbers.
- A Span is a contiguous (line_start, line_end). Use a number of Spans on one merchandise
  when the supporting proof is cut up throughout non-adjacent areas.
- IMPORTANT: Span.line_start and Span.line_end MUST be the GLOBAL_LINE worth
  (the FIRST column of every passage row), NOT the per-page line_in_page.
- If the passages don't include the requested reply, return gadgets=[],
  answer_found=False, and clarify in caveats what was or wasn't discovered.
- Set complete_answer_found=False when the reply exists however is partial.
- If passages battle, set conflicting_evidence=True and floor in caveats.
- If a passage seems to be malformed (damaged desk, OCR garble), set context_structured=False.
"""

The form fragment doesn’t exchange the schema alternative: the schema is already enforced by responses.parse(text_format=...). The fragment steers the mannequin’s extraction technique: “return one AnswerItem per component of the checklist”, “foreign money MUST be legitimate ISO 4217”, “don’t convert if the doc quotes a unique foreign money, set answer_found=False as an alternative”. Schema enforces kind at decoding; fragment steers extraction at immediate time.

SHAPE_FRAGMENTS = {
    "textual content":    "Use `textual content=...` per merchandise. Keep near supply phrasing.",
    "quantity":  "Fill `quantity = Quantity(worth, foreign money, unit)`. ISO 4217.",
    "date":    "Fill `date = DateValue(iso, authentic)`. iso = YYYY-MM-DD.",
    "checklist":    "ONE merchandise per component. Every merchandise carries its personal Spans.",
    "desk":   "TableValue(headers, rows). Rectangular. ONE merchandise per desk.",
    "boolean": "True / False. Conditional solutions go in caveats.",
}

def format_fragment(constraint: dict[str, str]) -> str | None:
    if not constraint:
        return None
    guidelines = []
    if "foreign money" in constraint:    guidelines.append(f"Foreign money MUST be {foreign money!r}.")
    if "interval" in constraint:      guidelines.append(f"Interval MUST be {interval!r}.")
    if "date_format" in constraint: guidelines.append(f"Dates as {date_format!r}.")
    return "Format constraints:n- " + "n- ".be a part of(guidelines)

def distinguish_fragment(distinctions: checklist[Distinction]) -> str | None:
    if not distinctions:
        return None
    strains = [f"- {d.this!r} is NOT {d.not_!r}. Return only {d.this!r}." for d in distinctions]
    return "Watch out with these distinctions:n" + "n".be a part of(strains)

The dispatcher reads the parsed query, picks the schema from ANSWER_REGISTRY[parsed_q.expected_answer_shape] (or an specific answer_schema=... override, for customized shapes: see part 1.5), composes BASE + the related fragments, and returns the (immediate, utilized) pair. The utilized checklist goes into the hint so a unsuitable format six months later is traceable to the precise set of fragments that had been composed. By no means to “the agent determined”.

def build_system_prompt(parsed_q: ParsedQuestion) -> tuple[str, list[str]]:
    elements:   checklist[str] = [BASE]
    utilized: checklist[str] = ["BASE"]

    elements.append(SHAPE_FRAGMENTS[parsed_q.expected_answer_shape])
    utilized.append(f"SHAPE:{parsed_q.expected_answer_shape}")

    transient = parsed_q.era
    if frag := format_fragment(transient.format_constraint):
        elements.append(frag); utilized.append("FORMAT")
    if frag := distinguish_fragment(transient.must_distinguish):
        elements.append(frag); utilized.append("DISTINGUISH")
    if transient.disambiguation:
        elements.append(f"Disambiguation: {transient.disambiguation}")
        utilized.append("DISAMBIGUATION")
    if parsed_q.decomposed_subquestions:
        elements.append("This query decomposes into ...")
        utilized.append("DECOMPOSITION")

    return "nn".be a part of(elements), utilized

The consumer immediate is a skinny shell: the query, the unique key phrases (so the mannequin can mark which had been present in passages), and the candidate strains. The column header GLOBAL_LINEtpagetline_in_pagettext is repeated as a one-line reminder proper earlier than the information: identical kind the BASE rule makes use of, in caps. Two reminders in two locations sound redundant, however they’re what mounted the per-page-vs-global numbering bug from part 1.3. Low cost insurance coverage for a recurring failure.

def build_user_prompt(parsed_q: ParsedQuestion, filtered_line_df: pd.DataFrame) -> str:
    df = filtered_line_df
    if "overall_line_num" not in df.columns:
        df = df.reset_index(drop=False).rename(columns={"index": "overall_line_num"})
    strains = "n".be a part of(
        f"{int(r.overall_line_num)}t{int(r.page_num)}t{int(r.line_num)}t{r.textual content}"
        for r in df.itertuples()
    )
    keyword_strs = [k.text for k in parsed_q.keywords]
    return (
        f"Query: {parsed_q.original_question}nn"
        f"Authentic question key phrases: {keyword_strs}nn"
        "Passages (TAB-separated: GLOBAL_LINEtpagetline_in_pagettext).n"
        "Cite through Span.line_start = GLOBAL_LINE (first column).nn"
        f"{strains}"
    )

One name vs okay calls: A small architectural alternative hides inside build_user_prompt. When retrieval returns okay=3 candidate chunks, we will hand all three to the mannequin in one mixed name, or name the dispatcher sequentially, chunk by chunk, stopping the second we have now what we want.

The 2 modes have very completely different value profiles:

• Mixed (one name with all okay chunks). The mannequin sees every part directly, can cross-reference throughout passages, emits per-passage citations naturally through gadgets: checklist[XItem] (one merchandise per discovering, every with its personal Span). Price: one round-trip, with the total context dimension. That is the default when the reply may be synthesized throughout chunks (an inventory of exclusions scattered throughout pages, a definition plus its instance footnote).
• Sequential with early termination (one name per chunk, cease on success). Course of chunks in retrieval rank order. After every name, verify answer_found=True and complete_answer_found=True: if each, ship and skip the remaining. Price in one of the best case: one chunk’s price of context. That is the fitting transfer when the reply is a single truth prone to reside in a single place (an quantity, a date, an individual’s title, a sure/no), so the top-ranked chunk nearly at all times has it. Saves 2/3 of the tokens at okay=3.

Two different circumstances power sequential no matter query form: every chunk is giant sufficient that combining would push the context previous the 70-80% margin (part 1.5), or the chunks are heterogeneous in a approach that breaks one schema (a contract part vs its modification vs its schedule, every with its personal validation guidelines).

The combined-vs-sequential choice is made upstream, in query parsing, not right here. Query parsing already classifies the query (answer_shape, answer_type, ambiguity, and answer_context for the way a lot surrounding textual content to learn). The routing trace sits alongside on the identical ParsedQuestion, in a top-level chunk_strategy: "mixed" | "sequential" subject. Retrieval then reads that trace and decides what to drag (one tight anchor vs broader span). Era, on this article, receives the ultimate checklist of chunks plus the technique, and simply executes it. All selections are upstream; this text solely runs the loop.

Three concrete payoffs from this composition design:

• Modifying a format touches one file: Change the rendering of quantities (at all times two decimals, at all times trailing foreign money) → edit SHAPE_FRAGMENTS["amount"]. Nothing else strikes. The Quantity kind construction is enforced individually by Pydantic.
• Including a constraint touches one file. A brand new query kind begins asking for “the latest worth when the doc offers a historical past” → add a prefer_recent_fragment(transient.prefer_recent), department within the dispatcher. Different shapes are untouched.
• Auditability is a free output: end result.meta["fragments_applied"] lists precisely which fragments had been composed for this name. A unsuitable format six months from now’s traceable to a mis-detected form (query parsing subject) or a buggy fragment (this text’s subject).

Temperature 0. RAG era is extraction, not inventive writing. Reproducibility issues greater than variation.

At all times persist the total uncooked response on the pipeline hint. Tokens, mannequin model, request id, end cause, system fingerprint, the rest the SDK exposes: all of it lives in utilization, mannequin, id, system_fingerprint. Holding solely output_text is the bug; storing the entire payload prices just a few KB per name and saves hours of forensic work later. The other mistake is frequent: estimating tokens with an area tokenizer earlier than the decision. tiktoken drifts throughout mannequin variations, and also you burn native CPU to recompute what the API returns without spending a dime.

Preserve margin. Cap utilization at 70-80%. Fashions degrade effectively earlier than the exhausting restrict: fill 127k of a 128k window and reply high quality, instruction-following, and reasoning fall off collectively. In case you’re constantly filling greater than that, the issue is upstream: retrieval is returning too many passages, the schema is just too giant, or the query wanted extra aggressive scope filtering. The sign to look at is utilization["input_tokens"] / model_max_input over a window of latest calls, not a one-shot pre-call estimate.

In manufacturing, the uncooked responses go to a responses desk or object retailer keyed by request id. Six months later, when a consumer reviews “the reply for query X was completely different”, you may pull the precise request, the precise response, the precise mannequin model, and reconstruct what occurred. With out the uncooked payload, that dialog is inconceivable.

def generate(parsed_q: ParsedQuestion, filtered_line_df, consumer,
             answer_schema: kind[BaseModel] | None = None) -> GenerationResult:
    Reply = answer_schema or ANSWER_REGISTRY[parsed_q.expected_answer_shape]
    system, utilized = build_system_prompt(parsed_q)
    consumer = build_user_prompt(parsed_q, filtered_line_df)
    resp = consumer.responses.parse(
        mannequin=model_chat,
        enter=[{"role": "system", "content": system},
               {"role": "user",   "content": user}],
        text_format=Reply,
        temperature=0.0,
        retailer=False,
    )
    reply = Reply.model_validate_json(resp.output_text)
    return GenerationResult(reply=reply, meta={
        "schema_used": Reply.__name__,
        "fragments_applied": utilized,
        "template_version": "v1",
        "raw_response": resp.model_dump(mode="json"),
    })

When the reply is twenty typed fields as an alternative of 1 sentence, push the AnswerWithEvidence wrapper right down to the sector degree. The schema turns into a Pydantic class whose each subject is a FieldExtraction[T]: every carries its personal typed worth plus the span the mannequin learn it from. The LLM fills the entire tree in a single structured name; downstream code reads profile.electronic mail.worth for the sector and profile.electronic mail.web page / line_start / line_end for the quotation. The one-claim AnswerWithEvidence stays the default; that is the escalation path for multi-field extraction in regulated contexts (HR triage, credit score selections, healthcare consumption, bill ingestion).

The collection already does this on the merchandise degree. AddressItem from Article 8A (the reply contract) wraps an Handle worth with its spans. AmountItem wraps an Quantity with its spans. DateItem wraps a DateValue with its spans. The merchandise is worth plus the place the worth was learn. The identical sample, utilized on the subject degree moderately than the list-element degree, is what this part makes specific.

The primary reflex is one name per subject. Every name is a qa.ask(pdf, "what is the candidate's electronic mail?", response_format=str). Audit is clear (one log line per subject) however the doc context is distributed N occasions and the invoice scales linearly. On a one-page CV with many fields, that is the costliest form that exists.

The second reflex is one name returning a flat JSON with out proof. {title: "...", electronic mail: "...", cellphone: "..."}. The price drops by an element of N as a result of the doc is distributed as soon as. The provenance disappears solely; a unsuitable subject can’t be traced again to a particular line. A candidate who appeals has no reply to which line of my CV did the mannequin learn?

The third reflex is one name with a top-level proof block. {profile: {...all fields...}, spans: [...]}. Higher, however the quotation is per-answer, not per-field. The reviewer sees an inventory of pages however can’t map “this web page backs the e-mail” to “that web page backs the present position”. When a single subject is unsuitable, the entire proof block turns into suspect.

Not one of the three extends AnswerWithEvidence actually. The proper transfer is to push the wrapper right down to the sector degree.

The primitive is a generic Pydantic class that wraps a typed worth with its personal proof:

from typing import Generic, TypeVar
from pydantic import BaseModel, Subject
T = TypeVar("T")

class FieldExtraction(BaseModel, Generic[T]):
    """Per-field analogue of AnswerWithEvidence. Every subject carries
    its personal typed worth plus the span the LLM learn it from."""
    worth: T | None = Subject(..., description="Typed worth or null if not discovered.")
    quote: str = Subject(default="", description="Verbatim line(s) from the parsed supply.")
    web page: int | None = Subject(default=None)
    line_start: int | None = Subject(default=None)
    line_end: int | None = Subject(default=None)
    discovered: bool = Subject(default=True)
    caveat: str = Subject(default="")

The output schema turns into a Pydantic class whose each subject is a FieldExtraction[T] occasion. The LLM fills the entire tree in a single structured name. For the CV case from a bonus article on rule-based fields:

class CandidateProfile(BaseModel):
    title:          FieldExtraction[str]
    electronic mail:         FieldExtraction[str]
    cellphone:         FieldExtraction[str]
    linkedin_url:  FieldExtraction[str]
    current_role:  FieldExtraction[str]
    years_experience: FieldExtraction[int]
    # ... and so forth, twenty-odd fields

Downstream code reads profile.electronic mail.worth for the sector and profile.electronic mail.web page / line_start / line_end for the quotation. The bbox highlighter from Article 8C (validation) runs per subject moderately than per request; every subject will get its personal yellow rectangle. The audit log writes one row per subject if it desires to, one row per request if it doesn’t. Similar primitive, two granularities.

The post-LLM work is the place a JSON turns into a reliable object: model_validate_json parses it, field-level validators implement area codecs, and a model_validator(mode="after") fills any derived fields. The layer that earns its maintain right here is confirm. For each FieldExtraction[T] whose worth isn't None, the quote should truly seem (modulo whitespace) someplace within the parsed supply. A hallucinated quotation reads plausibly to a reviewer however doesn’t exist on the web page. The verifier walks the schema, substring-checks every quote towards the parsed strains, and flags each one that doesn’t seem. Low cost, and it catches the failure mode enterprises concern most: the mannequin invented a justification. It’s the per-field model of the validator inbuilt Article 8C (validation).

def verify_citations(profile: BaseModel, line_df: pd.DataFrame) -> checklist[str]:
    """Return an inventory of (field_path, quote) for citations that do not seem
    in line_df. Empty checklist = each cited quote is within the supply."""
    flat = " ".be a part of(line_df["text"].astype(str)).decrease()
    misses = []
    for field_path, subject in _walk_fields_of_type(profile, FieldExtraction):
        if subject.worth is None or not subject.quote:
            proceed
        if subject.quote.decrease().strip() not in flat:
            misses.append((field_path, subject.quote))
    return misses

A profile that returns with misses == [] is safer than the identical profile returned with out the verify. A profile that returns with two or three hallucinated quotes is one the reviewer should learn alongside the supply. The failure mode is strictly the one the suggestions loop of Article 8C (validation) already handles, utilized per subject.

The remainder of the multi-field story is its personal matter and lives in a devoted bonus article: decomposing a worth into typed slots so a SQL filter can hit it (postcode as its personal subject, not buried in an tackle string), computing derived fields after the LLM learn the uncooked one (an ISO nation code from “Italian”), coalescing mixed-shape questions behind one qa.ask(...) entry level, and mixing this LLM extraction with rule-based fields for regulated circumstances (HR triage, credit score selections, healthcare consumption). The piece that belongs to era is the one above: push the proof wrapper right down to the sector degree, then confirm each quote towards the supply.

Yet one more fragment, added at question time: the validated examples closest to the brand new query, pulled from a financial institution and dropped into the immediate earlier than the decision. It’s an FAQ turned towards the mannequin. A traditional FAQ prepares Q&A pairs for individuals; this financial institution prepares them for the mannequin, aimed on the reply format and the tough extractions. The mechanism reuses the retrieval brick. The dispatcher of part 1 assembles the immediate from BASE plus build-time fragments; right here yet another fragment is added at question time: the examples closest to the brand new query, pulled from a financial institution and dropped into the immediate earlier than the decision. The identical retrieve_pages runs towards example_bank_df as an alternative of the corpus’s line_df, returning 3-5 examples (extra dilutes the brand new query). The financial institution grows because the staff curates validated examples; the immediate picks them up on the following question, no deploy wanted.

Take a concrete failure. A consumer asks “what’s the annual premium?” on a contract whose line 212 prints the premium with a greenback signal ($1,850.50). With no instance, the mannequin copies the supply too actually: it returns the foreign money as "$" as an alternative of the ISO-4217 code "USD". The principles are all in BASE, however the mannequin nonetheless slips on the symbol-versus-code formatting.

The financial institution holds previous questions the staff already answered and checked, one row every: the query textual content, the validated reply JSON, and some tags. Retrieval (towards the query textual content, with the identical embedder the corpus makes use of) pulls the closest previous row, the place precisely that normalization was already settled:

{
  "query": "What was final 12 months's premium?",
  "answer_json": {
    "answer_type": "quantity",
    "gadgets": [
      {"amount": {"value": 1850.5, "currency": "USD"},
       "spans": [{"line_start": 212, "line_end": 212,
                  "quote": "Annual premium: $1,850.50"}]}
    ],
    "answer_found": true
  },
  "tags": ["amount", "premium", "usd"]
}

The dispatcher drops that row into the consumer immediate as a labored instance, proper earlier than the brand new query:

Here's a previous reply within the actual form anticipated.

Q: What was final 12 months's premium?
   (supply line: "Annual premium: $1,850.50")
A: {"answer_type": "quantity",
    "gadgets": [{"amount": {"value": 1850.5, "currency": "USD"},
               "spans": [{"line_start": 212, "line_end": 212}]}],
    "answer_found": true}

Now reply this query in the identical form.
Q: What's the annual premium?

The instance does the educating. It pairs a messy supply ($1,850.50) with the clear validated reply (worth 1850.5, foreign money "USD"), so the mannequin copies that actual mapping as an alternative of re-deriving it from a guidelines paragraph it simply demonstrably ignored. The identical row additionally carries the gadgets/spans nesting and the answer_found flag by demonstration.

Every instance is price its tokens solely when it prevents a particular, recurring mistake:

• Format examples repair normalization slips. The premium case above: a clear previous quantity stops the mannequin from returning "$" for the foreign money as an alternative of the ISO code "USD". Attain for one at any time when the supply format and the goal format differ and the mannequin retains copying the supply.
• Extraction examples repair a behaviour. An IBAN printed throughout two strains, with a previous row that joined the halves into one worth, stops the mannequin returning solely the primary line. The instance is the spec for how to extract, not simply what form to return.
• Query-parsing examples repair disambiguation. A previous obscure query with its resolved parse (“the guarantee bit” → target_field="warranty_duration") reveals a brand new obscure query which subject to land on, as an alternative of guessing warranty_terms or warranty_exclusions.

The price is actual, so maintain it conditional. Skip the fragment when there is no such thing as a financial institution but, when the brand new query is in contrast to something within the financial institution (an out-of-distribution pull provides noise, not sign), or when the financial institution holds delicate knowledge the immediate shouldn’t carry. The bonus article on FAQ-as-RAG develops the acute model, the place your entire corpus is the instance financial institution.

A ParsedQuestion in, a typed name out: the registry picks the schema, BASE plus fragments compose the system immediate, the consumer immediate carries the query, the key phrases, and the labeled passage strains, and the hint data each choice so a unsuitable format six months later is traceable to the precise set of fragments. Nothing right here trusts the mannequin but. The reply that comes again is checked towards the contract of Article 8A (the reply contract) by Article 8C (validation) earlier than anybody reads it.

The dispatcher composes the immediate at construct time from typed elements; the literature’s primary various fingers that management to the mannequin at runtime. Studying the 2 aspect by aspect is one of the best ways to see what the engineered change buys (reproducibility, bounded value, an auditable hint) and what it offers up (open-ended flexibility). The agentic line, runtime tool-picking on prime of this dispatcher, is follow-up work past this collection.

Similar path because the article: