In my RAG-ING Forward collection I cloud-native retrieval stack: speech and doc processing, chunking, embeddings, Azure AI Search, and an assistant layer sitting on high of it. That collection answered the query I had on the time, which was basically ‘how do I get a language mannequin to reply questions on paperwork it was by no means skilled on’?
The stack nonetheless works. Retrieval-Augmented Era stays probably the most sensible solution to floor a mannequin in personal, domain-specific or lately modified data with out retraining something.[1] You probably have a corpus and also you want solutions from it, RAG continues to be the place you begin.
However I’ve been working that sample for some time now, on initiatives that lasted longer than a demo, and a distinct query began to hassle me:
The system retrieves the identical paragraph, causes over it, produces a superb reply — after which throws all that reasoning away. Tomorrow, somebody asks a associated query, and it does the similar work once more, from scratch, on the identical price, with no assure of reaching the identical conclusion.
My first intuition was to tune the equipment moderately than query it. I experimented with totally different caching mechanisms based mostly on embeddings: recognizing that an incoming query was semantically shut to at least one the system had already answered, and serving the sooner response as an alternative of paying for the complete retrieval-and technology move once more. Semantic caching genuinely helps with price and latency, and I’d nonetheless advocate it. Nevertheless it took me some time to confess what it truly is. It caches solutions, not understanding. The cached response is strictly as disposable as the unique one. Nothing concerning the system’s mannequin of the area has improved, and the second a query falls exterior the similarity threshold, the work begins from zero once more. No matter I tweaked, the primary RAG architectural idea beneath remained the identical.
That isn’t a retrieval drawback. Retrieval is doing precisely what it was designed to do. It’s an structure drawback. There may be nowhere in a regular RAG system for understanding to build up. No quantity of caching, re-ranking or chunking technique fixes that, as a result of all of them optimize the lookup, none of them provides the system a reminiscence.
This text is about constructing that lacking place. It’s the results of my newest work and experimentation round RAG, GraphRAG and agentic reasoning over a corpus of paperwork, the purpose the place the incremental tweaks stopped being sufficient and the design itself needed to change. I’ll current it in three components.
Half I is vendor-neutral. It describes the structure as a design sample: the layers, the item mannequin, the failure modes it exists to outlive, and the governance it calls for. None of it is dependent upon Azure, or on any specific database or mannequin supplier. If you’re on AWS, GCP, or working Postgres with pgvector and an area mannequin, the design nonetheless holds, and I would really like it to be helpful to you.
Half II is the Azure implementation. Service by service, with the reasoning for every selection, actual infrastructure-as-code, and a working FastAPI software you may clone and deploy.
Half III is the demonstration. An artificial property insurer known as Ostermere Mutual, twenty-one interconnected paperwork, and three walkthroughs that present the sample doing one thing a retrieval system genuinely can not.
All the things within the dataset is artificial. Ostermere Mutual doesn’t exist. Neither does the regulator, the coverage, the claims, the folks, the wind zones or the figures. Nothing right here is insurance coverage, authorized, underwriting or claims recommendation, and no web page within the demo represents an actual interpretation of any actual coverage.
Naming notice: present Microsoft documentation makes use of Microsoft Foundry for the unified platform beforehand known as Azure AI Foundry. I exploit Foundry all through, whereas retaining the acquainted Azure service names the place they make the structure simpler to comply with.
Contents
Half I – The design:
1. The place basic RAG works, and the place it stops
2. Retrieval is just not amassed understanding
4. What truly lives within the data layer
6. Writing data is a distinct danger class
8. When you shouldn’t construct this
Half II – Implementing it on Azure:
9. Mapping the layers to providers
Half III – The demonstration:
25. What I’d construct subsequent
The entire mission (software, infrastructure, dataset and a ready-to-open Obsidian vault) is obtainable within the accompanying GitHub repository at github.com/mcekikj/persistent-knowledge-layer, underneath the MIT license.
Half I – The design
1. The place basic RAG works, and the place it stops
A standard RAG circulation is easy sufficient to attract in a single line.
Paperwork are chunked, embedded, and saved in a vector-capable index. A query arrives, the system finds semantically or lexically related chunks, and palms them to the mannequin as context. The mannequin, which is aware of nothing about your small business, is quickly made to appear to be it does.
This solves an actual and vital drawback, and I don’t need to undersell it. The group’s personal data doesn’t must dwell within the mannequin’s parameters. It’s fetched when wanted. That could be a genuinely good thought and it’s why the sample unfold so quick.
However take a look at what the structure is optimized for. It’s optimized for lookup at question time. Each query is handled as the primary query anybody has ever requested.
Now think about an actual person, engaged on an actual drawback, over a number of weeks:
- What’s Precise Money Worth?
- How does it differ from Alternative Value Worth?
- When can recoverable depreciation be paid?
- Which earlier resolution outlined how we deal with depreciation?
- Which doc launched the exception, and why?
- A colleague advised me the brink is 15 years. Is it?
A easy RAG software solutions every of those independently. It retrieves chunks once more, rebuilds context once more, and asks the mannequin to purpose once more. Every reply might nicely be good. However the synthesis is disposable, that means that when the response is delivered, the understanding evaporates. Nothing concerning the sixth query is less complicated as a result of the system already answered the primary 5.
And on the final query – is the brink 15 years? – a retrieval system will do one thing worse than fail. It should discover the chunk that claims 15 years, and it’ll confidently let you know sure.
2. Retrieval is just not amassed understanding
The analogy I hold coming again to is a researcher with a submitting cupboard.
You ask a query. The researcher goes to the cupboard, pulls 4 paperwork, reads the related passages, and provides you a thought-about reply. That is genuinely helpful. Then they put the paperwork again, throw away their notes, and neglect the complete train. Tomorrow you ask a follow-up, they usually begin once more on the cupboard.
A greater researcher does one thing else whereas studying. They:
- hold summaries of the vital sources;
- preserve a web page for every recurring idea, and join the ideas that become associated;
- write down selections, and the reasoning behind them;
- replace a comparability when new proof modifications it;
- document contradictions moderately than quietly resolving them;
- hold a working listing of what they nonetheless can not reply;
- and hold a hyperlink from each declare again to the doc it got here from.
The primary researcher is a query-time RAG system. The second is what I need to construct.
That is near the LLM Wiki sample that Andrej Karpathy sketched out: uncooked sources keep the place they’re, whereas an agent maintains a set of Markdown pages (entities, ideas, comparisons, cross-references) {that a} human can learn and navigate. Obsidian occurs to be a handy window onto the outcome.[2]
It’s simple to get distracted by the Markdown right here, so let me be exact about what the vital thought truly is.
The vital thought is just not Obsidian. It isn’t even Markdown. It’s that data is compiled as soon as right into a sturdy artifact, as an alternative of being reconstructed from uncooked chunks on each request.
That’s an architectural declare, and it has architectural penalties.
3. The three layers
I don’t need to exchange RAG. I need to give it someplace to place what it learns.
The uncooked supply continues to be the strongest factor you’ve everytime you want precise wording, a citation, a clause reference, a newly uploaded doc, or verification of one thing disputed. A generated web page, nonetheless fastidiously maintained, is a derived interpretation. It isn’t proof, and the second you let it fake to be proof, you’ve constructed one thing harmful.
So the design has three layers, they usually reply three totally different questions.
| Layer | The query it solutions | Optimized for |
| Proof | What supply materials is related to this query, proper now? | Recall, precise wording, quotation, freshness |
| Information | What has this method already labored out, and what does it at present consider? | Continuity, relationships, synthesis, reuse |
| Orchestrator | Which of these do I must reply this safely? | Routing, danger, temporal scope |
The excellence is sensible, not philosophical. The proof layer is a retrieval index. The data layer is a maintained, structured, human-readable mannequin of the area. The orchestrator is the factor that is aware of a query about precise coverage wording ought to go to the primary, and a query about why we determined this ought to go to the second.
One rule holds the entire thing collectively, and it’s price stating by itself line:
A data web page isn’t a supply. It’s all the time traceable to at least one.
Break that rule and also you not have a data base. As a substitute, you’ve a group of assured claims that no one can confirm.
4. What truly lives within the data layer
The phrase wiki invitations folks to image a folder of Markdown recordsdata. For a private data base that’s genuinely fantastic. For an software, I desire a structured retailer beneath, with Markdown as a view generated from it.
The thing mannequin I’ve settled on:
Most of those are unsurprising. Three of them are the entire level, and I need to dwell on them.
Resolution – as a result of the why dies first
A call object holds the rule, its scope, its efficient date, its accountable proprietor, and its rationale, with a pointer to the place the rationale got here from.
That final subject issues greater than it appears. In my artificial corpus, an underwriting replace says a roof inspection is triggered above 15 years in a single particular wind zone. The replace doesn’t say why 15. The reasoning exists in precisely one place: an e mail thread between an analyst and a head of underwriting, which explains that between 15 and 20 years, roof displacement within the severe-wind band runs about 3.4x the usual band, and which explicitly warns that folks will learn the quantity with out the qualifier and apply it to the complete guide.
An e mail is just not a coverage doc. No retrieval system ranks it extremely. And in eighteen months, when somebody asks “why is it 15?”, that reasoning is gone — except one thing intentionally preserved it.
Contradiction – a first-class object, not an error
That is the thought I’d most encourage you to remove, no matter else you employ from this text.
Actual corpora contradict themselves. Two groups write two paperwork, each present, neither superseding the opposite, they usually disagree. In any doc set maintained by a couple of workforce for greater than a yr, that is the traditional state of affairs, not an edge case.
A retrieval system handles this catastrophically badly. It retrieves one chunk, or the opposite, or each, after which asks a language mannequin to reconcile them in a single ahead move, underneath a system immediate that advised it to be useful. The mannequin will produce a solution that will probably be fluent, assured, however it can have silently picked a aspect.
Worse, the heuristic it most naturally reaches for is the more moderen doc wins. That sounds smart, and it’s unsuitable. Recency is just not applicability. A more recent doc might have narrower scope, might tackle a distinct product, or might have been written by a workforce with no authority over the query.
So, the contradiction will get its personal object, with a standing, each statements verbatim, their efficient dates, an accountable proprietor, and an express subject: why_not_resolved. The system’s job is to detect the battle and refuse to settle it.
Open query – understanding what you don’t know
The pure companion. Some questions can’t be answered but, actually because they’re blocked by contradiction. An open query that’s seen is secure. The identical query, quietly answered unsuitable, is the factor that leads to a criticism file.
5. The six failure modes this structure exists to outlive
Right here is the sincere check of any structure: what does it do this the less complicated factor can not?
I constructed the demonstration corpus particularly to reply that. It incorporates six distinct traps. A pure retrieval system falls into each one in all them — and falls fluently, producing a solution that reads completely nicely.
5.1 Scoped supersession
A basic rule says examine roofs above 20 years. A later replace says examine above 15 years, however solely in a single wind zone, and just for new enterprise.
Retrieval returns the 15-year chunk. The mannequin says the brink is 15 years. It’s now demanding inspections on tens of 1000’s of strange roofs, and the dealer complaints are totally justified.
The data layer shops a choice with scope: “New enterprise solely. Zone H3 solely.” The quantity by no means travels with out its qualifier.
5.2 Real contradiction
The Claims Dealing with Handbook says trace-and-access prices are coated as normal as much as €5,000 and handlers might authorize with out referral. The Endorsement Catalogue says hint and entry is an elective paid endorsement, not payable except it’s on the schedule.
Each paperwork are present. Neither supersedes the opposite. Completely different groups wrote them.
A RAG system picks one. The data layer raises a contradiction, names an proprietor, and states that no reply is obtainable.
5.3 Terminology drift
Throughout the corpus, the identical idea seems as precise money worth, ACV, money settlement foundation and depreciated worth. A dealer e mail within the dataset actually lists six such phrases and asks whether or not they’re six issues or one.
With out entity decision, your wiki grows 4 separate pages that disagree with one another by omission. With it, one web page, 4 aliases, and a question for any of them lands in the proper place.
5.4 Efficient-date scoping
A declare has a date of loss of 20 February 2026. A rule took impact on 1 March 2026. The rule can not apply to that declare.
This one is my favorite, as a result of a retrieval system has no defence in opposition to it in any respect. Semantic similarity doesn’t encode time. The chunk concerning the 15-year threshold is maximally related to a query about roof age on that declare – and fully inapplicable. The system is just not merely unsuitable, it’s unsuitable in probably the most convincing potential approach.
The repair requires the orchestrator to know that the query is about a date, and to pick out the documentation in drive on that date, together with retaining a outdated doc that was dwell on the time.
5.5 Rationale loss
Coated above. The reasoning lives in an e mail, the rule lives in a tenet, whereas the connection between them lives nowhere.
5.6 Multi-hop
“Why was this declare triaged Degree 1?” requires the declare notes, then the triage guideline, then the water-damage idea, then the coverage clause. 4 hops. Prime-k similarity search doesn’t traverse, it ranks. Typed relationships do traverse.
Put collectively, these six are the argument — not that the wiki is nicer, however that there’s a class of query retrieval solutions confidently and wrongly, and the data layer catches it.
6. Writing data is a distinct danger class than answering
Right here is the factor that took me longest to internalize, and it modified how I take into consideration the entire design.
A unsuitable chat reply impacts one dialog. A unsuitable canonical idea web page impacts each reply that’s later constructed on high of it, for so long as it stays unsuitable, and no one notices, as a result of it appears like data.
The second your system begins writing persistent data, it has crossed from “retrieval software” into “system of document”, and it wants the controls that include that.
All the things is a patch
The mannequin by no means writes to the shop. It proposes a patch. The appliance validates it and, the place the change is consequential, a human approves it.
Notice the place “resolve a contradiction” sits: by no means computerized. If the system might resolve contradictions by itself authority, the contradiction object could be pointless.
Provenance is a series, not a subject
An object whose sources can’t be recognized needs to be deleted, not corrected. You can not repair one thing while you don’t know the place it got here from.
Staleness is a property it’s essential to monitor
A web page may be appropriate on Monday and unsuitable on Friday as a result of a supply was outdated beneath it. So each derived object carries last_validated_at, and the supply it was derived from carries superseded_by. When a supply is outdated, all the pieces derived from it’s marked stale and should not be introduced as present till it’s re-derived.
7. Routing: which layer solutions this?
Not each query wants each layers, and sending all the pieces to each is the way you construct one thing costly and sluggish.
Two issues about this diagram are deliberate.
First, the temporal test belongs earlier than retrieval, not after. For those who rank by similarity first and discard inapplicable outcomes afterwards, the inapplicable paperwork have already consumed your top-k. In manufacturing, put the efficient and outdated dates on the index and filter within the question itself, so the candidate set is constrained earlier than rating. The demo takes a shortcut right here that I ought to come clean with: it applies the date filter instantly after retrieval, which behaves identically at this corpus measurement however would quietly starve top-k on a big one. The precept stands that the demo trades it for an easier index schema.
Second, the contradiction test is a gate on the best way out. It doesn’t matter which route the query took. If the subject is contested, the system stops. In code, that’s roughly:
def question(self, query, requested_mode, top_k, as_of=None):
mode = self.choose_mode(query, requested_mode)
wiki_items = self._search_wiki(query, top_k) if mode in {"wiki", "hybrid"} else []
proof = self.proof.search(query, top_k) if mode in {"proof", "hybrid"} else []
if as_of:
# The date the query is ABOUT - not the date it's requested.
proof = self._filter_by_date(proof, as_of)
# Pull in each contradiction touching a retrieved idea, even when the
# contradiction object itself didn't rank. Somebody asking about hint and
# entry will get the battle whether or not or not they used the phrase "contradiction".
contradictions = self._contradictions_for(wiki_items)
warnings = self._warnings(proof, contradictions, as_of)
context = self._build_context(wiki_items, proof, contradictions, as_of)
reply = self.mannequin.reply(query, context)
... And the instruction that goes to the mannequin is unambiguous:
UNRESOLVED CONTRADICTIONS. You MUST current each positions with their sources and state that the place is unresolved. You MUST NOT select between them, and also you MUST NOT want the more moderen doc - recency is just not applicability.
8. When it’s best to not construct this
I’d moderately you skip this structure than misapply it, so let me be direct concerning the instances the place easy RAG is the higher engineering selection.
Stick with plain RAG when:
- the corpus is small and queried hardly ever, so there may be nothing to amortize;
- customers overwhelmingly need precise supply lookup, not synthesis;
- paperwork churn so quick that any derived synthesis is stale earlier than it’s used;
- there isn’t a cross-session data price preserving;
- it’s a prototype with a brief life;
- ingestion latency has to be minimal;
- your group can not but govern AI-generated persistent data. This one is just not a technical constraint and it’s the one folks ignore.
Construct the data layer when:
- the identical area is queried repeatedly, by folks whose work continues throughout classes;
- cross-source synthesis is regular, not distinctive;
- selections and their rationale should survive employees turnover;
- exceptions, scopes and contradictions truly matter;
- area consultants must see and proper what the system believes;
- an audit path from reply to supply is a requirement, not a nice-to-have.
This can be a workload resolution moderately than a matter of new good, previous unhealthy, and the sincere reply for lots of functions is that you don’t want this.
Half II – Implementing it on Azure
All the things above is intentionally transportable. Now let me construct it correctly on Azure, the platform I work with every day.
9. Mapping the layers to providers
| Accountability | Azure service |
| Immutable uncooked sources | Azure Blob Storage |
| Scanned PDFs, tables, kinds, format | Azure AI Doc Intelligence |
| Chunk, key phrase, vector and hybrid retrieval | Azure AI Search |
| Chat and embedding mannequin deployments | Microsoft Foundry |
| Structured wiki state | Azure Cosmos DB for NoSQL |
| API and orchestration | FastAPI on Azure Container Apps |
| Occasion-driven ingestion | Occasion Grid → Container Apps Jobs |
| Id and secrets and techniques | Entra ID, managed identification, Key Vault |
| Telemetry | Utility Insights |
| Human inspection of the data | Obsidian, over exported Markdown |
10. Blob Storage – the one factor you can’t regenerate
All the things else on this structure is derived. Chunks may be re-chunked. Embeddings may be re-embedded. All the wiki can, in precept, be recompiled from scratch. The unique paperwork can’t be recovered from something.
So they get handled accordingly:
raw-sources/
{workspace-id}/
{document-id}/
original-file.pdf ← by no means rewritten
extracted-content.json ← Doc Intelligence output
ingestion-metadata.json ← what ran, when, which mannequin model
wiki-export/
{workspace-id}/
House.md
Ideas/ · Choices/ · Contradictions/ · Sources/
In Bicep, the half that issues is three properties:
useful resource blobService 'Microsoft.Storage/storageAccounts/blobServices@2023-05-01' = {
guardian: storage
title: 'default'
properties: {
// Originals should survive an ingestion bug. Versioning and gentle delete are the
// most cost-effective insurance coverage obtainable on the one artifact the system can not regenerate.
isVersioningEnabled: true
deleteRetentionPolicy: { enabled: true, days: 30 }
containerDeleteRetentionPolicy: { enabled: true, days: 30 }
}
}
And on the storage account itself, one line that I'd argue for in any manufacturing deployment:
allowSharedKeyAccess: false // no connection strings, everThe wiki-writing course of must not ever have the ability to contact the originals. That could be a provenance, audit and deletion-workflow requirement, and it’s a lot simpler to implement with separate containers and slim position assignments than with good intentions.
11. Doc Intelligence – used selectively
My demo corpus is .txt and .md, so the applying reads it straight. Actual insurance coverage paperwork are scans, kinds, tables and signatures, and the format itself is usually load-bearing. A desk of endorsement limits flattened right into a paragraph is worse than ineffective.
Azure AI Doc Intelligence provides you prebuilt and customized fashions that return textual content, tables, choice marks and construction.[3] My rule of thumb:
- clear textual content and Markdown → a easy parser, no cost;
- machine-readable PDFs → a PDF parser, whether it is genuinely enough;
- scans, kinds, complicated format, tables → Doc Intelligence;
- all the time persist the extracted JSON subsequent to the authentic and hold the web page and span offsets so a quotation can level at a location, not only a doc.
That final level pays for itself the primary time a compliance reviewer asks “the place precisely does it say that?”
12. Azure AI Search – the proof layer
Every listed document carries each searchable textual content and its vector:
{
"id": "INS-SYN-004-0",
"workspace_id": "ostermere-insurance-demo",
"source_id": "INS-SYN-004",
"title": "Excessive-Wind Zone Underwriting Replace",
"content material": "For brand new Hearthmere enterprise in zone H3...",
"chunk_number": 0,
"content_vector": [0.012, -0.008, 0.031, "..."]
} Azure AI Search suits this design nicely for one particular purpose: textual content and vector fields coexist in a single index, and a hybrid question runs the full-text and vector queries in parallel, fusing the rankings with Reciprocal Rank Fusion. Semantic rating can then reorder the highest outcomes.[4]
That issues greater than it’d sound. In insurance coverage, half the queries are conceptual (“what counts as sudden water injury”) and half are lexical (“what does HS-TA-01 cowl”). Vector search is nice at first and unreliable at second: embeddings are identified to be weak with precise identifiers, clause numbers and product codes. BM25 handles these nicely however can not deal with paraphrasing. You need each, and also you need them fused moderately than chosen between.
from azure.search.paperwork.fashions import VectorizedQuery
vector_query = VectorizedQuery(
vector=self.mannequin.embed(question),
k_nearest_neighbors=max(top_k, 10),
fields="content_vector",
)
outcomes = self.shopper.search(
search_text=question, # BM25 leg
vector_queries=[vector_query], # vector leg
filter=f"workspace_id eq '{self.workspace_id}'", # safety trim
choose=["id", "source_id", "title", "content", "chunk_number"],
high=top_k,
) Notice the filter, as a result of it’s doing safety work, not relevance tuning.
⚠️ Vector similarity is just not authorization. Nothing about cosine distance respects your permission mannequin. Safety trimming have to be a tough filter on an listed subject, utilized at question time, on each single question. It have to be utilized identically to the data layer and to the exported Markdown. A wiki web page that synthesizes three paperwork the person can not learn continues to be an information leak, only a properly formatted one.
For bigger programs, built-in vectorization can transfer chunking and embedding into the indexer pipeline.[5] I hold the embedding calls within the software right here purely, so the circulation is seen and explainable in an article.
One sizing notice from truly deploying this: the demo runs on the free Search tier, and vector plus hybrid search work fantastic there for a 21-document corpus. What the free tier provides up is the semantic ranker and managed-identity help on the service itself, each are conditionals within the Bicep, so deal with primary as the ground for manufacturing and free as a superbly great way to validate the design for nothing.
13. Cosmos DB – the data layer
Cosmos DB for NoSQL holds the structured wiki. The entire design suits in three selections.
Partition key’s /workspace_id. Each wiki object carries a kind discriminator, so ideas, selections, contradictions and open questions all dwell in a single container. Meaning a single-partition question can pull a whole workspace’s data with no cross-partition fan-out, which is strictly the entry sample this method has.
partitionKey: {
paths: ['/workspace_id']
form: 'Hash'
} No graph database — but. Folks attain for Gremlin or Neo4j the second they hear “relationships”. I’d push it again. Express relationship information in a doc retailer deal with all the pieces this method truly does: discover neighbours, comply with a typed edge, render an idea web page with its hyperlinks. That’s one or two hops.
A graph retailer earns its place when deep traversal is itself the workload: multi-hop affect evaluation, centrality, path-finding throughout a big community. If you’re not doing that, you’re paying for a second database and a second question language to keep away from writing WHERE c.source_id = @id.
Serverless, for now. Billing follows consumed request models, which fits a demo and a spiky early workload. Transfer to provisioned throughput after you have measured actual RU consumption, not earlier than, and never as a result of the phrase serverless is trendy.[6]
Knowledge-plane entry makes use of Cosmos’s personal RBAC system (SQL position assignments), which is separate from Azure RBAC and catches folks out:
useful resource cosmosDataRole 'Microsoft.DocumentDB/databaseAccounts/sqlRoleAssignments@2024-11-15' = {
guardian: cosmos
title: guid(cosmos.id, identification.id, 'data-contributor')
properties: {
principalId: identification.properties.principalId
// 00000000-...-000000000002 is the built-in Cosmos DB Knowledge Contributor
roleDefinitionId: '${cosmos.id}/sqlRoleDefinitions/00000000-0000-0000-0000-000000000002'
scope: cosmos.id
}
} Mixed with disableLocalAuth: true, there isn’t a key to leak.
14. Microsoft Foundry – fashions now, brokers later
Foundry gives the chat and embedding deployments. The appliance talks to it by means of the Azure OpenAI v1 interface utilizing the usual OpenAI Python SDK, which suggests no api-version parameter to chase each few months:[7]
from openai import OpenAI
from azure.identification import DefaultAzureCredential, get_bearer_token_provider
credential = get_bearer_token_provider(
DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default"
)
shopper = OpenAI(base_url="https://YOUR-RESOURCE.openai.azure.com/openai/v1/", api_key=credential)
response = shopper.responses.create(mannequin="YOUR-CHAT-DEPLOYMENT", enter=immediate) For this demo, the FastAPI app orchestrates explicitly: embed, retrieve, search the wiki, construct context, name the mannequin, validate, write. I did that on objective since each step is seen, and you’ll put a breakpoint in any of them.
The pure evolution is Foundry Agent Service, the place the identical orchestration turns into an agent with a constrained toolset:[8]
search_evidence()
search_wiki()
get_concept()
check_contradictions() ← the gate, as a software
propose_wiki_patch() ← proposes solely; can not apply
apply_approved_patch() ← requires an approval token
export_obsidian_vault() I’d not hand an agent unrestricted database entry on day one. Every software enforces its personal validation, authorization, logging and slim enter schema. Notice that propose_wiki_patch and apply_approved_patch are separate instruments: the agent can attain the primary and can’t attain the second and not using a human in between. That separation is the entire governance mannequin, expressed as an API floor.
For extraction, use structured outputs, which constrain the response to a JSON Schema as an alternative of merely requesting legitimate JSON.[9] The distinction turns into apparent the primary time a manufacturing extraction returns almost-valid JSON.
I can report that from expertise now, as a result of deploying this stack produced two subject notes price passing on:
- Truncated JSON, on the very first doc. gpt-5-mini is a reasoning mannequin, and its reasoning tokens are spent from the identical max_output_tokens price range as the reply. With a 5,000-token cap, the extraction JSON arrived minimize off mid-string. The repair within the repo: a 16,000-token price range, reasoning: {“effort”: “low”} for extraction work, JSON mode (textual content.format: json_object) to constrain the decoder, and one retry. A element that price me two failed seeding runs: the primary full seed succeeded with out JSON mode, then two later runs failed on totally different paperwork. Immediate-only JSON doesn’t fail reliably – it fails intermittently, which is worse. Schema-constrained structured outputs stay the actual repair; that is the pragmatic one.
- A deployment naming quirk. An embedding deployment named identically to its mannequin (text-embedding-3-small) got here up wholesome in each standing test after which returned unknown_model on each single name, on each the v1 and the basic route. An similar deployment named embed-3-small labored on the primary request. Chat deployments don’t present the issue. The Bicep now retains the deployment title and the mannequin title as separate parameters, with this story within the description.
15. FastAPI on Container Apps
The API floor:
GET /well being
GET /wiki listing objects, filterable by kind
GET /wiki/contradictions the contradiction register
GET /wiki/{item_id}
POST /question { query, mode, top_k, as_of }
POST /ingest/textual content
POST /ingest/file
POST /cost-estimate
POST /export/obsidian
POST /export/obsidian.zip One implementation element that price me some debugging time and is price passing on: /wiki/contradictions should be declared earlier than /wiki/{item_id}, or FastAPI matches the trail parameter first and cheerfully appears for a wiki object with the id “contradictions”. Route order is critical.
Container Apps is the proper runtime right here as a result of it’s container-based with out asking me to run Kubernetes.[10] The scaling selection is price a phrase:
scale: {
// For manufacturing, 1 heat duplicate avoids chilly begins; for a demo, 0 prices nothing.
minReplicas: minReplicas
maxReplicas: 3
} minReplicas is a parameter (default 1) as a result of the proper reply is dependent upon what you’re working. In manufacturing, scale-to-zero appears like free cash after which expenses you a chilly begin plus a mannequin handshake on the primary request after each scale-in. For a demo that’s exercised from a developer machine, zero is strictly proper — the validated deployment on this article ran at zero and value nothing whereas idle.
16. Id – no keys, anyplace
The entire deployment runs on a user-assigned managed identification with narrowly scoped roles, and each service has native auth disabled. The Container App will get AZURE_CLIENT_ID in its setting, DefaultAzureCredential picks it up, and no secret is ever issued to the applying.
{ title: 'AZURE_CLIENT_ID', worth: identification.properties.clientId } This issues extra on this structure than in a plain RAG app, and it’s price spelling out why. A retrieval system reads. This technique writes persistent data that different solutions will probably be constructed on. The blast radius of a compromised credential is just not “somebody learn your paperwork” – it’s “somebody edited what your group believes”. Deal with the write path accordingly.
The complete infra/essential.bicep within the repo provisions all the pieces: storage with versioning, AI Search with disableLocalAuth, Cosmos with serverless and SQL position assignments, Foundry with each mannequin deployments, Key Vault, Log Analytics, Utility Insights, the Container Apps setting and the app itself, plus the six position assignments that wire them collectively – and, while you move your personal deployerPrincipalId, a mirrored set on your person account, which is what lets the seeding script and the checks run from a developer machine and not using a single key.
az group create -n rg-wikirag-demo -l swedencentral
az deployment group create -g rg-wikirag-demo -f infra/essential.bicep -p namePrefix=wikirag 17. The ingestion lifecycle
That element within the center is the one folks miss. When the mannequin extracts ideas from a brand new doc, it should see what the wiki already is aware of. In any other case, it invents “Money Settlement Foundation” as a brand-new idea, with out understanding that actual-cash-value already exists with that precise alias, and your data base quietly forks.
Entity decision within the demo:
def _resolve_concept(self, title, aliases):
"""Precise id → title → alias.
With out this, 'money settlement foundation', 'depreciated worth' and 'ACV' every develop into
their very own web page and the wiki fragments into synonyms. The demo stops at alias
matching. A manufacturing system provides embedding similarity and an LLM adjudication
step for the ambiguous center, which is the place the fascinating failures dwell.
"""
candidates = {title.decrease(), *(a.decrease() for a in aliases)}
for idea in self.repository.list_items("idea"):
if idea["id"] == slugify(title):
return idea
identified = {idea["title"].decrease(), *(a.decrease() for a in idea.get("aliases", []))}
if candidates & identified:
return idea
return None The demo ships with a passing check that ingests a doc utilizing the phrase “depreciated worth” and asserts that no new idea web page is created, but it surely resolves onto the prevailing actual-cash-value.
And here’s what occurs with out the stronger decision steps, measured moderately than argued. After I ran all twenty-one paperwork by means of dwell gpt-5-mini extraction on the deployed stack, the mannequin proposed ideas that alias matching might resolve solely partially. The outcome was 149 idea objects the place the curated graph has 19 – a roughly 7x fragmentation issue. The machine-extracted ideas usually are not unsuitable, they’re simply named in methods no alias listing anticipated (“Roof Inspection Requirement”, “EUR 5,000 authorization restrict”). That quantity is the concrete argument for embedding similarity and LLM adjudication within the decision chain.
Half III – The demonstration
18. Ostermere Mutual
The artificial corpus is twenty-one paperwork for an imaginary property insurer. It’s sufficiently small to learn in a day and intentionally engineered in order that the six failure modes from stage 5 are all dwell in it.
| Class | Paperwork |
| Coverage & product | Coverage overview v1.2, outdated v1.1, Part 4 wording (escape of water), endorsement catalogue |
| Underwriting | Roof guideline (20 yrs), H3 replace (15 yrs), wind-zone register, referral matrix, surveyor panel notice |
| Claims | Triage guideline, claims handbook ch.7, three declare recordsdata (CLM-1042, CLM-1108, CLM-1155) |
| Regulatory & compliance | Veyland round (fictional regulator), honest claims dealing with normal, data provenance normal |
| Casual | Two e mail threads, working-group assembly minutes, buyer FAQ |
These casual sources usually are not ornament. The assembly minutes are the place the contradiction is formally recorded as unresolved. The e-mail thread is the place the solely clarification of the 15-year threshold exists. In my expertise, that is precisely how actual organizations work; the principles are within the paperwork, and the explanations are in somebody’s inbox.
A notice on knowledge provenance: each doc within the corpus is artificial and written by me (polished with AI) for this text, so there aren’t any licensing constraints on its use. The dataset ships with the repository underneath the identical MIT license because the code.
Compiled, that corpus produces:
21 sources · 19 ideas · 28 typed relationships · 4 comparisons · 5 selections · 2 contradictions · 4 open questions · 2 processes
19. Walkthrough one – the scoped rule
curl -X POST http://localhost:8000/question -H 'Content material-Kind: software/json' -d '{
"query": "What's the roof inspection threshold?",
"mode": "hybrid"
}'
Plain retrieval finds the H3 replace, which says 15 years, and says “15 years”.
The data layer returns the choice object:
{
"id": "h3-roof-inspection-threshold",
"kind": "resolution",
"abstract": "For brand new Hearthmere enterprise in zone H3, request a roof inspection when the
main roof masking is greater than 15 years previous. Outdoors H3, the final
threshold of greater than 20 years stays in drive...",
"scope": "New enterprise solely. Zone H3 solely. Doesn't apply to in-force insurance policies
written earlier than the zone mannequin existed.",
"rationale": "Between 15 and 20 years, the frequency of whole or near-total masking
displacement within the severe-wind band runs at roughly 3.4x the usual
band on the identical roof age...",
"rationale_source": "INS-SYN-018",
"accountable_owner": "D. Lindqvist (Head of Property Underwriting)"
}The quantity by no means seems with out its scope. And the rationale, recovered from an e mail, is true there, which implies that in eighteen months, when somebody asks why 15, the reply exists.
The e-mail’s creator, by the way, predicted this precise failure in writing:
“Please don't let this develop into a basic 15-year rule. If somebody reads the replace with out the zone qualifier they may apply it to the entire guide, we are going to demand inspections on tens of 1000's of completely strange roofs, and the dealer complaints will probably be totally justified.”
That could be a artificial quote I wrote to make a degree, however I don’t suppose it’s an unrealistic one.
20. Walkthrough two – the contradiction
That is the one I’d put in entrance of a sceptical architect.
curl -X POST http://localhost:8000/question -H 'Content material-Kind: software/json' -d '{
"query": "Is hint and entry coated underneath Hearthmere?",
"mode": "wiki"
}' A retrieval system solutions this. It retrieves a bit (from the claims handbook, or from the endorsement catalogue) and tells you both “sure, as much as €5,000 as normal” or “provided that to procure HS-TA-01”. Each solutions are supported by an actual doc. Each are unsuitable, as a result of the agency doesn’t have a place.
The hybrid system responds:
{
"reply": "The place is NOT SETTLED. The data base holds an unresolved
contradiction masking this query, so no reply is given...",
"warnings": [
"UNRESOLVED CONTRADICTION (con-001): Trace and Access - Standard Cover or Paid
Endorsement? The system will not choose between the conflicting sources.
Owner: Y. Tanaka (Product)."
],
"contradictions": [{
"id": "con-001",
"status": "unresolved",
"accountable_owner": "Y. Tanaka (Product)",
"statements": [
{ "source_id": "INS-SYN-012", "locator": "Chapter 7.3",
"effective_date": "2025-11-01",
"statement": "Trace-and-access costs are covered as standard under Hearthmere,
up to EUR 5,000..." },
{ "source_id": "INS-SYN-008", "locator": "HS-TA-01",
"effective_date": "2026-01-01",
"statement": "Trace and access is an optional endorsement (HS-TA-01)...
Where it does not appear on the policy schedule, trace-and-access
costs are not payable." }
],
"why_not_resolved": "Each paperwork are present. Neither supersedes the opposite. They
have been written by totally different groups. The Endorsement Catalogue is the more moderen
doc, however recency is just not applicability..."
}]
} Each statements. Each sources. Each dates. An accountable human being. And no reply, as a result of there may be no one.
That JSON is demo mode’s deterministic scaffold. Here’s what the deployed stack does with the identical query – gpt-5-mini, dwell, over Cosmos DB and AI Search:
There may be an UNRESOLVED CONTRADICTION related to this query. Don’t deal with the difficulty as settled.
- INS-SYN-012 (Chapter 7.3, efficient 2025-11-01): “Hint-and-access prices are coated as normal underneath Hearthmere, as much as EUR 5,000. Handlers ought to authorize affordable expenditure as much as this restrict with out referral.”
- INS-SYN-008 (HS-TA-01, efficient 2026-01-01): “Hint and entry is an elective endorsement (HS-TA-01), restrict EUR 5,000, indicative premium EUR 24. The place it doesn’t seem on the coverage schedule, trace-and-access prices usually are not payable.”
This contradiction is UNRESOLVED. Accountable proprietor: Y. Tanaka (Product) [con-001].
I can not resolve or state whether or not trace-and-access is roofed underneath Hearthmere due to the unresolved battle above.
The mannequin was not requested to be cautious on the whole. It was handed the contradiction object and one rule, and it adopted the rule.
The second-order impact is the half I discover genuinely fascinating. As soon as a contradiction is a saved object, different issues can rely on it. The corpus incorporates a policyholder query on declare CLM-1155 (is the leak-detection survey price payable?) which can’t be answered till this battle is resolved. So, it’s saved as an open query with blocked_by: “con-001”.
The system is aware of why it can not reply, and it is aware of who has to determine earlier than it might probably.
21. Walkthrough three – the date of loss
Declare CLM-1108: storm injury, roof masking displaced, date of loss 20 February 2026. The roof is eighteen years previous. The property, underneath right this moment’s classification, could be in zone H3.
Ask a retrieval system whether or not an inspection was required, and it’ll discover the H3 rule (15 years, and 18 > 15) and let you know the roof was over threshold, and no inspection was on file. It seems like a discovering. It’s the starting of a wrongful declinature.
As a result of the H3 rule took impact on 1 March 2026. 9 days after the loss. And the coverage was written in September 2025, earlier than the zone mannequin existed in any respect, so the property carried no zone classification on the day the roof got here off.
curl -X POST http://localhost:8000/question -H 'Content material-Kind: software/json' -d '{
"query": "What roof inspection threshold utilized to this property?",
"mode": "proof",
"as_of": "2026-02-20"
}' The as_of parameter is the date the query is about – not the date it’s requested. With it set, INS-SYN-004 is excluded from the candidate set totally. It was not in drive. The relevant threshold was the final 20-year rule, the roof was 18 years previous, it was underneath threshold, and no requirement was breached.
The identical mechanism runs within the different course. Ask about settlement foundation with as_of: “2025-08-01” and the system retrieves the outdated coverage v1.1, as a result of that’s the model that ruled a loss on that date, and marks it:
{
"source_id": "INS-SYN-009",
"in_force_at_as_of": true,
"notice": "outdated right this moment, however in drive on 2025-08-01 - that is the model that
governs a loss on that date"
} Beneath v1.1, depreciation was not recoverable. Beneath right this moment’s v1.2, it’s. Similar time period, identical coverage, reverse reply – and the one factor that distinguishes them is a date that similarity search can not see.
Each behaviours are coated by checks within the repo, and each reproduce on the dwell deployment. At as_of: 2026-02-20 the deployed mannequin answered:
Quick reply: The final >20-year inspection threshold utilized. [INS-SYN-003]
Why: On 2026-02-20 the Hearthmere guideline required an inspection when the first roof masking is greater than 20 years previous [INS-SYN-003]. The H3 15-year threshold … didn’t take impact till 1 March 2026; this coverage was written 14 Sep 2025 (so the H3 rule didn’t apply to it) and the roof was ~18 years previous at loss. [INS-SYN-018] [INS-SYN-013]
And at as_of: 2025-08-01:
- Settlement foundation: Precise Money Worth (ACV) – outlined as alternative price much less depreciation – for eligible constructing claims underneath Ostermere Hearthmere v1.1. [INS-SYN-009]
- Is depreciation recoverable? No. Beneath v1.1 depreciation is just not recoverable after restore. [INS-SYN-009]
They’re the 2 checks I’d hold if I needed to delete each different one.
Yet another, dwell and unscripted: the multi-hop query
Part 5.6 claimed that typed relationships let the system traverse the place similarity search solely ranks. I had not scripted a demo for it, so I requested the deployed stack straight: “Clarify why the CLM-1042 water loss was positioned at triage Degree 1 underneath the property declare triage course of.” The reply walked the entire chain:
- Governing Degree-1 standards: Degree 1 – Normal is outlined as “Minor seen injury, no security concern, no persevering with water entry, and estimated publicity beneath EUR 5,000” [INS-SYN-002].
- How CLM-1042 matches these standards: property remained liveable, no reported electrical fault [INS-SYN-005]; water circulation stopped inside roughly 40 minutes [INS-SYN-005]; injury confined to the water path [INS-SYN-005; INS-SYN-014]; preliminary restore estimate EUR 3,800, underneath EUR 5,000 [INS-SYN-005; INS-SYN-002].
- Supporting operational steerage: the Claims Dealing with Handbook lists indicators of a sudden escape of water … which align with CLM-1042’s info [INS-SYN-012]. CLM-1155 explicitly contrasts CLM-1042 as a datable, sudden failure [INS-SYN-014].
- Notice on provisional standing: protection and exclusions stay topic to a proper protection willpower by a certified claims skilled [INS-SYN-005].
4 sources, 4 hops, and the human-decision boundary preserved on the finish with out being requested for.
One sincere statement from the identical session: my first phrasing of that query (“…and the way does the water injury cowl apply to it?”) received gated by the contradiction – CLM-1042’s proof touches the contested trace-and-access idea, so the system introduced con-001 and declined to make a settled willpower as an alternative of answering the triage half. The gate errs on the aspect of stopping. For a claims workflow I think about that the proper default, however it’s a actual trade-off: an aggressive gate will generally maintain again a solution the person legitimately wanted and tuning that boundary is a part of working the system.
22. Seeing it: the Obsidian vault
The appliance exports the complete Cosmos DB state right into a vault. Fifty-five pages, all generated, nothing hand-written:
obsidian_vault/
House.md
Open Questions.md
Ideas/ 19 pages
Sources/ 21 pages
Choices/ 5 pages
Comparisons/ 4 pages
Contradictions/ 2 pages ← con-001, con-002
Processes/ 2 pages 
Open the folder in Obsidian, and you’ll navigate hyperlinks, examine backlinks, comply with the graph, spot orphan pages, and, most significantly, see what the system believes and inform it that it’s unsuitable.
That final functionality is, I feel, the strongest argument for this whole structure. A vector index is operationally wonderful and fully opaque to a website skilled. You can not hand an underwriter a 1,536-dimension embedding and ask, “does this look proper to you?” You can hand them a Markdown web page that claims the brink is 15 years, however solely in H3, and just for new enterprise, and right here is why, and listed here are the 4 paperwork it got here from.
They may let you know inside thirty seconds whether or not it’s proper. Markdown makes the reminiscence auditable by the individuals who truly know the area. No different a part of the stack does that.
23. The price argument, actually
The data layer prices extra at ingestion. I’m not going to fake in any other case.
A RAG pipeline extracts and embeds every doc as soon as. The hybrid pipeline moreover summarizes, extracts ideas and claims, resolves entities in opposition to the prevailing wiki, generates relationship and comparability patches, validates, and regenerates Markdown. The write amplification is actual, and it isn’t small.
The argument is that this front-loaded price buys down repeated query-time reasoning. So, the query is just not whether or not the wiki prices extra to construct (it plainly does) however whether or not the compilation is amortized throughout sufficient future use.
The mannequin in cost_model/cost_model.py:
Compilation = D × Td × M
Easy RAG = Q × Tr
Hybrid = Q × (Tw + V × Television) With the illustrative defaults – 50 paperwork, 6,000 tokens every, 1,000 questions, 6,000 retrieved context tokens per RAG query versus 1,350 wiki context tokens, uncooked verification on 25% of questions:
| Measure | Tokens |
| Supply corpus | 300,000 |
| Wiki compilation | 501,000 |
| Easy RAG, 1,000 questions | 6,000,000 |
| Hybrid, 1,000 questions | 1,725,000 |
| Context saved | 4,275,000 |
| Break-even | ~117 questions |
Now the caveats, as a result of a chart like this could simply mislead:
- That is token quantity, not value. It ignores output tokens, embedding prices, AI Search capability, Cosmos RUs, Container Apps compute, and Doc Intelligence pages.
- It treats each token as equal. In follow, ingestion and answering can use totally different mannequin lessons, and that’s the place a lot of the actual saving lives, as a result of a small mannequin can reply from concise wiki context whereas a stronger one is reserved for reconciliation and updates.
- It doesn’t assure something. A badly ruled agent that rewrites the entire wiki on each ingestion will erase any saving you modelled. The economics rely totally on disciplined replace insurance policies.
One measured knowledge level, from deploying this precise stack: ingesting all twenty-one paperwork by means of dwell gpt-5-mini extraction and embeddings, working each walkthrough on this article, and regenerating the vault from Cosmos DB price roughly $0.20-0.30 in whole. The idle stack – free-tier search, scale-to-zero Container App, serverless Cosmos – burns about $0.05 a day. At this corpus measurement the compilation price is espresso cash. The economics solely develop into fascinating at scale, which is what the mannequin above is for.
The context-size assumption additionally survived contact with the deployed system. Measured throughout a set of conceptual questions on the dwell stack, the wiki context averaged roughly 500 tokens in opposition to roughly 1,750 for the equal proof context – a 3.5x ratio, in the identical vary because the 4.4x the mannequin assumes. Absolutely the numbers are smaller than the mannequin’s, as a result of the artificial paperwork are brief. The ratio is the half that transfers to an actual corpus. If something, the measured ratio is barely extra conservative than the assumed one, which strikes the break-even later, not earlier — price understanding earlier than you quote the mannequin at a price range assembly.
Run /cost-estimate with your personal assumptions. Then throw them away and use telemetry out of your precise corpus and question combine, priced with the present Azure calculator.[11]
And actually, the token argument is the weakest argument for this structure. The true returns are:
- constant terminology throughout classes and throughout folks;
- selections and rationale that survive the one that made them go away;
- contradictions which can be seen as an alternative of silently resolved;
- an audit path from any reply to its supply;
- a data artifact a website skilled can overview;
- continuity throughout agent classes and throughout mannequin upgrades.
I’d take over a couple of million enter tokens.
24. Governance, restated
As a result of the system writes, it wants to regulate a read-only system doesn’t.
Protect provenance. Each assertion traces to a bit, to a span, to a doc, on the model it was derived from.
Patch, by no means write. The mannequin proposes; deterministic validation and, for something consequential, a human disposes.
Detect staleness. last_validated_at, superseded_by, source_effective_date. When a supply is outdated, all the pieces derived from it’s stale till re-derived.
Trim safety at each layer. Blob paths, search information, Cosmos objects, Markdown exports, agent instruments, caches, telemetry. Constantly. A synthesized web page must not ever floor data the reader couldn’t entry within the supply.
Hold the human resolution human. Within the demo, the AI might summarize an consumption, retrieve coverage proof, determine lacking data, suggest a provisional triage stage and flag conflicting guidelines. It might not decide protection, decline a declare, assess fraud, fee a danger, resolve a contradiction, or inform a buyer a choice has been made.
The artificial working-group minutes comprise the perfect articulation of this that I managed to jot down, and I’ll let it stand because the governance precept for the entire structure:
Handlers at present undertake the assistant’s proposed triage stage in roughly 90% of instances, which is ok, however solely as a result of they’re studying the consumption themselves. Eradicating the handler from the loop removes the factor that makes the 90% reliable.
That’s the entice in a single sentence. An automatic system earns credibility underneath human overview, after which that credibility is used because the argument for eradicating the overview.
25. What I’d construct subsequent
The repo is a baseline, not a product. The gaps I’m most acutely aware of:
- Occasion Grid and queue-driven async ingestion (the demo ingests synchronously as a result of it’s simpler to run regionally).
- Doc Intelligence with web page and span preservation, so citations level at places.
- Strict JSON Schema structured outputs on each extraction and patch.
- Entity decision with embedding similarity and LLM adjudication, not simply alias matching.
- A human approval UI for high-risk patches: proper now the lifecycle exists within the design and the low-risk path exists within the code.
- Foundry Agent Service instruments, with suggest and apply as separately-permissioned surfaces.
- Analysis units for retrieval, synthesis and the arduous one: replace accuracy. How do you check {that a} data base modified accurately?
- Freshness and contradiction dashboards. A contradiction register no one appears at is only a log file.
- Per-tenant safety trimming, finish to finish.
- Mannequin routing by activity complexity and danger.
Quantity 7 is the genuinely open analysis drawback, and I would not have a superb reply to it but.
26. To sum all of it up
RAG is the proof engine of this structure, and nothing right here is its obituary. It provides the mannequin entry to authentic, related, present supply materials, and there’s no substitute for that.
What it doesn’t do is keep in mind.
The data layer provides the factor that was lacking: a maintained, structured, inspectable illustration of what the system has already labored out, with the scopes intact, the rationale preserved, the contradictions seen, and a line again to the proof for each declare.
RAG asks: What ought to I retrieve for this query?
The data layer: What needs to be durably true after processing all the pieces to date?
The orchestrator: Which of these do I must reply this safely, proper now?
On Azure that separation maps cleanly:
- Blob Storage preserves the originals — the one factor you can’t regenerate.
- Doc Intelligence extracts the troublesome content material and retains the spans.
- Azure AI Search shops retrievable, security-trimmed proof.
- Cosmos DB shops the evolving ideas, relationships, selections and contradictions.
- Microsoft Foundry gives the fashions, and the trail to brokers.
- FastAPI on Container Apps runs the orchestration, the temporal scoping and the contradiction gate.
- Obsidian makes the entire thing seen to the individuals who know whether or not it’s proper.
It’s extra work than easy RAG, and it prices extra to ingest. In change you get organizational reminiscence, constant synthesis, express relationships, a visual resolution historical past, and, the half I hold coming again to, a system that can let you know “two of our paperwork disagree and no one has determined but” as an alternative of confidently making one thing up.
That final functionality is just not a characteristic. It’s the purpose to construct it.
The appliance is not looking a pile of paperwork. It’s slowly constructing a reviewable mannequin of a website, whereas retaining the unique proof shut sufficient to test each vital conclusion in opposition to.
Thanks for taking the time to discover this structure with me. It changed into an extended piece than I supposed, as a result of the design saved having another half price explaining. The FastAPI mission, the Bicep templates, the twenty-one artificial paperwork and the Obsidian vault are all in the repository, and I strongly consider they provide you with a sensible place to begin for constructing a persistent data layer of your personal. Clone it, run it in demo mode with none Azure credentials, and attempt to break it — I’d genuinely like to listen to the place it fails.
Disclosure: I’m a Microsoft MVP. This text displays my very own impartial work and opinions; Microsoft had no involvement in or overview of its content material. All Azure utilization described is predicated on public documentation and my very own deployment.
References
[1] P. Lewis et al., Retrieval-Augmented Era for Information-Intensive NLP Duties (2020), NeurIPS 2020
[2] A. Karpathy, LLM Wiki (2025), GitHub Gist
[3] Microsoft, Doc Intelligence Structure Mannequin (2026), Microsoft Study
[4] Microsoft, Hybrid Search Overview – Azure AI Search (2026), Microsoft Study
[5] Microsoft, Built-in Vectorization in Azure AI Search (2026), Microsoft Study
[6] Microsoft, Azure Cosmos DB Serverless (2026), Microsoft Study
[7] Microsoft, Azure OpenAI v1 API Lifecycle (2026), Microsoft Study
[8] Microsoft, Foundry Agent Service Overview (2026), Microsoft Study
[9] Microsoft, Structured Outputs with Azure OpenAI (2026), Microsoft Study
[10] Microsoft, Deploy a Flask or FastAPI Internet App on Azure Container Apps (2026), Microsoft Study
[11] Microsoft, Plan and Handle Prices of Azure AI Search (2026), Microsoft Study
