Constructing multi-tenant brokers with Amazon Bedrock AgentCore

Software program as a service (SaaS) suppliers constructing multi-tenant agentic functions should deal with architectural challenges past the standard issues of safety, governance, and response accuracy. These embody tenant isolation, tenant id, tenant observability, knowledge isolation, price attribution, and noisy neighbor mitigation. Closing the hole between a working demo and a manufacturing deployment requires infrastructure constructed for multi-tenant environments.Amazon Bedrock AgentCore is a managed, serverless service for constructing, deploying and securely working agentic functions on AWS. It gives constructs for deploying brokers and internet hosting MCP servers, with built-in help for id administration, reminiscence, observability, and evaluations, all designed to make multi-tenant agent architectures simple to construct.

This submit, half 1 of the weblog collection, explores design issues for architecting multi-tenant agentic functions and the framework wanted to handle SaaS structure challenges with Amazon Bedrock AgentCore.

Design issues for constructing a multi-tenant agent

Constructing safe multi-tenant agentic functions with robust isolation requires cautious architectural choices throughout sure key parts, as proven in Determine 1. Every part should steadiness tenant isolation, operational effectivity, and value optimization whereas sustaining safety and compliance requirements. These design issues revolve round three tenant isolation patterns: Silo, Pool, and Bridge, with tiering technique as a key consideration when selecting amongst them.

Determine 1: Design issues for a multi-tenant agent

Within the following part, we elaborate how multi-tenancy impacts every of those parts.

1. Agent Runtime Deployment: Devoted in comparison with Shared

A key choice in a multi-tenant agentic structure is how the agent runtime is provisioned relative to tenants. A devoted runtime per tenant instantiates a separate execution atmosphere for every tenant, with its personal container picture, course of area, and lifecycle. This silo method provides the strongest noisy-neighbor safety and streamlines compliance audits. A shared runtime hosts brokers for all tenants inside the identical container picture and course of pool, decreasing infrastructure prices and operational overhead however requiring strict in-process tenant context propagation.

Amazon Bedrock AgentCore Runtime resolves this rigidity via session-isolated microVM-based compute. AgentCore Runtime launches light-weight microVMs on a per-session foundation, with out the price or latency of spinning up a full digital machine for each tenant. Every session carries its personal persistent file system, so brokers can learn and write session-scoped information, preserve intermediate computation artifacts, and protect state throughout multi-step interactions, lowering the chance of cross-session knowledge leakage. The structure is an efficient match for internet hosting multi-tenant MCP servers, brokers, and AG-UI servers.Tenant context flows into the remoted execution atmosphere via customized HTTP headers. When the SaaS platform forwards a request to an AgentCore Runtime session, it attaches headers carrying tenant-specific metadata reminiscent of tenant identifier, tier, regional preferences, characteristic flags, or entitlements, alongside commonplace authorization tokens. The agent reads these headers at invocation time to ascertain full tenant consciousness, so it may run workflows tuned to that tenant’s enterprise logic, invoke solely licensed instruments, and name tenant-specific API endpoints with out hardcoded routing logic.

2. Shared in comparison with Tier-Particular in comparison with. Nice-Tuned Fashions

Shared basis fashions (FMs) function the really helpful start line for many multi-tenant deployments, providing streamlined operations with single mannequin upkeep. Tenants sometimes profit from computerized mannequin updates with out per-tenant customizations. The choice to pick out the mannequin based mostly on tenant tier (Tier-specific mannequin) permits flexibility and balances price, efficiency, and accuracy throughout tenant tiers. Tenant-specific fine-tuned fashions grow to be vital for specialised use instances requiring tenant-specific terminology, regulatory compliance, or efficiency SLAs, although they introduce greater operational complexity and per-tenant pipelines. A hybrid method, utilizing much less succesful fashions for normal tiers and fine-tuned or extra succesful fashions for premium enterprise clients, balances price effectivity with customization wants.Amazon Bedrock gives a selection of enormous language fashions (LLMs) from main suppliers, permitting SaaS suppliers to select a mannequin appropriate for tenant and tier-specific wants. Amazon Bedrock fine-tuning helps the customization of FMs utilizing your individual labeled datasets to enhance efficiency for domain-specific duties. With Amazon Bedrock Customized Mannequin Import, you’ll be able to convey your individual fine-tuned fashions and deploy them utilizing the Amazon Bedrock managed infrastructure.

3. Workflows: Silo, Pool, and Bridge patterns

Multi-tenant agentic functions require versatile workflow administration the place every agent executes totally different sequences of steps based mostly on tenant necessities and enterprise logic. Workflows might be applied via a number of mechanisms: as MCP instruments that encapsulate step-by-step processes, as API endpoints that outline enterprise logic flows, or as agent abilities that embed domain-specific workflow patterns.

Three major patterns handle tenant-specific workflows. The silo sample makes use of devoted tenant-specific abilities the place every tenant’s full workflow, together with all enterprise logic, validation guidelines, and integration steps, is embedded in remoted agent abilities. This offers most customization and full independence however requires separate talent upkeep per tenant. The pool sample makes use of shared agent abilities. The bridge sample embeds frequent workflow steps reminiscent of authentication, logging, and error dealing with in shared agent abilities that invoke tenant-specific abilities at runtime for business-critical logic. The result’s reusable infrastructure that coexists with tenant-specific customization.

4. Multi-tenant RAG

Retrieval Augmented Technology (RAG) methods require knowledge isolation choices. The silo sample makes use of devoted vector databases per tenant, offering most safety and full knowledge separation. That is really helpful for regulated industries and enterprise clients requiring devoted infrastructure. The pool sample makes use of shared vector databases with metadata-based tenant filtering and namespace-based entry management, which helps cost-efficient operations for SaaS platforms serving many small-to-medium tenants. Retrieval operations ought to embody computerized tenant filter injection and consequence sanitization to assist forestall cross-tenant knowledge leakage.

Amazon Bedrock Information Bases gives totally managed RAG capabilities that join FMs to your knowledge sources, robotically dealing with knowledge ingestion, chunking, embedding era, and vector storage. It helps a number of vector databases and gives the flexibility to create siloed or shared vector database (utilizing meta-data filtering).

For detailed steerage on implementing multi-tenant RAG architectures with Amazon Bedrock Information Bases, see Multi-tenant RAG with Amazon Bedrock Information Bases for silo, pool, and bridge deployment patterns, and Multi-tenancy in RAG functions in a single Amazon Bedrock data base with metadata filtering for metadata-based tenant isolation inside a shared data base.

5. Tenant context, act-on-behalf patterns, and token propagation

Multi-tenant id administration requires cautious dealing with of tenant context all through the service chain. Tenant context, representing the entire id, and request-specific state should move via each architectural layer utilizing dependable and safe mechanisms. Not like deterministic software program APIs with predictable execution paths, AI brokers are non-deterministic and might be doubtlessly autonomous, making safety issues totally different in essential methods. Rogue or compromised brokers might doubtlessly make unauthorized calls to downstream companies, resulting in stolen credentials, privilege escalation, and the Confused Deputy drawback. When brokers function with full consumer credentials (impersonation), a single compromised agent beneficial properties full entry to all consumer permissions throughout all downstream methods. This threat grows as brokers grow to be extra autonomous and make unbiased choices about which instruments to invoke, when to invoke them, and with what parameters. The act-on-behalf sample issues as a result of it establishes a transparent distinction between the consumer and the agent, with brokers making calls on behalf of the consumer with explicitly restricted, scoped permissions for every particular operation.

Encode tenant context inside JSON Internet Tokens (JWT) capturing three dimensions: Safety Context (commonplace claims: iss, sub, exp, aud), Tenant Context (tenant_id and tenant-specific scopes), and Request Context (domain-specific attributes for enterprise logic). Encoding tenant context this manner gives a robust and versatile basis for multi-tenant operations.

Select between two patterns with distinct safety implications: Impersonation permits brokers to function with full consumer id and permissions, providing simple implementation however violating the least privilege rules and creating safety dangers. Act-on-Behalf (Delegation), the really helpful method, implements true delegation the place tokens are remodeled at every service boundary with scope-limited credentials and an act declare (per OAuth 2.0 RFC 8693) figuring out the agent. Use the On-behalf-of token trade in AgentCore Id, enabling brokers and different workloads, reminiscent of MCP servers, to trade an inbound consumer entry token for a brand new, scoped entry token that targets a downstream useful resource server. Because the trade converts a token issued for one viewers instantly right into a token for a distinct downstream viewers, your brokers can entry protected assets on behalf of authenticated customers with out triggering extra consent flows. The exchanged token carries each the agent’s personal id and the unique caller’s id, giving useful resource servers the alerts they should implement fine-grained, zero-trust authorization at each hop.

6. Nice-grained entry management for MCP instruments and APIs

Multi-tenant agentic functions require limiting MCP server entry utilizing insurance policies, fine-grained entry management on the device invocation layer, and tenant isolation on the knowledge entry layers. On the authorization layer, insurance policies consider tenant context at runtime to make enable/deny authorization choices, and to evaluate tenant quotas, tier-based permissions, and utilization limits earlier than permitting device invocations based mostly on present tenant state fairly than relying solely on static permissions embedded in tokens. Decoupled and centralized coverage shops enable dynamic updates with out redeployment, with coverage versioning supporting audit trails and rollback capabilities. AgentCore Coverage intercepts and evaluates all agent requests towards outlined insurance policies earlier than permitting device entry, offering fine-grained management based mostly on consumer id and power enter parameters, with insurance policies authored utilizing pure language or instantly in Cedar.

On the invocation layer, MCP servers implement fine-grained entry management by filtering obtainable instruments based mostly on tenant tier, characteristic flags, and quota limits earlier than brokers can invoke them. Software interceptors validate JWT claims to substantiate that the requesting principal has acceptable permissions for the particular operation. Schema translation capabilities adapt device interfaces based mostly on tenant configurations and entitlements. AgentCore Gateway permits brokers to securely entry instruments by remodeling APIs and AWS Lambda capabilities into agent-compatible instruments and connecting to present MCP servers, with help for Amazon API Gateway, OpenAPI schemas, Smithy fashions, Lambda capabilities, and MCP servers. You possibly can implement entry management via gateway interceptors for customized logic or use resource-based insurance policies for normal AWS-style entry management.On the knowledge entry layer, Attribute-Based mostly Entry Management (ABAC) insurance policies implement tenant isolation for knowledge entry, with tenant identification occurring via JWT claims. ABAC insurance policies use AWS Id and Entry Administration (IAM) circumstances to limit knowledge entry based mostly on principal tags and attributes, so brokers can solely question assets matching their tenant context via row-level safety or storage insurance policies.

7. Reminiscence: Hierarchical namespace isolation

Multi-tenant reminiscence administration requires cautious architectural design in order that brokers can preserve context and realized info whereas stopping cross-tenant knowledge leakage. Reminiscence methods ought to implement 5 logical ranges:

• World (cross-tenant shared data)
• Technique (agent-type-specific patterns and behaviors)
• Tenant (tenant-scoped conversational historical past and preferences)
• Consumer (particular person consumer context inside a tenant)
• Session (ephemeral short-term reminiscence for energetic conversations)

Entry management enforces isolation via attribute-based insurance policies that validate principal identities towards requested namespace paths, so brokers can solely learn and write reminiscence inside their allowed scope. The pool sample makes use of shared infrastructure with hierarchical namespace-based logical isolation for operational and value effectivity, storing all tenant knowledge in a standard knowledge retailer with strict filtering based mostly on namespace prefixes. The silo sample deploys devoted reminiscence shops per tenant for optimum isolation, lowering cross-tenant entry threat at a better operational price. Implementation entails developing composite identifiers from tenant and consumer info (for instance, tenant_123:user_456), authenticating with scoped credentials that carry tenant context as claims or tags, and prefixing all reminiscence operations with the suitable namespace path.

AgentCore Reminiscence gives hierarchical namespace isolation throughout international, technique, tenant, consumer, and session ranges, supporting context-aware agent experiences with each short-term reminiscence for multi-turn conversations and long-term reminiscence that persists throughout periods. It helps useful resource based mostly insurance policies and attribute-based entry management for fine-grained entry.

8. Agent id, belief, and discovery

As agentic functions work together with exterior brokers throughout organizational boundaries, three foundational issues emerge: agent id, agent belief, and agent discovery. Whereas associated, every addresses a definite drawback.

Agent Id solutions “Who is that this agent, and may it show it?” – establishing a verifiable, distinctive id tied to a company.

Agent Belief solutions “Ought to I belief this agent?” – evaluating trustworthiness based mostly on a mixture of alerts, not a single credential.

Agent Discovery solutions “How do I discover the appropriate agent?” – finding brokers by functionality or affiliation with out prior data of endpoints.

Agent id with AgentCore Id

Amazon Bedrock AgentCore Id implements agent identities as workload identities, a sample well-established in cloud-native safety. Every agent receives a cryptographically verifiable id anchored to the group’s AWS account and IAM infrastructure. Brokers can securely entry AWS assets and third-party instruments on behalf of customers utilizing OAuth 2.0 flows, and AgentCore Id integrates with present company id suppliers reminiscent of Okta, Microsoft Entra ID, and Amazon Cognito with out requiring consumer migration.

Agent belief

Id alone doesn’t reply whether or not an agent must be trusted. The trade is actively engaged on this drawback. The Agent Naming Service (ANS) v2, presently an IETF Web-Draft (work in progress), which anchors each agent id to a DNS area identify. Shoppers can select assurance ranges which are acceptable to their transaction threat with three verification tiers, Bronze (PKI), Silver (PKI + DANE), and Gold (PKI + DANE + Transparency Log).

Agent discovery with AWS Agent Registry

AWS Agent Registry, obtainable via Amazon Bedrock AgentCore, gives a centralized catalog for locating brokers, abilities, MCP servers, and customized assets throughout a company. Groups can publish, model, and share reusable agent capabilities. Customers uncover brokers via pure language or structured search without having prior data of identifiers or endpoints. Constructed-in governance controls decide how shoppers entry the registry and whether or not information require approval earlier than changing into discoverable.In abstract, AgentCore Id gives the foundational proof of id, Agent Registry solves discovery, and rising belief frameworks like ANS purpose to shut the hole on multi-signal belief analysis.

9. Value monitoring per tenant and observability

Correct multi-tenant price attribution requires application-level instrumentation that emits tenant-tagged metrics to a logging resolution for each agent invocation, capturing I/O tokens, device invocations, and execution length. Structured logging with tenant context permits detailed evaluation of utilization patterns, efficiency bottlenecks, and capability planning. AgentCore Observability gives real-time visibility into agent workflows with OpenTelemetry-compatible integration powered by Amazon CloudWatch, providing detailed visualizations of every step of agent execution.

10. Guardrails: Content material security

Multi-tenant guardrails implement security and compliance at three enforcement factors. Pre-processing enter guardrails validate consumer enter earlier than agent processing, blocking malicious prompts, immediate injections, and sanitizing PII based mostly on tenant-specific compliance necessities reminiscent of HIPAA for healthcare and PCI-DSS for finance. Publish-processing output guardrails validate agent responses for factual accuracy, detect hallucinations, affirm format compliance, and scan for delicate knowledge leakage throughout tenant boundaries. You possibly can apply guardrails by tenant or tier, offering configurations for toxicity detection, content material filtering, and customized blocked phrases, with observability metrics monitoring set off charges, blocked requests by class, and false optimistic charges for steady enchancment. Amazon Bedrock Guardrails gives content material filtering and security controls with configurable insurance policies for denied matters, content material filters, phrase filters, and delicate info redaction, supporting accountable AI deployment throughout all mannequin interactions.

These ten parts present a complete framework for designing multi-tenant brokers. Within the following sections, we discover the implementation of the silo, pool, and bridge fashions inside AgentCore, retaining these core parts in thoughts.

Implementing Silo mannequin with AgentCore

As described within the following Determine, the silo mannequin permits every tenant to function inside a completely remoted stack with its personal devoted Bedrock AgentCore Runtime, Bedrock AgentCore Gateway, and Bedrock AgentCore Reminiscence, all scoped behind separate AWS IAM boundaries. There are a number of classifications of reminiscence supported reminiscent of long-term, short-term, and episodic, which must be configured as per the tenant requirement.

Key architectural parts

• Siloed Agent Layer – Devoted AgentCore Runtime every deployed with separate IAM execution roles for tenant particular permissions.
• Siloed Gateway – Devoted AgentCore Gateway for device orchestration utilizing MCP, scoped entry to knowledge layer based mostly on execution roles.
• Siloed Agent Reminiscence – Devoted AgentCore Reminiscence with hierarchical namespace isolation, eradicating the necessity to embody tenant IDs in each namespace path. Brokers entry tenant-specific reminiscence via IAM roles.
• Siloed Knowledge Layer – Devoted instruments, data bases, databases, and backend assets for optimum knowledge isolation.

Request move

1. Authentication – Customers authenticate utilizing the Id Supplier, receiving JWT tokens containing tenant context (tenant ID and subscription tier).
2. SaaS utility proxy routing – The SaaS utility proxy decides which agent to invoke based mostly on the tenant context. This requires a mapping configuration to be established between tenant and agent deployment, a operate sometimes a part of the SaaS management airplane. The proxy transforms application-level requests into AgentCore Runtime API calls (InvokeAgent), attaching the tenant JWT token.
3. Agent execution – The AgentCore Runtime validates the JWT utilizing AgentCore Id, creates an remoted microVM session, and begins agent reasoning. Moreover, it validates if the tenant id is allowed to invoke this agent (for instance, “enable provided that tenant_id = Tenant A”) by configuring customized claims within the JWT Authorizer of AgentCore Id. The agent accesses tenant-specific AgentCore Reminiscence utilizing runtime IAM execution roles.
4. Software entry utilizing AgentCore Gateway – When the agent should invoke instruments, it calls the devoted AgentCore Gateway, which is particularly scoped to entry MCP instruments for a particular tenant. The Gateway:
   1. Validates the JWT utilizing AgentCore Id.
   2. Extracts tenant context from the validated token and verifies the Gateway is mapped to the tenant in context utilizing customized interceptors.
   3. Integrates with siloed tenant-specific backend assets (APIs, databases, data bases).
5. Response move – Software responses move again via the Gateway to the agent, which completes its reasoning. The siloed agent applies tenant-specific formatting earlier than returning to the SaaS utility proxy. The proxy returns the response to the consumer.

The Silo sample is designed so that every buyer’s agent periods, device entry, and reminiscence are totally contained, and prices are attributed on to the client whose alert triggered the work.The trade-off is greater operational overhead, since every buyer runs devoted assets fairly than sharing them. However for security-critical and compliance-sensitive workflows, the restricted scope of potential impression makes it the appropriate selection.

Determine 2: Silo Mannequin with AgentCore

Implementing pool mannequin with AgentCore

As described within the following Determine, the pool mannequin permits useful resource sharing throughout a number of tenants, so you’ll be able to design architectures that maximize useful resource utilization and ship operational effectivity.

Key architectural parts

• Pooled Agent Layer – Shared AgentCore Runtime and agent logic throughout a number of tenants.
• Pooled Gateway – Centralized AgentCore Gateway for device orchestration utilizing MCP.
• Pooled Agent Reminiscence – Shared AgentCore Reminiscence partitioned based mostly on tenant context.
• Pooled Knowledge Layer – Shared instruments, data bases, databases, and backend assets.
• Pooled Id Administration – Pooled Id Supplier with JWT-based tenant context propagation.

Request move

1. Authentication– Customers authenticate utilizing the Id Supplier, receiving JWT tokens containing tenant context (tenant ID and subscription tier).
2. SaaS utility proxy routing – The SaaS utility acts as cross via the place it routes enter request with tenant context to brokers operating in pooled AgentCore Runtime. The SaaS utility proxy transforms application-level requests into AgentCore Runtime API calls (InvokeAgent), attaching the tenant JWT token.
3. Agent execution – The AgentCore Runtime validates the JWT utilizing AgentCore Id, creates an remoted microVM session, extracts the tenant context from the JWT and begins agent reasoning. The agent accesses tenant-scoped AgentCore Reminiscence utilizing namespace-based partitioning (for instance, actor_id: “tenant-a:user-123”).
4. Software entry utilizing AgentCore Gateway – When the agent should invoke instruments, it calls the pooled AgentCore Gateway, which is particularly designed for MCP device orchestration, not generic routing. The Gateway:
   1. Validates the JWT utilizing AgentCore Id.
   2. Extracts tenant context from the validated token.
   3. Routes device calls to pooled backend assets (APIs, databases, data bases).
   4. Enforces tool-level isolation via tenant-scoped credentials and configuration.
   5. Applies coverage enforcement and interceptors for cross-cutting issues.
5. Response move – Software responses move again via the Gateway to the agent, which completes its reasoning. The agent response returns via the Runtime to the Vendor proxy, which applies tenant-specific formatting earlier than returning to the consumer.

The pool mannequin is extremely environment friendly and is likely to be the one choice when you will have massive variety of small tenants.The trade-off is extra rigor round testing fine-grained entry management, and extra instrumentation is required to attribute price to tenants.

Determine 3: Pooled Mannequin with AgentCore

Implementing bridge mannequin with AgentCore

The bridge mannequin (the hybrid mannequin) represents a strategic center floor between the silo and pool deployment patterns. This method combines the price effectivity of shared infrastructure with the safety advantages of remoted knowledge assets.

Relying in your wants, you’ll be able to select to implement the bridge sample in varied methods:

1. Siloed AgentCore Runtime/gateway/device/reminiscence for premium tier tenant and pooled shared AgentCore Runtime/gateway/device/reminiscence for normal tier
2. Siloed Runtime with pooled gateway/instruments and reminiscence
3. Others

The thought is to have the ability to select the tenancy at every layer and part, fairly than tied to a particular tenant isolation sample.This method combines the advantages of each approaches, relying in your implementation. For instance, within the SOC analyst use case, the gateway may very well be siloed to deal with e mail API interactions and different downstream tenant assets, whereas the pooled agent runtime hosts the agent and performs reasoning, since every investigation runs in its personal remoted microVM.

Determine 4: Bridge Mannequin with AgentCore (variation 1)

Determine 5: Bridge Mannequin with AgentCore (variation 2)

What’s subsequent

On this submit, we coated the foundational ideas for constructing multi-tenant brokers. Within the upcoming posts, we are going to take a deeper look into the implementation features of those ideas. Particularly, we are going to stroll via an end-to-end working implementation of each the pool and silo deployment fashions, incorporating the parts outlined within the design issues part.

Conclusion

Constructing production-ready multi-tenant agentic functions requires extra than simply practical AI brokers. It calls for a complete architectural method that addresses tenant isolation, id administration, price attribution, and safety at each layer. Amazon Bedrock AgentCore gives the foundational primitives wanted to deal with these challenges, providing versatile deployment patterns via silo, pool, and bridge fashions that may be tailor-made to your particular tiering technique and compliance necessities. Whether or not you’re serving enterprise clients requiring devoted infrastructure or optimizing prices throughout a whole lot of smaller tenants, you should utilize the built-in Runtime, Gateway, Reminiscence, Id, and Observability parts of AgentCore to construct safe, scalable multi-tenant agentic workflows with out reinventing the wheel. These primitives work collectively to assist preserve tenant knowledge isolation, scoped device entry, correct price attribution, and safety boundaries, remodeling the complexity of multi-tenant agent structure right into a manageable, production-ready resolution that scales along with your SaaS enterprise.

We encourage readers to discover the multi-tenant brokers workshop for hands-on expertise constructing these multi-tenant brokers with Amazon Bedrock AgentCore.

Concerning the authors