Scalable voice agent design with Amazon Nova Sonic: multi-agent, instruments, and session segmentation

Design patterns for scalable voice brokers matter for organizations that must ship quick, pure, and dependable voice experiences. Many groups face challenges like excessive latency, managing real-time audio, and coordinating a number of brokers in advanced workflows.

On this put up, you’ll learn to use Amazon Nova Sonic, Amazon Bedrock AgentCore, and Strands BidiAgent to construct scalable, maintainable voice brokers that deal with these challenges effectively, leading to extra responsive and clever buyer interactions.

We’ll discover three well-liked architectural patterns for voice brokers, highlighting their trade-offs and finest practices for minimizing latency.

The constructing blocks

Earlier than diving deeper into the structure patterns, right here’s a fast overview of the three key parts used because the pattern resolution on this put up.

Amazon Nova Sonic is a basis mannequin that creates pure, human-like speech-to-speech conversations for generative AI purposes. Customers can work together with AI by voice in actual time, with capabilities for understanding tone, pure conversational circulate, and performing actions.

Amazon Bedrock AgentCore Runtime is a serverless internet hosting atmosphere for AI brokers. You bundle your agent as a container, deploy to AgentCore Runtime, and it handles scaling, session isolation, and billing. For voice brokers, it gives bidirectional WebSocket streaming with SigV4 auth, microVM-level session isolation to keep away from noisy-neighbor latency spikes, AgentCore Gateway for shared software internet hosting utilizing the Mannequin Context Protocol (MCP) open supply protocol, persistent reminiscence throughout periods, and telemetry for voice-specific metrics like time-to-first-audio.

Strands Brokers is an open supply framework for constructing AI brokers. Its BidiAgent class is one integration choice between Nova Sonic and your software. It manages the bidirectional stream lifecycle, routes software calls, and handles session administration, simplifying the voice agent software by the mannequin SDK interface.

Three integration patterns: software, agent-as-tool (sub-agent), and session segmentation

As a substitute of constructing one omnipotent agent, fashionable voice methods are more and more composed of tool-driven brokers, sub-agents performing as instruments and session segmentation methods that isolate prompts, reminiscence, and permissions. These patterns enable groups to decompose massive assistants into smaller, specialised, and reusable parts whereas sustaining clear safety boundaries.

Earlier than operating the samples within the following sections, set up Python and the required dependencies, together with strands-agents and boto3, and ensure your IAM setup has the mandatory permissions for the required companies. For the total instance, discuss with the GitHub repository.

Sample 1: AgentCore Gateway – software choice for low latency

A software name is when a voice agent sends enter to an exterior operate or service, which processes it and returns output. It lets the agent carry out duties like querying a database or triggering a service shortly and securely, with out further reasoning steps.

With AgentCore Gateway, you expose your current enterprise logic as instruments, discrete capabilities that Nova Sonic can name straight throughout a dialog. The voice mannequin selects which software to invoke, passes parameters, will get a consequence, and speaks it again. There’s no intermediate reasoning layer between the mannequin and the software.

AgentCore Gateway hosts MCP servers as managed endpoints. MCP is the protocol, AgentCore Gateway is the AWS characteristic that runs them. The voice agent connects by way of Gateway ARNs.

# Nova Sonic calls instruments straight by way of AgentCore Gateway
mannequin = BidiNovaSonicModel(
    model_id="amazon.nova-2-sonic-v1:0",
    mcp_gateway_arn=[
        "arn:aws:bedrock-agentcore:us-east-1:123456789012:gateway/auth-tools",
        "arn:aws:bedrock-agentcore:us-east-1:123456789012:gateway/banking-tools",
        "arn:aws:bedrock-agentcore:us-east-1:123456789012:gateway/mortgage-tools",
    ],
)

When a person says “What’s my account stability?”, Nova Sonic:

1. Understands the intent from speech.
2. Selects get_account_balance from the out there MCP instruments.
3. Calls the software with the appropriate parameters.
4. Speaks the consequence again.

Commerce-off: Nova Sonic makes all the selections. If a software name requires multi-step validation, conditional logic, or chaining a number of operations collectively, that reasoning burden falls totally on the voice mannequin’s system immediate. For easy instruments that is high-quality. For advanced workflows, it will get brittle.

Sample 2: Sub-agent – extra reasoning with decoupled brokers

With the sub-agent or agent-as-tool sample, your current enterprise logic runs in autonomous brokers, every with its personal mannequin, system immediate, instruments, and reasoning capabilities. The voice orchestrator delegates entire duties to those sub-agents as an alternative of calling particular person instruments.

There are various methods to connect with a sub-agent out of your voice agent. Agent-to-Agent (A2A) and Strands Agent-as-Device are two frequent approaches:

• Native agent-as-tool: The sub-agent runs in-process, wrapped as a @software operate utilizing the Brokers as Instruments sample in Strands. That is probably the most easy method with no community hop and no separate deployment. The trade-off is that the sub-agent shares the identical course of and scales with the orchestrator.
• Distant agent by way of A2A protocol: The sub-agent is deployed as an impartial A2A server on AgentCore Runtime (or a distant server) and invoked over the community. A2A is an open protocol for agent-to-agent communication. As MCP connects brokers to instruments, A2A connects brokers to different brokers. Because the AWS weblog on A2A protocol help in AgentCore Runtime explains, brokers constructed with completely different frameworks (Strands, OpenAI, LangGraph, Google ADK) can share context and reasoning in a typical format. This gives full deployment independence and cross-framework interoperability.

Strands Brokers has built-in help for each protocols, MCP for software entry and A2A for agent-to-agent communication. For a hands-on walkthrough, see the group information on Agent Collaboration: Strands Brokers, MCP, and the Agent2Agent Protocol.

Right here’s the native agent-as-tool method, every sub-agent is a @software wrapping a full Strands Agent:

# sub_agents.py — Outline sub-agents as Strands instruments utilizing the Brokers-as-Instruments sample
from strands import Agent, software
from strands.fashions import BedrockModel

# Every sub-agent is a full Strands Agent wrapped as a @software
# The BidiAgent orchestrator calls these by way of Nova Sonic's software use

@software
def authenticate_customer(account_id: str, date_of_birth: str) -> str:
    """Authenticate a buyer utilizing their account ID and date of start.
    Handles the total verification circulate together with id checks and retry logic.
    Returns authentication standing and token."""
    auth_agent = Agent(
        mannequin=BedrockModel(model_id="amazon.nova-lite-v1:0"),
        system_prompt="""You might be an authentication agent. Confirm the shopper's id
        utilizing the supplied account ID and date of start. Name verify_identity to examine
        credentials. Return a transparent auth standing in 1-2 sentences.""",
        instruments=[verify_identity, check_account_exists],  # Sub-agent's personal instruments
    )
    consequence = auth_agent(f"Authenticate account {account_id}, DOB: {date_of_birth}")
    return str(consequence)


@software
def handle_banking_inquiry(question: str, auth_token: str) -> str:
    """Deal with banking questions — balances, transactions, transfers.
    Validates permissions and returns a conversational abstract."""
    banking_agent = Agent(
        mannequin=BedrockModel(model_id="amazon.nova-lite-v1:0"),
        system_prompt="""You're a banking agent. Use the supplied instruments to reply
        the shopper's question. Summarize ends in 2-3 pure sentences.
        Don't return uncooked JSON.""",
        instruments=[get_account_balance, get_recent_transactions, transfer_funds],
    )
    consequence = banking_agent(question)
    return str(consequence)


@software
def handle_mortgage_inquiry(question: str) -> str:
    """Deal with mortgage questions — charges, calculations, eligibility, software standing.
    Performs its personal calculations and reasoning."""
    mortgage_agent = Agent(
        mannequin=BedrockModel(model_id="amazon.nova-lite-v1:0"),
        system_prompt="""You're a mortgage specialist. Assist with fee inquiries,
        fee calculations, and eligibility assessments. Hold responses concise
        and conversational — this can be spoken aloud.""",
        instruments=[get_mortgage_rates, calculate_payment, check_eligibility],
    )
    consequence = mortgage_agent(question)
    return str(consequence)

The voice orchestrator then makes use of BidiAgent with these sub-agent instruments:

# voice_orchestrator.py — BidiAgent with sub-agents as instruments
from strands.experimental.bidi.agent import BidiAgent
from strands.experimental.bidi.fashions.nova_sonic import BidiNovaSonicModel
from sub_agents import authenticate_customer, handle_banking_inquiry, handle_mortgage_inquiry

mannequin = BidiNovaSonicModel(
    area="us-east-1",
    model_id="amazon.nova-2-sonic-v1:0",
    provider_config={"audio": {"voice": "tiffany", "input_sample_rate": 16000, "output_sample_rate": 16000}},
)

agent = BidiAgent(
    mannequin=mannequin,
    instruments=[authenticate_customer, handle_banking_inquiry, handle_mortgage_inquiry],
    system_prompt="""You're a banking voice assistant. Route buyer requests to the
    applicable specialist. All the time authenticate earlier than accessing account information.
    Hold your personal responses temporary — the sub-agents deal with the main points.""",
)

await agent.run(inputs=[ws_input], outputs=[ws_output])

The sub-agent does its personal pondering. Nova Sonic doesn’t must orchestrate the person steps. It delegates and speaks the consequence.

Commerce-off: Every sub-agent name provides latency: the sub-agent’s personal mannequin inference plus its software calls. In a voice dialog, this implies longer silence whereas the sub-agent causes. The AWS weblog on multi-agent voice assistants recommends beginning with smaller, environment friendly fashions like Amazon Nova 2 Lite for sub-agents to scale back latency whereas nonetheless dealing with specialised duties successfully.

Amazon Nova 2 Sonic helps asynchronous software calling, so the dialog continues naturally whereas instruments run within the background. It retains accepting enter, can run a number of instruments in parallel, and gracefully adapts if the person adjustments their request mid-process, delivering all outcomes whereas specializing in what’s nonetheless related.

Sample 3: Session segmentation for ultra-low latency

There’s a 3rd method value contemplating. It doesn’t map neatly to the MCP or sub-agent patterns, however is purpose-built for voice eventualities the place latency is the overriding concern.

As a substitute of delegating exterior instruments or sub-agents, you section the dialog into logical phases, every with its personal Nova Sonic session, system immediate, and gear set. When the dialog transitions from one section to the following (for instance, from authentication to account inquiry), you shut the present session and open a brand new one with a special immediate and instruments, throughout the identical WebSocket connection. Every sub-voice-agent can use its personal MCP gateways, instruments, and even sub-agents — the variations that it operates with a centered immediate and minimal software floor, lowering reasoning overhead and latency.

Consider a banking voice assistant with three dialog phases: authentication, account administration, and mortgage inquiry. Fairly than loading one huge system immediate with each software, you run every section as a centered Nova Sonic session:

# Section 1: Authentication
auth_session = BidiNovaSonicModel(
    model_id="amazon.nova-2-sonic-v1:0",
    mcp_gateway_arn=["arn:...gateway/auth-tools"],  # Solely auth instruments
)
auth_agent = BidiAgent(
    mannequin=auth_session,
    instruments=[],
    system_prompt="""You might be an authentication assistant. 
    Accumulate the person's account ID and date of start. 
    Name verify_identity to authenticate. 
    As soon as verified, say 'You are all set' and cease.""",
)
# Run till authentication completes
await auth_agent.run(inputs=[ws_input], outputs=[ws_output])

# Section 2: Account administration (new session, new immediate, new instruments)
banking_session = BidiNovaSonicModel(
    model_id="amazon.nova-2-sonic-v1:0",
    mcp_gateway_arn=["arn:...gateway/banking-tools"],  # Solely banking instruments
)
banking_agent = BidiAgent(
    mannequin=banking_session,
    instruments=[],
    system_prompt="""You're a banking assistant. The person is already authenticated.
    Assist with stability inquiries, transactions, and transfers.
    Hold responses to at least one or two sentences.""",
)
await banking_agent.run(inputs=[ws_input], outputs=[ws_output])

Every section will get a clear Nova Sonic session with:

• A centered system immediate: Shorter, extra particular, much less room for the mannequin to get confused.
• Solely the related instruments: by way of MCP gateways, native instruments, or each. The mannequin doesn’t waste reasoning cycles selecting between 15 instruments when it solely wants 3.
• Optionally its personal sub-agents: a section that requires deeper reasoning can use Sample 2 internally, whereas less complicated phases keep tool-only.
• The earlier session context could be handed into the brand new session as chat historical past, so the general dialog retains continuity.

In comparison with software, sub-agent, and session segmentation patterns

Latency finest practices for voice brokers

Latency is a key consideration when constructing voice versus textual content brokers. Listed here are sensible methods to maintain response instances quick and responsive:

Begin with small fashions for sub-agents. Your voice orchestrator makes use of Nova Sonic for the dialog, however sub-agents don’t want a big mannequin. Begin with Amazon Nova 2 Lite or Nova 2 Micro. They’re quick, value optimized, and deal with most specialised duties properly. You may at all times improve a particular sub-agent to a bigger mannequin if high quality requires it, however default to small.

Design stateful sub-agents with caching. A stateless sub-agent that hits a database or API on each name provides latency each time. As a substitute, design sub-agents to cache outcomes from information sources (APIs, AWS Lambda capabilities, databases) inside a session. If the banking sub-agent fetches account particulars as soon as, it ought to maintain that information in reminiscence and serve subsequent questions (stability, transactions, abstract) from cache quite than making repeated backend calls.

Prefetch information after authentication. That is particularly worthwhile for contact middle eventualities. After a buyer authenticates, you already know who they’re. Don’t look ahead to them to ask earlier than pulling their information. Instantly fetch account balances, current transactions, pending alerts, and mortgage standing within the background. When the shopper asks “What’s my stability?”, the reply is already in reminiscence.

Parallelize impartial software calls. If the person asks “Give me an outline of my accounts”, don’t name get_checking_balance, then get_savings_balance, then get_credit_card_balance sequentially. Use concurrent execution so three calls occur without delay. Strands helps this natively. The agent’s software executor runs impartial calls in parallel by default.

Use filler phrases to masks software latency. When a software name or sub-agent delegation is unavoidable, instruct the voice mannequin to talk a short filler whereas ready: “Let me examine that for you…” or “One second whereas I look that up…” This retains the dialog feeling alive as an alternative of dropping into silence.

Decrease software rely per session. Device choice will get slower because the variety of out there instruments grows. In case your agent has 15 instruments however a typical dialog solely makes use of 3 to 4, think about the session segmentation sample to load solely the related instruments per section.

Clear up

After you end testing the pattern, bear in mind to scrub up the sources you created to keep away from pointless prices. Comply with the repository directions to cease companies and delete any deployed infrastructure.

Conclusion

Migrating a textual content chatbot to a voice assistant isn’t an easy wrapper job. The interplay mannequin is basically completely different, from response design to latency budgets to turn-taking conduct. However with a well-structured multi-agent structure and Amazon Bedrock AgentCore, the enterprise logic layer stays intact.

The sub-agents you’ve already constructed are your greatest asset. Reuse them.

For a working instance of a Strands BidiAgent voice assistant deployed on AgentCore Runtime with WebSocket streaming, see the AgentCore bidirectional streaming pattern.

Subsequent steps

Subsequent, you may prolong the pattern to suit your personal use case, combine your online business instruments, refine prompts for voice interactions, and take a look at the agent in real-world eventualities to organize for manufacturing deployment. To study extra about voice brokers on AWS, go to:

In regards to the authors