Eating places miss a median of 150 telephone calls per location each month, and about 60 p.c of these are clients making an attempt to put an order or e-book a desk. Most of those calls are available throughout dinner service, precisely when the host is seating company, servers are turning tables, and the telephone turns into an afterthought. Pulling somebody off the ground to reply doesn’t repair the issue. It solely makes two experiences worse. Including an app or an internet site helps clients preferring to order on-line, but it surely does nothing for the one who solely needs to name.
On this publish, we present you the best way to construct a voice ordering system that solutions a telephone quantity and takes the order from greeting to affirmation. The system makes use of Amazon Bedrock AgentCore to host and run the agent and Amazon Nova 2 Sonic for real-time speech, related to a restaurant backend by the Mannequin Context Protocol (MCP). The walkthrough covers deploying the total stack with AWS Cloud Growth Package (AWS CDK) and bridging a telephone name into the agent by a Session Initiation Protocol (SIP) gateway on Amazon Elastic Container Service (Amazon ECS) and AWS Fargate. It additionally warms the agent session whereas the telephone remains to be ringing, so the caller by no means hears useless air.
Answer overview
The system has three layers. The telephony layer handles phone-specific considerations. Audio arrives over the telephone community moderately than a browser, and the system identifies the caller by telephone quantity moderately than by a login. A SIP gateway streams that audio to the agent layer over a signed WebSocket connection, the place the agent runs the dialog with Amazon Nova 2 Sonic. The agent reaches the backend layer by MCP instruments. The backend holds the menu, carts, orders, and areas. Conserving these layers aside means the ordering logic stays unbiased from the channel that calls it. A brand new channel like a cellular app or a kiosk can connect with the identical agent with out rewriting the backend. And since MCP is an open customary for connecting an agent to exterior instruments, the backend can change with out touching the agent. The agent helps each textual content and audio as enter and output, dealing with transcription, turn-taking, and interruptions in a single bidirectional stream.
The answer deploys the next:
On the telephony aspect, Amazon Chime SDK Voice Connector gives the SIP trunk and toll-free quantity that accepts inbound calls. Amazon ECS on AWS Fargate runs the SIP gateway behind a Community Load Balancer. For the agent layer, AgentCore Runtime hosts the dialog logic, with every name working in its personal microVM for isolation. Amazon Nova 2 Sonic handles the speech-to-speech interplay. AgentCore Gateway exposes the backend APIs as MCP instruments the agent can uncover and name by identify. The backend layer makes use of Amazon API Gateway to entrance REST endpoints secured by AWS Id and Entry Administration (IAM). AWS Lambda runs the enterprise logic for menus, carts, orders, and placement lookups. Amazon DynamoDB shops the information, and Amazon Location Service handles geocoding and route calculation. Amazon Elastic Container Registry (Amazon ECR), AWS CodeBuild, and Amazon Easy Storage Service (Amazon S3) construct and retailer the agent container picture.
Structure diagram
The next diagram reveals the answer, which is organized into 4 sections.
The restaurant’s backend infrastructure deploys first in Part A. Amazon DynamoDB holds the client, order, menu, cart, and placement information, and Amazon Location Service handles addresses and routing. AWS Lambda runs the enterprise logic, and Amazon API Gateway exposes it externally with IAM authorization. Assets deploy in dependency order.
Part B creates the AgentCore Gateway, units up its IAM permissions, and configures the gateway to reveal the backend endpoints as agent-accessible MCP instruments. That is the layer that decouples the agent from the backend. With out it, including or altering a device would require redeploying the agent itself.
Part C provisions the agent. It creates the Amazon ECR repository and makes use of Amazon S3 and AWS CodeBuild to construct and push the container picture. It then deploys AgentCore Runtime. It additionally deploys the supporting items that permit the agent personalize every name, together with a prompt-renderer Lambda operate and a set of AWS Methods Supervisor Parameter Retailer entries that maintain the immediate templates and a secret used for caller identification.
Part D provisions the telephone path. It units up the Amazon Chime SDK Voice Connector and a toll-free quantity, a SIP Media Software Lambda that decides what to do with an incoming name, a shared Amazon Digital Personal Cloud (Amazon VPC), and the SIP gateway (drachtio-server), which runs on Amazon ECS on AWS Fargate behind a Community Load Balancer. The gateway bridges the calls between the Chime SDK Voice Connector and AgentCore Runtime.
The numbered callouts within the previous diagram hint the answer finish to finish:
- A name is initiated to the telephone quantity provisioned by Amazon Chime SDK, both by a buyer or forwarded from one other line.
- Amazon Chime SDK solutions the decision and invokes AWS Lambda to arrange the bridge.
- The Lambda creates a session identifier and opens a connection to AgentCore Runtime to heat up the microVM, which avoids a chilly begin when media streaming begins.
- After a profitable response from the Lambda, Amazon Chime SDK initiates a bridge motion in opposition to the Community Load Balancer by sending a SIP invite over TCP port 5060.
- The SIP service working on AWS Fargate accepts the invite and allocates a free port within the Actual-time Transport Protocol (RTP) service on the identical container to obtain media on the assigned public IP tackle.
- The RTP service receives the media on the UDP port from the Voice Connector and connects to the AgentCore Runtime WebSocket to start media translation, utilizing the identical session identifier created by the SIP Media Software handler.
- AgentCore Runtime invokes an AWS Lambda operate to construct the system immediate, saved on AWS Methods Supervisor Parameter Retailer, based mostly on the session identifier and the client’s Amazon DynamoDB report.
- AgentCore Runtime creates a session with Amazon Nova 2 Sonic and greets the client over the established connection, following the system immediate directions.
- AgentCore Runtime lists and calls the obtainable instruments from AgentCore Gateway utilizing the MCP protocol.
- AWS CDK deploys the answer to Amazon S3, triggering AWS CodeBuild to construct the container photographs and retailer them in Amazon ECR. AgentCore Runtime and AWS Fargate use these photographs to deploy the agent and the SIP and RTP servers.
- Amazon CloudWatch gives centralized monitoring, logging, and alerting throughout all companies, and all information at relaxation is encrypted utilizing AWS Key Administration Service (AWS KMS).
Callouts 1–9 occur throughout a single telephone name, callout 10 covers how the answer deploys the SIP server and AgentCore Runtime, and callout 11 covers the way it’s monitored and secured. The next part units deployment and operations apart and will get nearer to the decision itself.
Inbound name stream
This part follows one name from the caller’s aspect from the primary ring to the spoken reply. It’s the identical runtime path as callouts 1–9 within the previous structure diagram. The next diagram reveals it as a sequence so the order of occasions is extra easy to see.
The numbered steps within the previous diagram correspond to those levels of the decision:
- The caller dials the toll-free quantity and Amazon Chime SDK Voice Connector solutions.
- The Voice Connector invokes the SIP Media Software Lambda, which computes a session identifier for the decision.
- The Lambda sends a warmup request to AgentCore Runtime, so the agent prepares its session whereas the telephone remains to be ringing.
- The Lambda tells the Voice Connector to bridge the decision to the SIP gateway on Amazon ECS on AWS Fargate, passing the session identifier alongside.
- The SIP gateway opens a SigV4-signed WebSocket to AgentCore Runtime utilizing that very same session identifier. The decision attaches to the warmed session, caller audio flows to the agent, and the agent’s audio flows again to the caller.
- The agent runs the dialog with Amazon Nova 2 Sonic and calls backend instruments by AgentCore Gateway when it wants menu, cart, order, or location information.
Conditions
Earlier than you start, confirm that you’ve the next in place:
- An AWS account.
- Amazon Bedrock mannequin entry for Amazon Nova 2 Sonic within the AWS Area the place you deploy, requested on the mannequin entry web page within the Amazon Bedrock console.
- Amazon Chime SDK PSTN Audio entry, with a phone-number quota improve requested within the Amazon Chime SDK console in case you have by no means ordered a quantity on this account.
- Node.js 24.x or later.
- AWS Command Line Interface (AWS CLI) 2.x configured with credentials.
- git to clone the repository.
- AWS CDK bootstrapped in your goal account and Area (
npx cdk bootstrap aws://)./
The agent container is constructed with Python, however the construct runs inside AWS CodeBuild, so you do not want Python by yourself machine. Deploy in a Area the place Amazon Nova 2 Sonic, Amazon Chime SDK PSTN Audio, and AgentCore Runtime are all obtainable. US East (N. Virginia), us-east-1, is an efficient place to start out.
Deploy the answer with AWS CDK
The total answer is out there within the pattern repository on GitHub. Clone the repository and alter into the mission listing.
Run the preflight verify. It confirms that Node.js, the AWS CLI, git, the AWS CDK bootstrap, and Amazon Bedrock mannequin entry are all in place and stories something lacking.
Then run the deployment script with a deployment prefix. The prefix is added to each useful resource identify, which you should use to deploy the answer greater than as soon as in the identical account.
The script deploys every AWS CDK stack so as and passes the outputs of 1 stack to the following stack. It builds the backend first, then provides the AgentCore Gateway in entrance of the backend APIs. Subsequent, it builds and pushes the agent container picture with AWS CodeBuild and deploys the agent on AgentCore Runtime. Lastly, it brings up the SIP gateway on Amazon ECS on AWS Fargate and provisions the Amazon Chime SDK Voice Connector and toll-free quantity. It additionally seeds pattern menu and placement information so you may place an actual order as quickly because it finishes. The construct of the agent container takes a number of minutes the primary time, so count on the run to pause whereas AWS CodeBuild works.
When the script completes, it prints the quantity to dial:
How the SIP gateway works
The telephony layer has one job. It turns a telephone name right into a media stream the agent can learn and write. This issues as a result of the telephone community and the agent communicate completely different protocols. The telephone delivers audio as RTP packets over UDP. The agent expects a WebSocket carrying frames within the format Amazon Nova 2 Sonic understands. With out a translation layer in between, the agent would want to find out about SIP, codec negotiation, and network-level media routing, all of which might couple it to a single channel. Two parts deal with that translation.
The Amazon Chime SDK Voice Connector accepts the inbound name and invokes the SIP Media Software Lambda. That Lambda is the place the decision is steered. It returns an instruction to bridge the decision to the SIP gateway and carries the session identifier alongside so the following hop is aware of which session this name belongs to.
The SIP gateway runs on Amazon ECS on AWS Fargate. It makes use of drachtio-server for SIP signaling, with a Node.js bridge for the audio. The gateway solutions the decision and converts between the audio format the telephone community makes use of and the format Amazon Nova 2 Sonic expects. For prime availability (HA), it runs as two duties throughout two Availability Zones (AZs), which supplies it redundancy and a spot to publish a call-count metric to Amazon CloudWatch for scaling. The signaling for the decision passes by the Community Load Balancer, however the audio itself flows straight between the Voice Connector and the Fargate activity, which retains the load balancer off the media path.
Warming the agent whereas the telephone rings
A voice name is unforgiving about silence. If a caller connects and hears nothing for a number of seconds, the decision feels damaged regardless that the system is working. The gradual a part of beginning a name is the one-time setup. The agent has to resolve the system immediate for this caller, open the Amazon Nova 2 Sonic stream, and uncover the obtainable MCP instruments.
Reasonably than make the caller wait by that, the SIP Media Software Lambda kicks it off whereas the telephone remains to be ringing. As quickly as the decision arrives, the Lambda sends a warmup request to AgentCore Runtime with a session identifier for the decision. AgentCore Runtime allocates a microVM and runs the setup through the ringing window. A second later, when the SIP gateway opens its WebSocket connection utilizing the identical session identifier, AgentCore Runtime routes it to that already-warm microVM, and the agent is able to discuss.
The session identifier is what connects the 2 requests. The Lambda computes it from the decision itself, so the warmup request and the later audio connection resolve to the identical microVM with none further state to trace. The next diagram reveals how this overlaps with the ringing window so the agent is prepared when the decision connects.
Storing menus, carts, and orders
5 Amazon DynamoDB tables cowl the ordering workflow. The Prospects desk shops profiles, together with identify, telephone, and loyalty data, which the agent makes use of to acknowledge a returning caller. The Orders desk retains order historical past together with the pickup location. The Menu desk holds objects, costs, and availability, which may differ by location. The Carts desk holds in-progress carts and makes use of a time-to-live worth so deserted carts clear themselves up. The Places desk holds restaurant particulars resembling coordinates, hours, and tax charges that the agent makes use of for totals and proposals. DynamoDB on-demand capability scales with visitors, so there’s no throughput to handle.
Discovering a pickup location
Amazon Location Service helps a caller discover a handy pickup spot with out typing something. A telephone caller has no browser to share a location, so the agent asks for a ZIP code or a cross-street and makes use of Amazon Location Service to show that into coordinates. From there, the backend can do a number of issues with these coordinates. It may possibly discover the closest eating places, rank them by driving time moderately than straight-line distance so it favors a brief detour alongside the caller’s route, or geocode a selected tackle. That lets the agent say one thing a caller can act on, resembling “the closest location is on Predominant Road, about 5 minutes away,” as an alternative of studying again an inside code.
Processing voice with Amazon Bedrock AgentCore and Amazon Nova 2 Sonic
The agent runs on AgentCore Runtime. Every name runs in its personal microVM, so one caller’s session can’t have an effect on one other’s. AgentCore Runtime handles scaling and gives the WebSocket connection that carries audio each methods.
Inside the agent, the agent framework defines the system immediate, the instruments it will probably name, and the stream of the dialog. Amazon Nova 2 Sonic does the speech work throughout the name. It acknowledges speech throughout a spread of accents and handles the various audio high quality that comes with a telephone line. It streams audio in each instructions with low latency, and it calls instruments asynchronously with out stopping the dialog (see Async device calls in Amazon Nova Sonic), so the caller isn’t left ready whereas information is fetched. It additionally handles interruptions, so a caller can discuss over the agent the way in which folks do on human-to-human calls.
One element is restricted to telephone calls. A backend lookup can take a number of seconds, and Amazon Nova 2 Sonic ends a session that sits idle on the server aspect for too lengthy. To carry the session open throughout a gradual device name, the agent sends a silent body at a brief interval. The session stays alive and the caller hears nothing uncommon.
The agent by no means calls the backend Lambda capabilities straight. AgentCore Gateway sits in between and presents the backend endpoints as MCP instruments that the agent discovers and calls by identify, overlaying menu lookups, cart operations, order placement, buyer and order historical past, geocoding, and placement search. This implies the agent doesn’t must know which Lambda operate lives behind a given operation or the best way to authenticate to it.
That layer is what retains the design loosely coupled. When the agent calls a device resembling PlaceOrder, the gateway turns it right into a REST request to Amazon API Gateway, which routes it to the matching AWS Lambda operate. As a result of the agent talks to named instruments moderately than to particular capabilities, you may change a backend handler or add a device with out altering the agent. The identical backend may serve different channels, as a result of all of them place orders in opposition to the identical instruments and information.
The next diagram reveals how a single device name travels from the agent to the backend.
Recognizing a caller and not using a login
A telephone caller doesn’t register, so the system makes use of the caller’s telephone quantity as the idea for identification. The system hashes the quantity with a secret worth held on AWS Methods Supervisor Parameter Retailer, and the outcome turns into the session identifier. The uncooked telephone quantity doesn’t seem in logs or session state. This hashing method helps meet personally identifiable data (PII) dealing with necessities by guaranteeing that delicate caller data is rarely saved or transmitted in plain textual content.
If that identifier matches a recognized buyer, the agent greets the caller by identify and might supply their final order. If it doesn’t match, the caller orders as a visitor, and the order remains to be positioned and saved. Every order data the channel it got here from and whether or not the caller was a visitor, so you may determine telephony orders later.
This acknowledges a returning caller with out asking for a login, but it surely’s not identification verification, and the phone-number hash shouldn’t be handled as proof of who is asking. Anybody with entry to the identical telephone can place an order beneath that identification. A manufacturing deployment that wants verified identification can add a step resembling a one-time passcode despatched by SMS earlier than the agent opens the caller’s account historical past.
Ordering walkthrough
Dial the quantity from the deploy output. The agent greets you and asks what you prefer to. You’ll be able to communicate naturally, ask in regards to the menu, give a ZIP code for pickup, and make sure the order, all by voice. Whereas the caller talks, the agent calls backend instruments within the background, so there aren’t any pauses whereas information hundreds. The next video reveals a typical order from greeting to affirmation.
After the decision ends, you may hint the total path in Amazon CloudWatch Logs. The SIP gateway logs present how the decision was arrange and bridged to the agent. The agent logs present every agent occasion and gear name because the dialog progressed. As soon as the order is confirmed, it lands within the DynamoDB Orders desk with its whole and estimated prepared time.
Value
You pay for the AWS companies the system makes use of. The toll-free per-minute cost and the always-on Fargate duties are the 2 largest line objects. You’ll be able to decrease each if it matches your wants. A neighborhood Direct Inward Dialing quantity prices much less per minute than a toll-free quantity, and scaling the Fargate duties down exterior enterprise hours cuts the compute cost as soon as you understand your visitors. Test the pricing web page for every service for present charges, and arrange a finances in AWS Value Explorer to trace spend.
Clear up
To keep away from ongoing costs, take away the assets if you end up executed. The always-on Fargate duties and the toll-free quantity proceed to accrue price whether or not or not calls are coming in. The cleanup script deletes the stacks in reverse order so every stack is eliminated earlier than those it relies on.
Cleanup is harmful. It releases the toll-free quantity, deletes the order historical past in Amazon DynamoDB, removes the key on AWS Methods Supervisor Parameter Retailer, and deletes the container photographs in Amazon ECR. Again up something you need to preserve first. When the script finishes, verify within the AWS CloudFormation console that the stacks are gone (see Viewing AWS CloudFormation stack information and assets on the AWS Administration Console) and within the Amazon Chime SDK console that the toll-free quantity has been launched (see Managing telephone numbers in Amazon Chime SDK).
Conclusion
On this publish, we walked by a telephony voice ordering system from deployment to a accomplished order. The structure retains telephony, agent, and backend unbiased so each bit can evolve by itself. That separation is what makes the system sensible to function, as a result of you may replace menu information, swap in a brand new voice mannequin, or add a channel with out coordinating adjustments throughout the entire stack.
For the restaurant, this implies telephone orders stream in with out pulling employees away from the counter. Callers get a right away greeting as an alternative of maintain music, the agent confirms every merchandise out loud to scale back errors, and the system handles peak hours with out further headcount. As a result of the agent connects to the backend by MCP instruments, you may add capabilities like reservations or loyalty factors by registering new instruments with out altering the agent or the telephony layer.
To get began, clone the pattern repository on GitHub and run the preflight verify to verify your surroundings is prepared. From there, substitute the seed information within the DynamoDB desk definitions with your personal menu objects, areas, and pricing. Replace the system immediate templates within the Parameter Retailer entries to replicate your restaurant’s identify, greeting type, and any ordering guidelines. Then run the deploy script to deliver the system dwell by yourself toll-free quantity.
Concerning the authors
