Deploy a full stack voice AI agent with Amazon Nova Sonic

AI-powered speech options are remodeling contact facilities by enabling pure conversations between clients and AI brokers, shortening wait instances, and dramatically lowering operational prices—all with out sacrificing the human-like interplay clients count on. With the current launch of Amazon Nova Sonic in Amazon Bedrock, now you can construct refined conversational AI brokers that talk naturally via voice, with out the necessity for separate speech recognition and text-to-speech parts. Amazon Nova Sonic is a speech-to-speech mannequin in Amazon Bedrock that permits real-time, human-like voice conversations.

Whereas many early Amazon Nova Sonic implementations centered on native improvement, this resolution gives an entire cloud-deployed structure that you should use as a basis for constructing actual proof of idea purposes. This asset is deployable via the AWS Cloud Improvement Equipment (AWS CDK) and gives a basis for constructing additional Amazon Nova use instances utilizing preconfigured infrastructure parts, whereas permitting you to customise the structure to handle your particular enterprise necessities.

On this submit, we present the best way to create an AI-powered name heart agent for a fictional firm known as AnyTelco. The agent, named Telly, can deal with buyer inquiries about plans and providers whereas accessing real-time buyer information utilizing customized instruments applied with the Mannequin Context Protocol (MCP) framework.

Answer overview

The next diagram gives an summary of the deployable resolution.

The answer consists of the next layers:

• Frontend layer – The frontend layer of this method is constructed with scalability and efficiency in thoughts:
• Communication layer – The communication layer facilitates seamless real-time interactions:
  • Community Load Balancer manages WebSocket connections. WebSockets allow two-way interactive communication periods between a person’s browser and the server, which is crucial for real-time audio streaming purposes.
  • Amazon Cognito gives person authentication and JSON net token (JWT) validation. Amazon Cognito gives person authentication, authorization, and person administration for net and cellular purposes, assuaging the necessity to construct and keep your personal identification methods.
• Processing layer – The processing layer types the computational spine of the system:
  • Amazon Elastic Container Service (Amazon ECS) runs the containerized backend service.
  • AWS Fargate gives the serverless compute backend. Orchestration is offered by the Amazon ECS engine.
  • The Python backend processes audio streams and manages Amazon Nova Sonic interactions.
• Intelligence layer – The intelligence layer makes use of AI and information applied sciences to energy the core functionalities:
  • The Amazon Nova Sonic mannequin in Amazon Bedrock handles speech processing.
  • Amazon DynamoDB shops buyer data.
  • Amazon Bedrock Information Bases connects basis fashions (FMs) together with your group’s information sources, permitting AI purposes to reference correct, up-to-date data particular to your corporation.

The next sequence diagram highlights the stream when a person initiates dialog. The person solely indicators in a single time, however authentication Steps 3 and 4 occur each time the person begins a brand new session. The conversational loop in Steps 6–12 is repeated all through the conversational interplay. Steps a–c solely occur when the Amazon Nova Sonic agent decides to make use of a device. In situations with out device use, the stream goes immediately from Step 9 to Step 10.

Conditions

Earlier than getting began, confirm that you’ve the next:

Deploy the answer

You’ll find the answer and full deployment directions on the GitHub repository. The answer makes use of the AWS CDK to automate infrastructure deployment. Use the next code terminal instructions to get began in your AWS Command Line Interface (AWS CLI) atmosphere:

git clone https://github.com/aws-samples/sample-sonic-cdk-agent.git 
cd nova-s2s-call-center 

# Configure atmosphere variables
cp template.env .env

# Edit .env together with your settings

# Deploy the answer 
./deploy.sh 

The deployment creates two AWS CloudFormation stacks:

• Community stack for digital personal cloud (VPC) and networking parts
• Stack for software assets

The output of the second stack provides you a CloudFront distribution hyperlink, which takes you to the login web page.

You possibly can create an Amazon Cognito admin person with the next AWS CLI command:

aws cognito-idp admin-create-user 
  --user-pool-id YOUR_USER_POOL_ID 
  --username USERNAME 
  --user-attributes Identify=electronic mail,Worth=USER_EMAIL 
  --temporary-password TEMPORARY_PASSWORD 
  --region YOUR_AWS_REGION

The previous command makes use of the next parameters:

• YOUR_USER_POOL_ID: The ID of your Amazon Cognito person pool
• USERNAME: The specified person title for the person
• USER_EMAIL: The e-mail handle of the person
• TEMPORARY_PASSWORD: A short lived password for the person
• YOUR_AWS_REGION: Your AWS Area (for instance, us-east-1)

Log in together with your non permanent password from the CloudFront distribution hyperlink, and you may be requested to set a brand new password.

You possibly can select Begin Session to start out a dialog together with your assistant. Experiment with prompts and completely different instruments in your use case.

Customizing the applying

A key function of this resolution is its flexibility—you may tailor the AI agent’s capabilities to your particular use case. The pattern implementation demonstrates this extensibility via customized instruments and information integration:

• Buyer data lookup – Retrieves buyer profile information from DynamoDB utilizing telephone numbers as keys
• Information base search – Queries an Amazon Bedrock information base for firm data, plan particulars, and pricing

These options showcase the best way to improve the performance of Amazon Nova Sonic with exterior information sources and domain-specific information. The structure is designed for seamless customization in a number of key areas.

Modifying the system immediate

The answer features a UI in which you’ll regulate the AI agent’s conduct by modifying its system immediate. This permits speedy iteration on the agent’s character, information base, and dialog model with out redeploying your entire software.

Including new instruments

You can too lengthen the AI agent’s capabilities by implementing further instruments utilizing the MCP framework. The method includes:

• Implementing the device logic, usually as a brand new Python module
• Registering the device with the MCP server through the use of the @mcp_server.device customized decorator and defining the device specification, together with its title, description, and enter schema in /backend/instruments/mcp_tool_registry.py

For instance, the next code illustrates the best way to add a information base lookup device:

@mcp_server.device(
    title="lookup",
    description="Runs question in opposition to a information base to retrieve data."
)
async def lookup_tool(
    question: Annotated[str, Field(description="the query to search")]
) -> dict:
    """Lookup data within the information base"""
    outcomes = knowledge_base_lookup.principal(question)
    return outcomes

The decorator handles registration with the MCP server, and the operate physique comprises your device’s implementation logic.

Increasing the information base

The answer makes use of Amazon Bedrock Information Bases to supply the AI agent with company-specific data. You possibly can replace this data base with:

• Regularly requested questions and their solutions
• Product catalogs and specs
• Firm insurance policies and procedures

Clear up

You possibly can take away the stacks with the next command:

# transfer to the cdk folder, assuming you might be within the challenge root folder
cd cdk
# Removes each stacks sequentially
npx cdk destroy --all

Conclusion

AI brokers are remodeling how organizations method customer support, with options providing the flexibility to deal with a number of conversations concurrently, present constant service across the clock, and scale immediately whereas sustaining high quality and lowering operational prices. This resolution makes these advantages accessible by offering a deployable basis for Amazon Nova Sonic purposes on AWS. The answer demonstrates how AI brokers can successfully deal with buyer inquiries, entry real-time information, and supply customized service—all whereas sustaining the pure conversational stream that clients count on.

By combining the Amazon Nova Sonic mannequin with a strong cloud structure, safe authentication, and versatile device integration, organizations can rapidly transfer from idea to proof of idea. This resolution isn’t just serving to construct voice AI purposes, it’s serving to firms drive higher buyer satisfaction and productiveness throughout a spread of industries.

To study extra, seek advice from the next assets:

Concerning the authors

Reilly Manton is a Options Architect in AWS Telecoms Prototyping. He combines visionary pondering and technical experience to construct modern options. Specializing in generative AI and machine studying, he empowers telco clients to reinforce their technological capabilities.

Shuto Araki is a Software program Improvement Engineer at AWS. He works with clients in telecom business specializing in AI safety and networks. Outdoors of labor, he enjoys biking all through the Netherlands.

Ratan Kumar is a Principal Options Architect at Amazon Net Providers.A trusted know-how advisor with over 20 years of expertise working throughout a spread of business domains, Ratan’s ardour lies in empowering enterprise clients innovate and remodel their enterprise by unlocking the potential of AWS cloud.

Chad Hendren is a Principal Options Architect at Amazon Net Providers. His ardour is AI/ML and Generative AI utilized to Buyer Expertise. He’s a broadcast creator and inventor with 30 years of telecommunications expertise.