This submit is co-written with Mark Berkeland, Oscar Rodriguez and Marina Gerzon from Vonage.
Voice-based applied sciences are reworking the way in which companies have interaction with clients throughout buyer help, digital assistants, and clever brokers. Nonetheless, creating real-time, expressive, and extremely responsive voice interfaces nonetheless requires navigating a posh stack of communication protocols, AI fashions, and media infrastructure. To simplify this course of, Vonage has built-in Amazon Nova Sonic, our speech-to-speech basis mannequin (FM), with the Vonage Voice API, a part of their Communications Platform as a Service (CPaaS) providing.
With this integration, builders can deploy AI voice brokers to allow extra human-like voice conversations over cellphone calls, SIP connections, WebRTC, and cell apps. The answer makes it simple to convey clever, real-time conversations into workflows for a wide range of use instances, reminiscent of a small auto restore store utilizing voice AI to e book appointments and observe down elements, a worldwide retail model dealing with a excessive quantity of customer support calls, or a developer constructing a scalable voice interface.
On this submit, we discover how builders can combine Amazon Nova Sonic with the Vonage communications service to construct responsive, natural-sounding voice experiences in actual time. By combining the Vonage Voice API with the low-latency and expressive speech capabilities of Amazon Nova Sonic, companies can deploy AI voice brokers that ship extra human-like interactions than conventional voice interfaces. These brokers can be utilized as buyer help, digital assistants, and extra.
Amazon Nova Sonic for real-time conversational AI
Amazon Nova Sonic is a speech-to-speech FM designed to construct real-time conversational AI purposes in Amazon Bedrock, with industry-leading price-performance and low latency. Its structure unifies speech understanding and technology right into a single mannequin, to allow extra human-like voice conversations in AI purposes. The mannequin can perceive speech in numerous talking types and generate speech in expressive voices, together with each masculine-sounding and feminine-sounding voices. Amazon Nova Sonic can adapt the intonation, prosody, and elegance of the generated speech response to align with the context and content material of the speech enter and gracefully deal with interruptions. Moreover, Amazon Nova Sonic permits for perform calling and information grounding with enterprise information utilizing Retrieval Augmented Technology (RAG).
Vonage Voice APIs, powered by AI
Vonage, an AWS associate, supplies a developer-friendly platform for constructing voice, messaging, video, and authentication experiences. With its wide-ranging Voice APIs, Vonage affords WebRTC help, multi-channel communication instruments, normal cellphone name integrations, in-app softphones, front-ending contact facilities, and voice-over-browser performance. The software program additionally affords important constructing blocks reminiscent of inbound and outbound voice name dealing with, voicemail help, and programmable logic for name routing and queuing. Vonage’s resolution builder and SDKs enable for quick, low-code integration, whereas its interoperability with enterprise purposes and productiveness instruments permits groups to embed communication immediately into their present workflows.
Resolution overview
Vonage collaborated with Amazon Nova Sonic to construct low-latency, voice-first purposes that may perceive and reply like a human agent over normal telephony or WebRTC channels. This new instrument can join inbound and outbound Vonage calls on to Amazon Nova Sonic for conversational AI processing, utilizing expressive, real-time speech synthesis to ship fluid, pure interactions. Amazon Nova Sonic’s integration into Vonage Voice API seamlessly manages audio buffering, customized media infrastructure, and protocol translation, so groups can concentrate on constructing participating experiences.
With built-in dialog management logic and noise cancellation, Vonage’s integration with Amazon Nova Sonic makes it simple for companies to quickly construct and deploy responsive AI voice brokers. These brokers can deal with real-time voice conversations and scale voice interactions with out counting on conventional contact facilities.
Vonage is making this integration obtainable as a GitHub repository for builders to deploy and customise to their wants.
“As an AWS Amazon Accomplice Community (APN) member, Vonage has an extended historical past of working carefully with the AWS innovation staff to create new options to profit enterprise clients,” stated Christophe Van de Weyer, President and Head of Enterprise Unit API for Vonage. “This newest collaboration with AWS permits organizations to rework how they have interaction with clients by adopting generative AI options that create added worth for inner and exterior communication. By combining Vonage’s communications APIs with AWS’s superior AI, this new voice AI agent expertise permits companies to streamline the adoption of clever brokers, speed up the modernization of legacy voice techniques, and supply a strong service to ship distinctive buyer experiences with measurable enhancements in satisfaction and operational effectivity.”
The next video showcases a demo of Diana, an AI voice agent constructed utilizing Vonage’s integration with Amazon Nova Sonic.
The next structure diagram supplies an outline of Amazon Nova Sonic deployed as a voice agent within the Vonage Voice API framework on AWS.
The answer routes various kinds of incoming calls to Amazon Nova Sonic over a WebSocket connection. The architectural elements embrace (left to proper):
- Calls – Incoming voice connections that may come from international cellphone numbers, SIP connections with contact facilities or enterprise techniques, or WebRTC connections from net browsers and cell apps.
- Vonage Voice API – Gives programmatic management over these kind of calls and voice connections, permitting them to be built-in with AI techniques, routed elsewhere, or given speech and different remedies. As a result of Amazon Nova Sonic is a full speech-to-speech AI service, the real-time voice streams are linked immediately, in contrast to different AI integrations which may use text-based integration.
- Amazon Nova Sonic connector – A Vonage integration that connects calls to Amazon Nova Sonic over a WebSocket connection, offering low-latency, real-time, bi-directional voice streaming immediately with Amazon Nova Sonic. The connector additionally manages voice isolation to raised deal with noisy environments, conversational parts like “barge in” the place the caller interrupts the dialog, and fallback choices if wanted.
- Amazon Nova Sonic – A part of the Amazon Nova household of FMs obtainable in Amazon Bedrock. Amazon Nova Sonic unifies speech understanding and technology right into a single mannequin, streamlining improvement and decreasing complexity when constructing conversational purposes.
- Retrieval Augmented Technology (RAG) – Instruments inside Amazon Bedrock that optimize the output of an underlying massive language mannequin (LLM). Amazon Nova Sonic can reference enterprise-authorized information sources. Attribution and supply visibility will be configured primarily based on buyer necessities.
- Customizable immediate – Offered to the AI mannequin and permits the voice agent’s persona and conversational capabilities to be outlined and the precise information base for use.
- Consumer context – Maintained by Amazon Nova Sonic all through interplay sequences to permit a pure steady dialog. Personally identifiable data (PII) is processed in actual time and never retained by Amazon Nova Sonic. AWS safeguards your information by way of complete safety controls, encryption at relaxation and in transit, and compliance certifications, whereas additionally providing you with the flexibleness to configure extra logging, safety, and compliance measures by way of AWS companies.
These elements work collectively to create a versatile, clever voice agent service that may dynamically adapt to totally different communication eventualities and enterprise use instances with totally different information bases and prompts.
Instance use instances
The next are only a few of the high-impact methods companies are already utilizing this integration to rework voice interactions:
- Buyer help automation – Deploy voice brokers that reply inbound buyer queries, take appointments, and escalate calls solely when vital.
- Proactive outbound calling – Generate dynamic, expressive outbound messages like reminders, confirmations, or follow-ups with voicemail fallback.
- Multilingual voice assistants – Construct voice experiences that seamlessly change between English and Spanish relying on the caller, enabled by Vonage’s language detection and multilingual synthesis with Amazon Nova Sonic.
Conclusion
By combining Amazon Nova Sonic with Vonage’s versatile communication infrastructure, builders can construct clever, responsive AI voice brokers. With this resolution, you may present proactive voice engagement, create multilingual assistants, deal with buyer help, and extra. This integration makes voice-first AI purposes extra accessible and scalable than ever.
To start out constructing with Amazon Nova Sonic, go to the Amazon Bedrock console. For Vonage integration, discover the Vonage API Developer Portal or use the Vonage Resolution Builder to configure your voice agent in minutes.
To be taught extra about Amazon Nova Sonic, try the AWS Information Weblog, Amazon Nova Sonic product web page, or Amazon Bedrock Consumer Information.
Concerning the authors
Divyesha Malhotra is a Senior Product Supervisor Technical Intern on the AGI Nova Sonic staff. She leads the client adoption and integrations of cutting-edge speech-to-speech basis fashions for next-generation voice-based applied sciences.
Mark Berkeland is a Senior Options Engineer within the API Enterprise Unit at Vonage. He designs and implements technical options together with demos and proofs of idea to assist clients convey voice and messaging purposes to life. With an expert programming profession that started in 1979, his expertise ranges from FORTRAN on punched playing cards to trendy cloud-native stacks like React Native, combining deep technical experience with a ardour for making advanced concepts accessible.
Oscar Rodriguez is Senior Director of International Accomplice Options within the API Enterprise Unit at Vonage, the place he leads strategic initiatives to empower companions by way of scalable communications options. He brings deep technical experience and a sensible understanding of real-world utility improvement with over 20 years expertise in net applied sciences and the final 10 in CPaaS.
Marina Gerzon is a Accomplice Options Architect at Vonage with over 20 years of expertise in real-time communications, specializing in Video and Voice over IP options. Identified for her skill to bridge technical depth with enterprise affect, her work spans Telecom, Training, Healthcare, Fintech, and Insurance coverage industries, the place she has persistently delivered enterprise-grade SaaS and PaaS architectures tailor-made to advanced enterprise wants.
