This put up is cowritten with Harrison Hunter is the CTO and co-founder of MaestroQA.
MaestroQA augments name middle operations by empowering the standard assurance (QA) course of and buyer suggestions evaluation to extend buyer satisfaction and drive operational efficiencies. They help with operations akin to QA reporting, teaching, workflow automations, and root trigger evaluation.
On this put up, we dive deeper into one in all MaestroQA’s key options—dialog analytics, which helps assist groups uncover buyer issues, tackle factors of friction, adapt assist workflows, and establish areas for teaching by means of using Amazon Bedrock. We talk about the distinctive challenges MaestroQA overcame and the way they use AWS to construct new options, drive buyer insights, and enhance operational inefficiencies.
Amazon Bedrock is a totally managed service that gives a alternative of high-performing basis fashions (FMs) from main AI firms, akin to AI21 Labs, Anthropic, Cohere, Meta, Mistral, Stability AI, and Amazon by means of a single API, together with a broad set of capabilities to construct generative AI functions with safety, privateness, and accountable AI.
The chance for open-ended dialog evaluation at enterprise scale
MaestroQA serves a various clientele throughout numerous industries, together with ecommerce, marketplaces, healthcare, expertise acquisition, insurance coverage, and fintech. All of those prospects have a standard problem: the necessity to analyze a excessive quantity of interactions with their prospects. Analyzing these buyer interactions is essential to enhancing their product, enhancing their buyer assist, offering buyer satisfaction, and figuring out key business alerts. Nonetheless, buyer interplay information akin to name middle recordings, chat messages, and emails are extremely unstructured and require superior processing strategies to be able to precisely and robotically extract insights.
When prospects obtain incoming calls at their name facilities, MaestroQA employs its proprietary transcription expertise, constructed by enhancing open supply transcription fashions, to transcribe the conversations. After the information is transcribed, MaestroQA makes use of expertise they’ve developed together with AWS companies akin to Amazon Comprehend to run numerous kinds of evaluation on the client interplay information. For instance, MaestroQA affords sentiment evaluation for patrons to establish the sentiment of their finish buyer in the course of the assist interplay, enabling MaestroQA’s prospects to kind their interactions and manually examine the very best or worst interactions. MaestroQA additionally affords a logic/keyword-based guidelines engine for classifying buyer interactions based mostly on different components akin to timing or course of steps together with metrics like Common Deal with Time (AHT), compliance or course of checks, and SLA adherence.
MaestroQA’s prospects love these evaluation options as a result of they permit them to constantly enhance the standard of their assist and establish areas the place they’ll enhance their product to higher fulfill their finish prospects. Nonetheless, they had been additionally considering extra superior evaluation, akin to asking open-ended questions like “What number of instances did the client ask for an escalation?” MaestroQA’s current guidelines engine couldn’t at all times reply these kinds of queries as a result of end-users might ask for a similar final result in many alternative methods. For instance, “Can I converse to your supervisor?” and “I wish to converse to somebody increased up” don’t share the identical key phrases, however are each asking for an escalation. MaestroQA wanted a solution to precisely classify buyer interactions based mostly on open-ended questions.
MaestroQA confronted an extra hurdle: the immense scale of buyer interactions their purchasers handle. With purchasers dealing with wherever from 1000’s to tens of millions of buyer engagements month-to-month, there was a urgent want for complete evaluation of assist group efficiency throughout this huge quantity of interactions. Consequently, MaestroQA needed to develop an answer able to scaling to fulfill their purchasers’ in depth wants.
To begin growing this product, MaestroQA first rolled out a product known as AskAI. AskAI allowed prospects to run open-ended questions on a focused record of as much as 1,000 conversations. For instance, a buyer may use MaestroQA’s filters to search out buyer interactions in Oregon throughout the previous two months after which run a root trigger evaluation question akin to “What are prospects pissed off about in Oregon?” to search out churn threat anecdotes. Their prospects actually preferred this characteristic and stunned MaestroQA with the breadth of use instances they lined, together with analyzing advertising campaigns, service points, and product alternatives. Prospects began to request the flexibility to run such a evaluation throughout all of their transcripts, which might quantity within the tens of millions, so they may quantify the impression of what they had been seeing and discover cases of essential points.
Answer overview
MaestroQA determined to make use of Amazon Bedrock to handle their prospects’ want for superior evaluation of buyer interplay transcripts. Amazon Bedrock’s broad alternative of FMs from main AI firms, together with its scalability and security measures, made it a really perfect answer for MaestroQA.
MaestroQA built-in Amazon Bedrock into their current structure utilizing Amazon Elastic Container Service (Amazon ECS). The client interplay transcripts are saved in an Amazon Easy Storage Service (Amazon S3) bucket.
The next structure diagram demonstrates the request circulate for AskAI. When a buyer submits an evaluation request by means of MaestroQA’s internet software, an ECS cluster retrieves the related transcripts from Amazon S3, cleans and codecs the immediate, sends them to Amazon Bedrock for evaluation utilizing the client’s chosen FM, and shops the leads to a database hosted in Amazon Elastic Compute Cloud (Amazon EC2), the place they are often retrieved by MaestroQA’s frontend internet software.
MaestroQA affords their prospects the flexibleness to select from a number of FMs out there by means of Amazon Bedrock, together with Anthropic’s Claude 3.5 Sonnet, Anthropic’s Claude 3 Haiku, Mistral 7b/8x7b, Cohere’s Command R and R+, and Meta’s Llama 3.1 fashions. This permits prospects to pick out the mannequin that most closely fits their particular use case and necessities.
The next screenshot reveals how the AskAI characteristic permits MaestroQA’s prospects to make use of the big variety of FMs out there on Amazon Bedrock to ask open-ended questions akin to “What are among the widespread points in these tickets?” and generate helpful insights from customer support interactions.
To deal with the excessive quantity of buyer interplay transcripts and supply low-latency responses, MaestroQA takes benefit of the cross-Area inference capabilities of Amazon Bedrock. Initially, they had been doing the load balancing themselves, distributing requests between out there AWS US Areas (us-east-1, us-west-2, and so forth) and out there EU Areas (eu-west-3, eu-central-1, and so forth) for his or her North American and European prospects, respectively. Now, the cross-Area inference functionality of Amazon Bedrock permits MaestroQA to attain twice the throughput in comparison with single-Area inference, a important think about scaling their answer to accommodate extra prospects. MaestroQA’s group not has to spend effort and time to foretell their demand fluctuations, which is particularly key when utilization will increase for his or her ecommerce prospects across the vacation season. Cross-Area inference dynamically routes visitors throughout a number of Areas, offering optimum availability for every request and smoother efficiency throughout these high-usage durations. MaestroQA screens this setup’s efficiency and reliability utilizing Amazon CloudWatch.
Advantages: How Amazon Bedrock added worth
Amazon Bedrock has enabled MaestroQA to innovate quicker and achieve a aggressive benefit by providing their prospects highly effective generative AI options for analyzing buyer interplay transcripts. With Amazon Bedrock, MaestroQA can now present their prospects with the flexibility to run open-ended queries throughout tens of millions of transcripts, unlocking precious insights that had been beforehand inaccessible.
The broad alternative of FMs out there by means of Amazon Bedrock permits MaestroQA to cater to their prospects’ various wants and preferences. Prospects can choose the mannequin that finest aligns with their particular use case, discovering the fitting steadiness between efficiency and worth.
The scalability and cross-Area inference capabilities of Amazon Bedrock allow MaestroQA to deal with excessive volumes of buyer interplay transcripts whereas sustaining low latency, no matter their prospects’ geographical places.
MaestroQA takes benefit of the strong security measures and moral AI practices of Amazon Bedrock to bolster buyer confidence. These measures make it possible for shopper information stays safe throughout processing and isn’t used for mannequin coaching by third-party suppliers. Moreover, Amazon Bedrock availability in Europe, coupled with its geographic management capabilities, permits MaestroQA to seamlessly prolong AI companies to European prospects. This enlargement is achieved with out introducing extra complexities, thereby sustaining operational effectivity whereas adhering to Regional information laws.
The adoption of Amazon Bedrock proved to be a sport changer for MaestroQA’s compact growth group. Its serverless structure allowed the group to quickly prototype and refine their software with out the burden of managing advanced {hardware} infrastructure. This shift enabled MaestroQA to channel their efforts into optimizing software efficiency slightly than grappling with useful resource allocation. Furthermore, Amazon Bedrock affords seamless compatibility with their current AWS setting, permitting for a easy integration course of and additional streamlining their growth workflow. MaestroQA was ready to make use of their current authentication course of with AWS Identification and Entry Administration (IAM) to securely authenticate their software to invoke giant language fashions (LLMs) inside Amazon Bedrock. They had been additionally ready to make use of the acquainted AWS SDK to shortly and effortlessly combine Amazon Bedrock into their software.
Total, through the use of Amazon Bedrock, MaestroQA is ready to present their prospects with a strong and versatile answer for extracting precious insights from their buyer interplay information, driving steady enchancment of their merchandise and assist processes.
Success metrics
The early outcomes have been outstanding.
A lending firm makes use of MaestroQA to detect compliance dangers on 100% of their conversations. Earlier than, brokers would elevate inner escalations if a shopper complained concerning the mortgage or expressed being in a susceptible state. Nonetheless, this course of was handbook and error susceptible, and the lending firm would miss many of those dangers. Now, they can detect compliance dangers with nearly 100% accuracy.
A medical gadget firm, who’s required to report gadget points to the FDA, not depends solely on brokers to report internally customer-reported points, however makes use of this service to investigate all of their conversations to verify all complaints are flagged.
An training firm has been in a position to substitute their handbook survey scores with an automatic buyer sentiment rating that elevated their pattern dimension from 15% to 100% of conversations.
One of the best is but to come back.
Conclusion
Utilizing AWS, MaestroQA was in a position to innovate quicker and achieve a aggressive benefit. Firms from totally different industries akin to monetary companies, healthcare and life sciences, and EdTech all share the widespread want to supply higher buyer companies for his or her purchasers. MaestroQA was in a position to allow them to do this by shortly pivoting to supply highly effective generative AI options that solved tangible enterprise issues and enhanced general compliance.
Take a look at MaestroQA’s characteristic AskAI and their LLM-powered AI Classifiers in case you’re considering higher understanding your buyer conversations and survey scores. For extra about Amazon Bedrock, see Get began with Amazon Bedrock and study options akin to cross-Area inference to assist scale your generative AI options globally.
In regards to the Authors
Carole Suarez is a Senior Options Architect at AWS, the place she helps information startups by means of their cloud journey. Carole makes a speciality of information engineering and holds an array of AWS certifications on a wide range of matters together with analytics, AI, and safety. She is obsessed with studying languages and is fluent in English, French, and Tagalog.
Ben Gruher is a Generative AI Options Architect at AWS, specializing in startup prospects. Ben graduated from Seattle College the place he obtained bachelor’s and grasp’s levels in Laptop Science and Knowledge Science.
Harrison Hunter is the CTO and co-founder of MaestroQA the place he leads the engineer and product groups. Previous to MaestroQA, Harrison studied laptop science and AI at MIT.
