This publish is co-authored by Jay Kshirsagar and Ronald Quan from Qualtrics. The content material and opinions on this publish are these of the third-party creator and AWS is just not answerable for the content material or accuracy of this publish.
Qualtrics, based in 2002, is a pioneering software program firm that has spent over 20 years creating distinctive frontline experiences, constructing high-performing groups, and designing merchandise that folks love. Because the creators and stewards of the Expertise Administration (XM) class, Qualtrics serves over 20,000 purchasers globally, bringing humanity, connection, and empathy again to companies throughout numerous industries, together with retail, authorities, and healthcare.
Qualtrics’s complete XM platform allows organizations to persistently perceive, measure, and enhance the experiences they ship for patrons, workers, and the broader market. With its three core product suites—XM for Buyer Expertise, XM for Worker Expertise, and XM for Analysis & Technique—Qualtrics supplies actionable insights and purpose-built options that empower firms to ship distinctive experiences.
Qualtrics harnesses the ability of generative AI, cutting-edge machine studying (ML), and the most recent in pure language processing (NLP) to offer new purpose-built capabilities which might be precision-engineered for expertise administration (XM). These AI capabilities are purpose-built to assist organizations of all sizes deeply perceive and deal with the wants of each buyer, worker, and stakeholder—driving stronger connections, elevated loyalty, and sustainable progress.
On this publish, we share how Qualtrics constructed an AI platform powered by Amazon SageMaker and Amazon Bedrock.
AI at Qualtrics
Qualtrics has a deep historical past of utilizing superior ML to energy its industry-leading expertise administration platform. Early 2020, with the push for deep studying and transformer fashions, Qualtrics created its first enterprise-level ML platform referred to as Socrates. Constructed on prime of SageMaker, this new platform enabled ML scientists to effectively construct, check, and ship new AI-powered capabilities for the Qualtrics XM suite. This robust basis in ML and AI has been a key driver of Qualtrics’s innovation in expertise administration.
Qualtrics AI, a strong engine that sits on the coronary heart of the corporate’s XM platform, harnesses the most recent advances in ML, NLP, and AI. Skilled on Qualtrics’s expansive database of human sentiment and expertise information, Qualtrics AI unlocks richer, extra personalised connections between organizations and their clients, workers, and stakeholders. Qualtrics’s unwavering dedication to innovation and buyer success has solidified its place as the worldwide chief in expertise administration.
To study extra about how AI is remodeling expertise administration, go to this weblog from Qualtrics.
Socrates platform: Powering AI at Qualtrics
Qualtrics AI is powered by a custom-built ML platform, a synergistic suite of instruments and providers designed to allow a various set of Qualtrics personae—researchers, scientists, engineers, and data employees—to harness the transformative energy of AI and ML. Qualtrics refers to it internally because the “Socrates” platform. It makes use of managed AWS providers like SageMaker and Amazon Bedrock to allow your entire ML lifecycle. Information employees can supply, discover, and analyze Qualtrics information utilizing Socrates’s ML workbenches and AI Information Infrastructure. Scientists and researchers are enabled to conduct analysis, prototype, develop, and practice fashions utilizing a bunch of SageMaker options. ML engineers can check, productionize, and monitor a heterogeneous set of ML fashions possessing a variety of capabilities, inference modes, and manufacturing visitors patterns. Accomplice utility groups are supplied with an abstracted mannequin inference interface that makes the combination of an ML mannequin into the Qualtrics product a seamless engineering expertise. This holistic strategy allows inner groups to seamlessly combine superior AI and ML capabilities into their workflows and decision-making processes.
Science Workbench
The Socrates Science Workbench, purpose-built for Qualtrics Information and Information Staff, supplies a strong platform for mannequin coaching and hyperparameter optimization (HPO) with a JupyterLab interface, assist for a spread of programming languages, and safe, scalable infrastructure by SageMaker integration, giving customers the flexibleness and reliability to deal with their core ML duties. Customers can reap the benefits of the strong and dependable infrastructure of SageMaker to take care of the confidentiality and integrity of their information and fashions, whereas additionally benefiting from the scalability that SageMaker supplies to deal with even probably the most demanding ML workloads.
AI Information Infrastructure
Socrates’s AI Information Infrastructure is a complete and cohesive end-to-end ML information ecosystem. It encompasses a safe and scalable information retailer built-in with the Socrates Science Workbench, enabling customers to effortlessly retailer, handle, and share datasets with capabilities for anonymization, schematization, and aggregation. The AI Information Infrastructure additionally supplies scientists with interfaces for distributed compute, information pulls and enrichment, and ML processing.
AI Playground
The AI Playground is a user-friendly interface that gives Socrates customers with direct entry to the highly effective language fashions and different generative AI capabilities hosted on the Socrates platform utilizing backend instruments like SageMaker Inference, Amazon Bedrock, and OpenAI GPT, permitting them to experiment and quickly prototype new concepts with out in depth coding or technical experience. By constantly integrating the most recent fashions, the AI Playground empowers Socrates customers to remain on the forefront of developments in massive language fashions (LLMs) and different cutting-edge generative AI applied sciences, exploring their potential and discovering new methods to drive innovation.
Mannequin deployment for inference
The Socrates platform encompasses a subtle mannequin deployment infrastructure that’s important for the scalable implementation of ML and AI fashions. This infrastructure permits customers to host fashions throughout the number of {hardware} choices accessible for SageMaker endpoints, offering the flexibleness to pick out a deployment atmosphere that optimally meets their particular wants for inference, whether or not these wants are associated to efficiency optimization, cost-efficiency, or explicit {hardware} necessities.
One of many defining traits of the Socrates mannequin deployment infrastructure is its functionality to simplify the complexities of mannequin internet hosting. This enables customers to focus on the important process of deploying their fashions for inference inside the bigger Socrates ecosystem. Customers profit from an environment friendly and user-friendly interface that allows them to effortlessly bundle their fashions, modify deployment settings, and put together them for inference use.
By providing an adaptable mannequin deployment resolution, the Socrates platform makes certain ML fashions created inside the system are easily built-in into real-world purposes and workflows. This integration not solely hurries up the transition to manufacturing but additionally maximizes the utilization of Qualtrics’s AI-driven options, fostering innovation and offering important enterprise worth to its clients.
Mannequin capability administration
Mannequin capability administration is a essential element that gives environment friendly and dependable supply of ML fashions to Qualtrics customers by offering oversight of mannequin entry and the allocation of computing sources throughout a number of customers. The Socrates workforce intently displays useful resource utilization and units up charge limiting and auto scaling insurance policies, the place relevant, to fulfill the evolving calls for of every use case.
Unified GenAI Gateway
The Socrates platform’s Unified GenAI Gateway simplifies and streamlines entry to LLMs and embedding fashions throughout the Qualtrics ecosystem. The Unified GenAI Gateway is an API that gives a typical interface for customers to work together with all the platform-supported LLMs and embedding fashions, no matter their underlying suppliers or internet hosting environments. Which means that Socrates customers can use the ability of cutting-edge language fashions with out having to fret concerning the complexities of integrating with a number of distributors or managing self-hosted fashions.
The standout function of the Unified GenAI Gateway is its centralized integration with inference platforms like SageMaker Inference and Amazon Bedrock. which permits the Socrates workforce to deal with the intricate particulars of mannequin entry, authentication, and attribution on behalf of customers. This not solely simplifies the consumer expertise but additionally allows price attribution and management mechanisms, ensuring the consumption of those highly effective AI sources is rigorously monitored and aligned with particular use circumstances and billing codes. Moreover, the Unified GenAI Gateway boasts capabilities like rate-limiting assist, ensuring the system’s sources are effectively allotted, and an upcoming semantic caching function that may additional optimize mannequin inference and improve general efficiency.
Managed Inference APIs (powered by SageMaker Inference)
The Socrates Managed Inference APIs present a complete suite of providers that simplify the combination of superior ML and AI capabilities into Qualtrics purposes. This infrastructure, constructed on prime of SageMaker Inference, handles the complexities of mannequin deployment, scaling, and upkeep, boasting a rising catalog of production-ready fashions.
Managed Inference APIs supply each asynchronous and synchronous modes to accommodate a variety of utility use circumstances. Importantly, these managed APIs include assured production-level SLAs, offering dependable efficiency and cost-efficiency as utilization scales. With available pre-trained Qualtrics fashions for inference, the Socrates platform empowers Qualtrics utility groups to deal with delivering distinctive consumer experiences, with out the burden of constructing and sustaining AI infrastructure.
GenAI Orchestration Framework
Socrates’s GenAI Orchestration Framework is a group of instruments and patterns designed to streamline the event and deployment of LLM-powered purposes inside the Qualtrics ecosystem. The framework consists of such instruments/frameworks akin to:
- Socrates Agent Platform, constructed on prime of LangGraph Platform offering a versatile orchestration framework to develop brokers as graphs that expedite supply of agentic options whereas centralizing core infrastructure and observability elements.
- A GenAI SDK, offering simple coding comfort for interacting with LLMs and third-party orchestration packages
- Immediate Lifecycle Administration Service (PLMS) for sustaining the safety and governance of prompts
- LLM guardrail tooling, enabling LLM customers to outline the protections they need utilized to their mannequin inference
- Synchronous and asynchronous inference gateways
These instruments all contribute to the general reliability, scalability, and efficiency of the LLM-powered purposes constructed upon it. Capabilities of the Socrates AI App Framework are anticipated to develop and evolve alongside the fast developments within the subject of LLMs. Which means that Qualtrics customers at all times have entry to the most recent and most cutting-edge AI capabilities from generative AI inference platforms like SageMaker Inference and Amazon Bedrock, empowering them to harness the transformative energy of those applied sciences with better ease and confidence.
Ongoing enhancements to the Socrates platform utilizing SageMaker Inference
Because the Socrates platform continues to evolve, Qualtrics is constantly integrating the most recent developments in SageMaker Inference to additional improve the capabilities of their AI-powered ecosystem:
- Improved price, efficiency, and usefulness of generative AI inference – One distinguished space of focus is the combination of price and efficiency optimizations for generative AI inference. The SageMaker Inference workforce has launched modern methods to optimize using accelerators, enabling SageMaker Inference to cut back basis mannequin (FM) deployment prices by 50% on common and latency by 20% on common with inference elements. Utilizing this function, we’re engaged on reaching important price financial savings and efficiency enhancements for Qualtrics clients working their generative AI workloads on the Socrates platform. As well as, SageMaker has streamlined deployment of open supply LLMs and FMs with simply three clicks. This user-friendly performance removes the complexity historically related to deploying these superior fashions, empowering extra Qualtrics clients to harness the ability of generative AI inside their workflows and purposes.
- Improved auto scaling speeds – The SageMaker workforce has developed a sophisticated auto scaling functionality to higher deal with the scaling necessities of generative AI fashions. These enhancements scale back considerably (from a number of minutes to below a minute), lowering auto scaling instances by as much as 40% and auto scaling detection by six instances for Meta Llama 3 8B, enabling Socrates customers to quickly scale their generative AI workloads on SageMaker to fulfill spikes in demand with out compromising efficiency.
- Simple deployment of self-managed OSS LLMs – Utilizing the brand new functionality from SageMaker Inference for a extra streamlined and intuitive course of to bundle your generative AI fashions reduces the technical complexity that was historically related to this process. This, in flip, empowers a wider vary of Socrates customers, together with utility groups and subject material specialists, to make use of the transformative energy of those cutting-edge AI applied sciences inside their workflows and decision-making processes.
- Generative AI inference optimization toolkit – Qualtrics can be actively utilizing the most recent developments within the SageMaker Inference optimization toolkit inside the Socrates platform, which provides two instances increased throughput whereas lowering prices by as much as 50% for generative AI inference. By integrating utilizing capabilities, Socrates is engaged on reducing the price of generative AI inference. This breakthrough is especially impactful for Qualtrics’s clients, who depend on the Socrates platform to energy AI-driven purposes and experiences.
“By seamlessly integrating SageMaker Inference into our Socrates platform, we’re in a position to ship inference developments in AI to our world buyer base. The generative AI inference from capabilities in SageMaker like inference elements, quicker auto scaling, simple LLM deployment, and the optimization toolkit have been a sport changer for Qualtrics to cut back the fee and enhance the efficiency for our generative AI workloads. The extent of sophistication and ease of use that SageMaker Inference brings to the desk is outstanding.”
– James Argyropoulos, Sr AI/ML Engineer at Qualtrics.
Partnership with SageMaker Inference
Since adopting SageMaker Inference, the Qualtrics Socrates workforce has been a key collaborator within the growth of AI capabilities in SageMaker Inference. Constructing on experience to serve Socrates customers, Qualtrics has labored intently with the SageMaker Inference workforce to boost and develop the platform’s generative AI functionalities. From the early levels of generative AI, they provided invaluable insights and experience to the SageMaker workforce. This has enabled the introduction of a number of new options and optimizations which have strengthened the platform’s generative AI choices, together with:
- Value and efficiency optimizations for generative AI inference – Qualtrics helped the SageMaker Inference workforce construct a new inference functionality for SageMaker Inference to cut back FM deployment prices by 50% on common and latency by 20% on common with inference elements. This function delivers important price financial savings and efficiency enhancements for patrons working generative AI inference on SageMaker.
- Quicker auto scaling for generative AI inference – Qualtrics has helped the SageMaker workforce develop These enhancements have diminished auto scaling instances by as much as 40% for fashions like Meta Llama 3 and elevated auto scaling detection velocity by six instances quicker. With this, generative AI inference can scale with altering visitors with out compromising efficiency.
- Inference optimization toolkit for generative AI inference – Qualtrics has been instrumental in giving suggestions for AWS to launch the inference optimization toolkit, which will increase throughput by as much as two instances quicker and reduces latency by 50%.
- Launch of multi-model endpoint (MME) assist for GPU – MMEs permit clients to cut back inference prices by as much as 90%. Qualtrics was instrumental in serving to AWS with the launch of this function by offering worthwhile suggestions.
- Launch of asynchronous inference – Qualtrics was a launch accomplice for and has performed a key function in serving to AWS enhance the providing to provide clients optimum price-performance.
The partnership between Qualtrics and the SageMaker Inference workforce has been instrumental in advancing the state-of-the-art in generative AI inside the AWS ecosystem. Qualtrics’s deep area data and technical proficiency have performed a vital function in shaping the evolution of this quickly creating subject on the SageMaker Inference.
“Our partnership with the SageMaker Inference product workforce has been instrumental in delivering unbelievable efficiency and value advantages for Socrates platform customers working AI Inference workloads. By working hand in hand with the SageMaker workforce, we’ve been in a position to introduce sport altering optimizations which have diminished AI inference prices a number of folds for a few of our use circumstances. We look ahead to continued innovation by worthwhile partnership to enhance state-of-the-art AI inference capabilities.”
– Jay Kshirsagar, Senior Supervisor, Machine Studying
Conclusion
The Socrates platform underscores Qualtrics’s dedication to advancing innovation in expertise administration by flawlessly integrating superior AI and ML applied sciences. Due to a powerful partnership with the SageMaker Inference workforce, the platform has seen enhancements that increase efficiency, scale back prices, and enhance the accessibility of AI-driven options inside the Qualtrics XM suite. As AI expertise continues to develop quickly, the Socrates platform is geared to empower Qualtrics’s AI groups to innovate and ship distinctive buyer experiences.
Concerning the Authors
Jay Kshirsagar is a seasoned ML chief driving GenAI innovation and scalable AI infrastructure at Qualtrics. He has constructed high-impact ML groups and delivered enterprise-grade LLM options that energy key product options.
Ronald Quan is a Employees Engineering Supervisor for the Information Intelligence Platform workforce inside Qualtrics. The workforce’s constitution is to allow, expedite and evolve AI and Agentic developments on the Socrates platform. He focuses on the workforce’s technical roadmap and strategic alignment with the enterprise wants.
Saurabh Trikande is a Senior Product Supervisor for Amazon Bedrock and SageMaker Inference. He’s captivated with working with clients and companions, motivated by the purpose of democratizing AI. He focuses on core challenges associated to deploying advanced AI purposes, inference with multi-tenant fashions, price optimizations, and making the deployment of Generative AI fashions extra accessible. In his spare time, Saurabh enjoys mountaineering, studying about modern applied sciences, following TechCrunch, and spending time along with his household.
Michael Nguyen is a Senior Startup Options Architect at AWS, specializing in utilizing AI/ML to drive innovation and develop enterprise options on AWS. Michael holds 12 AWS certifications and has a BS/MS in Electrical/Pc Engineering and an MBA from Penn State College, Binghamton College, and the College of Delaware.
Ranga Malaviarachchi is a Sr. Buyer Options Supervisor within the ISV Strategic Accounts group at AWS. He has been intently related to Qualtrics over the previous 4 years in supporting their AI initiatives. Ranga holds a BS in Pc Science and Engineering and an MBA from Imperial Faculty London.
