Fanatics Betting and Gaming (FBG) constructed a multi-agent buyer assist system on AWS to unravel a problem distinctive to sports activities betting. Clients anticipate prompt, correct solutions, particularly throughout stay occasions when each minute counts. Clients ask about account points, deposit limits, state-specific laws, and accountable gaming assets. The foundations differ throughout each jurisdiction the place an operator is licensed. Conventional chatbot options constructed on determination bushes wrestle with this complexity, typically irritating clients and driving up prices as human agent queues develop.
Fanatics Betting and Gaming (FBG) is a sports activities betting platform that mixes superior know-how with deep sports activities experience. As a part of the Fanatics household of manufacturers, FBG operates throughout a number of U.S. states, serving a quickly rising consumer base that calls for across the clock assist, notably throughout high-traffic occasions like NFL playoffs and the Tremendous Bowl.
Dealing with exponential development in assist quantity, FBG’s engineering group constructed a multi-agent AI system on AWS that resolves buyer points sooner, extra precisely, and at a fraction of the price of human-only assist. On this put up, we stroll by way of the structure, the AWS companies concerned, and the patterns you possibly can take into account when designing your individual multi-agent buyer assist answer.
The problem
As FBG scaled, their present assist mannequin required extra human touches per interplay, creating greater operational prices that grew proportionally with their buyer base. The group acknowledged a possibility to enhance their clients’ expertise whereas getting ready the assist infrastructure for the following part of development. A number of components made the issue particularly tough.
Each U.S. state has its personal guidelines for fee strategies, deposit limits, withdrawal timelines, and accountable gaming necessities. A buyer in Indiana will get totally different solutions than one in New Jersey. Throughout main sporting occasions, assist requests can surge to over 40 inquiries each two minutes, and the system must scale immediately with out degrading response high quality.
The range of queries compounds the issue. Clients ask about every thing from transaction historical past and account settings to betting guidelines and self-exclusion choices. No single mannequin or data base covers all of it. On high of that, operators should establish and reply to indicators of downside playing in actual time. This requires a nuanced understanding of conversational context, not simply key phrase matching.
FBG wanted a system that would deal with this complexity autonomously whereas understanding precisely when to escalate to a human agent.
“As we scaled, we knew our assist expertise wanted to evolve with us. We wished to offer our clients sooner, extra correct solutions whereas ensuring we by no means compromised on accountable gaming or compliance. The purpose was to construct one thing that received higher over time, not simply larger.”
— Ian Botts, CTO, Fanatics Betting and Gaming
Resolution overview
Slightly than counting on a single monolithic chatbot, FBG designed a multi-agent system the place specialised brokers deal with totally different elements of the client interplay. As a result of the group had already constructed deep operational experience on Amazon Elastic Kubernetes Service (Amazon EKS), they may construct on their present container platform to deploy, scale, and iterate on every agent independently.
FBG selected Amazon Bedrock for its model-agnostic entry to a number of basis fashions by way of a single API, which lets the group match every activity to the very best mannequin and swap fashions as higher choices emerge. As a result of Bedrock runs inside their present AWS surroundings, the system additionally inherits FBG’s established safety and governance controls, and Amazon Bedrock Guardrails supplies the accountable AI safeguards their compliance necessities demand.
The structure follows an orchestrator sample. A main orchestration agent receives every buyer message, coordinates with specialised instruments and sub-agents, and returns a unified response. This design permits the group so as to add new capabilities, akin to new instruments, new data domains, and new enterprise models, with out rewriting the core system.
“We designed the system so that every agent has a transparent accountability and could be improved independently. That modularity is what permits us to maneuver quick. When we have to assist a brand new case kind or a brand new enterprise unit, we add a brand new device or agent with out touching the remainder of the system.”
— Luis Fernandez Rocha, Sr. Supervisor of Software program Engineering, Fanatics Betting and Gaming
Determine 1 illustrates the high-level structure.
Determine 1: Finish-to-end structure of the multi-agent buyer assist system on AWS
A buyer message enters by way of the FBG cell app, passes by way of Salesforce Einstein to the Spring AI service on Amazon EKS, then flows by way of Amazon Bedrock Guardrails and a Accountable Gaming classifier earlier than reaching the Supervisor Agent. The Supervisor Agent invokes specialised instruments, together with a Retrieval Augmented Era (RAG) pipeline, account and transaction Mannequin Context Protocol (MCP) servers, and a transfer-to-agent device to generate a response.
How a request flows by way of the system
A buyer sends a message by way of the FBG cell app, which connects to Salesforce Einstein because the chat interface layer. The request is routed by way of normal REST calls to the Spring AI service operating on Amazon EKS. This service validates the client token and invokes the AI agent.
The request then passes by way of Amazon Bedrock Guardrails to assist detect immediate injection earlier than reaching the AI layer. A Accountable Gaming classification agent, powered by Amazon Nova 2 Lite, evaluates each message towards a compliance-approved classification framework. Excessive-severity classifications set off a direct switch to a human agent with full dialog context.
The Supervisor Agent, operating Anthropic Claude on Amazon Bedrock, determines which instruments to invoke primarily based on the client’s intent. Relying on the question, the supervisor calls a number of specialised instruments, some by way of MCP and others native to the service:
- Retrieval Augmented Era (RAG) Software retrieves related data from a vector retailer for FAQ-style questions.
- Account Instruments (MCP) queries inside account companies for customer-specific data.
- Transaction Instruments (MCP) retrieves current transaction historical past together with deposits, withdrawals, and betting exercise.
- Switch-to-Agent Software escalates to a human agent when the client explicitly requests it or when the state of affairs requires human judgment.
The supervisor synthesizes the device responses and returns a pure language response to the client.
Deep dive: Key architectural parts
On this part, we look at the 4 parts that make the system work: the Amazon EKS internet hosting platform, the customized RAG pipeline, the accountable gaming classifier, and the guardrails that assist maintain conversations secure.
Amazon EKS for agent internet hosting and MCP servers
FBG runs their whole AI stack on Amazon EKS internet hosting their MCP server and Spring AI service as Kubernetes companies. The MCP server exposes instruments that make REST calls to exterior companies just like the account service and transaction-history service. Native instruments stay instantly within the Spring AI service alongside the Claude massive language mannequin (LLM). These embrace the RAG device and Switch-to-Human device.
This strategy supplies a number of benefits for multi-agent techniques. The MCP server and Spring AI service scale independently primarily based on demand. When FBG must assist further enterprise domains or options, including a brand new MCP server is simply one other Kubernetes deployment. The group can even replace particular person instruments with out redeploying the whole system. New MCP instruments are added to an present MCP server with out requiring new pod deployments.
FBG makes use of Spring AI as its utility framework, which supplies native MCP assist. The MCP server defines instruments that the supervisor agent can uncover and invoke dynamically. The group selected Spring AI as a result of their builders had deep Java experience, which allow them to transfer shortly. For groups working in Python, Strands Brokers is an open supply SDK from AWS that gives related agent orchestration and MCP assist.
For groups contemplating an identical strategy, Amazon EKS supplies the container orchestration wanted to handle a number of agent companies at scale. MCP supplies the standardized protocol for device communication between brokers. Groups that want a managed expertise can even discover Amazon Bedrock AgentCore, a platform to construct, join, and optimize brokers at scale, with any framework or mannequin. AgentCore additionally helps MCP for device integration.
Customized RAG with Amazon Titan embeddings
Essentially the most ceaselessly used device within the system is the RAG pipeline. FBG constructed a customized implementation fairly than utilizing a managed data base, giving them exact management over the ingestion, chunking, and retrieval course of.
The pipeline works on this means. Assist documentation is collected from upstream sources, together with state-specific fee technique guides, FAQ articles, accountable gaming assets, and account administration guides. Paperwork are break up utilizing a token-based chunking technique, which means every doc is split into segments of a hard and fast variety of tokens (the models of textual content a mannequin processes) fairly than by sentences or paragraphs. This provides the group fine-grained management over chunk boundaries. Chunks are then embedded utilizing Amazon Titan V2, producing vector representations saved in MongoDB Atlas.
When a buyer asks a query, the system converts the question right into a vector search-optimized kind utilizing an LLM, then performs a similarity search towards the doc retailer. For jurisdiction-specific questions, the system performs each a state-specific search and a common search, combining the outcomes earlier than passing them to the supervisor agent for response era.
This practice strategy is especially invaluable when your data base has advanced retrieval necessities, like needing to mix state-specific and common paperwork in a single response. The data base is constantly increasing, with a whole bunch of latest paperwork added month-to-month because the group identifies gaps by way of dialog evaluation.
“Constructing our personal RAG pipeline gave us full management over what the mannequin sees and the way it retrieves data. Each state has totally different guidelines, so we would have liked the flexibility to mix state-specific and common paperwork in a single response. That degree of management made all of the distinction in accuracy.”
— Sharoze Amir, Software program Engineer, Fanatics Betting and Gaming
Accountable gaming classification with Amazon Nova
Accountable gaming is a regulatory requirement and a core worth for FBG. The group labored with their compliance division to construct a classification system. The system evaluates buyer interactions to substantiate accountable gaming requirements are met and connects clients with the correct assets when wanted.
The system makes use of Amazon Nova 2 Lite, a light-weight, quick classification mannequin. The group selected a smaller mannequin intentionally. The duty is well-defined with clear examples and a restricted set of outcomes, so a bigger, dearer mannequin would add latency with out enhancing accuracy.
The mannequin receives each the present message and the complete dialog historical past, enabling it to detect escalating patterns fairly than counting on single-message key phrase matching. When the system identifies a high-severity concern, it instantly transfers the client to a human agent with full dialog context. Decrease-severity flags are recorded for compliance evaluation whereas permitting the dialog to proceed.
This can be a sample that applies broadly: use the smallest mannequin that meets your accuracy necessities for well-scoped classification duties, and reserve bigger fashions for open-ended reasoning.
“Off-the-shelf assist brokers deal with each dialog the identical. Ours can’t — a query a few withdrawal would possibly actually be a accountable gaming second, and recognizing that requires deep integration with our compliance framework. That’s why we constructed on AWS in-house: no vendor was going to deal with these delicate areas the way in which our trade calls for.”
— Trevor Gurgick, Head of Utilized AI, Fanatics Betting and Gaming
Amazon Bedrock Guardrails for safety
FBG makes use of Amazon Bedrock Guardrails to assist defend towards immediate injection and assist maintain conversations inside applicable boundaries. The group tuned their guardrail configuration to assist steadiness safety with the realities of customer support interactions, the place overly restrictive filters can create friction within the buyer expertise.
The important thing perception: tune your guardrails to your precise use case fairly than making use of most restrictions by default. For buyer assist, immediate injection safety is important, however overly aggressive content material filtering creates false positives that frustrate clients.
A multi-model structure
FBG runs a multi-model structure on Amazon Bedrock, taking a deliberate, bottom-up strategy to mannequin choice. For classification duties like accountable gaming, they use Amazon Nova 2 Lite. It’s quick, cost-effective, and adequate for well-defined classification the place clear examples exist. For supervisor and orchestration duties akin to dialog administration, they use Anthropic Claude Sonnet on Amazon Bedrock. Claude handles advanced reasoning, device orchestration, and pure dialog. For embeddings within the RAG pipeline, they use Amazon Titan V2, which generates high-quality vector representations. For mannequin availability by AWS Area, confer with Supported fashions by AWS Area in Amazon Bedrock.
The group makes use of a round-robin technique throughout mannequin Areas for the supervisor agent, making certain they by no means hit throughput limits throughout peak occasions. As a result of each mannequin is accessed by way of the identical Amazon Bedrock API, routing totally different workloads to totally different fashions requires no adjustments to the underlying infrastructure.
Outcomes
Inside the first two months of deployment, the multi-agent system delivered measurable enhancements over FBG’s earlier assist expertise, primarily based on FBG’s inside metrics. The containment fee improved by roughly 56 p.c, which means extra buyer points at the moment are resolved with out human agent involvement. Decision charges improved by roughly 53 p.c, with clients getting their issues really solved fairly than deflected. The system has resolved hundreds of instances autonomously, representing important value financial savings as a result of AI-powered interactions value a fraction of human agent interactions. Buyer satisfaction can also be trending upward. Dialog high quality has improved so considerably that clients ceaselessly don’t understand they’re interacting with AI.
Throughout peak sporting occasions, the system handles excessive request volumes whereas sustaining constant efficiency and response high quality. Amazon EKS autoscaling ensures that the MCP servers and Spring AI service add capability robotically as visitors spikes, with out requiring handbook intervention. Which means whether or not it’s a quiet Tuesday or the Tremendous Bowl, clients obtain the identical quick, correct responses no matter what number of others are asking questions on the similar time.
Steady enchancment: Analysis and iteration
Constructing the agent is only the start. What makes FBG’s strategy stand out is the funding they’ve made in constantly enhancing the system after deployment. The group treats their multi-agent system as a residing product, not a one-time implementation. They evaluation dialog logs, observe decision accuracy, and use actual buyer interactions to refine prompts, alter device conduct, and establish gaps of their data base. This ongoing funding ensures that the system will get smarter over time fairly than degrading as buyer wants evolve.
On the core of their analysis technique is an LLM-as-a-Choose system that robotically opinions each accomplished dialog, classifying whether or not the AI efficiently resolved the case or whether or not it fell quick. An operations group opinions these evaluations each day, figuring out patterns in the place the agent struggles and submitting enchancment tickets to shut these gaps. This creates a suggestions loop the place the agent will get measurably higher over time.
On the engineering aspect, the group displays system well being by way of real-time observability metrics, together with hallucination detection, latency, and price monitoring. When creating new options or testing adjustments, they pull precise buyer conversations from manufacturing and replay them towards the up to date structure to floor edge instances earlier than something reaches clients.
One self-discipline that has been notably efficient is the group’s strategy to immediate engineering. Slightly than leaping to a extra highly effective (and dearer) mannequin when efficiency dips, they first have a look at whether or not the system immediate could be improved or whether or not there are contradictions within the directions. This retains prices low whereas driving steady high quality enhancements, they usually solely improve fashions when the immediate has been absolutely optimized for the duty. Groups seeking to systematize this self-discipline can use superior immediate optimization in Amazon Bedrock, which refines prompts towards analysis standards and compares outcomes throughout a number of fashions earlier than committing to a migration.
Getting began: Constructing your individual multi-agent assist system
Should you’re seeking to construct an identical multi-agent buyer assist system, right here’s a sensible beginning path.
Outline your scope narrowly
FBG launched with simply 4 of their over 20 case sorts. Beginning slender helps you to show worth shortly, construct analysis infrastructure, and be taught what works earlier than increasing. Decide the case sorts with the best quantity and clearest decision standards.
Arrange your infrastructure
Deploy your agent companies on Amazon EKS for full management or use Amazon Bedrock AgentCore for a managed expertise. Use MCP for communication between your orchestrator and gear companies.
Construct your RAG pipeline
Begin with Amazon Bedrock Data Bases, the absolutely managed RAG functionality, or construct a customized pipeline with Amazon Titan embeddings in the event you want fine-grained management over retrieval logic. Both means, make investments time in your chunking technique. It has extra influence on response high quality than mannequin choice.
Implement guardrails and compliance from day one
Use Amazon Bedrock Guardrails to assist defend towards immediate injection. In case your trade has compliance necessities (gaming, healthcare, finance), construct classification brokers early. They’re simpler to combine from the beginning than to retrofit later.
Begin small with mannequin choice
Use the smallest mannequin that meets your accuracy necessities for every activity. Reserve bigger fashions on your orchestration agent the place advanced reasoning is required. Amazon Bedrock makes it easy to swap fashions as you iterate.
Put money into analysis early
Construct your analysis pipeline alongside your agent, not after. Observe containment charges, decision charges, and buyer satisfaction from day one. Use LLM-as-a-Choose patterns to automate dialog evaluation at scale.
Conclusion
Fanatics Betting and Gaming’s multi-agent structure demonstrates how combining Amazon EKS, Amazon Bedrock, MCP, and purpose-built AWS AI companies can ship buyer assist that scales with your enterprise whereas sustaining high quality and compliance. Through the use of specialised brokers for various duties, together with orchestration, data retrieval, classification, and gear execution, the system handles the complexity of state-specific laws, real-time accountable gaming detection, and high-traffic sporting occasions.
The patterns on this put up apply broadly. Outline your scope narrowly, use specialised brokers with clear obligations, select the smallest mannequin that meets your accuracy necessities, and put money into analysis infrastructure from the beginning.
Should you’re able to construct an identical multi-agent buyer assist system, begin by defining your agent boundaries and figuring out the instruments every agent wants. You’ll be able to deploy your brokers on Amazon EKS for full management over scaling and orchestration or use Amazon Bedrock AgentCore for a managed runtime that handles infrastructure for you. For the AI layer, Amazon Bedrock offers you entry to fashions like Anthropic Claude and Amazon Nova by way of a single API, and Amazon Bedrock Guardrails may also help you add security checks with out customized code. To get began, see the Amazon Bedrock Getting Began information and the Strands Brokers documentation for an open-source Python framework that helps MCP-based device orchestration.
Concerning the authors
