Construct a generative AI Slack chat assistant utilizing Amazon Bedrock and Amazon Kendra

Regardless of the proliferation of data and knowledge in enterprise environments, staff and stakeholders typically discover themselves trying to find data and struggling to get their questions answered shortly and effectively. This may result in productiveness losses, frustration, and delays in decision-making.

A generative AI Slack chat assistant might help handle these challenges by offering a available, clever interface for customers to work together with and procure the data they want. Through the use of the pure language processing and era capabilities of generative AI, the chat assistant can perceive consumer queries, retrieve related data from numerous knowledge sources, and supply tailor-made, contextual responses.

By harnessing the ability of generative AI and Amazon Internet Companies (AWS) companies Amazon Bedrock, Amazon Kendra, and Amazon Lex, this answer offers a pattern structure to construct an clever Slack chat assistant that may streamline data entry, improve consumer experiences, and drive productiveness and effectivity inside organizations.

Why use Amazon Kendra for constructing a RAG software?

Amazon Kendra is a completely managed service that gives out-of-the-box semantic search capabilities for state-of-the-art rating of paperwork and passages. You should utilize Amazon Kendra to shortly construct high-accuracy generative AI functions on enterprise knowledge and supply essentially the most related content material and paperwork to maximise the standard of your Retrieval Augmented Era (RAG) payload, yielding higher massive language mannequin (LLM) responses than utilizing standard or keyword-based search options. Amazon Kendra gives simple-to-use deep studying search fashions which are pre-trained on 14 domains and don’t require machine studying (ML) experience. Amazon Kendra can index content material from a variety of sources, together with databases, content material administration methods, file shares, and net pages.

Additional, the FAQ function in Amazon Kendra enhances the broader retrieval capabilities of the service, permitting the RAG system to seamlessly change between offering prewritten FAQ responses and dynamically producing responses by querying the bigger data base. This makes it well-suited for powering the retrieval element of a RAG system, permitting the mannequin to entry a broad data base when producing responses. By integrating the FAQ capabilities of Amazon Kendra right into a RAG system, the mannequin can use a curated set of high-quality, authoritative solutions for generally requested questions. This may enhance the general response high quality and consumer expertise, whereas additionally lowering the burden on the language mannequin to generate these fundamental responses from scratch.

This answer balances retaining customizations when it comes to mannequin choice, immediate engineering, and including FAQs with not having to take care of phrase embeddings, doc chunking, and different lower-level complexities usually required for RAG implementations.

Resolution overview

The chat assistant is designed to help customers by answering their questions and offering data on a wide range of subjects. The aim of the chat assistant is to be an internal-facing Slack software that may assist staff and stakeholders discover the data they want.

The structure makes use of Amazon Lex for intent recognition, AWS Lambda for processing queries, Amazon Kendra for looking by means of FAQs and net content material, and Amazon Bedrock for producing contextual responses powered by LLMs. By combining these companies, the chat assistant can perceive pure language queries, retrieve related data from a number of knowledge sources, and supply humanlike responses tailor-made to the consumer’s wants. The answer showcases the ability of generative AI in creating clever digital assistants that may streamline workflows and improve consumer experiences primarily based on mannequin decisions, FAQs, and modifying system prompts and inference parameters.

Structure diagram

The next diagram illustrates a RAG method the place the consumer sends a question by means of the Slack software and receives a generated response primarily based on the info listed in Amazon Kendra. On this submit, we use Amazon Kendra Internet Crawler as the info supply and embody FAQs saved on Amazon Easy Storage Service (Amazon S3). See Information supply connectors for a listing of supported knowledge supply connectors for Amazon Kendra.

The step-by-step workflow for the structure is the next:

1. The consumer sends a question corresponding to What's the AWS Properly-Architected Framework? by means of the Slack app.
2. The question goes to Amazon Lex, which identifies the intent.
3. Presently two intents are configured in Amazon Lex (Welcome and FallbackIntent).
4. The welcome intent is configured to reply with a greeting when a consumer enters a greeting corresponding to “hello” or “good day.” The assistant responds with “Good day! I might help you with queries primarily based on the paperwork supplied. Ask me a query.”
5. The fallback intent is fulfilled with a Lambda operate.
   1. The Lambda operate searches Amazon Kendra FAQs by means of the search_Kendra_FAQ technique by taking the consumer question and Amazon Kendra index ID as inputs. If there’s a match with a excessive confidence rating, the reply from the FAQ is returned to the consumer.

      def search_Kendra_FAQ(query, kendra_index_id):
          """
          This operate takes within the query from the consumer, and checks if the query exists within the Kendra FAQs.
          :param query: The query the consumer is asking that was requested by way of the frontend enter textual content field.
          :param kendra_index_id: The kendra index containing the paperwork and FAQs
          :return: If present in FAQs, returns the reply together with any related hyperlinks. If not, returns False after which calls kendra_retrieve_document operate.
          """
          kendra_client = boto3.shopper('kendra')
          response = kendra_client.question(IndexId=kendra_index_id, QueryText=query, QueryResultTypeFilter="QUESTION_ANSWER")
          for merchandise in response['ResultItems']:
              score_confidence = merchandise['ScoreAttributes']['ScoreConfidence']
              # Taking solutions from FAQs which have a really excessive confidence rating solely
              if score_confidence == 'VERY_HIGH' and len(merchandise['AdditionalAttributes']) > 1:
                  textual content = merchandise['AdditionalAttributes'][1]['Value']['TextWithHighlightsValue']['Text']
                  url = "None"
                  if merchandise['DocumentURI'] != '':
                      url = merchandise['DocumentURI']
                  return (textual content, url)
          return (False, False)

   2. If there isn’t a match with a excessive sufficient confidence rating, related paperwork from Amazon Kendra with a excessive confidence rating are retrieved by means of the kendra_retrieve_document technique and despatched to Amazon Bedrock to generate a response because the context.

      def kendra_retrieve_document(query, kendra_index_id):
          """
          This operate takes within the query from the consumer, and retrieves related passages primarily based on default PageSize of 10.
          :param query: The query the consumer is asking that was requested by way of the frontend enter textual content field.
          :param kendra_index_id: The kendra index containing the paperwork and FAQs
          :return: Returns the context to be despatched to the LLM and doc URIs to be returned as related knowledge sources.
          """
          kendra_client = boto3.shopper('kendra')
          paperwork = kendra_client.retrieve(IndexId=kendra_index_id, QueryText=query)
          textual content = ""
          uris = set()
          if len(paperwork['ResultItems']) > 0:
              for i in vary(len(paperwork['ResultItems'])):
                  score_confidence = paperwork['ResultItems'][i]['ScoreAttributes']['ScoreConfidence']
                  if score_confidence == 'VERY_HIGH' or score_confidence == 'HIGH':
                      textual content += paperwork['ResultItems'][i]['Content'] + "n"
                      uris.add(paperwork['ResultItems'][i]['DocumentURI'])
          return (textual content, uris)

   3. The response is generated from Amazon Bedrock with the invokeLLM technique. The next is a snippet of the invokeLLM technique inside the achievement operate. Learn extra on inference parameters and system prompts to change parameters which are handed into the Amazon Bedrock invoke mannequin request.

      def invokeLLM(query, context, modelId):
          """
          This operate takes within the query from the consumer, together with the Kendra responses as context to generate a solution
          for the consumer on the frontend.
          :param query: The query the consumer is asking that was requested by way of the frontend enter textual content field.
          :param paperwork: The response from the Kendra doc retrieve question, used as context to generate a greater
          reply.
          :return: Returns the ultimate reply that might be supplied to the end-user of the applying who requested the unique
          query.
          """
          # Setup Bedrock shopper
          bedrock = boto3.shopper('bedrock-runtime')
          # configure mannequin specifics corresponding to particular mannequin
          modelId = modelId
      
          # physique of knowledge with parameters that's handed into the bedrock invoke mannequin request
          physique = json.dumps({"max_tokens": 350,
                  "system": "You're a truthful AI assistant. Your objective is to supply informative and substantive responses to queries primarily based on the paperwork supplied. In case you have no idea the reply to a query, you honestly say you have no idea.",
                  "messages": [{"role": "user", "content": "Answer this user query:" + question + "with the following context:" + context}],
                  "anthropic_version": "bedrock-2023-05-31",
                      "temperature":0,
                  "top_k":250,
                  "top_p":0.999})
      
          # Invoking the bedrock mannequin together with your specs
          response = bedrock.invoke_model(physique=physique,
                                          modelId=modelId)
          # the physique of the response that was generated
          response_body = json.masses(response.get('physique').learn())
          # retrieving the precise completion area, the place you reply might be
          reply = response_body.get('content material')
          # returning the reply as a ultimate outcome, which finally will get returned to the top consumer
          return reply

   4. Lastly, the response generated from Amazon Bedrock together with the related referenced URLs are returned to the top consumer.

   When deciding on web sites to index, adhere to the AWS Acceptable Use Coverage and different AWS phrases. Keep in mind which you can solely use Amazon Kendra Internet Crawler to index your individual net pages or net pages that you’ve got authorization to index. Go to the Amazon Kendra Internet Crawler knowledge supply information to be taught extra about utilizing the online crawler as an information supply. Utilizing Amazon Kendra Internet Crawler to aggressively crawl web sites or net pages you don’t personal is not thought of acceptable use.

   Supported options

   The chat assistant helps the next options:

   1. Help for the next Anthropic’s fashions on Amazon Bedrock:
      • claude-v2
      • claude-3-haiku-20240307-v1:0
      • claude-instant-v1
      • claude-3-sonnet-20240229-v1:0
   2. Help for FAQs and the Amazon Kendra Internet Crawler knowledge supply
   3. Returns FAQ solutions provided that the boldness rating is VERY_HIGH
   4. Retrieves solely paperwork from Amazon Kendra which have a HIGH or VERY_HIGH confidence rating
   5. If paperwork with a excessive confidence rating aren’t discovered, the chat assistant returns “No related paperwork discovered”

   Stipulations

   To carry out the answer, you might want to have following stipulations:

   • Fundamental data of AWS
   • An AWS account with entry to Amazon S3 and Amazon Kendra
   • An S3 bucket to retailer your paperwork. For extra data, see Step 1: Create your first S3 bucket and the Amazon S3 Person Information.
   • A Slack workspace to combine the chat assistant
   • Permission to put in Slack apps in your Slack workspace
   • Seed URLs for the Amazon Kendra Internet Crawler knowledge supply
     • You’ll want authorization to crawl and index any web sites supplied
   • AWS CloudFormation for deploying the answer sources

   Construct a generative AI Slack chat assistant

   To construct a Slack software, use the next steps:

   1. Request mannequin entry on Amazon Bedrock for all Anthropic fashions
   2. Create an S3 bucket within the us-east-1 (N. Virginia) AWS Area.
   3. Add the AIBot-LexJson.zip and SampleFAQ.csv information to the S3 bucket
   4. Launch the CloudFormation stack within the us-east-1 (N. Virginia) AWS Area.
   5. Enter a Stack title of your selection
   6. For S3BucketName, enter the title of the S3 bucket created in Step 2
   7. For S3KendraFAQKey, enter the title of the SampleFAQs uploaded to the S3 bucket in Step 3
   8. For S3LexBotKey, enter the title of the Amazon Lex .zip file uploaded to the S3 bucket in Step 3
   9. For SeedUrls, enter as much as 10 URLs for the online crawler as a comma delimited listing. Within the instance on this submit, we give the publicly obtainable Amazon Bedrock service web page because the seed URL
   10. Go away the remainder as defaults. Select Subsequent. Select Subsequent once more on the Configure stack choices
   11. Acknowledge by deciding on the field and select Submit, as proven within the following screenshot
       
   12. Look forward to the stack creation to finish
   13. Confirm all sources are created
   14. Take a look at on the AWS Administration Console for Amazon Lex
       1. On the Amazon Lex console, select your chat assistant ${YourStackName}-AIBot
       2. Select Intents
       3. Select Model 1 and select Take a look at, as proven within the following screenshot
          
       4. Choose the AIBotProdAlias and select Verify, as proven within the following screenshot. If you wish to make modifications to the chat assistant, you should use the draft model, publish a brand new model, and assign the brand new model to the AIBotProdAlias. Study extra about Versioning and Aliases.
          
       5. Take a look at the chat assistant with questions corresponding to, “Which AWS service has 11 nines of sturdiness?” and “What’s the AWS Properly-Architected Framework?” and confirm the responses. The next desk exhibits that there are three FAQs within the pattern .csv file.
          

          _question	_answer	_source_uri
          Which AWS service has 11 nines of sturdiness?	Amazon S3	https://aws.amazon.com/s3/
          What’s the AWS Properly-Architected Framework?	The AWS Properly-Architected Framework allows prospects and companions to evaluate their architectures utilizing a constant method and offers steerage to enhance designs over time.	https://aws.amazon.com/structure/well-architected/
          In what Areas is Amazon Kendra obtainable?	Amazon Kendra is at the moment obtainable within the following AWS Areas: Northern Virginia, Oregon, and Eire	https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/

       6. The next screenshot exhibits the query “Which AWS service has 11 nines of sturdiness?” and its response. You possibly can observe that the response is identical as within the FAQ file and features a hyperlink.
          
       7. Primarily based on the pages you could have crawled, ask a query within the chat. For this instance, the publicly obtainable Amazon Bedrock web page was crawled and listed. The next screenshot exhibits the query, “What are brokers in Amazon Bedrock?” and and a generated response that features related hyperlinks.
          
   15. For integration of the Amazon Lex chat assistant with Slack, see Integrating an Amazon Lex V2 bot with Slack. Select the AIBotProdAlias beneath Alias within the Channel Integrations

   Run pattern queries to check the answer

   1. In Slack, go to the Apps part. Within the dropdown menu, select Handle and choose Browse apps.
      
   2. Seek for ${AIBot} in App Listing and select the chat assistant. This can add the chat assistant to the Apps part in Slack. Now you can begin asking questions within the chat. The next screenshot exhibits the query “Which AWS service has 11 nines of sturdiness?” and its response. You possibly can observe that the response is identical as within the FAQ file and features a hyperlink.
      
   3. The next screenshot exhibits the query, “What's the AWS Properly-Architected Framework?” and its response.
      
   4. Primarily based on the pages you could have crawled, ask a query within the chat. For this instance, the publicly obtainable Amazon Bedrock web page was crawled and listed. The next screenshot exhibits the query, “What are brokers in Amazon Bedrock?” and and a generated response that features related hyperlinks.
      
   5. The next screenshot exhibits the query, “What's amazon polly?” As a result of there is no such thing as a Amazon Polly documentation listed, the chat assistant responds with “No related paperwork discovered,” as anticipated.
      

   These examples present how the chat assistant retrieves paperwork from Amazon Kendra and offers solutions primarily based on the paperwork retrieved. If no related paperwork are discovered, the chat assistant responds with “No related paperwork discovered.”

   Clear up

   To wash up the sources created by this answer:

   1. Delete the CloudFormation stack by navigating to the CloudFormation console
   2. Choose the stack you created for this answer and select Delete
   3. Verify the deletion by coming into the stack title within the supplied area. This can take away all of the sources created by the CloudFormation template, together with the Amazon Kendra index, Amazon Lex chat assistant, Lambda operate, and different associated sources.

   Conclusion

   This submit describes the event of a generative AI Slack software powered by Amazon Bedrock and Amazon Kendra. That is designed to be an internal-facing Slack chat assistant that helps reply questions associated to the listed content material. The answer structure consists of Amazon Lex for intent identification, a Lambda operate for fulfilling the fallback intent, Amazon Kendra for FAQ searches and indexing crawled net pages, and Amazon Bedrock for producing responses. The submit walks by means of the deployment of the answer utilizing a CloudFormation template, offers directions for operating pattern queries, and discusses the steps for cleansing up the sources. General, this submit demonstrates find out how to use numerous AWS companies to construct a strong generative AI–powered chat assistant software.

   This answer demonstrates the ability of generative AI in constructing clever chat assistants and search assistants. Discover the generative AI Slack chat assistant: Invite your groups to a Slack workspace and begin getting solutions to your listed content material and FAQs. Experiment with completely different use circumstances and see how one can harness the capabilities of companies like Amazon Bedrock and Amazon Kendra to boost your online business operations. For extra details about utilizing Amazon Bedrock with Slack, seek advice from Deploy a Slack gateway for Amazon Bedrock.

   ---

   Concerning the authors

   Kruthi Jayasimha Rao is a Companion Options Architect with a give attention to AI and ML. She offers technical steerage to AWS Companions in following greatest practices to construct safe, resilient, and extremely obtainable options within the AWS Cloud.

   Mohamed Mohamud is a Companion Options Architect with a give attention to Information Analytics. He focuses on streaming analytics, serving to companions construct real-time knowledge pipelines and analytics options on AWS. With experience in companies like Amazon Kinesis, Amazon MSK, and Amazon EMR, Mohamed allows data-driven decision-making by means of streaming analytics.