Construct RAG-based generative AI purposes in AWS utilizing Amazon FSx for NetApp ONTAP with Amazon Bedrock

The publish is co-written with Michael Shaul and Sasha Korman from NetApp.

Generative synthetic intelligence (AI) purposes are generally constructed utilizing a method referred to as Retrieval Augmented Era (RAG) that gives basis fashions (FMs) entry to extra knowledge they didn’t have throughout coaching. This knowledge is used to complement the generative AI immediate to ship extra context-specific and correct responses with out repeatedly retraining the FM, whereas additionally enhancing transparency and minimizing hallucinations.

On this publish, we display an answer utilizing Amazon FSx for NetApp ONTAP with Amazon Bedrock to supply a RAG expertise to your generative AI purposes on AWS by bringing company-specific, unstructured person file knowledge to Amazon Bedrock in a simple, quick, and safe method.

Our answer makes use of an FSx for ONTAP file system because the supply of unstructured knowledge and repeatedly populates an Amazon OpenSearch Serverless vector database with the person’s present information and folders and related metadata. This permits a RAG state of affairs with Amazon Bedrock by enriching the generative AI immediate utilizing Amazon Bedrock APIs along with your company-specific knowledge retrieved from the OpenSearch Serverless vector database.

When growing generative AI purposes reminiscent of a Q&A chatbot utilizing RAG, prospects are additionally involved about retaining their knowledge safe and stopping end-users from querying data from unauthorized knowledge sources. Our answer additionally makes use of FSx for ONTAP to permit customers to increase their present knowledge safety and entry mechanisms to enhance mannequin responses from Amazon Bedrock. We use FSx for ONTAP because the supply of related metadata, particularly the person’s safety entry management listing (ACL) configurations hooked up to their information and folders and populate that metadata into OpenSearch Serverless. By combining entry management operations with file occasions that notify the RAG utility of recent and altered knowledge on the file system, our answer demonstrates how FSx for ONTAP allows Amazon Bedrock to solely use embeddings from licensed information for the precise customers that hook up with our generative AI utility.

AWS serverless providers make it simple to give attention to constructing generative AI purposes by offering automated scaling, built-in excessive availability, and a pay-for-use billing mannequin. Occasion-driven compute with AWS Lambda is an efficient match for compute-intensive, on-demand duties reminiscent of doc embedding and versatile giant language mannequin (LLM) orchestration, and Amazon API Gateway offers an API interface that permits for pluggable frontends and event-driven invocation of the LLMs. Our answer additionally demonstrates the way to construct a scalable, automated, API-driven serverless utility layer on high of Amazon Bedrock and FSx for ONTAP utilizing API Gateway and Lambda.

Resolution overview

The answer provisions an FSx for ONTAP Multi-AZ file system with a storage digital machine (SVM) joined to an AWS Managed Microsoft AD area. An OpenSearch Serverless vector search assortment offers a scalable and high-performance similarity search functionality. We use an Amazon Elastic Compute Cloud (Amazon EC2) Home windows server as an SMB/CIFS shopper to the FSx for ONTAP quantity and configure knowledge sharing and ACLs for the SMB shares within the quantity. We use this knowledge and ACLs to check permissions-based entry to the embeddings in a RAG state of affairs with Amazon Bedrock.

The embeddings container part of our answer is deployed on an EC2 Linux server and mounted as an NFS shopper on the FSx for ONTAP quantity. It periodically migrates present information and folders together with their safety ACL configurations to OpenSearch Serverless. It populates an index within the OpenSearch Serverless vector search assortment with company-specific knowledge (and related metadata and ACLs) from the NFS share on the FSx for ONTAP file system.

The answer implements a RAG Retrieval Lambda operate that permits RAG with Amazon Bedrock by enriching the generative AI immediate utilizing Amazon Bedrock APIs along with your company-specific knowledge and related metadata (together with ACLs) retrieved from the OpenSearch Serverless index that was populated by the embeddings container part. The RAG Retrieval Lambda operate shops dialog historical past for the person interplay in an Amazon DynamoDB desk.

Finish-users work together with the answer by submitting a pure language immediate both by a chatbot utility or instantly by the API Gateway interface. The chatbot utility container is constructed utilizing Streamlit and fronted by an AWS Software Load Balancer (ALB). When a person submits a pure language immediate to the chatbot UI utilizing the ALB, the chatbot container interacts with the API Gateway interface that then invokes the RAG Retrieval Lambda operate to fetch the response for the person. The person may instantly submit immediate requests to API Gateway and acquire a response. We display permissions-based entry to the RAG paperwork by explicitly retrieving the SID of a person after which utilizing that SID within the chatbot or API Gateway request, the place the RAG Retrieval Lambda operate then matches the SID to the Home windows ACLs configured for the doc. As a further authentication step in a manufacturing atmosphere, it’s possible you’ll need to additionally authenticate the person towards an identification supplier after which match the person towards the permissions configured for the paperwork.

The next diagram illustrates the end-to-end move for our answer. We begin by configuring knowledge sharing and ACLs with FSx for ONTAP, after which these are periodically scanned by the embeddings container. The embeddings container splits the paperwork into chunks and makes use of the Amazon Titan Embeddings mannequin to create vector embeddings from these chunks. It then shops these vector embeddings with related metadata in our vector database by populating an index in a vector assortment in OpenSearch Serverless. The next diagram illustrates the end-to-end move.

The next structure diagram illustrates the assorted parts of our answer.

Stipulations

Full the next prerequisite steps:

1. Be sure to have mannequin entry in Amazon Bedrock. On this answer, we use Anthropic Claude v3 Sonnet on Amazon Bedrock.
2. Set up the AWS Command Line Interface (AWS CLI).
3. Set up Docker.
4. Set up Terraform.

Deploy the answer

The answer is out there for obtain on this GitHub repo. Cloning the repository and utilizing the Terraform template will provision all of the parts with their required configurations.

1. Clone the repository for this answer:

   sudo yum set up -y unzip
   git clone https://github.com/aws-samples/genai-bedrock-fsxontap.git
   cd genai-bedrock-fsxontap/terraform

2. From the terraform folder, deploy your entire answer utilizing Terraform:

   terraform init
   terraform apply -auto-approve

This course of can take 15–20 minutes to finish. When completed, the output of the terraform instructions ought to seem like the next:

api-invoke-url = "https://9ng1jjn8qi.execute-api..amazonaws.com/prod"
fsx-management-ip = toset([
"198.19.255.230",])
fsx-secret-id = "arn:aws:secretsmanager:::secret:AmazonBedrock-FSx-NetAPP-ONTAP-a2fZEdIt-0fBcS9"
fsx-svm-smb-dns-name = "BRSVM.BEDROCK-01.COM"
lb-dns-name = "chat-load-balancer-2040177936..elb.amazonaws.com"

Load knowledge and set permissions

To check the answer, we’ll use the EC2 Home windows server (ad_host) mounted as an SMB/CIFS shopper to the FSx for ONTAP quantity to share pattern knowledge and set person permissions that can then be used to populate the OpenSearch Serverless index by the answer’s embedding container part. Carry out the next steps to mount your FSx for ONTAP SVM knowledge quantity as a community drive, add knowledge to this shared community drive, and set permissions primarily based on Home windows ACLs:

1. Get hold of the ad_host occasion DNS from the output of your Terraform template.
2. Navigate to AWS Methods Supervisor Fleet Supervisor in your AWS console, find the ad_host occasion and comply with directions right here to login with Distant Desktop. Use the area admin person bedrock-01Admin and acquire the password from AWS Secrets and techniques Supervisor. You’ll find the password utilizing the Secrets and techniques Supervisor fsx-secret-id secret id from the output of your Terraform template.
3. To mount an FSx for ONTAP knowledge quantity as a community drive, below This PC, select (right-click) Community after which select Map Community drive.
4. Select the drive letter and use the FSx for ONTAP share path for the mount
   (.c$):
   
5. Add the Amazon Bedrock Person Information to the shared community drive and set permissions to the admin person solely (just be sure you disable inheritance below Superior):
6. Add the Amazon FSx for ONTAP Person Information to the shared drive and ensure permissions are set to Everybody:
7. On the ad_host server, open the command immediate and enter the next command to acquire the SID for the admin person:

   wmic useraccount the place title="Admin" get sid

Take a look at permissions utilizing the chatbot

To check permissions utilizing the chatbot, receive the lb-dns-name URL from the output of your Terraform template and entry it by your net browser:

For the immediate question, ask any normal query on the FSx for ONTAP person information that’s accessible for entry to everybody. In our state of affairs, we requested “How can I create an FSx for ONTAP file system,” and the mannequin replied again with detailed steps and supply attribution within the chat window to create an FSx for ONTAP file system utilizing the AWS Administration Console, AWS CLI, or FSx API:

Now, let’s ask a query in regards to the Amazon Bedrock person information that’s accessible for admin entry solely. In our state of affairs, we requested “How do I take advantage of basis fashions with Amazon Bedrock,” and the mannequin replied with the response that it doesn’t have sufficient data to supply an in depth reply to the query.:

Use the admin SID on the person (SID) filter search within the chat UI and ask the identical query within the immediate. This time, the mannequin ought to reply with steps detailing the way to use FMs with Amazon Bedrock and supply the supply attribution utilized by the mannequin for the response:

Take a look at permissions utilizing API Gateway

You may as well question the mannequin instantly utilizing API Gateway. Get hold of the api-invoke-url parameter from the output of your Terraform template.

curl -v '/bedrock_rag_retreival' -X POST -H 'content-type: utility/json' -d '{"session_id": "1","immediate": "What's an FSxN ONTAP filesystem?", "bedrock_model_id": "anthropic.claude-3-sonnet-20240229-v1:0", "model_kwargs": {"temperature": 1.0, "top_p": 1.0, "top_k": 500}, "metadata": "NA", "memory_window": 10}'

Then invoke the API gateway with Everybody entry for a question associated to the FSx for ONTAP person information by setting the worth of the metadata parameter to NA to point Everybody entry:

curl -v '/bedrock_rag_retreival' -X POST -H 'content-type: utility/json' -d '{"session_id": "1","immediate": "what's bedrock?", "bedrock_model_id": "anthropic.claude-3-sonnet-20240229-v1:0", "model_kwargs": {"temperature": 1.0, "top_p": 1.0, "top_k": 500}, "metadata": "S-1-5-21-4037439088-1296877785-2872080499-1112", "memory_window": 10}'

Cleanup

To keep away from recurring prices, clear up your account after making an attempt the answer. From the terraform folder, delete the Terraform template for the answer:

terraform apply --destroy

Conclusion

On this publish, we demonstrated an answer that makes use of FSx for ONTAP with Amazon Bedrock and makes use of FSx for ONTAP assist for file possession and ACLs to supply permissions-based entry in a RAG state of affairs for generative AI purposes. Our answer lets you construct generative AI purposes with Amazon Bedrock the place you possibly can enrich the generative AI immediate in Amazon Bedrock along with your company-specific, unstructured person file knowledge from an FSx for ONTAP file system. This answer lets you ship extra related, context-specific, and correct responses whereas additionally ensuring solely licensed customers have entry to that knowledge. Lastly, the answer demonstrates the usage of AWS serverless providers with FSx for ONTAP and Amazon Bedrock that allow automated scaling, event-driven compute, and API interfaces to your generative AI purposes on AWS.

For extra details about the way to get began constructing with Amazon Bedrock and FSx for ONTAP, consult with the next assets:

Concerning the authors

Kanishk Mahajan is Principal, Options Structure at AWS. He leads cloud transformation and answer structure for ISV prospects and companion at AWS. Kanishk focuses on containers, cloud operations, migrations and modernizations, AI/ML, resilience and safety and compliance. He’s a Technical Subject Group (TFC) member in every of these domains at AWS.

Michael Shaul is a Principal Architect at NetApp’s workplace of the CTO. He has over 20 years of expertise constructing knowledge administration methods, purposes, and infrastructure options. He has a novel in-depth perspective on cloud applied sciences, builder, and AI options.

Sasha Korman is a tech visionary chief of dynamic growth and QA groups throughout Israel and India. With 14-years at NetApp that started as a programmer, his hands-on expertise and management have been pivotal in steering complicated initiatives to success, with a give attention to innovation, scalability, and reliability.