Summarize name transcriptions securely with Amazon Transcribe and Amazon Bedrock Guardrails

Given the quantity of conferences, interviews, and buyer interactions in trendy enterprise environments, audio recordings play an important function in capturing precious info. Manually transcribing and summarizing these recordings could be a time-consuming and tedious activity. Thankfully, developments in generative AI and computerized speech recognition (ASR) have paved the best way for automated options that may streamline this course of.

Customer support representatives obtain a excessive quantity of calls every day. Beforehand, calls had been recorded and manually reviewed later for compliance, rules, and firm insurance policies. Name recordings needed to be transcribed, summarized, after which redacted for private identifiable info (PII) earlier than analyzing calls, leading to delayed entry to insights.

Redacting PII is a crucial apply in safety for a number of causes. Sustaining the privateness and safety of people’ private info will not be solely a matter of moral duty, but additionally a authorized requirement. On this put up, we present you the way to use Amazon Transcribe to get close to real-time transcriptions of calls despatched to Amazon Bedrock for summarization and delicate information redaction. We’ll stroll by way of an structure that makes use of AWS Step Features to orchestrate the method, offering seamless integration and environment friendly processing

Amazon Bedrock is a completely managed service that provides a alternative of high-performing basis fashions (FMs) from main mannequin suppliers equivalent to AI21 Labs, Anthropic, Cohere, Meta, Stability AI, Mistral AI, and Amazon by way of a single API, together with a broad set of capabilities you want to construct generative AI purposes with safety, privateness, and accountable AI. You should use  Amazon Bedrock Guardrails to redact delicate info equivalent to PII discovered within the generated name transcription summaries. Clear, summarized transcripts are then despatched to analysts. This supplies faster entry to name traits whereas defending buyer privateness.

Resolution overview

The structure of this resolution is designed to be scalable, environment friendly, and compliant with privateness rules. It consists of the next key parts:

1. Recording – An audio file, equivalent to a gathering or assist name, to be transcribed and summarized
2. Step Features workflow – Coordinates the transcription and summarization course of
3. Amazon Transcribe – Converts audio recordings into textual content
4. Amazon Bedrock – Summarizes the transcription and removes PII
5. Amazon SNS – Delivers the abstract to the designated recipient
6. Recipient – Receives the summarized, PII-redacted transcript

The next diagram reveals the structure overflow –

The workflow orchestrated by Step Features is as follows:

1. An audio recording is offered as an enter to the Step Features workflow. This could possibly be carried out manually or mechanically relying on the precise use case and integration necessities.
2. The workflow invokes Amazon Transcribe, which converts the multi-speaker audio recording right into a textual, speaker-partition transcription. Amazon Transcribe makes use of superior speech recognition algorithms and machine studying (ML) fashions to precisely partition audio system and transcribe the audio, dealing with varied accents, background noise, and different challenges.
3. The transcription output from Amazon Transcribe is then handed to Anthropic’s Claude 3 Haiku mannequin on Amazon Bedrock by way of AWS Lambda. This mannequin was chosen as a result of it has comparatively decrease latency and price than different fashions. The mannequin first summarizes the transcript in keeping with its abstract directions, after which the summarized output (the mannequin response) is evaluated by Amazon Bedrock Guardrails to redact PII. To be taught the way it blocks dangerous content material, check with How Amazon Bedrock Guardrails works. The directions and transcript are each handed to the mannequin as context.
4. The output from Amazon Bedrock is saved in Amazon Easy Storage Service (Amazon S3) and despatched to the designated recipient utilizing Amazon Easy Notification Service (Amazon SNS). Amazon SNS helps varied supply channels, together with e-mail, SMS, and cell push notifications, ensuring that the abstract reaches the meant recipient in a well timed and dependable method

The recipient can then evaluation the concise abstract, rapidly greedy the important thing factors and insights from the unique audio recording. Moreover, delicate info has been redacted, sustaining privateness and compliance with related rules.

The next diagram reveals the Step Features workflow –

Conditions

Comply with these steps earlier than beginning:

1. Amazon Bedrock customers have to request entry to fashions earlier than they’re obtainable to be used. This can be a one-time motion. For this resolution, you want to allow entry to Anthropic’s Claude 3 Haiku mannequin on Amazon Bedrock. For extra info, check with Entry Amazon Bedrock basis fashions. Deployment, as described under, is at the moment supported solely within the US West (Oregon) us-west-2 AWS Area. Customers might discover different fashions if desired. You may want some customizations to deploy to various Areas with totally different mannequin availability (equivalent to us-east-1, which hosts Anthropic’s Claude 3.5 Sonnet). Be sure to take into account mannequin high quality, velocity, and price tradeoffs earlier than selecting a mannequin.
2. Create a guardrail for PII redaction. Configure filters to dam or masks delicate info. This feature might be discovered on the Amazon Bedrock console on the Add delicate info filters web page when making a guardrail. To discover ways to configure filters for different use instances, check with Take away PII from conversations by utilizing delicate info filters.

Deploy resolution assets

To deploy the answer, obtain an AWS CloudFormation template to mechanically provision the mandatory assets in your AWS account. The template units up the next parts:

• A Step Features workflow
• Lambda capabilities
• An SNS subject
• An S3 bucket
• AWS Key Administration Service (AWS KMS) keys for information encryption and decryption

By utilizing this template, you possibly can rapidly deploy the pattern resolution with minimal handbook configuration. The template requires the next parameters:

• E mail deal with used to ship abstract – The abstract shall be despatched to this deal with. You should acknowledge the preliminary Amazon SNS affirmation e-mail earlier than receiving further notifications.
• Abstract directions – These are the directions given to the Amazon Bedrock mannequin to generate the abstract
• Guardrail ID – That is the ID of your not too long ago created guardrail, which might be discovered on the Amazon Bedrock Guardrails console in Guardrail overview

The Abstract directions are learn into your Lambda perform as an setting variable.

 
# Use the offered directions to supply the abstract. Use a default if no intructions are offered.
SUMMARY_INSTRUCTIONS = os.getenv('SUMMARY_INSTRUCTIONS')
 
These are then used as a part of your payload to Anthropic’s Claude 3 Haiku mannequin. That is shared to provide you an understanding of the way to move the directions and textual content to the mannequin.
 
# Create the payload to supply to the Anthropic mannequin.
        user_message = {"function": "consumer", "content material": f"{SUMMARY_INSTRUCTIONS}{transcript}"}
        messages = [user_message]
response = generate_message(bedrock_client, 'anthropic.claude-3-haiku-20240307-v1:0"', "", messages, 1000)
 
The generate_message() perform comprises the invocation to Amazon Bedrock with the guardrail ID and different related parameters.
 
def generate_message(bedrock_runtime, model_id, system_prompt, messages, max_tokens):
    physique = json.dumps(
        {
            "anthropic_version": "bedrock-2023-05-31",
            "max_tokens": max_tokens,
            "system": system_prompt,
            "messages": messages
        }
    )
print(f'Invoking mannequin: {BEDROCK_MODEL_ID}')
 
    response = bedrock_runtime.invoke_model(
        physique=physique,
        modelId=BEDROCK_MODEL_ID,
        # contentType=contentType,
        guardrailIdentifier =BEDROCK_GUARDRAIL_ID,
        guardrailVersion ="1",
        hint ="ENABLED")
    response_body = json.masses(response.get('physique').learn())
    print(f'response: {response}')
    return response_body

Deploy the answer

After you deploy the assets utilizing AWS CloudFormation, full these steps:

1. Add a Lambda layer.

Though AWS Lambda repeatedly updates the model of AWS Boto3 included, on the time of scripting this put up, it nonetheless supplies model 1.34.126. To make use of Amazon Bedrock Guardrails, you want model 1.34.90 or greater, for which we’ll add a Lambda layer that updates the Boto3. You may comply with the official developer information on the way to add a Lambda layer.

There are other ways to create a Lambda layer. A easy methodology is to make use of the steps outlined in Packaging the layer content material, which references a pattern software repo. It is best to be capable to exchange requests==2.31.0 inside necessities.txt content material to boto3, which is able to set up the newest obtainable model, then create the layer.

So as to add the layer to Lambda, guarantee that the parameters laid out in Creating the layer match the deployed Lambda. That’s, you want to replace compatible-architectures to x86_64.

2. Acknowledge the Amazon SNS e-mail affirmation that it is best to obtain a couple of moments after creating the CloudFormation stack
3. On the AWS CloudFormation console, discover the stack you simply created
4. On the stack’s Outputs tab, search for the worth related to AssetBucketName. It should look one thing like summary-generator-assetbucket-xxxxxxxxxxxxx.
5. On the Amazon S3 console, discover your S3 property bucket.

That is the place you’ll add your recordings. Legitimate file codecs are MP3, MP4, WAV, FLAC, AMR, OGG, and WebM.

6. Add your recording to the recordings folder in Amazon S3

Importing recordings will mechanically set off the AWS Step Features state machine. For this instance, we use a pattern workforce assembly recording from the pattern recording.

7. On the AWS Step Features console, discover the summary-generator state machine. Select the identify of the state machine run with the standing Operating.

Right here, you possibly can watch the progress of the state machine because it processes the recording. After it reaches its Success state, it is best to obtain an emailed abstract of the recording. Alternatively, you possibly can navigate to the S3 property bucket and consider the transcript there within the transcripts folder.

Develop the answer

Now that you’ve got a working resolution, listed below are some potential concepts to customise the answer to your particular use instances:

• Strive altering the method to suit your obtainable supply content material and desired outputs:
  • For conditions the place transcripts can be found, create an alternate AWS Step Features workflow to ingest present text-based or PDF-based transcriptions
  • As an alternative of utilizing Amazon SNS to inform recipients by way of e-mail, you should utilize it to ship the output to a unique endpoint, equivalent to a workforce collaboration web site or to the workforce’s chat channel
• Strive altering the abstract directions for the AWS CloudFormation stack parameter offered to Amazon Bedrock to provide outputs particular to your use case. The next are some examples:
  • When summarizing an organization’s earnings name, you would have the mannequin give attention to potential promising alternatives, areas of concern, and issues that it is best to proceed to observe
  • Should you’re utilizing the mannequin to summarize a course lecture, it might determine upcoming assignments, summarize key ideas, checklist information, and filter out small speak from the recording
• For a similar recording, create totally different summaries for various audiences:
  • Engineers’ summaries give attention to design selections, technical challenges, and upcoming deliverables
  • Venture managers’ summaries give attention to timelines, prices, deliverables, and motion gadgets
  • Venture sponsors get a short replace on venture standing and escalations
  • For longer recordings, strive producing summaries for various ranges of curiosity and time dedication. For instance, create a single sentence, single paragraph, single web page, or in-depth abstract. Along with the immediate, you may need to alter the max_tokens_to_sample parameter to accommodate totally different content material lengths.

Clear up

Clear up the assets you created for this resolution to keep away from incurring prices. You should use an AWS SDK, the AWS Command Line Interface (AWS CLI), or the console.

1. Delete Amazon Bedrock Guardrails and the Lambda layer you created
2. Delete the CloudFormation stack

To make use of the console, comply with these steps:

3. On the Amazon Bedrock console, within the navigation menu, choose Guardrails. Select your guardrail, then choose Delete.
4. On the AWS Lambda console, within the navigation menu, choose Layers. Select your layer, then choose Delete.
5. On the AWS CloudFormation console, within the navigation menu, choose Stacks. Select the stack you created, then choose Delete.

Deleting the stack received’t delete the related S3 bucket. Should you not require the recordings or transcripts, you possibly can delete the bucket individually. Amazon Transcribe is designed to mechanically delete transcription jobs after 90 days. Nonetheless, you possibly can choose to manually delete these jobs earlier than the 90-day retention interval expires.

Conclusion

As companies flip to information as a basis for decision-making, being able to effectively extract insights from audio recordings is invaluable. By utilizing the ability of generative AI with Amazon Bedrock and Amazon Transcribe, your group can create concise summaries of audio recordings whereas sustaining privateness and compliance. The proposed structure demonstrates how AWS companies might be orchestrated utilizing AWS Step Features to streamline and automate advanced workflows, enabling organizations to give attention to their core enterprise actions.

This resolution not solely saves effort and time, but additionally makes certain that delicate info is redacted, mitigating potential dangers and selling compliance with information safety rules. As organizations proceed to generate and course of giant volumes of audio information, options like it will turn into more and more necessary for gaining insights, making knowledgeable selections, and sustaining a aggressive edge.

In regards to the authors

Yash Yamsanwar is a Machine Studying Architect at Amazon Net Companies (AWS). He’s chargeable for designing high-performance, scalable machine studying infrastructure that optimizes the total lifecycle of machine studying fashions, from coaching to deployment. Yash collaborates intently with ML analysis groups to push the boundaries of what’s potential with LLMs and different cutting-edge machine studying applied sciences.

Sawyer Hirt is a Options Architect at AWS, specializing in AI/ML and cloud architectures, with a ardour for serving to companies leverage cutting-edge applied sciences to beat advanced challenges. His experience lies in designing and optimizing ML workflows, enhancing system efficiency, and making superior AI options extra accessible and cost-effective, with a selected give attention to Generative AI. Exterior of labor, Sawyer enjoys touring, spending time with household, and staying present with the newest developments in cloud computing and synthetic intelligence.