Monitor on-premises and multi-cloud AI brokers with AgentCore Observability

While you deploy AI brokers constructed with frameworks like Strands Brokers, LangGraph, and CrewAI, you want observability into their efficiency. This holds true whether or not they run on Amazon Elastic Kubernetes Service (Amazon EKS), Amazon Elastic Container Service (Amazon ECS), AWS Lambda, on-premises, or one other cloud supplier akin to Google Cloud Platform (GCP) or Microsoft Azure.

Amazon Bedrock AgentCore is a platform to construct, join, and optimize brokers at scale, with any framework or mannequin. Though Amazon Bedrock AgentCore Observability, a functionality of Amazon Bedrock AgentCore, gives native tracing, monitoring, and analytics that native cloud monitoring instruments don’t provide out of the field, it natively helps solely brokers deployed on AgentCore runtime within the AWS Cloud. In case your brokers run wherever else, you want extra configuration to ship telemetry to the dashboard.

On this publish, we present you learn how to arrange observability for brokers operating exterior AWS. You discover ways to configure the AWS Distro for OpenTelemetry (ADOT) auto-instrumentation in non-AWS environments, route telemetry to the AgentCore Observability dashboard, and validate the setup finish to finish.

The next diagram exhibits the end-to-end observability pipeline and the way telemetry flows from brokers to the AgentCore Observability dashboard.

Determine 1: Finish-to-end observability pipeline from brokers to the AgentCore Observability dashboard

Answer overview

The answer makes use of the AWS Distro for OpenTelemetry (ADOT) operating in-process with the agent utility. ADOT auto-instruments the agent framework and captures generative AI semantic conference spans, then exports the telemetry on to the Amazon CloudWatch OpenTelemetry Protocol (OTLP) endpoint utilizing SigV4 authentication with AWS Identification and Entry Administration (IAM) credentials.

Sending telemetry out of your AI agent to Amazon Bedrock AgentCore Observability requires three core parts:

• ADOT auto-instrumentation: The AWS Distro for OpenTelemetry handles the complexities of exporting telemetry from non-AWS environments.
• IAM credentials: The ADOT makes use of these entry keys to authenticate with CloudWatch and ahead your agent’s telemetry (traces, metrics, and logs) to the AgentCore Observability dashboard.
• Setting variables: These include particular OpenTelemetry settings associated to routing and authentication.

As seen within the following diagram, this cross-platform observability answer integrates a number of AWS companies. Amazon CloudWatch serves as the inspiration, dealing with telemetry ingestion and storage. Amazon Bedrock AgentCore Observability provides specialised monitoring dashboards for AI brokers. AWS Distro for OpenTelemetry (ADOT) gives the cross-platform instrumentation capabilities. IAM secures the authentication between your exterior environments and AWS.

Determine 2: Cross-platform observability structure and the AWS companies concerned

Observability is a foundational pillar of accountable AI. By routing telemetry to AgentCore Observability, you acquire visibility into agent reasoning chains, software invocations, and mannequin outputs. This lets you detect hallucinations, monitor for dangerous or off-topic responses, observe token utilization for price governance, and audit agent habits throughout environments. That is particularly crucial for brokers operating exterior AWS, the place problematic outputs would possibly go unnoticed with out centralized observability.

Stipulations

Earlier than you start, confirm that you’ve got:

• An AWS account:
  • with Amazon Bedrock mannequin entry configured (this walkthrough makes use of Claude Haiku). For mannequin availability by AWS Area, check with supported fashions by AWS Area in Amazon Bedrock.
  • for designated AgentCore Observability and designated log group(s).
• CloudWatch Transaction Search turned on in your account (one-time setup)
• Python 3.10 or later put in in your non-AWS atmosphere.
• IAM consumer credentials (entry key ID and secret entry key) with permissions for:
  • bedrock:InvokeModel.
  • logs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents.
  • xray:PutTraceSegments, xray:PutTelemetryRecords, xray:GetSamplingRules, and xray:GetSamplingTargets.
  • cloudwatch:PutMetricData.
• Outbound HTTPS entry to AWS endpoints out of your atmosphere.

Activate CloudWatch Transaction Search

In the event you haven’t turned on Transaction Search, run the next (one-time per account):

aws xray update-trace-segment-destination --destination CloudWatchLogs --region us-east-1

Confirm it’s lively:

aws xray get-trace-segment-destination --region us-east-1
# Anticipated: {"Vacation spot": "CloudWatchLogs", "Standing": "ACTIVE"}

The way it works

The ADOT auto-instrumentation (aws-opentelemetry-distro) handles the complexity of exporting telemetry from non-AWS environments to CloudWatch:

1. Auto-instrumentation: The opentelemetry-instrument command injects the ADOT into the Python runtime. It mechanically patches boto3 (for Amazon Bedrock calls) and the Strands framework (for agent reasoning spans) to emit OpenTelemetry traces.
2. SigV4 authentication: The aws_configurator makes use of the boto3 credential chain to signal OTLP export requests with SigV4. From non-AWS environments, this makes use of the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY atmosphere variables.
3. CloudWatch OTLP endpoint: The ADOT exports traces and logs to the CloudWatch native OTLP ingestion endpoint. The OTEL_EXPORTER_OTLP_LOGS_HEADERS header directs logs to the precise AgentCore log group, which is how CloudWatch indexes the information below the generative AI observability dashboard. For particulars on how the CloudWatch OTLP endpoint URL is decided and configured, see CloudWatch OTLP endpoint.
4. Generative AI semantic conventions: The Strands [otel] package deal emits spans following the OpenTelemetry generative AI semantic conventions, together with agent reasoning steps, software invocations, and mannequin calls with token utilization.

The next diagram exhibits how telemetry export by means of ADOT auto-instrumentation works from non-AWS environments to CloudWatch.

Determine 3: Telemetry export by means of ADOT auto-instrumentation from non-AWS environments to CloudWatch

Walkthrough

Comply with these steps to configure and run a Strands agent in a non-AWS atmosphere, with telemetry routed to AgentCore Observability.

Step 1: Set up dependencies

In your non-AWS atmosphere (on-premises server, GCP VM, Azure VM, or a compute with web entry):

pip set up "aws-opentelemetry-distro>=0.10.0" boto3 "strands-agents[otel]"

The aws-opentelemetry-distro package deal contains the ADOT auto-instrumentation with OTLP exporters particular to AWS and the aws_configurator that handles SigV4 authentication. The strands-agents[otel] package deal gives OpenTelemetry hint emission from the Strands framework.

Step 2: Configure AWS credentials

Set your IAM consumer credentials as atmosphere variables.

export AWS_ACCESS_KEY_ID=
export AWS_SECRET_ACCESS_KEY=
export AWS_REGION=us-east-1

Safety observe: For manufacturing deployments, think about using IAM Roles Anyplace as an alternative of long-lived entry keys. With IAM Roles Anyplace, on-premises workloads can get hold of non permanent credentials utilizing X.509 certificates.

Step 3: Set OpenTelemetry atmosphere variables

These atmosphere variables configure the ADOT to route telemetry to the AgentCore Observability dashboard:

export AGENT_OBSERVABILITY_ENABLED=true
export OTEL_PYTHON_DISTRO=aws_distro
export OTEL_PYTHON_CONFIGURATOR=aws_configurator
export OTEL_RESOURCE_ATTRIBUTES="service.identify=my-external-agent,aws.log.group.names=/aws/bedrock-agentcore/runtimes/my-external-agent"
export OTEL_EXPORTER_OTLP_LOGS_HEADERS="x-aws-log-group=/aws/bedrock-agentcore/runtimes/my-external-agent,x-aws-log-stream=runtime-logs,x-aws-metric-namespace=bedrock-agentcore"
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_TRACES_EXPORTER=otlp

Key configuration particulars:

• AGENT_OBSERVABILITY_ENABLED=true prompts generative AI-specific telemetry processing within the ADOT.
• OTEL_PYTHON_DISTRO=aws_distro and OTEL_PYTHON_CONFIGURATOR=aws_configurator activate the OpenTelemetry configuration particular to AWS, together with SigV4 signing for the CloudWatch OTLP endpoint.
• OTEL_RESOURCE_ATTRIBUTES with aws.log.group.names tells CloudWatch to index the telemetry below the AgentCore Observability dashboard. With out this, traces go to generic Amazon CloudWatch Logs.
• OTEL_EXPORTER_OTLP_LOGS_HEADERS with x-aws-metric-namespace=bedrock-agentcore routes metrics in embedded metric format to the right CloudWatch namespace.

Step 4: Create the agent utility

Create a file named agent_test.py with a Strands agent:

from strands import Agent
from strands.fashions.bedrock import BedrockModel
from opentelemetry import baggage
from opentelemetry.context import connect
import time

# Configure the Bedrock mannequin
mannequin = BedrockModel(
    model_id="us.anthropic.claude-haiku-4-5-20251001-v1:0",
    region_name="us-east-1"
)
# Create the agent
agent = Agent(
    mannequin=mannequin,
    system_prompt="You're a useful journey assistant."
)

# Set session ID for AgentCore session monitoring
# All agent calls after connect() share similar session ID for a number of requests/responses
session_id = f"external-session-{int(time.time())}"
ctx = baggage.set_baggage("session.id", session_id)
connect(ctx)

# Run the agent
response = agent("What are the highest 3 issues to do in Tokyo?")
print(response)

Step 5: Run with ADOT auto-instrumentation

The opentelemetry-instrument command wraps your Python course of with the ADOT, mechanically instrumenting Amazon Bedrock calls and Strands framework operations:

opentelemetry-instrument python3.12 agent_test.py

The agent’s response seems within the terminal. Behind the scenes, the ADOT captures traces, spans, and logs, and exports them to CloudWatch.

Step 6: Confirm in AgentCore Observability

You see telemetry information inside two to 3 minutes of execution. Open the Amazon CloudWatch console:

1. Select GenAI Observability, then Bedrock AgentCore.
2. Within the Brokers tab, search for my-external-agent.
3. Select the agent to view classes, traces, and span metrics.

The next screenshot exhibits the telemetry from the Strands agent (my-external-agent) operating in a non-AWS atmosphere, as seen within the AgentCore Observability dashboard in CloudWatch.

Determine 4: The my-external-agent telemetry within the AgentCore Observability dashboard

The console exhibits:

• Agent identify: my-external-agent.
• Periods: no less than one session.
• Traces: hint spans exhibiting the agent’s reasoning and Amazon Bedrock mannequin invocations.
• Span particulars: invoke_agent, chat, execute_event_loop_cycle, and chat.us.anthropic.claude-haiku spans with latency and token metrics.

The next screenshot exhibits a profitable hint from the Strands agent (my-external-agent) with 4 spans, mannequin data, and latency and token particulars within the AgentCore Observability dashboard.

Determine 5: Hint element for my-external-agent with span, latency, and token metrics

Validating from Google Cloud Platform

To verify the answer works from a third-party cloud supplier, we examined the identical setup from Google Cloud Shell, a browser-based terminal operating on GCP infrastructure.

Arrange the atmosphere on Google Cloud Shell:

# Create a digital atmosphere
python3.12 -m venv venv
supply venv/bin/activate
# Set up dependencies
pip set up "aws-opentelemetry-distro" boto3 "strands-agents[otel]"
# Set AWS credentials
export AWS_ACCESS_KEY_ID=
export AWS_SECRET_ACCESS_KEY=
export AWS_REGION=us-east-1
# Set ADOT atmosphere variables
export AGENT_OBSERVABILITY_ENABLED=true
export OTEL_PYTHON_DISTRO=aws_distro
export OTEL_PYTHON_CONFIGURATOR=aws_configurator
export OTEL_RESOURCE_ATTRIBUTES="service.identify=gcp-hosted-agent,aws.log.group.names=/aws/bedrock-agentcore/runtimes/gcp-hosted-agent"
export OTEL_EXPORTER_OTLP_LOGS_HEADERS="x-aws-log-group=/aws/bedrock-agentcore/runtimes/gcp-hosted-agent,x-aws-log-stream=runtime-logs,x-aws-metric-namespace=bedrock-agentcore"
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_TRACES_EXPORTER=otlp

Run the agent from GCP:

cat > agent_test.py << 'EOF'
from strands import Agent
from strands.fashions.bedrock import BedrockModel
from opentelemetry import baggage
from opentelemetry.context import connect
import time

mannequin = BedrockModel(
    model_id="us.anthropic.claude-haiku-4-5-20251001-v1:0",
    region_name="us-east-1"
)
agent = Agent(mannequin=mannequin, system_prompt="You're a useful assistant.")

# Set session ID for AgentCore session monitoring
# All agent calls after connect() share similar session ID for a number of requests/responses
session_id = f"gcp-session-{int(time.time())}"
ctx = baggage.set_baggage("session.id", session_id)
connect(ctx)

response = agent("What are the highest 3 issues to do in Paris?")
print(response)
EOF
opentelemetry-instrument python3.12 agent_test.py

The next screenshot exhibits the Strands agent (gcp-hosted-agent) operating on Google Cloud Shell (GCP) and returning a profitable response.

Determine 6: The gcp-hosted-agent operating in Google Cloud Shell

Confirm cross-cloud telemetry

Inside two to 3 minutes of execution, the gcp-hosted-agent seems within the AgentCore Observability dashboard alongside brokers operating on AgentCore runtime or different environments.

The next screenshot exhibits a profitable hint from the Strands agent (gcp-hosted-agent) operating on GCP with 4 spans, mannequin data, and latency and token particulars within the AgentCore Observability dashboard.

Determine 7: Hint element for gcp-hosted-agent operating on GCP

The telemetry is equivalent to what an AgentCore runtime-hosted agent produces. Periods, traces, span metrics, token utilization, and latency are all seen in the identical dashboard, no matter the place the agent runs.

Though this walkthrough makes use of Strands Brokers, the identical ADOT-based sample applies to different OpenTelemetry-compatible agent frameworks.

When selecting learn how to deploy your AI brokers, understanding the observability trade-offs between totally different runtime environments helps you make the suitable architectural resolution. Brokers deployed straight on Amazon Bedrock AgentCore runtime profit from automated observability configuration. Brokers operating in non-AWS environments require extra handbook setup however provide better deployment flexibility. The next comparability highlights the important thing variations in telemetry assortment, credential administration, and use circumstances that can assist you decide one of the best method on your necessities.

Validated environments

We examined the ADOT auto-instrumentation method throughout two non-AWS environments:

Finest practices

Primarily based on our testing, we suggest the next when organising cross-platform AgentCore Observability:

1. Use constant naming: The service.identify in OTEL_RESOURCE_ATTRIBUTES turns into the agent identify on the dashboard. Use descriptive names that determine the atmosphere (for instance, prod-onprem-support-agent and staging-gcp-research-agent).
2. Confirm with get-caller-identity first: Earlier than operating the agent, verify that your credentials work by operating python -c "import boto3; print(boto3.consumer('sts').get_caller_identity())". If this fails, the ADOT additionally fails silently.
3. Use Python 3.10 or later: The ADOT requires Python 3.10 or later. We suggest Python 3.12 for one of the best compatibility with all dependencies.
4. Set session IDs for multi-turn conversations: Use the OpenTelemetry baggage API to propagate session IDs:

from opentelemetry import baggage
from opentelemetry.context import connect
ctx = baggage.set_baggage("session.id", "my-session-123")
connect(ctx)

5. Rotate credentials commonly: For manufacturing deployments, keep away from long-lived entry keys. Take into account IAM Roles Anyplace for on-premises workloads, or use your cloud supplier’s id federation to imagine AWS IAM roles.

Clear up

To take away the sources created throughout this walkthrough:

# Delete the IAM entry key (if created for testing)
aws iam delete-access-key --user-name  --access-key-id 
# Optionally delete the auto-created CloudWatch log teams
aws logs delete-log-group --log-group-name /aws/bedrock-agentcore/runtimes/my-external-agent --region us-east-1
aws logs delete-log-group --log-group-name /aws/bedrock-agentcore/runtimes/gcp-hosted-agent --region us-east-1

This walkthrough makes use of Amazon Bedrock, Amazon CloudWatch, and AWS X-Ray, which incur prices. See the respective pricing pages for particulars.

Conclusion

Amazon Bedrock AgentCore Observability isn’t restricted to brokers operating on AgentCore runtime or inside AWS. Utilizing ADOT auto-instrumentation with IAM credentials and the right OpenTelemetry atmosphere variables, you’ll be able to ship telemetry out of your alternative of atmosphere with web entry. Your brokers can run on-premises, on GCP, on Azure, or wherever else and nonetheless report back to the identical AgentCore Observability dashboard.

The setup requires a pip set up and a set of atmosphere variables. The ensuing telemetry is equivalent to what AgentCore runtime-hosted brokers produce: classes, traces, span metrics, and token utilization, multi functional unified view.

To get began, clone the pattern code from GitHub and observe the directions within the README to configure and run the agent in your atmosphere.

For brokers already operating on AWS however exterior AgentCore runtime (EKS, ECS, Lambda), check with the AgentCore Observability for EKS-hosted brokers tutorial. For brokers on AgentCore runtime, observability is configured mechanically. See Add observability to your AgentCore sources.

In regards to the authors