Introducing cross-Area inference for OpenAI GPT-5.6 fashions on Amazon Bedrock

This publish is co-written with Chris Dickens from OpenAI.

Amazon Bedrock now gives OpenAI GPT-5.6 fashions on Amazon Bedrock in additional than 25 AWS Areas, with cross-Area inference. Three GPT-5.6 variants help cross-Area inference, Sol, Terra, and Luna, every tuned for a distinct steadiness of functionality and value.

Cross-Area inference (CRIS) in Amazon Bedrock works by inference profiles, which outline a mannequin and the AWS Areas to which Amazon Bedrock can route a request. You name the profile from a supply Area, and Amazon Bedrock routes the request to a vacation spot Area utilizing compute in that Area. CRIS is primarily a capability mechanism. By permitting requests to attract on a broader pool of compute relatively than being sure to at least one Area’s accessible capability, it improves throughput and helps keep constant efficiency beneath load. A geographic inference profile routes requests inside a single geography, so you’ll be able to scale whereas holding knowledge processed inside that geography. For GPT-5.6, this launch introduces US geographic (US cross-Area inference) and world CRIS. A world inference profile routes throughout all supported AWS industrial Areas the place the mannequin is deployed, based mostly on real-time capability, providing you with broad entry to Amazon Bedrock capability.

On this publish, we stroll by an outline of GPT-5.6 fashions on Amazon Bedrock, how geographic and world cross-Area inference work for these fashions, and the way to name them from the Amazon Bedrock console and in code with the OpenAI Responses API, the OpenAI Chat Completions API, and the Amazon Bedrock Converse API.

GPT-5.6 on Amazon Bedrock

The GPT-5.6 household on Amazon Bedrock consists of general-purpose and specialised cyber safety variants. This publish covers the three general-purpose variants that help cross-Area inference, Sol, Terra, and Luna. All three settle for textual content and picture inputs and return textual content, have a 1 million token context window, and help reasoning mode, server-side device calling, and immediate caching. You’ll be able to name them with the OpenAI Responses API, the OpenAI Chat Completions API, and the Amazon Bedrock Converse API. Streaming is supported by the Responses and Chat Completions API (stream=True), and ConverseStream.

Amazon Bedrock inference profiles are logical identifiers you cross as an alternative of a uncooked mannequin ID.

• A geographic inference profile (prefixed with a geography code equivalent to us., for instance us.openai.gpt-5.6-terra) retains inference processing inside the Areas of its predefined geography. Requests enter by your supply Area and may solely be routed to vacation spot Areas inside that geography, so workloads with knowledge residency necessities can scale throughout Areas whereas staying inside the boundary.
• A International inference profile (prefixed world. equivalent to world.openai.gpt-5.6-terra) can route a request to any supported industrial AWS Area the place the mannequin is deployed, based mostly on real-time capability. It gives the widest capability pool and is the suitable alternative when your workload has no geographic processing necessities.

Billing and quota consumption are tracked in opposition to your account no matter which backend Area dealt with the request, so a single spending and throughput image nonetheless applies. Knowledge processed by world CRIS could cross the Areas in that mannequin’s eligible set. In case your workload has knowledge residency necessities that prohibit processing to particular geographies, use the geographic profile for that geography (equivalent to us.openai.gpt-5.6-terra) or a direct name to at least one Area as an alternative of the worldwide profile. The Amazon Bedrock cross-Area inference documentation lists which Areas take part in every mannequin’s world and geographic profile units.

The next tables record the supply Areas the place you’ll be able to invoke the GPT-5.6 inference profiles and the vacation spot Areas the place your requests will be processed. The identical routing applies to all three variants: Sol, Terra, and Luna.

US cross-Area inference profile (us.openai.gpt-5.6-sol / -terra / -luna)

International cross-Area inference profile (world.openai.gpt-5.6-sol / -terra / -luna)

Attempt GPT-5.6 within the Amazon Bedrock console

The quickest strategy to strive GPT-5.6 is the textual content playground within the Amazon Bedrock console, which requires no coding or SDK setup. You’ll be able to ship prompts, alter inference parameters, and swap between variants to get a really feel for every mannequin earlier than you combine the API. The mannequin selector lists each the geographic and world cross-Area inference profiles, so you’ll be able to check both cross-Area inference choice earlier than writing code. Within the previous screenshot, the supply Area is US East (N. Virginia). The US entry within the mannequin selector is the geographic inference profile, and the International entry is the worldwide inference profile.

1. Open the Amazon Bedrock console in a Area the place the fashions can be found, equivalent to US East (N. Virginia).
2. Within the navigation pane, beneath Take a look at, select Playground.
3. Select Choose mannequin in the midst of the web page.
4. Seek for OpenAI GPT-5.6 Sol, choose both US OpenAI GPT-5.6 Sol or International OpenAI GPT-5.6 Sol, and select Apply.
5. Enter a immediate and select Run to generate a response.

Determine 1: Choosing a US or International GPT-5.6 inference profile within the Amazon Bedrock console mannequin selector

Getting began with the API

GPT-5.6 speaks the OpenAI Responses API codecs natively on Amazon Bedrock. In case your utility already calls OpenAI fashions, you’ll be able to level your present OpenAI SDK consumer on the Amazon Bedrock OpenAI-compatible endpoint. Then swap within the inference profile ID (world or geographic) because the mannequin parameter. For authentication, Amazon Bedrock accepts both commonplace AWS credentials or an Amazon Bedrock API key. The API key path is essentially the most direct match for the OpenAI SDK, which passes it because the bearer token. For manufacturing, generate short-term API keys programmatically utilizing the aws-bedrock-token-generator bundle, which derives a bearer token out of your present AWS credentials (long-term keys are really helpful just for exploration).

from aws_bedrock_token_generator import provide_token
from openai import OpenAI
area = "us-east-1"
# Level the OpenAI SDK at Amazon Bedrock's OpenAI-compatible endpoint to your Area. provide_token() generates a short-term Amazon Bedrock API key out of your present AWS credentials (legitimate as much as 12 hours), so no static key must be saved.
consumer = OpenAI(
    base_url= f"https://bedrock-runtime.{area}.amazonaws.com/openai/v1",
    api_key=provide_token(area=area),
)

# Use the worldwide inference profile ID for GPT-5.6 Terra.
model_id = "world.openai.gpt-5.6-terra"
response = consumer.responses.create(
    mannequin=model_id,
    enter="Summarize the distinction between horizontal and vertical scaling in two sentences.",
     max_output_tokens=512,
)

print(response.output_text)

For the complete record of supported parameters, see the OpenAI GPT mannequin parameters web page within the Amazon Bedrock Person Information. The identical consumer additionally works with the Chat Completions API, helpful in case your utility already makes use of this format.

response = consumer.chat.completions.create(
     # For US Geo CRIS, use "us.openai.gpt-5.6-terra".
     # Different variants: gpt-5.6-sol, gpt-5.6-luna
     mannequin="world.openai.gpt-5.6-terra",
     messages=[
         {
             "role": "user",
             "content": "In one sentence, what is cross-Region inference in Amazon Bedrock?",
         }
     ],
     max_completion_tokens=2000,
     reasoning_effort="low",
 )

print(response.decisions[0].message.content material)

For those who’re calling Amazon Bedrock immediately relatively than by the OpenAI SDK, use the Amazon Bedrock Converse API, which supplies you a similar request form you already use for different fashions on Bedrock:

import boto3

consumer = boto3.consumer("bedrock-runtime", region_name="us-east-1")

model_id = "world.openai.gpt-5.6-terra"
 response = consumer.converse(
     modelId=model_id,
     messages=[{"role": "user", "content": [{"text": "List three common uses for a message queue."}]}],
     inferenceConfig={"maxTokens": 512},
 )
 print(response["output"]["message"]["content"][0]["text"])

For streaming responses, name converse_stream with the identical arguments and iterate over the occasion stream:

stream_response = consumer.converse_stream(
     modelId=model_id,
     messages=[{"role": "user", "content": [{"text": "List three common uses for a message queue."}]}],
     inferenceConfig={"maxTokens": 512},
 )

for occasion in stream_response["stream"]:
     if "contentBlockDelta" in occasion:
         print(occasion["contentBlockDelta"]["delta"]["text"], finish="")

Safety and compliance

Cross-Area inference makes use of the identical Amazon Bedrock safety mannequin as direct in-Area calls. Requests are authenticated along with your AWS Id and Entry Administration (IAM) credentials, and IAM insurance policies management which inference profiles a job can invoke. Amazon Bedrock makes use of a zero-operator entry (ZOA) safety mannequin enforced on the chip, so no AWS operators can entry your prompts or completions. Each mannequin name runs beneath your IAM insurance policies, will be reached privately out of your digital non-public cloud (VPC) by a VPC endpoint, and is logged on AWS CloudTrail. Knowledge perimeter insurance policies assist forestall exfiltration throughout account and community boundaries.

For sure fashions, together with GPT-5.6, content material flagged by the Amazon Bedrock automated abuse-detection classifiers is retained for as much as 30 days for offline abuse detection. To see which fashions this is applicable to and the way it works, see Abuse detection within the Amazon Bedrock Person Information. To grasp extra about knowledge retention configuration on Bedrock, see the Amazon Bedrock knowledge retention documentation. For the authoritative record of Areas in every profile’s routing set, see the Amazon Bedrock cross-Area inference help web page.

Cross-Area inference requests seem on AWS CloudTrail in your supply Area, and the additionalEventData.inferenceRegion discipline data which Area processed every request. For those who allow mannequin invocation logging, the request and response payloads are delivered to Amazon Easy Storage Service (Amazon S3) or Amazon CloudWatch Logs in the identical account and Area.

Organising IAM permissions for cross-Area inference

To let a job invoke GPT-5.6 by an inference profile, grant it entry to the inference profile and to the inspiration mannequin in each Area the profile can path to. You need to use this managed coverage AmazonBedrockLimitedAccess or create your individual.

For a geographic inference profile, the coverage has three statements. The primary grants entry to the geographic inference profile and the default challenge in your supply Area. The second grants entry to the inspiration mannequin in your supply Area and in each vacation spot Area within the geography, with a situation that limits this entry to requests made by that profile. The third grants the bearer-token authentication the OpenAI-compatible APIs use.

{ 
  "Model": "2012-10-17", 
  "Assertion": [ 
    { 
      "Sid": "GrantGeoCrisProfileAndProjectAccess", 
      "Effect": "Allow", 
      "Action": ["bedrock:InvokeModel"], 
      "Useful resource": [ 
        "arn:aws:bedrock:::inference-profile/us.openai.gpt-5.6-terra", 
        "arn:aws:bedrock:::project/default" 
      ] 
    }, 
    { 
      "Sid": "GrantGeoCrisDestinationModelAccess", 
      "Impact": "Enable", 
      "Motion": ["bedrock:InvokeModel"], 
      "Useful resource": [ 
        "arn:aws:bedrock:us-east-1::foundation-model/openai.gpt-5.6-terra", 
        "arn:aws:bedrock:us-east-2::foundation-model/openai.gpt-5.6-terra", 
        "arn:aws:bedrock:us-west-2::foundation-model/openai.gpt-5.6-terra" 
      ], 
      "Situation": { 
        "StringLike": { 
          "bedrock:InferenceProfileArn": "arn:aws:bedrock:::inference-profile/us.openai.gpt-5.6-terra" 
        } 
      } 
    }, 
    { 
      "Sid": "AllowBearerTokenAuth", 
      "Impact": "Enable", 
      "Motion": ["bedrock:CallWithBearerToken"], 
      "Useful resource": "*" 
    } 
  ] 
} 

A world inference profile makes use of a four-part coverage. The primary grants entry to the worldwide inference profile and the default challenge in your supply Area. The second grants entry to the inspiration mannequin in your supply Area. The third grants entry to the inspiration mannequin by the Area-agnostic world ARN, which is what permits cross-Area routing. The fourth grants the bearer-token authentication utilized by the OpenAI-compatible APIs.

{ 
 "Model": "2012-10-17", 
 "Assertion": [ 
   { 
     "Sid": "GrantGlobalCrisProfileAndProjectAccess", 
     "Effect": "Allow", 
     "Action": ["bedrock:InvokeModel"], 
     "Useful resource": [ 
       "arn:aws:bedrock:::inference-profile/global.openai.gpt-5.6-terra", 
       "arn:aws:bedrock:::project/default" 
     ], 
     "Situation": { 
       "StringEquals": { "aws:RequestedRegion": "" } 
     } 
   }, 
   { 
     "Sid": "GrantGlobalCrisInRegionModelAccess", 
     "Impact": "Enable", 
     "Motion": ["bedrock:InvokeModel"], 
     "Useful resource": ["arn:aws:bedrock:::foundation-model/openai.gpt-5.6-terra"], 
     "Situation": { 
       "StringEquals": { 
         "aws:RequestedRegion": "", 
         "bedrock:InferenceProfileArn": "arn:aws:bedrock:::inference-profile/world.openai.gpt-5.6-terra" 
       } 
     } 
   }, 
   { 
     "Sid": "GrantGlobalCrisGlobalModelAccess", 
     "Impact": "Enable", 
     "Motion": ["bedrock:InvokeModel"], 
     "Useful resource": ["arn:aws:bedrock:::foundation-model/openai.gpt-5.6-terra"], 
     "Situation": { 
       "StringEquals": { 
         "aws:RequestedRegion": "unspecified", 
         "bedrock:InferenceProfileArn": "arn:aws:bedrock:::inference-profile/world.openai.gpt-5.6-terra" 
       } 
     } 
   }, 
   { 
     "Sid": "AllowBearerTokenAuth", 
     "Impact": "Enable", 
     "Motion": ["bedrock:CallWithBearerToken"], 
     "Useful resource": "*" 
   } 
 ] 
}

These insurance policies grant bedrock:CallWithBearerToken and bedrock:InvokeModel on the challenge/default useful resource, which the OpenAI Responses and Chat Completions APIs use to authenticate with an Amazon Bedrock API key and run inference. For those who use the streaming Converse API (ConverseStream), add bedrock:InvokeModelWithResponseStream to the inference-profile and foundation-model statements.

The statements work collectively, so eradicating one denies that profile’s cross-Area entry for the position, which additionally offers you a clear strategy to flip both functionality off for particular roles. Alternatively, you’ll be able to connect an express deny that targets world. or us. inference profiles.

Service management coverage consideration for world cross-Area inference

In case your group makes use of Area-restrictive service management insurance policies (SCPs), International CRIS requests set aws:RequestedRegion to unspecified relatively than a selected Area title. Geographic CRIS requests are evaluated in opposition to each vacation spot Area within the profile. The really helpful strategy is to exempt cross-Area inference with the bedrock:InferenceProfileArn situation relatively than widening your Area allowlist. Amazon Bedrock units that situation key when it authorizes the inspiration mannequin, so you’ll be able to enable CRIS routing whereas holding your allowlist tight for each different service. Your supply Area nonetheless must be within the allowlist. For those who favor to not use the situation, you’ll be able to as an alternative add the vacation spot Areas and unspecified to your allowlist, however that opens these Areas to each service, not solely Amazon Bedrock. For a ready-to-use SCP and step-by-step steering, see the Bedrock CRIS Area-control SCP pattern.

The next SCP denies Amazon Bedrock inference exterior your authorised Areas (Sydney and N. Virginia are proven) and exempts the us. and world. inference profiles that GPT-5.6 makes use of. The primary assertion retains the Area allowlist enforced for each different service, so embody the worldwide companies your group makes use of in its NotAction record.

{
  "Model": "2012-10-17",
  "Assertion": [
    {
      "Sid": "DenyOutsideApprovedRegionsExceptBedrockInvoke",
      "Effect": "Deny",
      "NotAction": [
        "iam:*", "sts:*", "organizations:*", "account:*",
        "route53:*", "cloudfront:*", "waf:*", "wafv2:*", "shield:*",
        "support:*", "trustedadvisor:*", "health:*",
        "budgets:*", "ce:*", "cur:*", "tax:*", "billing:*",
        "bedrock:Invoke*"
      ],
      "Useful resource": "*",
      "Situation": {
        "StringNotEquals": { "aws:RequestedRegion": ["ap-southeast-2", "us-east-1"] }
      }
    },
    {
      "Sid": "DenyBedrockInferenceOutsideApprovedRegionsUnlessCris",
      "Impact": "Deny",
      "Motion": [
        "bedrock:Invoke*"
      ],
      "Useful resource": "*",
      "Situation": {
        "StringNotEquals": { "aws:RequestedRegion": ["ap-southeast-2", "us-east-1"] },
        "ArnNotLike": {
          "bedrock:InferenceProfileArn": [
            "arn:aws:bedrock:*:*:inference-profile/us.*",
            "arn:aws:bedrock:*:*:inference-profile/global.*"
          ]
        }
      }
    }
  ]
}

Mannequin entry should be enabled for the account and Area for each profile sorts. For the complete guidelines, see Stipulations for inference profiles.

Working with immediate caching

All three GPT-5.6 variants help immediate caching on the bedrock-runtime endpoint. When your requests share a protracted immediate prefix, equivalent to a system immediate or a set of few-shot examples, Amazon Bedrock caches that prefix. Later requests reuse the cached prefix as an alternative of reprocessing it. This lowers enter value and latency for the cached portion.

GPT-5.6 helps two caching modes on Amazon Bedrock. Implicit caching is the default: the service locations a cache breakpoint on the newest person or device message, which fits conversations that develop by appending messages. In case your requests share a secure prefix adopted by content material that modifications on each request, you’ll be able to mark the tip of the secure content material with an express cache breakpoint as an alternative. In each modes, cross the elective prompt_cache_key parameter so requests with the identical prefix are routed to the identical cache. Each modes work with geographic and world inference profiles. Every cache breakpoint wants a immediate prefix of at the very least 1,024 tokens. For extra data, see Immediate caching for sooner mannequin inference.

Immediate caching additionally impacts quota utilization. Cache learn tokens are usually not counted towards your TPM quota, so requests that reuse a cached prefix devour much less of it. The subsequent part describes how quotas are calculated. The next instance makes use of express immediate caching with the consumer and model_id from the Getting began part:

# support_policy_manual is a protracted, secure prefix shared throughout requests.
# The breakpoint marks the tip of the secure content material; the person query
# after it modifications on each request with out invalidating the cached prefix.
completion = consumer.chat.completions.create(
     mannequin=model_id,
     messages=[
         {
             "role": "system",
             "content": [
                 {
                     "type": "text",
                     "text": support_policy_manual,
                     "prompt_cache_breakpoint": {"mode": "explicit"},
                 }
             ],
         },
         {"position": "person", "content material": "A buyer desires a refund for a broken laptop computer. What should I confirm first?"},
     ],
     max_completion_tokens=300,
     prompt_cache_key="support-policy-v1",
 )

To substantiate that caching is working, verify the utilization object of every response.

particulars = completion.utilization.prompt_tokens_details
print("Cached tokens:", particulars.cached_tokens)
print("Written tokens:", particulars.cache_write_tokens)

Quota administration

On-demand quotas for GPT-5.6 are managed as tokens per minute (TPM), they usually connect to the inference profile you name: a geographic profile and a worldwide profile for a similar mannequin carry separate quota allocations, so switching between them modifications the pool you draw from. To view your present allocations or request a rise, seek for the GPT-5.6 mannequin inference quotas within the AWS Service Quotas console from the Area your utility calls Amazon Bedrock in.

While you dimension a quota enhance request, account for the burndown fee: the speed at which enter and output tokens convert into token quota utilization for the throttling system. Enter tokens rely in opposition to your quota at 1:1, whereas output tokens can devour quota at the next a number of. For GPT-5.6 fashions, the burndown fee is 10x for output tokens, which means one output token consumes 10 tokens out of your TPM quota, so output-heavy workloads dissipate quota significantly sooner than uncooked token counts counsel. The per-request calculation is:

Enter token rely + Cache write enter tokens + (Output token rely x Burndown fee)

For instance, a request with 2,000 enter tokens and 1,000 output tokens depletes 12,000 tokens out of your quota. Cache learn tokens are usually not a part of this calculation, which is what makes immediate caching (earlier part) efficient for quota administration and value. For the present burndown charges by mannequin, see the Amazon Bedrock quotas web page. Three practices allow you to keep away from surprises:

• Request will increase earlier than deployment. For those who anticipate excessive utilization, request the rise by the Service Quotas console forward of your launch relatively than reacting to throttling in manufacturing.
• Monitor utilization. Amazon CloudWatch publishes quota utilization metrics in actual time per inference profile, so you’ll be able to set alerts when utilization approaches thresholds and monitor historic patterns to plan future will increase.
• Load check with lifelike visitors (together with peak patterns and production-size prompts) earlier than you commit a workload to manufacturing, and validate in opposition to the profile kind you’ll really use, since geographic and world quotas are separate.

Monitoring and logging

As a result of GPT-5.6 requests run by the Bedrock Runtime API, requests made by a geographic or world inference profile seem in Amazon Bedrock mannequin invocation logging the identical means on-demand requests do, with the inference profile ARN recorded alongside the request and response payloads (topic to your logging configuration). You’ll be able to ship invocation logs to Amazon S3 or Amazon CloudWatch Logs. Whichever profile kind you utilize, invocation logs and metrics are recorded in your supply Area, so your observability stays in a single place even when the request is processed elsewhere.

Amazon CloudWatch metrics masking invocation counts, token counts, latency, throttles, and errors are revealed per inference profile. As a result of geographic and world profiles are distinct sources, their metrics are reported individually. Per-request latency will differ considerably with the vacation spot Area a request lands in, so in case your dashboards break metrics down by Area at this time, take into account including a view that aggregates by inference profile ID as an alternative, since that’s the ID your utility code and quota consumption are tied to. Utilization can be itemized on AWS Value Explorer and the AWS Value and Utilization Report, so you’ll be able to attribute GPT-5.6 spend by mannequin and by profile the identical means you do for the remainder of your Bedrock workloads.

Conclusion

GPT-5.6 brings three OpenAI mannequin variants to Amazon Bedrock. With this launch you’ll be able to name every of them by two sorts of cross-Area inference profiles. You need to use a geographic profile when inference processing wants to remain inside a geography (scaling throughout the Areas inside it), and a world profile while you need the widest capability pool throughout supported industrial AWS Areas. Both means, your utility talks to at least one endpoint with one profile ID, logs, quotas and billing keep in your supply Area. The fashions communicate the OpenAI Chat Completions API and Responses API, together with streaming, and the Amazon Bedrock Converse API.

To get began, open the Amazon Bedrock console and ship a check immediate to GPT-5.6 Sol, Terra, or Luna by an inference profile, or name the fashions programmatically by the Responses API. Assessment the Amazon Bedrock pricing web page for present GPT-5.6 charges earlier than sizing a manufacturing workload. In case your utility wants long-context (1 million token) processing, verify the Amazon Bedrock mannequin help by Area web page for Area availability.

Concerning the authors