Introducing OpenAI fashions on Amazon Bedrock for in-country inferencing in India

Amazon Bedrock now helps the OpenAI GPT-5.6 fashions, Terra and Luna, in India, with India geographic cross-Area inference. You probably have native information processing necessities in India, together with in monetary providers, healthcare, and the general public sector, now you can use these OpenAI fashions at scale. Amazon Bedrock processes inference requests and information inside India.

Each fashions provide a 1-million-token context window, settle for textual content and picture enter, and produce textual content output. Your purposes can course of lengthy paperwork, massive code bases, and blended text-and-image workloads in a single request. The processing by no means leaves the nation.

On this publish, we stroll by means of how India geographic cross-Area inference works from the Mumbai and Hyderabad Areas. We additionally present how one can get began from the Amazon Bedrock console and with code, utilizing the OpenAI Responses API, OpenAI Chat Completions API, and the Amazon Bedrock Converse API.

India geographic cross-Area inference

Cross-Area inference mechanically routes inference requests throughout a number of AWS Areas to assist enhance throughput, with out you having to handle capability in every Area your self. It’s primarily a capability mechanism. As a substitute of being certain to 1 Area’s capability, your requests draw on a broader pool of compute. That helps you preserve throughput and constant efficiency beneath load, which issues most throughout site visitors peaks.

With India geographic cross-Area inference, Amazon Bedrock routes requests solely inside the India geography throughout Areas equivalent to Asia Pacific (Mumbai) Area (ap-south-1) and Asia Pacific (Hyderabad) Area (ap-south-2). You’ll be able to scale to fulfill demand whereas maintaining information processed inside India to fulfill information residency necessities. You name the profile from both India Area because the supply, and Amazon Bedrock routes the request to the vacation spot India Area primarily based on capability. For essentially the most present details about mannequin availability in every Area, see Regional availability by fashions within the Amazon Bedrock Person Information.

Cross-Area inference works by means of inference profiles. You name a profile ID because the mannequin, and it defines the mannequin and the AWS Areas Amazon Bedrock can route your request to. The India geographic inference profiles hold that routing inside India. There are two profiles:

in.openai.gpt-5.6-terra for GPT-5.6 Terra.
in.openai.gpt-5.6-luna for GPT-5.6 Luna.

With inference profiles, billing and quota consumption are tracked towards your account within the supply Area, no matter which backend Area dealt with the request. Amazon CloudWatch and AWS CloudTrail file log entries within the supply Area solely, so your monitoring stays in a single place.

Selecting between Amazon Bedrock Mantle and Runtime endpoints

For brand spanking new purposes, we suggest the bedrock-runtime endpoint. It helps the Bedrock-native InvokeModel and Converse APIs, the OpenAI-compatible Responses and Chat Completions APIs, and the Anthropic Messages API, and it’s the place Amazon Bedrock options equivalent to Guardrails, clever immediate routing, and cross-Area inference can be found.

Knowledge residency

The India geographic profile retains inference inside India. Requests route solely between ap-south-1 and ap-south-2. Your enter prompts and output outcomes would possibly transfer between these two Areas. Knowledge is encrypted in transit throughout the Amazon community. Amazon Bedrock makes use of a zero information retention (ZDR) information safety mannequin. Because of this by default, Amazon Bedrock doesn’t retailer mannequin inputs or outputs. Nonetheless, for sure fashions, together with GPT-5.6, content material flagged by the Amazon Bedrock automated abuse-detection classifiers is retained for offline abuse detection. Please see Abuse detection within the Amazon Bedrock Person Information for extra particulars.

World cross-Area inference

Amazon Bedrock additionally affords international cross-Area inference and international inference profiles (prefixed international.) in India that path to supported business AWS Areas worldwide for optimum capability. You ship your request to the India Area endpoint, both Asia Pacific (Mumbai) ap-south-1 or Asia Pacific (Hyderabad) ap-south-2, utilizing the worldwide profile ID because the mannequin ID. Amazon Bedrock then decides which vacation spot Area serves the request. World cross-Area inference helps OpenAI GPT-5.6 fashions, together with Sol, Terra, and Luna. Nonetheless, in case your workload has native information processing necessities, use the India (prefixed in.) profiles as an alternative, as a result of they hold inference inside the nation. To learn extra about international cross-Area inference, see Introduce cross-Area inference for OpenAI GPT-5.6 fashions on Amazon Bedrock.

Entry GPT-5.6 fashions from the Amazon Bedrock console

You’ll be able to strive GPT-5.6 within the textual content playground within the Amazon Bedrock console, which requires no coding or SDK setup. You’ll be able to ship prompts, modify inference parameters, and swap between variants to get a really feel for every mannequin earlier than you combine the API.

Open the Amazon Bedrock console in a Area the place the fashions can be found, equivalent to Asia Pacific (Mumbai) ap-south-1.
Within the navigation pane, beneath Check, select Playground.
Select Choose mannequin in the midst of the web page.
Seek for OpenAI GPT-5.6 Terra, choose IN OpenAI GPT-5.6 Terra, and select Apply.
Enter a immediate and select Run to generate a response.

Selecting the OpenAI GPT-5.6 Terra model from the Amazon Bedrock Playground console

Determine 1: The OpenAI GPT-5.6 Terra mannequin chosen within the Amazon Bedrock console playground

Name GPT-5.6 fashions with the OpenAI Responses API

GPT-5.6 fashions on Amazon Bedrock natively help the OpenAI Responses API format. In case your utility already calls OpenAI fashions, you possibly can level your current OpenAI SDK consumer on the Amazon Bedrock endpoint in both India Area, Mumbai or Hyderabad, and move an India geographic inference profile ID because the mannequin parameter.

For authentication, Amazon Bedrock accepts both normal AWS credentials or an Amazon Bedrock API key. The API key path is the simple match for the OpenAI SDK, which passes it because the bearer token. For manufacturing, generate short-term API keys programmatically with the aws-bedrock-token-generator package deal. It derives a bearer token out of your current AWS credentials, so no static key must be saved.

The next instance makes use of Asia Pacific (Mumbai) ap-south-1 endpoint and India geographic inference profile ID to name OpenAI GPT-5.6 Terra on Amazon Bedrock.

from aws_bedrock_token_generator import provide_token
from openai import OpenAI

# Level the OpenAI SDK on the Amazon Bedrock OpenAI-compatible
# endpoint within the Asia Pacific (Mumbai) Area.
consumer = OpenAI(
    base_url="https://bedrock-runtime.ap-south-1.amazonaws.com/openai/v1",
    api_key=provide_token(area="ap-south-1"),  # short-term Amazon Bedrock API key, legitimate as much as 12 hours
)

# Geographic (India) inference profile ID for GPT-5.6 Terra.
model_id = "in.openai.gpt-5.6-terra"  # "in.openai.gpt-5.6-luna" for luna
response = consumer.responses.create(
    mannequin=model_id,
    enter="Extract the cost due date and complete quantity from the bill textual content that follows, and return them as JSON. ",
    max_output_tokens=512,
)
print(response.output_text)

The Responses API makes use of a single enter discipline and returns the generated textual content in output_text, with the output restrict set by means of max_output_tokens. The identical consumer additionally works with the Chat Completions API, helpful in case your utility already makes use of this format.

Controlling reasoning depth

To manage reasoning depth, set the non-obligatory reasoning parameter, for instance reasoning={"effort": "low"}. GPT-5.6 fashions on Amazon Bedrock help the next reasoning effort ranges: none, low, medium, excessive, xhigh, and max. For extra data, see the associated publish Get began with OpenAI GPT-5.6 Sol, Terra, and Luna on Amazon Bedrock. Omitting the parameter makes use of the mannequin default.

response = consumer.responses.create(
    mannequin="in.openai.gpt-5.6-terra",
    enter="Extract the cost due date and complete quantity from the bill textual content that follows, and return them as JSON. ",
    reasoning={"effort": "excessive"},  # none | low | medium | excessive | xhigh | max
    max_output_tokens=512,
)

For the complete listing of supported parameters, see the OpenAI mannequin documentation within the Amazon Bedrock Person Information.

Multi-turn conversations with server-side state

Set retailer=True to have Amazon Bedrock retain the response server-side, then reference it on the following flip with previous_response_id. You ship solely the brand new flip. You don’t resend the prior dialog historical past.

first = consumer.responses.create(
    mannequin="in.openai.gpt-5.6-terra",
    enter="Bear in mind this: my favourite quantity is 42. Reply with simply 'saved'..",
    max_output_tokens=50,
    retailer=True,
)

second = consumer.responses.create(
    mannequin="in.openai.gpt-5.6-terra",
    previous_response_id=first.id,  # the mannequin remembers the sooner flip
    enter="what's my favorite quantity?",
    max_output_tokens=50,
    retailer=True,
)

print(second.output_text)

When you chain with previous_response_id, the response you chain from will need to have been created with retailer=True. Chaining from an unstored response returns an error.

Streaming responses

For streaming, set stream=True and iterate over the occasions:

stream = consumer.responses.create(
    mannequin="in.openai.gpt-5.6-terra",
    enter="Draft a brief standing replace for a delayed cargo, in a well mannered and direct tone.",
    max_output_tokens=512,
    stream=True,
)

for occasion in stream:
    if occasion.kind == "response.output_text.delta":
        print(occasion.delta, finish="")

Name GPT-5.6 fashions with the Converse API

When you choose the AWS SDK and SigV4 authentication over an API key, the Amazon Bedrock Converse API helps GPT-5.6 fashions with the identical unified interface it offers for different fashions on Amazon Bedrock. ConverseStream covers the streaming case.

import boto3

# Commonplace AWS credentials (SigV4), no API key wanted.
consumer = boto3.consumer("bedrock-runtime", region_name="ap-south-1")
model_id = "in.openai.gpt-5.6-terra"  # India Geo inference profile
response = consumer.converse(
    modelId=model_id,
    messages=[
        {
            "role": "user",
            "content": [{"text": "Classify this support ticket as billing, technical, or account: "}],
        }
    ],
    inferenceConfig={"maxTokens": 512},
)
print(response["output"]["message"]["content"]

Immediate caching

Your prompts would possibly share an extended, secure prefix, equivalent to a system instruction, a information base excerpt, or a set of few-shot examples. After they do, GPT-5.6 fashions on Amazon Bedrock help immediate caching. Cached reads are billed at a 90 % low cost in comparison with uncached enter tokens. This provides up rapidly for Retrieval Augmented Technology (RAG) and agent workloads that repeat the identical context throughout many turns. You could find particulars on Amazon Bedrock immediate caching documentation.

Immediate caching works with the India geographic inference profiles, so you retain the financial savings beneath the India data-residency boundary. Immediate caching runs in two modes, specific and implicit. With implicit caching, Amazon Bedrock locations the cache breakpoints for you mechanically. Notice that the minimal prefix size is 1,024 tokens. With specific caching, you mark the cache boundary your self for exact management, and the cached prefix stays heat for at the very least half-hour.

response = consumer.responses.create(
    mannequin="in.openai.gpt-5.6-terra",
    prompt_cache_key="ticket-agent-ver123",  # identical key throughout all requests
    enter=[
        {
            "type": "message",
            "role": "developer",
            "content": [{
                "type": "input_text",
                "text": SYSTEM_INSTRUCTIONS,  # long, static: guidelines, KB excerpts (>= 1,024 tokens)
                "prompt_cache_breakpoint": {"mode": "explicit"},
            }],
        },
        {
            "kind": "message",
            "function": "consumer",
            "content material": [{
                "type": "input_text",
                "text": user_question,  # changes on every request
            }],
        },
    ],
    extra_body={"prompt_cache_options": {"mode": "specific"}},
)

Each response tells you what the cache did, in utilization.input_tokens_details:

particulars = response.utilization.input_tokens_details
print(f"cached: {particulars.cached_tokens}, written: {particulars.cache_write_tokens}")

On the primary name, you will note cache_write_tokens populated because the prefix is saved. On subsequent calls, the identical tokens come again as cached_tokens, billed on the cache-read fee. As a result of these counts are a part of input_tokens relatively than added on prime, a cached token is counted and charged as soon as.

IAM permissions for India geographic cross-Area inference

To permit an AWS Id and Entry Administration (IAM) function to invoke GPT-5.6 fashions by means of the India geographic inference profile, grant the function entry to a few assets. These are the India geographic inference profile itself, the muse mannequin (FM) within the supply Area, and the muse mannequin in every vacation spot Area listed within the profile (ap-south-1 and ap-south-2). You need to use the AmazonBedrockLimitedAccess managed coverage or create your personal.

The next instance grants permission to make use of the GPT-5.6 Terra mannequin by means of the India geographic inference profile. The supply Area is ap-south-1, and the vacation spot Areas are ap-south-1 and ap-south-2. Substitute together with your account ID, and duplicate the assets for in.openai.gpt-5.6-luna if the function wants each fashions. When you supply from Hyderabad, use the ap-south-2 inference-profile ARN within the first assertion. The muse-model assets within the second assertion keep the identical, as a result of they already listing each India Areas.

{
    "Model": "2012-10-17",
    "Assertion": [
        {
            "Sid": "GrantGeoCrisInferenceProfileAccess",
            "Effect": "Allow",
            "Action": [
                "bedrock:InvokeModel*"
            ],
            "Useful resource": [
                "arn:aws:bedrock:ap-south-1::inference-profile/in.openai.gpt-5.6-terra",
                "arn:aws:bedrock:ap-south-1::project/default"
            ]
        },
        {
            "Sid": "GrantGeoCrisModelAccess",
            "Impact": "Enable",
            "Motion": [
                "bedrock:InvokeModel*"
            ],
            "Useful resource": [
                "arn:aws:bedrock:ap-south-1::foundation-model/openai.gpt-5.6-terra",
                "arn:aws:bedrock:ap-south-2::foundation-model/openai.gpt-5.6-terra"
            ],
            "Situation": {
                "StringEquals": {
                    "bedrock:InferenceProfileArn": "arn:aws:bedrock:ap-south-1::inference-profile/in.openai.gpt-5.6-terra"
                }
            }
        },
        {
            "Sid": "AllowBearerTokenAuth",
            "Impact": "Enable",
            "Motion": ["bedrock:CallWithBearerToken"],
            "Useful resource": "*"
        }
    ]
}

The primary assertion grants entry to the India geographic inference profile for requests originating from the supply Area. The second grants entry to the muse mannequin within the supply Area and in each vacation spot Area listed within the profile. This entry is scoped to that one profile by means of the bedrock:InferenceProfileArn situation. These insurance policies grant bedrock:CallWithBearerToken and bedrock:InvokeModel on the venture/default useful resource. The OpenAI Responses and Chat Completions APIs use this useful resource to authenticate with an Amazon Bedrock API key and run inference.

Service Management Coverage (SCP)

In case your group makes use of Service Management Insurance policies to dam unused Areas, your Area-specific SCP situations should permit each vacation spot Area listed within the geographic inference profile.

For the India geographic inference profile, each ap-south-1 and ap-south-2 should be permitted. Blocking both vacation spot Area causes cross-Area inference to fail regardless that the supply Area stays accessible.

Organising Codex with GPT-5.6 on Amazon Bedrock

Codex can use the geographic inference profiles by means of Amazon Bedrock Runtime. Set up the newest Codex CLI (codex-cli 0.149.1 or later) to make use of the Amazon Bedrock Runtime mannequin supplier. For organizations whose id supplier is Okta, Auth0, Microsoft Entra ID, Amazon Cognito, or AWS IAM Id Middle, the AWS OIDC Auth Helper repository offers a pattern credential helper.

First, comply with the information to configure your id supplier, the corresponding AWS federation useful resource, and an IAM function carrying the Bedrock permissions proven earlier. Then add a named profile to ~/.aws/config, so the helper doesn’t change credentials resolved by the default profile.

[profile ] 
credential_process =  --profile  
area = ap-south-1
output = json

This federation helper exchanges an OIDC token for non permanent AWS credentials, and Codex reads by means of the usual AWS credential chain with no additional configuration. Subsequent, create or replace ~/.codex/config.toml and reference the AWS profile, see the Codex configuration reference for different supported settings:

mannequin = "in.openai.gpt-5.6-terra" 
model_provider = "amazon-bedrock-runtime" 
model_reasoning_effort = "excessive" 
 
[model_providers.amazon-bedrock-runtime.aws] 
profile = "" 
area = "ap-south-1"

If the helper doesn’t have a legitimate cached session, it opens the configured sign-in web page in your browser. After you authenticate, the helper returns non permanent AWS credentials by means of credential_process. Requests are signed with AWS SigV4. When the profile is backed by AWS IAM Id Middle, the credentials are already short-term and rotate with the SSO session. To make use of this within the Asia Pacific (Hyderabad) Area, set the Area to ap-south-2 within the AWS profile and Codex configuration.

If you have already got AWS credentials configured domestically, Codex can use them immediately. Level the Amazon Bedrock Runtime supplier at an AWS profile and an India Area, and set the mannequin to an India geographic inference profile ID.

Create or replace ~/.codex/config.toml:

  mannequin = "in.openai.gpt-5.6-terra"
  model_provider = "amazon-bedrock-runtime"
  model_reasoning_effort = "excessive"

  [model_providers.amazon-bedrock-runtime.aws]
  profile = "default"
  area = "ap-south-1"

The profile worth is any profile in ~/.aws/config or ~/.aws/credentials. On this case, it’s proven as “default”, however you should utilize your personal named profile should you hold a number of accounts.

Monitoring and logging

The India geographic inference profiles work with the identical account-level controls you already use for different fashions on Amazon Bedrock. Requests seem in Amazon Bedrock mannequin invocation logging the identical manner on-demand requests do. You’ll be able to ship these logs to Amazon Easy Storage Service (Amazon S3) or Amazon CloudWatch Logs. CloudWatch metrics for invocation depend, token depend, latency, throttles, and errors are revealed per inference profile. You’ll be able to monitor GPT-5.6 fashions utilization from Mumbai or Hyderabad with out correlating logs throughout Areas your self.

To see which vacation spot Area processed a given request, CloudTrail occasions embody an additionalEventData discipline with an inferenceRegion key. For the India profile, that is all the time ap-south-1 or ap-south-2, which provides you an auditable file that inference stayed inside India. Utilization is itemized on AWS Value Explorer and the AWS Value and Utilization Report. Spend attribution by mannequin works the identical manner as for the remainder of your Amazon Bedrock workloads.

Conclusion

Amazon Bedrock brings the OpenAI GPT-5.6 Terra and Luna fashions, with a 1-million-token context window, to prospects constructing in India. The India geographic inference profiles can be found on the bedrock-runtime endpoint, which is the endpoint we suggest for brand spanking new purposes. With India geographic cross-Area inference, you possibly can scale throughout the Mumbai and Hyderabad Areas whereas maintaining inference inside the nation. You’ll be able to name the fashions by means of the OpenAI Responses API, the OpenAI Chat Completions API, or the Amazon Bedrock Converse API. You probably have an current workload on the bedrock-mantle endpoint, it stays totally supported and no modifications are required; to undertake the India geographic profiles, level that workload on the bedrock-runtime endpoint in Mumbai or Hyderabad. Evaluation the Amazon Bedrock pricing web page for present GPT-5.6 charges earlier than sizing a manufacturing workload.

To get began, evaluate the mannequin playing cards for GPT-5.6 Terra and Luna and the Cross-Area inference part within the Amazon Bedrock Person Information. Strive a immediate within the Amazon Bedrock console playground in Mumbai or Hyderabad, then wire up the code examples shared earlier on this publish. When you run into a problem or wish to ask the neighborhood, AWS re:Submit is an effective place to go looking or publish a query.