Introducing structured output for Customized Mannequin Import in Amazon Bedrock

With Amazon Bedrock Customized Mannequin Import, you’ll be able to deploy and scale fine-tuned or proprietary basis fashions in a completely managed, serverless surroundings. You possibly can carry your individual fashions into Amazon Bedrock, scale them securely with out managing infrastructure, and combine them with different Amazon Bedrock capabilities.

Right this moment, we’re excited to announce the addition of structured output to Customized Mannequin Import. Structured output constrains a mannequin’s technology course of in actual time so that each token it produces conforms to a schema you outline. Reasonably than counting on prompt-engineering tips or brittle post-processing scripts, now you can generate structured outputs instantly at inference time.

For sure manufacturing functions, the predictability of mannequin outputs is extra vital than their artistic flexibility. A customer support chatbot may profit from various, natural-sounding responses, however an order processing system wants precise, structured knowledge that conforms to predefined schemas. Structured output bridges this hole by sustaining the intelligence of basis fashions whereas verifying their outputs meet strict formatting necessities.

This represents a shift from free-form textual content technology to outputs which might be constant, machine-readable, and designed for seamless integration with enterprise techniques. Whereas free-form textual content excels for human consumption, manufacturing functions require extra precision. Companies can’t afford the paradox of pure language variations when their techniques depend upon structured outputs to reliably interface with APIs, databases, and automatic workflows.

On this publish, you’ll learn to implement structured output for Customized Mannequin Import in Amazon Bedrock. We’ll cowl what structured output is, how you can allow it in your API calls, and how you can apply it to real-world eventualities that require structured, predictable outputs.

Understanding structured output

Structured output, also referred to as constrained decoding, is a technique that directs LLM outputs to adapt to a predefined schema, similar to legitimate JSON. Reasonably than permitting the mannequin to freely choose tokens based mostly on likelihood distributions, it introduces constraints throughout technology that restrict selections to solely those who keep structural validity. If a specific token would violate the schema by producing invalid JSON, inserting stray characters, or utilizing an surprising discipline identify the structured output rejects it and requires the mannequin to pick out one other allowed possibility. This real-time validation helps maintain the ultimate output constant, machine readable, and instantly usable by downstream functions with out the necessity for extra post-processing.

With out structured output, builders typically try to implement construction by immediate directions like “Reply solely in JSON.” Whereas this strategy generally works, it stays unreliable as a result of inherently probabilistic nature of LLMs. These fashions generate textual content by sampling from likelihood distributions, introducing pure variability that makes responses really feel human however creates vital challenges for automated techniques.

Take into account a buyer assist utility that classifies tickets: if responses differ between “This looks like a billing difficulty,” “I’d classify this as: Billing,” and “Class = BILLING,” downstream code can not reliably interpret the outcomes. What manufacturing techniques require as an alternative is predictable, structured output. For instance:

{
  "class": "billing",
  "precedence": "excessive",
  "sentiment": "unfavorable"
}

With a response like this, your utility can mechanically route tickets, set off workflows, or replace databases with out human intervention. By offering predictable, schema-aligned responses, structured output transforms LLMs from conversational instruments into dependable system parts that may be built-in with databases, APIs, and enterprise logic. This functionality opens new potentialities for automation whereas sustaining the clever reasoning that underpin the worth of those fashions.

Past bettering reliability and simplifying post-processing, structured output gives extra advantages that strengthens efficiency, safety and security in manufacturing environments.

• Decrease token utilization and quicker responses: By constraining technology to an outlined schema, structured output removes pointless verbose, free-form textual content, leading to diminished token depend. As a result of token technology is sequential, shorter outputs instantly translate to quicker responses and decrease latency, bettering total efficiency and price effectivity.
• Enhanced safety in opposition to immediate injection: Structured output narrows the mannequin’s expression area and helps forestall it from producing arbitrary or unsafe content material. Dangerous actors can not inject directions, code or surprising textual content outdoors the outlined construction. Every discipline should match its anticipated kind and format, ensuring outputs stay inside protected boundaries.
• Security and coverage controls: Structured output lets you design schemas that inherently assist forestall dangerous, poisonous, or policy-violating content material. By limiting fields to authorised values, implementing patterns, and proscribing free-form textual content, schemas make certain outputs align with regulatory necessities.

Within the subsequent part, we are going to discover how structured output works with Customized Mannequin Import in Amazon Bedrock and walks by an instance of enabling it in your API calls.

Utilizing structured output with Customized Mannequin Import in Amazon Bedrock

Let’s begin by assuming you’ve gotten already imported a Hugging Face mannequin into Amazon Bedrock utilizing the Customized Mannequin Import function.

Stipulations

Earlier than continuing, be sure to have:

• An energetic AWS account with entry to Amazon Bedrock
• A customized mannequin created in Amazon Bedrock utilizing the Customized Mannequin Import function
• Applicable AWS Id and Entry Administration (IAM) permissions to invoke fashions by the Amazon Bedrock Runtime

With these conditions in place, let’s discover how you can implement structured output together with your imported mannequin.

To start out utilizing structured output with a Customized Mannequin Import in Amazon Bedrock, start by configuring your surroundings. In Python, this includes making a Bedrock Runtime shopper and initializing a tokenizer out of your imported Hugging Face mannequin.

The Bedrock Runtime shopper supplies entry to your imported mannequin utilizing the Bedrock InvokeModel API. The tokenizer applies the proper chat template that aligns with the imported mannequin, which defines how consumer, system, and assistant messages are mixed right into a single immediate, how the position markers (for instance, <|consumer|>, <|assistant|>) are inserted, and the place the mannequin’s response ought to start.

By calling tokenizer.apply_chat_template(messages, tokenize=False) you’ll be able to generate a immediate that matches the precise enter format your mannequin expects, which is crucial for constant and dependable inference, particularly when structured encoding is enabled.

import boto3
from transformers import AutoTokenizer
from botocore.config import Config

# HF mannequin identifier imported into Bedrock
hf_model_id = "<>" # Instance: "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
model_arn = "arn:aws:bedrock:<>:<>:imported-model/your-model-id"
area      = "<>"

# Initialize tokenizer aligned together with your imported mannequin 
tokenizer = AutoTokenizer.from_pretrained(hf_model_id)

# Initialize Bedrock shopper
bedrock_runtime = boto3.shopper(
    service_name="bedrock-runtime",
    region_name=area)

Implementing structured output

If you invoke a customized mannequin on Amazon Bedrock, you’ve gotten the choice to allow structured output by including a response_format block to the request payload. This block accepts a JSON schema that defines the structured of the mannequin’s response. Throughout inference, the mannequin enforces this schema in real-time, ensuring that every generated token conforms to the outlined construction. Beneath is a walkthrough demonstrating how you can implement structured output utilizing a easy handle extraction process.

Step 1: Outline the info construction

You possibly can outline your anticipated output utilizing a Pydantic mannequin, which serves as a typed contract for the info you wish to extract.

from pydantic import BaseModel, Area

class Tackle(BaseModel):
    street_number: str = Area(description="Road quantity")
    street_name: str = Area(description="Road identify together with kind (Ave, St, Rd, and so forth.)")
    metropolis: str = Area(description="Metropolis identify")
    state: str = Area(description="Two-letter state abbreviation")
    zip_code: str = Area(description="5-digit ZIP code")

Step 2: Generate the JSON schema

Pydantic can mechanically convert your knowledge mannequin right into a JSON schema:

schema = Tackle.model_json_schema()
address_schema = {
    "identify": "Tackle",
    "schema": schema
}

This schema defines every discipline’s kind, description, and requirement, making a blueprint that the mannequin will comply with throughout technology.

Step 3: Put together your enter messages

Format your enter utilizing the chat format anticipated by your mannequin:

messages = [{
    "role": "user",
    "content": "Extract the address: 456 Tech Boulevard, San Francisco, CA 94105"
}]

Step 4: Apply the chat template

Use your mannequin’s tokenizer to generate the formatted immediate:

immediate = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

Step 5: Construct the request payload

Create your request physique, together with the response_format that references your schema:

request_body = {
    'immediate': immediate,
    'temperature': 0.1,
    'max_gen_len': 1000,
    'top_p': 0.9,
    'response_format': {
        "kind": "json_schema",
        "json_schema": address_schema
    }
}

Step 6: Invoke the mannequin

Ship the request utilizing the InvokeModel API:

response = bedrock_runtime.invoke_model(
    modelId=model_arn,
    physique=json.dumps(request_body),
    settle for="utility/json",
    contentType="utility/json"
)

Step 7: Parse the response

Extract the generated textual content from the response:

consequence = json.hundreds(response['body'].learn().decode('utf-8'))
raw_output = consequence['choices'][0]['text']
print(raw_output)

As a result of the schema defines required fields, the mannequin’s response will comprise them:

{
"street_number": "456",
"street_name": "Tech Boulevard",
"metropolis": "San Francisco",
"state": "CA",
"zip_code": "94105"
}

The output is clear, legitimate JSON that may be consumed instantly by your utility with no further parsing, filtering, or cleanup required.

Conclusion

Structured output with Customized Mannequin Import in Amazon Bedrock supplies an efficient strategy to generate buildings, schema-aligned outputs out of your fashions. By shifting validation into the mannequin inference itself, structured output scale back the necessity for advanced post-processing workflows and error dealing with code.

Structured output generates outputs which might be predictable and easy to combine into your techniques and helps quite a lot of use instances, for instance, constructing monetary functions that require exact knowledge extraction, healthcare techniques that want structured medical documentation, or customer support techniques that demand constant ticket classification.

Begin experimenting with structured output together with your Customized Mannequin Import in the present day and rework how your AI functions ship constant, production-ready outcomes.

Concerning the authors

Manoj Selvakumar is a Generative AI Specialist Options Architect at AWS, the place he helps organizations design, prototype, and scale AI-powered options within the cloud. With experience in deep studying, scalable cloud-native techniques, and multi-agent orchestration, he focuses on turning rising improvements into production-ready architectures that drive measurable enterprise worth. He’s enthusiastic about making advanced AI ideas sensible and enabling prospects to innovate responsibly at scale—from early experimentation to enterprise deployment. Earlier than becoming a member of AWS, Manoj labored in consulting, delivering knowledge science and AI options for enterprise shoppers, constructing end-to-end machine studying techniques supported by sturdy MLOps practices for coaching, deployment, and monitoring in manufacturing.

Yanyan Zhang is a Senior Generative AI Information Scientist at Amazon Net Companies, the place she has been engaged on cutting-edge AI/ML applied sciences as a Generative AI Specialist, serving to prospects use generative AI to realize their desired outcomes. Yanyan graduated from Texas A&M College with a PhD in Electrical Engineering. Outdoors of labor, she loves touring, understanding, and exploring new issues.

Lokeshwaran Ravi is a Senior Deep Studying Compiler Engineer at AWS, specializing in ML optimization, mannequin acceleration, and AI safety. He focuses on enhancing effectivity, decreasing prices, and constructing safe ecosystems to democratize AI applied sciences, making cutting-edge ML accessible and impactful throughout industries.

Revendra Kumar is a Senior Software program Growth Engineer at Amazon Net Companies. In his present position, he focuses on mannequin internet hosting and inference MLOps on Amazon Bedrock. Previous to this, he labored as an engineer on internet hosting Quantum computer systems on the cloud and creating infrastructure options for on-premises cloud environments. Outdoors of his skilled pursuits, Revendra enjoys staying energetic by taking part in tennis and climbing.

Muzart Tuman is a software program engineer using his expertise in fields like deep studying, machine studying optimization, and AI-driven functions to assist remedy real-world issues in a scalable, environment friendly, and accessible method. His objective is to create impactful instruments that not solely advance technical capabilities but additionally encourage significant change throughout industries and communities.