A Sensible Information to OpenAI’s New Choices API

I not too long ago coated TypesafeAI’s JEV in TDS (hyperlink on the finish). JEV makes quick choices, corresponding to classifying content material or selecting the following step in a workflow, and returns typed solutions that software program can use immediately. The accompanying chances assist an utility resolve whether or not to behave on a solution or move it to an individual for overview.

The web went a bit loopy over JEV, so it was no shock to see rival merchandise seem after its launch. Among the many most outstanding was the Choices API that OpenAI previewed at its latest DevDay and made out there as a public beta shortly after.

On this article, I’ll have a look at the brand new API, the right way to get it, and a few sensible use-case examples, and I’ll let you realize whether or not I believe it’s a JEV killer or whether or not TypesafeAI can sleep soundly at evening.

The background to JEV-like fashions

Suppose you run a web based store. A buyer writes to say that their keyboard arrived with three damaged keys and asks for a alternative. Earlier than anybody replies, your utility must resolve which crew ought to deal with the message.

You can ask a language mannequin to categorise it and return JSON. However the utility solely wants a class and sufficient info to resolve whether or not to belief that class. That course of may be comparatively sluggish and expensive if it needed to deal with 1000s of requests.

OpenAI’s Choices API offers that operation its personal endpoint. Moreover, OpenAI claims its Choices API could make choices as much as ten occasions sooner than GPT-6 Luna, the LLM that Choices is predicated on, by way of the Responses API. To discover the way it works, we’ll stroll by way of three Python examples and examine OpenAI’s method with TypesafeAI’s JEV mannequin.

One main benefit of the Choices API over JEV is that it might probably deal with pictures. We’ll see an instance of that later.

What the OpenAI Choices API returns

A request provides a mannequin, some enter and an inventory of questions. The present beta helps the gpt-6-luna LLM. Questions share the enter, which may include textual content, pictures or each. 

There are three query sorts:

+-----------+--------------------------------------------------+------------------------------+| Kind      | Query                                         | Consequence                       |+-----------+--------------------------------------------------+------------------------------+| predicate | Does the doc include a return deadline?     | A likelihood between zero   ||           |                                                  | and one                      |+-----------+--------------------------------------------------+------------------------------+| selection    | Which assist queue ought to obtain this message? | A specific worth,            |       |           |                                                  | chances and confidence |+-----------+--------------------------------------------------+------------------------------+| rating     | How severely does this incident have an effect on prospects?| A rating throughout ordered       ||           |                                                  | ranges, chances and    |          |           |                                                  | confidence                   |+-----------+--------------------------------------------------+------------------------------+

These fields are outlined within the SDK’s response sorts. A predicate doesn’t return a Boolean: your code chooses the edge that turns its likelihood into an motion.

The API returns solutions in query order. It might probably additionally refuse particular person questions, so examine a solution’s kind earlier than studying its different fields. 

Defining the out there solutions is a part of designing the applying. Suppose your decisions are solely billing and supply, however the buyer asks about opening hours. Neither response is acceptable, however including a normal class offers the system a wise place to route that request. Check the classes in opposition to actual buyer messages, together with requests that don’t match any of them.

With the Choices API, you select from three query sorts: predicate, selection and rating. Every returns an outlined reply format, together with chances. If you could extract an bill quantity, a buyer identify and an inventory of bought gadgets into your personal JSON construction, use Structured Outputs with the Responses API.

When to make use of the Choices API and what it prices

A assist system processing hundreds of messages could solely want a queue identify earlier than passing every request to the proper crew. The Responses API can produce that reply, however the Choices API supplies a devoted interface for questions with outlined solutions: a sure/no likelihood, a selection from an inventory or a rating in opposition to a rubric.

You ship these inquiries to /v1/choices and use the returned solutions in your utility logic. That makes it a helpful choice for routing messages, choosing a search index or selecting an agent’s subsequent motion from a permitted checklist.

It additionally has separate pricing. At launch, OpenAI lists $0.10 per million enter tokens, with no costs for output tokens, cache reads, or cache writes. Lengthy-context multipliers and regional processing premiums nonetheless apply. 

As a easy calculation, a million requests averaging 1,000 billable enter tokens would value $100 at that base charge. Embody the questions and their descriptions when estimating enter measurement.

A process that wants a written clarification or an extracted object with arbitrary fields nonetheless wants a era interface. It doesn’t make sense to squeeze an bill extraction downside into twenty classification questions simply because the endpoint is quick.

Equally, preserve easy guidelines in Python. If precedence relies upon solely on an order whole exceeding £500, for instance, examine the quantity immediately in Python code itself. A mannequin turns into helpful when the enter expresses one thing your guidelines can’t simply recognise, corresponding to a buyer describing the identical fault in a number of alternative ways. Even then, the additional community name should save sufficient downstream work to justify its latency.

Arrange Python with the API

The official openai-python repository added Choices assist in model 3.26.0. Set up that model in your surroundings:

c:> python -m pip set up openai==3.26.0

You’ll want an OpenAI API key. Should you don’t have already got one, you could be sure to have registered with OpenAI and added a fee technique with some credit score to your account. Afterwards, go to https://platform.openai.com/residence. On the left aspect of the display, you’ll see an API keys hyperlink. Click on on that and comply with the directions to create a brand new secret key.

Set the OPENAI_API_KEY surroundings variable to your API key. Try this in PowerShell like this:

c:> $env:OPENAI_API_KEY="your-api-key"

Every instance under is an entire, standalone Python program with its personal imports and consumer setup.

The SDK implementation exposes consumer.choices.create() and sends the request to the /v1/choices endpoint. That is the consumer implementation; inference runs on OpenAI’s service.

Save every program utilizing the filename proven in its part, then run it immediately with Python. The three recordsdata work independently. Every program makes a billable API request if you run it.

Code instance 1: Routing a assist request

Our store has three specialist queues and a normal queue for something that doesn’t match. Create a file named decisions_route.py and add this code.

from openai import OpenAIMODEL = "gpt-6-luna"def predominant() -> None:    with OpenAI(timeout=20.0) as consumer:        consequence = consumer.choices.create(            mannequin=MODEL,            enter="My keyboard arrived with three damaged keys. Are you able to substitute it?",            questions=[{                "type": "choice",                "name": "queue",                "instructions": "Choose the queue for the customer's main request.",                "choices": [                    {"value": "returns", "description": "Damaged goods or replacements."},                    {"value": "billing", "description": "Charges or invoice errors."},                    {"value": "delivery", "description": "Missing or delayed deliveries."},                    {"value": "general", "description": "Everything else or unclear intent."},                ],            }],        )        reply = consequence.solutions[0]        if reply.kind == "refusal":            print("Ship to guide triage")        elif reply.kind == "selection":            print(reply.selection, reply.confidence)            queue = reply.selection if reply.confidence >= 0.8 else "manual_triage"            print("Queue:", queue)if __name__ == "__main__":    predominant()

The (right) output to my query a few damaged keyboard was:

c:> python decisions_route.pyreturns 1.0Queue: returns

After I requested a special query within the code,

Do you've got a web site I may have a look at?

I received this output, which is right once more.

c:> python decisions_route.pynormal 0.8Queue: normal

The selection objects include values and descriptions, as specified within the SDK’s request sorts. Descriptions allow us to distinguish broken items from a supply that by no means arrived.

The 0.8 threshold is illustrative. It isn’t an OpenAI advice, and it doesn’t set up an 80% success charge. Begin with messages folks have already categorised, and measure errors at a number of thresholds earlier than you resolve in your threshold.

Code Instance 2: Test a doc for lacking info

Suppose workers write return directions in a number of codecs. We wish to flag directions that omit both a deadline or a postal tackle. Create the file decisions_document.py with this content material.

from openai import OpenAI, RateLimitErrorMODEL = "gpt-6-luna"def predominant() -> None:    with OpenAI(timeout=20.0) as consumer:        checks = {            "deadline": "Does the textual content explicitly give a deadline for returning an merchandise?",            "tackle": "Does the textual content explicitly present a postal return tackle?",        }        consequence = consumer.choices.create(            mannequin=MODEL,            enter="Return the merchandise inside 30 days. E-mail assist to request our tackle.",            questions=[                {"type": "predicate", "name": name, "instructions": question}                for name, question in checks.items()            ],        )        for reply in consequence.solutions:            if reply.kind == "refusal":                print(reply.identify, "overview required")            elif reply.kind == "predicate":                standing = "current" if reply.likelihood >= 0.9 else "examine manually"                print(reply.identify, standing, reply.likelihood)if __name__ == "__main__":    attempt:        predominant()    besides RateLimitError as exc:        if exc.code == "credit_balance_exhausted":            increase SystemExit(                "Your OpenAI API credit score steadiness is exhausted. Add credit for "                "the organisation related together with your API key at "                "https://platform.openai.com/settings/group/billing/ "                "after which run this instance once more."            ) from None        increase

My output was:

c:> python decisions_document.pydeadline current 1.0tackle examine manually 0.0

The wording is essential. Being instructed to request an tackle isn’t the identical as being given one. A seek for the phrase tackle would miss that distinction.

Each questions can share one request as a result of neither will depend on the opposite’s reply. If a later query wants an earlier consequence, make one other request after inspecting that consequence.

This checks whether or not info seems within the textual content. Checking whether or not an tackle exists or whether or not a deadline matches your online business guidelines wants separate validation.

Code Instance 3: Examine a product {photograph}

Our final instance makes use of three pictures of a parcel, one closely broken, one with slight injury and the opposite fully undamaged. We’ll see if the mannequin can distinguish between broken/undamaged. Listed below are the photographs I used. All have been in .PNG format.

Place all three pictures in the identical location because the Python scripts, referred to as, say, undamaged.png, heavy_damage.png and slight_damage.png. Subsequent, create a file referred to as decisions_image.py with this code.

import base64from pathlib import Pathfrom openai import OpenAI, RateLimitErrorMODEL = "gpt-6-luna"PARCEL_IMAGES = ("undamaged.png", "heavy_damage.png", "slight_damage.png")def predominant() -> None:    with OpenAI(timeout=20.0) as consumer:        for filename in PARCEL_IMAGES:            image_path = Path(__file__).with_name(filename)            encoded = base64.b64encode(                image_path.read_bytes()            ).decode("ascii")            consequence = consumer.choices.create(                mannequin=MODEL,                enter=[{                    "role": "user",                    "content": [                        {                            "type": "input_text",                            "text": "Image of a delivered parcel.",                        },                        {                            "type": "input_image",                            "image_url": f"data:image/png;base64,{encoded}",                        },                    ],                }],                questions=[{                    "type": "predicate",                    "name": "visible_damage",                    "instructions": (                        "Does the packaging visibly have a tear, "                        "hole or crushed corner?"                    ),                }],            )            reply = consequence.solutions[0]            if reply.kind == "refusal":                print(f"{filename}: Examine the {photograph} manually")            elif reply.kind == "predicate":                print(                    f"{filename}: Chance of seen packaging "                    f"injury: {reply.likelihood}"                )if __name__ == "__main__":    attempt:        predominant()    besides RateLimitError as exc:        if exc.code == "credit_balance_exhausted":            increase SystemExit(                "Your OpenAI API credit score steadiness is exhausted. Add credit for "                "the organisation related together with your API key at "                "https://platform.openai.com/settings/group/billing/ "                "after which run this instance once more."            ) from None        increase

Photos should use inline information URLs; odd internet URLs and file IDs aren’t accepted. The endpoint helps as much as 128 pictures per request, and its message enter helps solely the consumer position with textual content and picture elements.

Listed below are my outputs:

c:> python decisions_image.pyundamaged.png: Chance of seen packaging injury: 0.05heavy_damage.png: Chance of seen packaging injury: 1.0slight_damage.png: Chance of seen packaging injury: 0.98

That final consequence stunned me. The package deal was solely barely broken, however the mannequin recognized it with excessive likelihood. That’s fairly spectacular, although I settle for that even slight injury might be seen and will result in a excessive rating.

Abstract: How Choices compares with JEV

JEV addresses an identical programming downside to the Choices API. Its interface evaluates shared state utilizing three primitives: Noul, Alternative, and Rating. Noul returns the likelihood of a sure/no reply, making it the closest equal to OpenAI’s predicate, which we utilized in examples 2 and three. JEV’s selection primitive is essentially the identical as OpenAI’s, which we utilized in our first instance.

The merchandise behind these interfaces differ. OpenAI exposes GPT-6 Luna by way of a specialised endpoint. TypeSafeAI describes JEV as a mannequin constructed for choices, with a parallel sampler and Reinforcement Studying for Calibrated Choices, or RLCD.

The sensible comparability, as of early October 2026, appears like this:

+-----------------------------+------------------+-------------------------------------------+| Characteristic                     | OpenAI Choices | Jev 1.13                                  |+-----------------------------+------------------+-------------------------------------------+| Mannequin                       | gpt-6-luna       | jev-1.13.0                                |+-----------------------------+------------------+-------------------------------------------+| Enter                       | Textual content and pictures  | Textual content, together with structured textual state  |+-----------------------------+------------------+-------------------------------------------+| Questions                   | Ordered array    | Map keyed by query identify                |+-----------------------------+------------------+-------------------------------------------+| Sure/no                      | predicate ->     | noul query -> noul                     ||                             | likelihood      |                                           |+-----------------------------+------------------+-------------------------------------------+| Select an choice            | selection -> selection | selection -> selection, chances,          ||                             | chances,   | confidence                                ||                             | confidence       |                                           |+-----------------------------+------------------+-------------------------------------------+| Rating in opposition to a rubric      | rating -> rating,  | rating -> rating, chances,            ||                             | chances,   | confidence, legend                        ||                             | confidence       |                                           |+-----------------------------+------------------+-------------------------------------------+|Worth per million tokens     | $0.10            | $0.042                                    |+-----------------------------+------------------+-------------------------------------------+| Output-token cost         | None             | None                                      |+-----------------------------+------------------+-------------------------------------------+

JEV’s listed enter token value is 58% decrease, though completely different tokenisation and request sizes have an effect on the invoice. Its present context limits are 64,000 tokens for the entire request and 32,000 for the state plus the longest query. 

TypeSafeAI studies 70–500 millisecond response occasions in its launch put up. Throughout my testing of the OpenAI Choices API, response occasions typically bumped into a number of seconds, however I didn’t time a direct comparability between the 2 merchandise.

For each programs, Python nonetheless controls what occurs subsequent after the fashions return their outcomes. Preserve arithmetic and glued enterprise guidelines in odd code, and use a generative mannequin when the duty is extra ambiguous and wishes a written clarification.

So, lastly, do I believe JEV must be nervous? No, not but, no less than. In my expertise, JEV has the benefit of pace; Choices API has the benefit of deciphering pictures. However how lengthy do you assume it is going to be earlier than JEV can course of pictures? 

You may learn my unique TDS article on JEV right here.