Migrate Chat Completions to the Responses API

Migrating from Chat Completions to the Responses API is mostly a rename-and-reshape job: POST /v1/chat/completions becomes POST /v1/responses, the messages array becomes input, the system prompt becomes instructions, and you read response.output_text instead of completion.choices[0].message.content. The parts that trip people up are the ones that change behavior rather than names — output items, statefulness, tool linkage, and structured-output fields. This guide walks the migration in the order you should actually do it, with a rollback plan so you can move one endpoint at a time instead of rewriting everything at once.

You do not have to migrate. Chat Completions remains supported. Migrate when you want reasoning continuity, server-managed conversation state, native tools, or the cleaner typed output that agent loops benefit from. If you are still deciding, read Responses API vs Chat Completions first.

Step 0: Establish a baseline you can diff against

Before touching code, capture what "correct" looks like on your current endpoint. Send a few representative prompts through Chat Completions and record the response text, token usage, and latency. Route them through your gateway so the traces land in its usage logs. You will re-run the same prompts against Responses and compare. Migrations that skip this step turn into guesswork the moment output differs.

Step 1: Rename the endpoint and move the system prompt

The smallest working migration is three edits. Point the SDK at the Responses method, move any system message into instructions, and collapse a single user turn into input.

Before:

completion = client.chat.completions.create(
    model="gpt-5.6",
    messages=[
        {"role": "system", "content": "You are a release-notes summarizer."},
        {"role": "user", "content": "Summarize the changelog below:\n" + changelog},
    ],
)
text = completion.choices[0].message.content

After:

response = client.responses.create(
    model="gpt-5.6",
    instructions="You are a release-notes summarizer.",
    input="Summarize the changelog below:\n" + changelog,
)
text = response.output_text

input accepts either a plain string (as above) or an array of typed items when you need multi-turn history or tool results. Start with the string form to get a green test, then reshape only where you need structure.

Step 2: Parse output items, not choices[0]

This is the step that changes behavior. Chat Completions gave you choices, and index 0 was almost always your answer. Responses returns an output array of typed items — message, reasoning, and function_call — and the answer is not guaranteed to be first. A reasoning model can emit a reasoning item ahead of the message, and a tool turn can emit a function_call with no text at all. (The function_call_output you send a tool result back with is an input item you construct, not one of these returned items — keep the two straight.)

The convenience accessor response.output_text handles the common "just give me the text" case. When you need more control, iterate by type:

answer, tool_calls = None, []
for item in response.output:
    if item.type == "message":
        answer = item.content[0].text
    elif item.type == "function_call":
        tool_calls.append(item)

Update your usage accounting at the same time: prompt_tokens/completion_tokens become input_tokens/output_tokens on the usage object. Any billing or dashboard code that reads token counts must handle both field names during a phased rollout.

Step 3: Decide how you manage conversation state

Chat Completions is stateless — you resend the full history every call. Responses are stored by default and return an id you can chain from. Pick one of three strategies and apply it consistently:

Strategy	How	Use when
previous_response_id	Send the prior response id on the next call	You want the server to hold history
Manual items	Pass prior output items back into input	You want full control of what is resent
Stateless	store: false and resend history yourself	Zero-retention or you already manage a store

Two facts to plan around. First, instructions are not carried across previous_response_id, so resend them every turn. Second, chaining does not lower your bill: all previous input tokens in the chain are billed as input tokens. Chaining saves bandwidth and boilerplate, not money. For real context-cost reduction, combine migration with prompt caching.

Step 4: Port tool calls and structured output

Function calling survives the migration, but the wiring changes. In Chat Completions you appended a role: "tool" message with a tool_call_id. In Responses you send a function_call_output item linked by call_id:

client.responses.create(
    model="gpt-5.6",
    previous_response_id=response.id,
    input=[{
        "type": "function_call_output",
        "call_id": call.call_id,
        "output": tool_result,
    }],
)

Sending the result without the matching call_id is one of the most common migration errors, so assert on it in tests. Structured output moves too: response_format becomes text.format, and you set strict: true explicitly on the schema — verify your JSON Schema still validates against the strict subset. (Do not carry the omit-strict degrade-to-non-strict behavior over from tool definitions; for structured outputs, be explicit.) Details are in structured outputs with JSON Schema, and portable definitions in portable tool calling across providers.

Step 5: Update streaming consumers

If you stream, the transport stays Server-Sent Events but the events change. Chat Completions streamed chat.completion.chunk objects with choices[].delta and ended with data: [DONE]. Responses streams semantic events such as response.created, response.output_text.delta, and response.completed. The semantic events are easier to route because each names itself, but your existing [DONE]-and-delta parser will not understand them without changes. The parsing patterns for both are in streaming LLM responses with SSE.

Step 6: Roll out behind a flag, keep a rollback

Do not flip every call at once. Put the endpoint choice behind a feature flag or route it per model at your gateway, register the upstreams behind it, and send a small slice of traffic to Responses. Compare the captured baseline from Step 0 against the new traces: same answers within tolerance, sane token counts, acceptable latency. If something regresses, the rollback is flipping the flag back to Chat Completions — not a redeploy — because the client difference is contained. When the slice looks healthy, widen it.

What the migration does not change

It helps to know where you can relax. Several things carry over unchanged, so you should not rewrite them:

• Authentication. The same API key and Authorization: Bearer header work; only the path changes.
• Model identifiers. A model like gpt-5.6 is named the same on both endpoints — you are changing the request envelope, not the model.
• Prompt content. Your actual instructions and user text are identical; they just move from messages entries into instructions and input.
• Temperature and sampling knobs. Common generation parameters keep their names and meanings.
• The transport for streaming. Both endpoints stream over Server-Sent Events, so your SSE reader stays; only the event shapes it parses differ.

Knowing this keeps the migration scoped. The behavioral surface that genuinely changes is small: output structure, state management, the tool-result field, and structured-output configuration. Everything else is a mechanical move. When you estimate the work, budget for those four areas and treat the rest as copy-paste, which is why a per-endpoint rollout is realistic rather than a big-bang rewrite.

Reading the common migration errors

Most migration bugs produce a handful of recognizable symptoms. Empty or missing text usually means you read output[0] and landed on a reasoning or function_call item instead of a message — iterate by type or use output_text. A schema-related 400 on structured output usually means you left response_format in place instead of switching to text.format. A tool loop that stalls or errors usually means a function_call_output went out without the matching call_id. And a conversation that "forgets" its role or tone across turns usually means you relied on instructions persisting through previous_response_id — it does not, so resend them. Keeping this short list next to your diff turns confusing failures into quick lookups.

Migration checklist

• Endpoint changed from /v1/chat/completions to /v1/responses.
• system message moved to instructions and resent on every chained turn.
• Output read via output_text or by iterating output items by type.
• Token accounting handles input_tokens/output_tokens.
• State strategy chosen: previous_response_id, manual items, or store: false.
• Tool results sent as function_call_output with a matching call_id.
• response_format replaced with text.format; schema re-validated.
• Streaming consumers updated for semantic events.
• Rollout is flag-gated with a one-switch rollback.

FAQ

Do I have to migrate before any deadline?

Chat Completions is not scheduled for sunset, so there is no forced deadline for it. The endpoint OpenAI has marked for deprecation is the Assistants API (deprecated 2025-08-26, with sunset announced for 2026-08-26), which is a different surface. Migrate to Responses for its capabilities, not out of urgency.

My assistant sometimes returns empty text after migrating. Why?

You are almost certainly reading output[0] and hitting a reasoning or function_call item instead of a message. Iterate output by type, or use response.output_text, which skips non-text items for you.

Can I keep resending full history instead of using previous_response_id?

Yes. Set store: false and pass your history as input items exactly like you did with messages. This keeps the endpoint stateless and is the right choice for zero-retention requirements.

Verify on one endpoint first

Take one non-critical endpoint, apply Steps 1–2 only (rename plus output parsing), and run your Step 0 baseline prompts through it. If the text matches and token counts look right in your gateway's usage logs, you have proven the migration mechanics on your own traffic before touching state, tools, or streaming.

References

• OpenAI: Migrate to the Responses API
• OpenAI Responses API reference
• OpenAI Chat Completions API reference