LLM structured output: preserve response_format through fallback

Symptoms

A primary adapter rejects or cannot implement a structured-output request, and a fallback returns plain text because the format argument disappeared. In the reviewed revision, the common factory forwards response_format; unsupported adapters reject it before their SDK is called. The observed application error text is provider does not support response_format, not a vendor error message. The dependency lock pins openai==2.46.0.

Cause

A common method signature is not proof that each adapter implements every parameter. The reviewed regression asserts that an unsupported Anthropic adapter makes no SDK call and that the OpenAI fallback receives the original format object. This concerns those application wrappers; it does not establish that Anthropic or Gemini APIs generally lack structured-output features. The official OpenAI guide distinguishes JSON mode from schema-constrained output.

Safe resolution

The reduced chain skips a capability that is explicitly unsupported and never removes the output contract. The injected send function represents an adapter that returns a parsed application object, not a raw SDK completion. Include a JSON instruction in messages when using OpenAI JSON mode.

def structured_fallback(providers, messages, response_format):
    if type(response_format) is not dict:
        raise ValueError("response_format must be an object")
    for supports_format, send in providers:
        if not supports_format:
            continue
        try:
            result = send(
                messages=messages,
                response_format=response_format,
            )
        except TimeoutError:
            continue
        if not isinstance(result, dict):
            raise ValueError("Expected parsed structured output")
        return result
    raise RuntimeError("No provider preserved response_format")

Verification

Synthetic Python examples were checked on 2026-10-16; the original application tests and live provider APIs were not run.

Pass a single format object such as {‘type’: ‘json_object’} and assert the fallback transport receives that exact object. Assert zero calls for an unsupported adapter, preserve the contract when a supported primary times out, and reject a text result rather than accepting silent degradation. If all adapters are unsupported, fail without making a provider call. Invalid format types must fail before any transport.

The checked-in tests extract this published Python block and exercise the same synthetic boundary cases. Re-run with npm test.

Limitations

A json_object fixture proves argument preservation and a parsed-object boundary; it does not prove strict JSON Schema adherence, every required business field, or live model compliance. A strict-schema requirement needs an explicitly supported adapter plus local validation; refusals and truncated responses remain separate outcomes. The example does not translate between different vendors’ schema dialects or migrate Chat Completions response_format to Responses text.format. See the LLM selection guide and Anthropic profile.

Implementation evidence was reviewed at pinned GetBeta revision 890c2531b76c. Source-repository links may require access; the public code above is a reduced adaptation without credentials or customer data. Search expressions are candidates, with no search-volume, indexing or ranking claim.