No Object Generated: Response Did Not Match Schema

Packing list. Visa notes. Insurance. Data plan. Destination picker. Discover hotels. Trip context. Itinerary judge. Regenerate day.

Nine features, one afternoon, all of them dead, all with the same string:

No object generated: response did not match schema

We’d just dropped the model picker from the UI and standardized on GLM-4.7 through Baseten. Cheaper, fast, good enough to orchestrate a booking conversation, and one less thing in the settings screen that nobody was touching.

Nine failures reading identically looks like nine bad schemas, so that’s where we went. we opened the Zod schemas for the broken ones looking for a shared shape, an .optional() that should’ve been .nullable(), a discriminated union the model wouldn’t fill correctly.

The schemas were fine. Every one of them. What the nine had in common wasn’t their shape at all, it’s that they all called generateObject.

What that function actually does

generateObject doesn’t ask a model for JSON and check the answer. It uses the provider’s strict json_schema mode, where the provider constrains decoding token by token so the output physically cannot violate the schema. That’s an OpenAI API feature. It’s a capability of the endpoint, not a feature of the SDK.

GLM through Baseten accepts the request, ignores the contract, and generates normally. So you get something JSON-shaped, usually correct, often wrapped in a markdown fence, occasionally carrying a field nobody asked for. The SDK validates it and throws.

The error message is true. It just points at your schema, when the actual problem is that the thing you delegated enforcement to never agreed to enforce anything. There’s no capability negotiation in that call and no warning when the provider silently isn’t doing the job. It works, until you change providers, and then nine things break at once and all of them blame your schema.

The twenty lines

export async function generateStructured<T>({ model, schema, prompt }) {
  const attempt = async (extra?: string) => {
    const { text } = await generateText({
      model,
      prompt: extra
        ? `${prompt}\n\nYour last reply was invalid: ${extra}`
        : prompt,
    })
    const cleaned = text
      .replace(/^```(?:JSON)?/m, '')
      .replace(/```$/m, '')
      .trim()
    return schema.parse(JSON.parse(cleaned))
  }

  try {
    return await attempt()
  } catch (err) {
    return await attempt(err instanceof Error ? err.message : String(err))
  }
}

generate text, strip the fence, parse it yourself, validate with Zod, and if that throws, go again with the validation error pasted into the prompt.

the retry does more than it looks like it does. handing the model its own error message is a repair loop, and it catches most near misses, a missing field or a number sent as a string or an enum value that’s almost right. one retry was enough. we never needed a second one, and i’d be a little suspicious of a design that did.

the rule now is generateStructured for Baseten and GLM and never generateObject. And .nullable() over .optional() everywhere, because open models handle an explicit null far better than they handle a key that just isn’t there. An absent key looks like the model forgot. A null looks like the model decided.

The schema was pulling the model into lying

Related, and it took longer to see because nothing errored.

Our hotel search tool has checkIn and checkOut as required fields, which is correct, since you can’t price a room without dates. But a required field in a tool schema is a kind of gravity. The model wants to call the tool, the tool won’t accept a call without dates, and the user hasn’t given any. So it picks tomorrow and searches.

No error. No failure. Just a confident set of results for dates the user never asked about, and a conversation that’s now quietly about the wrong week.

The fix isn’t in the schema, it’s in the prompt, and it has to be written as an override rather than a suggestion, because it’s arguing against the shape of the tool. The rule we landed on says if the user gave no date hint, use askField('dates'), and inventing tomorrow just to satisfy the schema is forbidden.

That phrasing is deliberate. “don’t invent dates” wasn’t enough on its own. Naming the reason the model wants to invent them is what made it stick.

Every chat was called Travel Planning

Months later, chat title generation was still on generateObject, against a Groq model that had since been decommissioned. It also wrapped the whole call in a try/catch that returned a static string on any error.

So the model went away, every call failed, the catch ate it, and every chat in the product got titled “Travel Planning”. No error rate moved. Nothing alerted. A generic title isn’t wrong the way a stack trace is wrong, it’s just unhelpful, and unhelpful doesn’t page anyone.

i don’t know how long that ran before someone noticed. Long enough that the fix was two changes rather than one: Move it to a supported model, and stop swallowing the error. The same shape shows up in the day we turned simulation off, where a silent fallback had been hiding two real checkout bugs for months.

The reverts

Worth saying that the model choice was never one decision. It was a sequence of corrections and two of them were reverts.

We moved summarization to a cheaper GPT tier and reverted it when quality dropped. We standardized on GLM everywhere, which caused everything above. Then we tried promoting Kimi-K2.6, the small model doing worker tasks, to be the main model as well, and reverted that one too.

Where it settled is GLM-4.7 doing the thinking and the talking, with Kimi-K2.6 running tool calls in parallel underneath it. A bigger model for the synthesis step the user actually reads, a cheap fast one for the fan out where nobody sees the prose.

Which sounds obvious written down. It cost two reverts to find, and the reason it cost two reverts is that “one model everywhere” is genuinely the simpler system right up until you measure it. None of this was survivable without an eval suite underneath catching the damage, because swapping the model under an agent and reading a few replies to see if they look fine is not testing.