generateObject fails occasionally due to invalid JSON

[bluefeet]

What is the name of the modal you’re running?

@hf/nousresearch/hermes-2-pro-mistral-7b

What is the error message?

[cause]: [SyntaxError: Unexpected token ‘`’, "```json { "… is not valid JSON]

What is the issue or error you’re encountering

The model is occasionally producing markdown formatted JSON, but generateObject only works with pure JSON.

What steps have you taken to resolve the issue?

I’ve tried different models.

It fails for about half the requests with the @hf/nousresearch/hermes-2-pro-mistral-7b model.
Every other model I’ve tried fails in a similar fashion.

I’m used to OpenAI’s structured output which guarantees the LLM will conform to your schema.
Is this possible with AI Workers?

What are the steps to reproduce the issue?

Follow the instructions to Generate Structured Objects:

AI SDK · Cloudflare Workers AI docs

Here is the code I am using:

  const workersAi = createWorkersAI({ binding });
  const model = workersAi("@hf/nousresearch/hermes-2-pro-mistral-7b", {});

  const { object: recipe } = await generateObject({
    model: model,
    prompt: "Generate a Lasagna recipe",
    schemaName: "recipe",
    schema: z.object({
      ingredients: z.array(z.string()),
      description: z.string(),
    }),
  });

Here is the full error:

✘ [ERROR] ⨯ Error [AI_NoObjectGeneratedError]: No object generated: could not parse the response.

      at fn (home/aran/projects/bluefeet.dev/.wrangler/tmp/dev-cMXj7u/worker.js:245088:21)
      at async (home/aran/projects/bluefeet.dev/.wrangler/tmp/dev-cMXj7u/worker.js:244524:25)
      at async ay (home/aran/projects/bluefeet.dev/.wrangler/tmp/dev-cMXj7u/worker.js:245811:82) {
    text: '```json\n' +
      '{\n' +
      '  "ingredients": [\n' +
      '    "Lasagna noodles",\n' +
      '    "Ground beef",\n' +
      '    "Tomato sauce",\n' +
      '    "Ricotta cheese",\n' +
      '    "Mozzarella cheese",\n' +
      '    "Parmesan cheese",\n' +
      '    "Eggs",\n' +
      '    "Salt",\n' +
      '    "Black pepper",\n' +
      '    "Fresh basil"\n' +
      '  ],\n' +
      '  "description": "Classic Italian lasagna recipe with layers of pasta, meat sauce, and
  melted cheese."\n' +
      '}\n' +
      '```',
    response: [Object],
    usage: [Object],
    digest: '2238823147',
    [cause]: Error [AI_JSONParseError]: JSON parsing failed: Text: ```json
    {
      "ingredients": [
        "Lasagna noodles",
        "Ground beef",
        "Tomato sauce",
        "Ricotta cheese",
        "Mozzarella cheese",
        "Parmesan cheese",
        "Eggs",
        "Salt",
        "Black pepper",
        "Fresh basil"
      ],
      "description": "Classic Italian lasagna recipe with layers of pasta, meat sauce, and melted
  cheese."
    }
    ```.
    Error message: Unexpected token '`', "```json
    {
    "... is not valid JSON
        at ew (home/aran/projects/bluefeet.dev/.wrangler/tmp/dev-cMXj7u/worker.js:243818:71)
        at fn (home/aran/projects/bluefeet.dev/.wrangler/tmp/dev-cMXj7u/worker.js:245086:22)
        at async (home/aran/projects/bluefeet.dev/.wrangler/tmp/dev-cMXj7u/worker.js:244524:25)
        at async ay (home/aran/projects/bluefeet.dev/.wrangler/tmp/dev-cMXj7u/worker.js:245811:82) {
      text: '```json\n' +
        '{\n' +
        '  "ingredients": [\n' +
        '    "Lasagna noodles",\n' +
        '    "Ground beef",\n' +
        '    "Tomato sauce",\n' +
        '    "Ricotta cheese",\n' +
        '    "Mozzarella cheese",\n' +
        '    "Parmesan cheese",\n' +
        '    "Eggs",\n' +
        '    "Salt",\n' +
        '    "Black pepper",\n' +
        '    "Fresh basil"\n' +
        '  ],\n' +
        '  "description": "Classic Italian lasagna recipe with layers of pasta, meat sauce, and
  melted cheese."\n' +
        '}\n' +
        '```',
      [cause]: [SyntaxError: Unexpected token '`', "```json
      {
      "... is not valid JSON]
    }
  }