Build Reliable AI Features with Structured Outputs using Schemas

What this is, in plain English

This workflow is about making Artificial Intelligence (AI) outputs more reliable for use in software. Instead of AI giving answers as plain text that a computer program might struggle to understand, we guide the AI to give answers in a specific, organized format. This is done by defining a ‘schema’, which is like a blueprint or a set of rules for the data the AI should produce.

This approach is more advanced because it requires understanding how to define these data structures and how to check if the AI’s output follows them. The exact steps to set this up can vary greatly depending on the specific AI tools and programming languages you use. The core idea, however, is consistent: define the expected structure, ask the AI to generate it, and then verify that the AI’s answer matches the structure.

This method is crucial when AI outputs are not just for humans to read, but need to be processed by other parts of a computer system. Examples include automatically sorting customer support tickets, filling in forms, or making decisions within an application. By using structured outputs, you reduce errors and make your AI-powered features more dependable.

What you can use it for

• Automate customer support routing: Automatically assign incoming support requests to the correct team (e.g., billing, technical) and set a priority level based on the customer’s message.
• Generate structured data for applications: Create data in a predictable format that your application code can easily use to display information or trigger actions.
• Build more reliable AI agents: Ensure that AI agents that perform tasks can consistently receive and provide information in a format that allows them to function correctly.
• Improve data consistency: Make sure that data generated by AI adheres to specific rules, like having certain fields present or values falling within a defined range.
• Enable programmatic decision-making: Allow AI to make decisions that can be directly interpreted and acted upon by software without manual review.

Tools you need

• OpenAI (paid): A company that provides powerful AI models, often accessed through an API (a way for software to talk to the AI).
• Google AI (paid): Similar to OpenAI, Google offers AI models and services that can be integrated into applications.
• Zod (free): A TypeScript-first schema declaration and validation library. It helps define the shape of data and check if data conforms to that shape.
• Pydantic (free): A data validation and settings management library for Python. It uses Python type hints to validate data and is often used with AI models that return structured data.

How it actually works

1. Define the data contract: Decide what information you need from the AI and define its structure. This is often done using a schema definition language like JSON Schema, or with tools like Zod (for TypeScript) or Pydantic (for Python).
2. Design the prompt: Create a prompt for the AI model that clearly asks it to return data in the specified structure. You might include the schema definition directly in the prompt or refer to it.
3. Call the AI model: Use the API of an AI provider like OpenAI or Google AI, sending your prompt and any necessary parameters to get a response.
4. Validate the output: Before using the AI’s response in your application, use your chosen validation tool (Zod or Pydantic) to check if the output matches the defined schema. This step catches errors where the AI might not have followed instructions perfectly.
5. Handle uncertainty: If the validation fails, have a plan for what to do next. This could involve retrying the prompt, asking the AI to correct its output, or flagging the issue for human review.

The exact code and configuration will depend on the specific AI model and programming language you are using. Refer to the documentation of your chosen AI provider (OpenAI, Google AI) and validation library (Zod, Pydantic) for detailed implementation instructions.

Words you’ll see, explained

• Schema: A blueprint or a set of rules that defines the expected structure, types, and constraints of data.
• Validation: The process of checking if data conforms to a defined schema or set of rules.
• JSON: JavaScript Object Notation, a lightweight data-interchange format that is easy for humans to read and write and easy for machines to parse and generate.
• API: Application Programming Interface. A set of rules and protocols that allows different software applications to communicate with each other.
• Agentic workflow: A type of AI system where an AI agent can autonomously plan and execute a series of tasks to achieve a goal.
• Enum: A data type that consists of a set of named values, often used to restrict a field to a specific set of options.

Original source

This guide is based on the insights shared by johnnylemonny on the DEV Community platform. The author explains how to move beyond simply parsing AI-generated text to building more robust AI features by ensuring AI outputs follow a predictable structure.

Notes & variations

• Do you even need this?: If you are only displaying AI output directly to a human user and not processing it with code, you might not need strict schema validation. Plain text or simple JSON might suffice.
• Free-tier limits: While Zod and Pydantic are free, the AI models from OpenAI and Google AI typically require payment for significant usage via their APIs. Always check their current pricing and any available free trial or limited free tiers.
• Common pitfall: A common mistake is assuming that asking an AI to “return JSON” is enough. AI models can still produce syntactically valid JSON that doesn’t match the specific structure or constraints your application needs. Always validate the output against a schema.