Structured Outputs

Introduction

Structured Outputs guarantee that the model consistently returns results matching a defined JSON Schema. This removes concerns about missing required fields or generating invalid enum values.

Key advantages include:

Type Safe: Responses are reliably formatted, eliminating the need for manual validation or retries
Simplified prompts: Lightens the load on prompt engineering

For structured outputs, we currently support the OpenAI client beta API as shown below.

Note: Latest guidelines recommend moving towards the Responses API for more advanced features. We currently do not support this, but will support it shortly.

Python


from openai import OpenAI
from pydantic import BaseModel
 
client = OpenAI(
    base_url="https://api.ncompass.tech/v1",
    api_key="YOUR_API_KEY",
)
 
class CalendarEvent(BaseModel):
    name: str
    date: str
    participants: list[str]
 
response = client.beta.chat.completions.parse(
    model="meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8",
    messages=[
        {"role": "system", "content": "Extract the event information."},
        {
            "role": "user",
            "content": "Alice and Bob are going to a science fair on Friday.",
        },
    ],
    response_format=CalendarEvent,
)
 
print(response)