Getting Structured Output from Large Language Models

Imagine you're building a data pipeline that needs to extract customer information from unstructured text, or you're creating a tool that generates SQL queries from natural language requests. In both cases, you need more than just conversational responses from an AI model—you need structured, predictable output that your code can reliably parse and use.

By default, large language models (LLMs) like GPT-4, Claude, or open-source models produce free-form text that's great for human reading but challenging for programmatic use. However, with the right techniques, you can consistently get JSON objects, formatted tables, executable code, and other structured formats that integrate seamlessly into your data workflows.

This lesson will transform how you work with LLMs by teaching you to extract reliable, structured data instead of hoping for the best with text parsing. You'll learn to turn AI responses into the exact format your applications need.

What you'll learn:

• How to use prompt engineering to request specific output formats
• Techniques for getting consistent JSON responses from any LLM
• Methods to generate properly formatted tables and code
• How to validate and handle structured output in your applications
• Debugging strategies when structured output fails

Prerequisites

You should be comfortable with basic programming concepts and have used an LLM through either an API or web interface. Familiarity with JSON format is helpful but not required—we'll cover the essentials.

Understanding Structured vs. Unstructured Output

When you ask an LLM "What's the weather like?", you typically get a conversational response: "The weather appears to be sunny with mild temperatures, perfect for outdoor activities." This unstructured text is natural for humans but problematic if you need to extract the temperature value or weather condition for a mobile app.

Structured output, by contrast, follows a predictable format. The same weather query might return:

{
  "condition": "sunny",
  "temperature": 72,
  "temperature_unit": "fahrenheit",
  "suitable_for_outdoor": true
}

This JSON structure allows your code to reliably access specific values without complex text parsing. The key insight is that LLMs can produce structured output—you just need to ask correctly.

Most LLMs are trained on vast amounts of structured data including JSON files, CSV tables, and code repositories. They understand these formats intimately. The challenge is communicating your exact requirements clearly enough that the model consistently produces what you need.

Basic Prompt Engineering for Structure

The foundation of getting structured output is explicit instruction. Instead of hoping the LLM will guess your preferred format, tell it exactly what you want.

Consider this ineffective prompt:

Extract the customer information from this email.

Versus this structured approach:

Extract the customer information from this email and return it as JSON with these exact fields:
- customer_name (string)
- email_address (string)
- phone_number (string, null if not provided)
- inquiry_type (string)
- priority_level (integer from 1-5)

Email: "Hi, this is Sarah Johnson at sarah.j@techcorp.com. My phone is 555-0123. I need urgent help with our API integration - it's been down for 2 hours and affecting our production system."

The second prompt eliminates ambiguity by specifying the output format, field names, data types, and how to handle missing information.

Here's the structured response you'd get:

{
  "customer_name": "Sarah Johnson",
  "email_address": "sarah.j@techcorp.com",
  "phone_number": "555-0123",
  "inquiry_type": "API integration issue",
  "priority_level": 5
}

Tip: Always specify data types in your prompts. "Return the count as an integer" is much clearer than "return the count."

Requesting JSON Output Reliably

JSON is the most versatile structured format because it's human-readable, widely supported, and easily parsed by virtually every programming language. However, getting consistent JSON requires specific techniques.

The JSON Schema Approach

The most reliable method is providing a JSON schema or example structure. Here's how to request product catalog data:

Analyze this product description and return information as JSON matching this exact structure:

{
  "product_name": "string",
  "category": "string",
  "price": number,
  "features": ["array", "of", "strings"],
  "availability": {
    "in_stock": boolean,
    "quantity": number
  }
}

Product description: "The UltraBoost Pro running shoe offers advanced cushioning and lightweight design. Features include responsive foam technology, breathable mesh upper, and continental rubber outsole. Available in sizes 6-13. Price: $180. Currently 23 units in stock."

This approach works because you're showing the LLM exactly what the output should look like, including nested objects and arrays.

Handling Complex Data Structures

For more complex scenarios, you might need nested JSON with multiple object types. Here's an example for processing a sales report:

Convert this sales data into JSON with the following structure:

{
  "report_date": "YYYY-MM-DD",
  "total_sales": number,
  "top_products": [
    {
      "name": "string",
      "units_sold": number,
      "revenue": number
    }
  ],
  "regional_breakdown": {
    "north": number,
    "south": number,
    "east": number,
    "west": number
  }
}

Sales Report March 15, 2024: Total sales reached $45,600. Best sellers were Laptop Model X (25 units, $25,000), Wireless Mouse Pro (150 units, $7,500), and Monitor Ultra (12 units, $9,600). Regional sales: North $12,300, South $8,900, East $15,200, West $9,200.

The response maintains perfect structure:

{
  "report_date": "2024-03-15",
  "total_sales": 45600,
  "top_products": [
    {
      "name": "Laptop Model X",
      "units_sold": 25,
      "revenue": 25000
    },
    {
      "name": "Wireless Mouse Pro", 
      "units_sold": 150,
      "revenue": 7500
    },
    {
      "name": "Monitor Ultra",
      "units_sold": 12,
      "revenue": 9600
    }
  ],
  "regional_breakdown": {
    "north": 12300,
    "south": 8900,
    "east": 15200,
    "west": 9200
  }
}

JSON Validation Instructions

Always include validation instructions to prevent common JSON errors:

Return valid JSON only. Ensure:
- All strings are properly quoted
- Numbers have no quotes
- Use true/false for booleans (not "true"/"false")
- No trailing commas
- Properly escaped quotes within strings

Generating Formatted Tables

Tables are essential for presenting comparative data, reports, and structured information. LLMs can generate tables in various formats including markdown, CSV, and HTML.

Markdown Tables

For documentation or reports, markdown tables offer excellent readability:

Create a comparison table of these database options in markdown format with columns: Database, Type, Best Use Case, Pricing Model, and Scalability Rating (1-5).

Information: PostgreSQL is a relational database, great for complex queries, open source, scales well (4/5). MongoDB is NoSQL, perfect for flexible documents, freemium model, good scaling (4/5). Redis is in-memory, ideal for caching, open source with enterprise options, excellent for specific uses (3/5). Amazon RDS is managed relational, best for AWS ecosystems, pay-per-use, scales automatically (5/5).

Result:

| Database | Type | Best Use Case | Pricing Model | Scalability Rating |
|----------|------|---------------|---------------|-------------------|
| PostgreSQL | Relational | Complex queries, ACID transactions | Open source | 4/5 |
| MongoDB | NoSQL | Flexible document storage | Freemium | 4/5 |
| Redis | In-memory | Caching, session storage | Open source + Enterprise | 3/5 |
| Amazon RDS | Managed Relational | AWS-integrated applications | Pay-per-use | 5/5 |

CSV Format for Data Processing

When you need data for spreadsheets or programmatic processing, CSV format works best:

Convert this employee data to CSV format with headers: Name, Department, Hire_Date, Salary, Status

Employee records: Jennifer Martinez joined Marketing on January 15, 2022, earning $65,000 annually, currently active. Robert Kim started in Engineering on March 3, 2021, makes $95,000 per year, active status. Lisa Wong began in Sales on November 8, 2023, salary of $58,000, currently on leave.

Output:

Name,Department,Hire_Date,Salary,Status
Jennifer Martinez,Marketing,2022-01-15,65000,Active
Robert Kim,Engineering,2021-03-03,95000,Active
Lisa Wong,Sales,2023-11-08,58000,On Leave

Warning: Be specific about date formats (YYYY-MM-DD vs MM/DD/YYYY) and number formatting (with or without commas) to ensure consistency.

Extracting Code from LLMs

Getting executable code from LLMs requires careful prompting to ensure the output follows proper syntax and includes necessary imports or setup.

Requesting Specific Code Formats

When asking for code, specify the programming language, any required libraries, and the expected input/output:

Write a Python function that takes a list of dictionaries representing customer orders and returns the total revenue. Include proper error handling and type hints. Use this structure:

Input: [{"order_id": "A123", "items": [{"name": "Widget", "price": 15.99, "quantity": 2}]}]
Output: Float representing total revenue

Include docstring and handle edge cases.

This produces clean, production-ready code:

from typing import List, Dict, Any

def calculate_total_revenue(orders: List[Dict[str, Any]]) -> float:
    """
    Calculate total revenue from a list of customer orders.
    
    Args:
        orders: List of dictionaries containing order information
        
    Returns:
        Total revenue as a float
        
    Raises:
        TypeError: If input is not a list or contains invalid data types
        KeyError: If required keys are missing from order dictionaries
    """
    if not isinstance(orders, list):
        raise TypeError("Orders must be a list")
    
    total_revenue = 0.0
    
    for order in orders:
        if not isinstance(order, dict) or "items" not in order:
            raise KeyError("Each order must be a dictionary with 'items' key")
            
        items = order["items"]
        if not isinstance(items, list):
            raise TypeError("Items must be a list")
            
        for item in items:
            try:
                price = float(item["price"])
                quantity = int(item["quantity"])
                total_revenue += price * quantity
            except (KeyError, ValueError, TypeError) as e:
                raise ValueError(f"Invalid item data: {e}")
    
    return total_revenue

SQL Query Generation

For database queries, specify the database system and schema information:

Generate a SQL query for PostgreSQL that finds customers who placed orders in the last 30 days but haven't placed any orders in the last 7 days. Include customer name and last order date.

Tables:
- customers (id, name, email, created_at)
- orders (id, customer_id, order_date, total_amount, status)

Return the query with proper formatting and comments.

Result:

-- Find customers with orders in last 30 days but not in last 7 days
SELECT DISTINCT 
    c.name,
    MAX(o.order_date) AS last_order_date
FROM customers c
INNER JOIN orders o ON c.id = o.customer_id
WHERE 
    -- Orders within last 30 days
    o.order_date >= CURRENT_DATE - INTERVAL '30 days'
    -- But not within last 7 days
    AND o.order_date < CURRENT_DATE - INTERVAL '7 days'
    -- Only completed orders
    AND o.status = 'completed'
GROUP BY c.id, c.name
ORDER BY last_order_date DESC;

Hands-On Exercise

Let's build a complete system that processes customer feedback and returns structured analysis. This exercise combines JSON output, table generation, and practical application.

Scenario: You're analyzing customer reviews for an e-commerce platform and need structured data for reporting and further processing.

Step 1: Design the Output Structure

First, define what structured output you need:

{
  "review_summary": {
    "total_reviews": number,
    "average_rating": number,
    "sentiment_distribution": {
      "positive": number,
      "neutral": number, 
      "negative": number
    }
  },
  "key_themes": [
    {
      "theme": "string",
      "frequency": number,
      "impact_on_rating": "positive|negative|neutral"
    }
  ],
  "actionable_insights": ["array of strings"],
  "sample_reviews_by_sentiment": {
    "positive": "string",
    "negative": "string"
  }
}

Step 2: Create the Analysis Prompt

Analyze these customer reviews and return structured insights as JSON matching this exact format:

{
  "review_summary": {
    "total_reviews": number,
    "average_rating": number,
    "sentiment_distribution": {
      "positive": number,
      "neutral": number,
      "negative": number
    }
  },
  "key_themes": [
    {
      "theme": "string",
      "frequency": number,
      "impact_on_rating": "positive|negative|neutral"
    }
  ],
  "actionable_insights": ["array of strings"],
  "sample_reviews_by_sentiment": {
    "positive": "string",
    "negative": "string"
  }
}

Customer Reviews:
1. "Great product! Fast delivery and excellent customer service. 5/5 stars." 
2. "Good quality but took 2 weeks to arrive. Product works as expected. 4/5 stars."
3. "Terrible experience. Product broke after 3 days and customer service was unhelpful. 1/5 stars."
4. "Love the features but the price is quite high. Worth it though. 4/5 stars."
5. "Perfect! Exactly what I needed. Quick shipping too. 5/5 stars."

Requirements:
- Count sentiment as positive (4-5 stars), neutral (3 stars), negative (1-2 stars)
- Identify themes mentioned multiple times
- Provide actionable business insights
- Include one sample review for positive and negative sentiment

Step 3: Process the Response

The LLM should return something like:

{
  "review_summary": {
    "total_reviews": 5,
    "average_rating": 3.8,
    "sentiment_distribution": {
      "positive": 4,
      "neutral": 0,
      "negative": 1
    }
  },
  "key_themes": [
    {
      "theme": "delivery_speed",
      "frequency": 4,
      "impact_on_rating": "positive"
    },
    {
      "theme": "customer_service",
      "frequency": 2,
      "impact_on_rating": "negative"
    },
    {
      "theme": "product_quality",
      "frequency": 3,
      "impact_on_rating": "positive"
    },
    {
      "theme": "pricing",
      "frequency": 1,
      "impact_on_rating": "negative"
    }
  ],
  "actionable_insights": [
    "Maintain fast delivery standards as it significantly impacts customer satisfaction",
    "Improve customer service training to handle product issues more effectively",
    "Consider pricing strategy review as cost concerns appear in feedback",
    "Implement better quality control to prevent early product failures"
  ],
  "sample_reviews_by_sentiment": {
    "positive": "Great product! Fast delivery and excellent customer service. 5/5 stars.",
    "negative": "Terrible experience. Product broke after 3 days and customer service was unhelpful. 1/5 stars."
  }
}

Step 4: Create a Summary Table

Now generate a summary table from this data:

Using the JSON analysis above, create a markdown table showing the key themes with columns: Theme, Frequency, Impact, and Recommendation Priority (High/Medium/Low based on frequency and negative impact).

This produces:

Theme	Frequency	Impact	Recommendation Priority
Delivery Speed	4	Positive	Medium
Customer Service	2	Negative	High
Product Quality	3	Positive	Medium
Pricing	1	Negative	Low

Common Mistakes & Troubleshooting

Invalid JSON Output

Problem: The LLM returns malformed JSON with syntax errors.

Solution: Add explicit validation requirements to your prompt:

Return ONLY valid JSON. Before responding:
1. Ensure all strings use double quotes, not single quotes
2. Check that all brackets and braces are properly closed
3. Remove any trailing commas
4. Verify numbers are not quoted
5. Use lowercase true/false for booleans

Inconsistent Field Names

Problem: The same data appears with different field names across requests (e.g., "user_name" vs "username" vs "name").

Solution: Provide explicit field naming in your schema:

Use these EXACT field names (case-sensitive):
- customer_name (not name, user_name, or customerName)
- email_address (not email or user_email)
- phone_number (not phone or phoneNumber)

Missing or Null Values

Problem: The LLM omits fields when data is unavailable instead of using null values.

Solution: Specify how to handle missing data:

For missing information:
- Use null for unavailable values
- Use empty string "" for text fields when no data exists
- Use 0 for counts when no items found
- Always include ALL fields in the response, even if empty

Overly Complex Nested Structures

Problem: Requesting deeply nested JSON that becomes unreliable.

Solution: Break complex structures into simpler, flatter formats:

Instead of:

{
  "data": {
    "analysis": {
      "sentiment": {
        "scores": {
          "positive": 0.8
        }
      }
    }
  }
}

Use:

{
  "sentiment_positive_score": 0.8,
  "sentiment_negative_score": 0.1,
  "sentiment_neutral_score": 0.1
}

Code Output Without Context

Problem: Generated code lacks imports, setup, or usage examples.

Solution: Request complete, runnable examples:

Provide a complete Python script including:
- All necessary imports
- Sample data for testing
- The main function
- Example usage with expected output
- Error handling for common issues

Pro Tip: Test your structured output prompts with edge cases like empty inputs, unusual characters, or missing data to ensure robustness.

Summary & Next Steps

You've learned to transform unpredictable LLM responses into reliable, structured data that integrates seamlessly into your applications. The key techniques include explicit format specification, JSON schema provision, and clear validation requirements.

The structured output approaches you've mastered—JSON objects, formatted tables, and executable code—form the foundation for building robust AI-powered applications. Instead of parsing free-form text and hoping for consistency, you can now extract exactly the data format your systems require.

Key takeaways:

• Always specify exact output formats and field names in your prompts
• Provide examples or schemas to eliminate ambiguity
• Include validation instructions to prevent common formatting errors
• Handle missing data explicitly with null values or empty strings
• Test edge cases to ensure robustness in production scenarios

Next steps to advance your LLM skills:

1. Function Calling: Learn how modern LLMs can call predefined functions with structured parameters
2. Output Validation: Implement automated validation and retry logic for malformed responses
3. Streaming Structured Data: Handle real-time structured output for interactive applications
4. Custom Output Parsers: Build reusable parsing utilities for your specific data formats
5. Multi-Model Orchestration: Combine multiple LLMs for complex structured data workflows

Start applying these techniques to your current projects. Whether you're building dashboards, data pipelines, or automated reports, structured output will make your AI integrations more reliable and maintainable.