Integrating AI APIs into Business Workflows: Calling OpenAI, Claude, and Gemini Programmatically Without a Data Engineering Background

Introduction

You've been using ChatGPT in a browser tab, copying outputs into spreadsheets, and manually pasting customer emails one at a time. It works, but it doesn't scale — and somewhere in the back of your mind, you know there's a better way. The better way is calling these models programmatically: sending data directly to the API, getting structured responses back, and plugging those responses into the tools your business already runs on. No browser tab. No copy-pasting. Just automated intelligence woven into your actual workflow.

Here's the good news: you don't need to be a data engineer or a backend developer to do this. If you can write a Python script (or you're willing to learn a bit of one), you can call OpenAI's GPT models, Anthropic's Claude, and Google's Gemini — and build workflows that would have required an engineering team two years ago. You do need to understand a few moving parts: API keys, HTTP requests, prompt design, response parsing, and basic error handling. This lesson covers all of them, concretely, with real examples you can adapt immediately.

By the end of this lesson, you'll have a working multi-model integration that processes business data — specifically, classifying and summarizing customer support tickets — using all three major AI providers. You'll understand the structural differences between each API, how to make them interchangeable in your code, and how to handle the inevitable hiccups that come with calling external services in production.

What you'll learn:

• How to authenticate with OpenAI, Anthropic, and Google's Gemini APIs using API keys and official Python SDKs
• How each provider structures requests and responses, and where they differ in ways that matter
• How to design prompts programmatically so the same logic works across multiple models
• How to parse structured JSON responses reliably so AI output flows cleanly into downstream systems
• How to handle errors, rate limits, and costs at a level that keeps your workflow stable without a DevOps team

---

Prerequisites

You should be comfortable with:

• Python basics: variables, functions, loops, dictionaries, and importing libraries
• Running Python scripts from a terminal or a notebook environment like Jupyter or Google Colab
• The general concept of what a large language model (LLM) does — generating text in response to a prompt

You don't need to understand: REST APIs at a deep level, async programming, cloud infrastructure, or machine learning theory. We'll cover everything else as we go.

Install the required libraries before starting:

pip install openai anthropic google-generativeai python-dotenv

---

Understanding What an API Call Actually Is

Before we touch any provider-specific code, let's nail down what's happening when you call an AI API, because it demystifies everything that follows.

When you type a message into ChatGPT's web interface and hit enter, the browser is making an HTTP POST request to OpenAI's servers with your message inside it, and the server sends back the model's response as JSON. The API is that same mechanism — but instead of the browser handling it, your Python code does.

Every AI API call, regardless of provider, involves the same three things:

1. Authentication: proving to the server that you're allowed to use it (your API key)
2. The request payload: your prompt, the model you want, and configuration like temperature or max tokens
3. The response payload: the model's output, plus metadata like how many tokens were used (which determines your cost)

The providers differ in their exact JSON structure, their Python SDK conventions, and their pricing — but the conceptual model is identical. Keep that in mind as we move through each one.

---

Setting Up API Keys Safely

API keys are credentials. If someone gets your key, they can run inference on your account and bill it to you. Before you write a single line of API code, set up a proper key management pattern.

The simplest safe approach for practitioners: use a .env file loaded by python-dotenv. This keeps keys out of your code files and out of version control.

Create a file named .env in your project directory:

OPENAI_API_KEY=sk-proj-your-actual-openai-key-here
ANTHROPIC_API_KEY=sk-ant-your-actual-anthropic-key-here
GOOGLE_API_KEY=your-actual-google-api-key-here

Add .env to your .gitignore file immediately:

echo ".env" >> .gitignore

Now load these keys at the top of every script:

import os
from dotenv import load_dotenv

load_dotenv()

OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
ANTHROPIC_API_KEY = os.getenv("ANTHROPIC_API_KEY")
GOOGLE_API_KEY = os.getenv("GOOGLE_API_KEY")

# Sanity check — catch missing keys before you get cryptic errors later
for key_name, key_value in [
    ("OPENAI_API_KEY", OPENAI_API_KEY),
    ("ANTHROPIC_API_KEY", ANTHROPIC_API_KEY),
    ("GOOGLE_API_KEY", GOOGLE_API_KEY),
]:
    if not key_value:
        raise ValueError(f"Missing environment variable: {key_name}")

Where to get the keys: OpenAI keys live at platform.openai.com under API keys. Anthropic keys are at console.anthropic.com. Google Gemini keys come from Google AI Studio at aistudio.google.com. Each provider gives you a limited free tier or trial credits; you won't need a credit card to follow this lesson if you're working with small volumes.

---

Calling OpenAI's API

OpenAI's Python SDK is the most widely documented of the three, which is why most tutorials start here. We will too.

The Basic Call

from openai import OpenAI

client = OpenAI(api_key=OPENAI_API_KEY)

def call_openai(system_prompt: str, user_message: str, model: str = "gpt-4o-mini") -> str:
    """
    Send a message to OpenAI and return the text response.
    
    system_prompt: Sets the model's behavior/role
    user_message: The actual input for this specific call
    model: Which OpenAI model to use
    """
    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": user_message}
        ],
        temperature=0.2,      # Lower = more consistent outputs; good for classification tasks
        max_tokens=500,       # Cap output length to control costs
    )
    return response.choices[0].message.content

Let's test this with a real business scenario: categorizing a customer support ticket.

system_prompt = """
You are a customer support classification assistant. 
Given a support ticket, respond with ONLY a JSON object in this exact format:
{
    "category": "<one of: billing, technical, account, shipping, other>",
    "priority": "<one of: low, medium, high, urgent>",
    "summary": "<one sentence summary of the issue>",
    "sentiment": "<one of: positive, neutral, negative, frustrated>"
}
Do not include any text outside the JSON object.
"""

ticket = """
Subject: Can't access my account - urgent!!
Message: Hi, I've been trying to log in since yesterday and keep getting 
'invalid credentials' even though I reset my password twice. I have a 
presentation tomorrow morning that requires files stored in your platform. 
This is completely unacceptable.
"""

result = call_openai(system_prompt, ticket)
print(result)

Expected output:

{
    "category": "account",
    "priority": "urgent",
    "summary": "Customer cannot log in despite two password resets and has time-sensitive need for stored files.",
    "sentiment": "frustrated"
}

Parsing the Response into a Python Dictionary

The response is a string. To use it in a workflow — routing the ticket to the right team, inserting it into a database, triggering a Slack alert — you need to parse it into a Python dictionary.

import json
import re

def parse_json_response(raw_response: str) -> dict:
    """
    Parse a JSON string from an LLM response, handling common formatting issues.
    LLMs sometimes wrap JSON in markdown code blocks, so we strip those first.
    """
    # Strip markdown code fences if present (```json ... ``` or ``` ... ```)
    cleaned = re.sub(r"```(?:json)?\s*|\s*```", "", raw_response).strip()
    
    try:
        return json.loads(cleaned)
    except json.JSONDecodeError as e:
        raise ValueError(f"Could not parse model response as JSON: {e}\nRaw response: {raw_response}")

# Use it:
parsed = parse_json_response(result)
print(f"Category: {parsed['category']}")
print(f"Priority: {parsed['priority']}")
print(f"Sentiment: {parsed['sentiment']}")

Why temperature=0.2 matters here: When you're extracting structured data, you want the model to behave consistently, not creatively. A temperature of 0 to 0.3 gives you the most deterministic outputs. Save higher temperatures for generative tasks like drafting marketing copy.

Checking Token Usage and Costs

def call_openai_with_usage(system_prompt: str, user_message: str, model: str = "gpt-4o-mini") -> tuple[str, dict]:
    """Returns the response text and a usage dict with token counts."""
    response = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": user_message}
        ],
        temperature=0.2,
        max_tokens=500,
    )
    usage = {
        "prompt_tokens": response.usage.prompt_tokens,
        "completion_tokens": response.usage.completion_tokens,
        "total_tokens": response.usage.total_tokens,
    }
    return response.choices[0].message.content, usage

result, usage = call_openai_with_usage(system_prompt, ticket)
print(f"Tokens used: {usage['total_tokens']} (prompt: {usage['prompt_tokens']}, completion: {usage['completion_tokens']})")

For gpt-4o-mini, as of this writing, that call would cost you a fraction of a cent. Run it 10,000 times per month and you're still talking about a few dollars. Understanding this math before you build a workflow keeps billing surprises from showing up at scale.

---

Calling Anthropic's Claude API

Claude's API has a different structure from OpenAI's, but once you see the pattern, it's straightforward. The most important structural difference: Claude separates the system prompt from the messages list at the top-level API call rather than embedding it as a message with role: system.

The Basic Call

import anthropic

client_anthropic = anthropic.Anthropic(api_key=ANTHROPIC_API_KEY)

def call_claude(system_prompt: str, user_message: str, model: str = "claude-3-5-haiku-20241022") -> str:
    """
    Send a message to Claude and return the text response.
    
    Note: Claude's system prompt is a top-level parameter, not a message role.
    """
    response = client_anthropic.messages.create(
        model=model,
        max_tokens=500,
        system=system_prompt,            # <-- Top-level, not inside messages list
        messages=[
            {"role": "user", "content": user_message}
        ],
        temperature=0.2,
    )
    # Claude returns a list of content blocks; text is in the first block
    return response.content[0].text

Use the exact same ticket and system prompt from the OpenAI section:

result_claude = call_claude(system_prompt, ticket)
parsed_claude = parse_json_response(result_claude)
print(parsed_claude)

Claude is generally very instruction-following with structured output requests — you'll typically get clean JSON without needing to wrestle with the response format.

Understanding Claude's Response Structure

When you print response from Claude's API, the full object looks something like this:

Message(
    id='msg_01XxXx...',
    content=[ContentBlock(text='{\n    "category": "account",\n    ...}', type='text')],
    model='claude-3-5-haiku-20241022',
    role='assistant',
    stop_reason='end_turn',
    stop_sequence=None,
    usage=Usage(input_tokens=187, output_tokens=62)
)

The content attribute is always a list of content blocks (Claude supports multi-modal responses with images and tool use), which is why you index into it with response.content[0].text. For pure text tasks, it's always the first block.

Accessing Usage Data from Claude

def call_claude_with_usage(system_prompt: str, user_message: str, model: str = "claude-3-5-haiku-20241022") -> tuple[str, dict]:
    response = client_anthropic.messages.create(
        model=model,
        max_tokens=500,
        system=system_prompt,
        messages=[{"role": "user", "content": user_message}],
        temperature=0.2,
    )
    usage = {
        "input_tokens": response.usage.input_tokens,
        "output_tokens": response.usage.output_tokens,
    }
    return response.content[0].text, usage

Claude model naming: Anthropic's model names include a date suffix (like claude-3-5-haiku-20241022). This is intentional — it pins you to a specific version so model updates don't silently change your workflow's behavior. When you want to upgrade, you do it deliberately. This is a smart convention that OpenAI has partially adopted with their model snapshot IDs.

---

Calling Google's Gemini API

Google's Gemini SDK works differently from both OpenAI and Anthropic. The library is called google-generativeai, the client setup involves configuring the library globally rather than instantiating a client object, and the conversation structure uses GenerativeModel instances.

The Basic Call

import google.generativeai as genai

genai.configure(api_key=GOOGLE_API_KEY)

def call_gemini(system_prompt: str, user_message: str, model: str = "gemini-1.5-flash") -> str:
    """
    Send a message to Gemini and return the text response.
    
    Gemini handles system instructions via a model-level parameter.
    """
    gemini_model = genai.GenerativeModel(
        model_name=model,
        system_instruction=system_prompt,    # System prompt goes here at model init
        generation_config=genai.types.GenerationConfig(
            temperature=0.2,
            max_output_tokens=500,
        )
    )
    response = gemini_model.generate_content(user_message)
    return response.text

result_gemini = call_gemini(system_prompt, ticket)
parsed_gemini = parse_json_response(result_gemini)
print(parsed_gemini)

A Note on Gemini's JSON Output

Gemini sometimes wraps JSON in markdown code fences even when you explicitly tell it not to — which is exactly why our parse_json_response function strips those fences before parsing. You can also use Gemini's built-in response MIME type control to force clean JSON:

def call_gemini_json(system_prompt: str, user_message: str, model: str = "gemini-1.5-flash") -> str:
    """
    Call Gemini with response_mime_type='application/json' to get cleaner output.
    This is a Gemini-specific feature that enforces JSON formatting at the API level.
    """
    gemini_model = genai.GenerativeModel(
        model_name=model,
        system_instruction=system_prompt,
        generation_config=genai.types.GenerationConfig(
            temperature=0.2,
            max_output_tokens=500,
            response_mime_type="application/json",   # Forces JSON output
        )
    )
    response = gemini_model.generate_content(user_message)
    return response.text

This is genuinely useful — it's the equivalent of OpenAI's response_format={"type": "json_object"} parameter, which you can also add to your OpenAI calls for the same benefit.

---

Building a Provider-Agnostic Wrapper

Now that you understand each provider's quirks, the smart move is to abstract them behind a single function your workflow calls. This way, if you want to switch from OpenAI to Claude for cost reasons (or run the same data through all three for comparison), you change one parameter rather than rewriting your workflow.

from typing import Literal
import time

ProviderType = Literal["openai", "claude", "gemini"]

def call_ai(
    system_prompt: str,
    user_message: str,
    provider: ProviderType = "openai",
    parse_json: bool = True,
) -> dict | str:
    """
    Provider-agnostic AI call. Routes to OpenAI, Claude, or Gemini.
    
    Returns a parsed dict if parse_json=True, otherwise returns raw string.
    """
    if provider == "openai":
        raw = call_openai(system_prompt, user_message)
    elif provider == "claude":
        raw = call_claude(system_prompt, user_message)
    elif provider == "gemini":
        raw = call_gemini(system_prompt, user_message)
    else:
        raise ValueError(f"Unknown provider: {provider}. Choose from: openai, claude, gemini")
    
    if parse_json:
        return parse_json_response(raw)
    return raw

Now your ticket classification becomes:

# Same call, any provider
result = call_ai(system_prompt, ticket, provider="claude")
result = call_ai(system_prompt, ticket, provider="openai")
result = call_ai(system_prompt, ticket, provider="gemini")

---

Adding Error Handling and Retry Logic

A workflow that crashes when an API call fails is not a workflow you can rely on. APIs return errors for legitimate reasons: rate limits when you're sending too many requests, temporary service outages, malformed requests, and expired keys. Your code needs to handle these gracefully.

import time
import logging

logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s")
logger = logging.getLogger(__name__)

def call_ai_with_retry(
    system_prompt: str,
    user_message: str,
    provider: ProviderType = "openai",
    parse_json: bool = True,
    max_retries: int = 3,
    base_delay: float = 1.0,
) -> dict | str | None:
    """
    Wraps call_ai with exponential backoff retry logic.
    
    On rate limit errors (429), waits progressively longer before retrying.
    On other errors, logs and retries up to max_retries times.
    Returns None if all retries fail — caller should handle the None case.
    """
    for attempt in range(max_retries):
        try:
            return call_ai(system_prompt, user_message, provider, parse_json)
        
        except openai.RateLimitError as e:
            wait_time = base_delay * (2 ** attempt)  # 1s, 2s, 4s
            logger.warning(f"OpenAI rate limit hit. Waiting {wait_time}s before retry {attempt + 1}/{max_retries}")
            time.sleep(wait_time)
        
        except anthropic.RateLimitError as e:
            wait_time = base_delay * (2 ** attempt)
            logger.warning(f"Anthropic rate limit hit. Waiting {wait_time}s before retry {attempt + 1}/{max_retries}")
            time.sleep(wait_time)
        
        except Exception as e:
            if attempt == max_retries - 1:
                logger.error(f"All {max_retries} attempts failed for provider {provider}. Last error: {e}")
                return None
            wait_time = base_delay * (2 ** attempt)
            logger.warning(f"Attempt {attempt + 1} failed: {e}. Retrying in {wait_time}s...")
            time.sleep(wait_time)
    
    return None

Why exponential backoff? Rate limits exist to prevent servers from being overwhelmed. If you immediately retry on a rate limit error, you make the problem worse. Waiting 1 second, then 2, then 4 gives the API time to clear the queue — and it's the pattern all major API providers recommend in their documentation.

---

Processing a Batch of Tickets: The Real Workflow

This is where everything comes together. Let's simulate a realistic scenario: you have a CSV of 20 support tickets that came in overnight, and you need to classify each one before your team's morning standup.

import csv
import json
import time
from pathlib import Path

# Simulate loading tickets from a CSV
# In production, this would be: df = pd.read_csv("tickets.csv")
sample_tickets = [
    {
        "ticket_id": "TKT-1001",
        "subject": "Invoice shows wrong amount",
        "message": "My invoice for March shows $450 but I should be on the $299 plan. Please fix this."
    },
    {
        "ticket_id": "TKT-1002",
        "subject": "API returns 503 errors intermittently",
        "message": "Our integration has been getting 503 errors from your API about 15% of the time since Tuesday. Logs attached. This is breaking our production pipeline."
    },
    {
        "ticket_id": "TKT-1003",
        "subject": "How do I export my data?",
        "message": "Hi, I'd like to download all my data before I cancel my subscription. Is there an export feature? No rush, just want to make sure I don't lose anything."
    },
    {
        "ticket_id": "TKT-1004",
        "subject": "Package not delivered",
        "message": "Tracking says delivered 3 days ago but nothing arrived. Checked with neighbors and building management. Order #8847291."
    },
    {
        "ticket_id": "TKT-1005",
        "subject": "Just wanted to say thank you!",
        "message": "Your support team helped me resolve my issue last week and I just wanted to say how impressed I was. The response time was incredible."
    },
]

CLASSIFICATION_SYSTEM_PROMPT = """
You are a customer support triage assistant for a SaaS business.
Analyze the support ticket and respond with ONLY a valid JSON object:
{
    "category": "<billing | technical | account | shipping | feedback | other>",
    "priority": "<low | medium | high | urgent>",
    "summary": "<one concise sentence describing the issue>",
    "sentiment": "<positive | neutral | negative | frustrated>",
    "requires_human": <true | false>,
    "suggested_team": "<billing_team | tech_support | customer_success | logistics | none>"
}
No markdown. No explanation. Only the JSON object.
"""

def process_ticket_batch(tickets: list[dict], provider: ProviderType = "openai") -> list[dict]:
    """
    Classify a list of tickets using the specified AI provider.
    Returns the original ticket data merged with the classification.
    """
    results = []
    
    for i, ticket in enumerate(tickets):
        logger.info(f"Processing ticket {i+1}/{len(tickets)}: {ticket['ticket_id']}")
        
        # Build the user message from ticket fields
        user_message = f"Subject: {ticket['subject']}\n\nMessage: {ticket['message']}"
        
        classification = call_ai_with_retry(
            system_prompt=CLASSIFICATION_SYSTEM_PROMPT,
            user_message=user_message,
            provider=provider,
            parse_json=True,
        )
        
        if classification is None:
            logger.error(f"Failed to classify ticket {ticket['ticket_id']} after all retries")
            classification = {
                "category": "other",
                "priority": "medium",
                "summary": "Classification failed — requires manual review",
                "sentiment": "neutral",
                "requires_human": True,
                "suggested_team": "customer_success",
            }
        
        result = {**ticket, **classification}  # Merge original ticket + classification
        results.append(result)
        
        # Be a good API citizen: small delay between calls to avoid rate limits
        if i < len(tickets) - 1:
            time.sleep(0.5)
    
    return results

# Run the batch
classified_tickets = process_ticket_batch(sample_tickets, provider="openai")

# Display results
for ticket in classified_tickets:
    print(f"\n{'='*60}")
    print(f"Ticket: {ticket['ticket_id']} | Priority: {ticket['priority'].upper()}")
    print(f"Category: {ticket['category']} | Team: {ticket['suggested_team']}")
    print(f"Summary: {ticket['summary']}")
    print(f"Sentiment: {ticket['sentiment']} | Human Required: {ticket['requires_human']}")

Saving Results to CSV

def save_results_to_csv(results: list[dict], output_path: str = "classified_tickets.csv"):
    """Write classified tickets to a CSV file for use in Excel, Sheets, or a database."""
    if not results:
        logger.warning("No results to save.")
        return
    
    fieldnames = list(results[0].keys())
    
    with open(output_path, "w", newline="", encoding="utf-8") as f:
        writer = csv.DictWriter(f, fieldnames=fieldnames)
        writer.writeheader()
        writer.writerows(results)
    
    logger.info(f"Saved {len(results)} classified tickets to {output_path}")

save_results_to_csv(classified_tickets)

Now you have a CSV you can drop into Google Sheets, load into Airtable, or pipe directly into a database insert. The AI processing is completely separated from the data handling — which is exactly the right structure.

---

Comparing Outputs Across Providers

One of the practical advantages of your provider-agnostic wrapper is being able to run the same ticket through multiple models and compare. This is useful when you're evaluating which provider to standardize on, or when you want to build a consensus classification for high-stakes routing.

def compare_providers(ticket: dict) -> dict:
    """
    Run the same ticket through all three providers and return a comparison.
    Useful for evaluating provider consistency or building consensus.
    """
    user_message = f"Subject: {ticket['subject']}\n\nMessage: {ticket['message']}"
    comparison = {"ticket_id": ticket["ticket_id"]}
    
    for provider in ["openai", "claude", "gemini"]:
        logger.info(f"Calling {provider} for ticket {ticket['ticket_id']}")
        result = call_ai_with_retry(
            system_prompt=CLASSIFICATION_SYSTEM_PROMPT,
            user_message=user_message,
            provider=provider,
            parse_json=True,
        )
        comparison[provider] = result
        time.sleep(0.5)
    
    return comparison

# Compare on a single ticket
comparison = compare_providers(sample_tickets[1])  # The 503 error ticket

print(f"\nComparison for {comparison['ticket_id']}:")
for provider in ["openai", "claude", "gemini"]:
    result = comparison[provider]
    if result:
        print(f"\n  [{provider.upper()}]")
        print(f"  Category: {result.get('category')} | Priority: {result.get('priority')}")
        print(f"  Team: {result.get('suggested_team')}")

In practice, the three models will often agree on category and priority for clear-cut tickets, but diverge on edge cases. Those divergences are informative — they tell you which ticket types need more specific prompt guidance, and they reveal where one model may be significantly better calibrated for your use case than others.

---

Hands-On Exercise

Build a meeting notes processor. Here's the scenario: your team records all-hands meetings and generates transcripts, but nobody wants to read 40 minutes of raw transcript. Your job is to build a script that takes a transcript (or section of one) and produces structured output: a summary, a list of action items with owners, key decisions made, and any questions left unresolved.

Step 1: Write a system prompt that instructs the AI to return a JSON object with these fields:

• summary: A 2-3 sentence overview of the meeting
• action_items: A list of objects, each with task, owner, and due_date (or "unspecified" if not mentioned)
• decisions: A list of strings describing decisions made
• open_questions: A list of unresolved questions from the meeting

Step 2: Write a process_transcript(transcript_text, provider) function using your call_ai_with_retry wrapper.

Step 3: Test it with a dummy transcript of at least 10 lines that includes at least 2 action items with owners, one decision, and one unresolved question.

Step 4: Run the same transcript through all three providers and compare the action_items output. Do they extract the same tasks? Do they attribute them to the same owners?

Stretch goal: Add a confidence_score field (0-100) to the JSON schema and see if the models will self-report how confident they are in their extraction. Examine how each model interprets that field — some will be calibrated, others will default to high confidence universally.

---

Common Mistakes & Troubleshooting

"I keep getting JSON parsing errors"

The most common cause: the model returned markdown-wrapped JSON like ```json\n{...}\n```. Your parse_json_response function handles this, but make sure you're using it. If errors persist, print the raw response before parsing to see exactly what the model returned.

A secondary cause is the model occasionally adding a preamble like "Here is the JSON you requested:" before the object. Make your system prompt aggressive: "Respond with ONLY the JSON object. No introduction. No explanation. No markdown." For production-critical workflows, use OpenAI's response_format={"type": "json_object"} or Gemini's response_mime_type="application/json".

"My API key isn't being recognized"

First, verify the .env file is in the same directory you're running your script from. Second, check that load_dotenv() is called before you access the environment variables. Third, confirm the key was copied completely — OpenAI keys start with sk-proj-, Claude keys with sk-ant-. A truncated key is one of the most common silent failures.

"I'm getting rate limit errors on every call"

You're almost certainly using a free tier account with very low rate limits. The fix is either to add longer delays between calls (2-5 seconds instead of 0.5), process tickets in smaller batches, or upgrade your account tier. Check your provider's rate limit documentation — OpenAI expresses limits as "requests per minute" (RPM) and "tokens per minute" (TPM), and both limits can trigger 429 errors independently.

"The model ignores my JSON format instructions sometimes"

This is a prompt engineering problem, not a code problem. Weaker models (especially free tiers or smaller variants) are less reliable instruction-followers. Three fixes: make the schema explicit in the prompt (show the exact keys and allowed values), add "Do not deviate from this format under any circumstances" to your system prompt, and switch to a larger model variant for production if the small one is too unreliable.

"Different runs give me different classifications for the same ticket"

Set temperature=0 for classification tasks if you want deterministic outputs. Note that even at temperature 0, there's a small amount of non-determinism in most LLM APIs due to how they handle floating-point math in parallel processing. For workflows where consistency is critical, log the outputs and periodically audit for drift.

"My costs are higher than expected"

Pull the token usage from each API response (all three providers expose this) and log it. Calculate your average tokens-per-call, multiply by your expected daily volume, and check the provider's current pricing page. Common oversights: your system prompt is much longer than you think (measure it), you're not setting max_tokens to cap completion length, or you're running expensive calls on test data without noticing.

---

Summary & Next Steps

You've built something real here. You have a working multi-provider AI integration that classifies business data, handles errors gracefully, parses structured responses reliably, and keeps API keys secure. Let's pull out the key principles worth carrying forward:

Authentication and key management: Always use environment variables, never hardcode keys, always load them at startup and fail loudly if they're missing.

Provider differences that matter: OpenAI puts the system prompt in the messages array with role: system. Claude puts it as a top-level system parameter. Gemini puts it as system_instruction at model initialization. The response parsing differs too — OpenAI uses response.choices[0].message.content, Claude uses response.content[0].text, Gemini uses response.text.

Structured output is the key to workflow integration: Raw text is useful for humans; JSON is useful for machines. Design your prompts to return JSON whenever you need to pipe AI output into anything else.

Error handling isn't optional: Rate limits, timeouts, and malformed responses will happen. Exponential backoff retry logic turns these from workflow-killers into speed bumps.

Provider abstraction pays off: A thin wrapper that routes to different providers by name means your workflow isn't locked in to any one vendor, and switching costs drop to editing a single function argument.

Next steps to deepen this skill:

• Learn how to use streaming responses for use cases where you want to display output progressively (customer-facing chatbots, real-time dashboards)
• Explore function calling / tool use — all three providers support structured tool definitions that make JSON extraction even more reliable than prompt-based approaches
• Add observability: log every API call with its provider, model, token count, latency, and a hash of the input so you can debug production issues and monitor cost over time
• Look into LangChain or LiteLLM as abstraction layers if your workflow grows to many providers and models — they handle the routing logic you built here at a framework level, with added features for caching, tracing, and model fallbacks
• Study prompt engineering patterns for extraction tasks specifically: chain-of-thought prompting, few-shot examples, and schema validation with Pydantic to enforce output structure before it reaches your downstream systems

The capability you've built in this lesson — taking raw data, running it through an AI model, and getting structured output back — is the foundational pattern for almost every practical AI integration in business. From here, the complexity you add should be in service of your specific use case, not in understanding the basics. You've got those.