Comparing LLM APIs: OpenAI vs Anthropic vs Gemini vs Open Source

The Story That Sets the Stage

The modern AI landscape offers developers a rich but confusing set of LLM API choices. OpenAI, Anthropic, Google Gemini, and open-source models (via Hugging Face, Ollama, Together AI, etc.) each have distinct strengths, pricing models, safety postures, context windows, and API designs. Understanding these differences is not just academic — it directly impacts cost, latency, reliability, and the quality of what you build.

The Four Players — Quick Landscape Map

API Design — Hello World Across All Four

The best way to feel an API's design philosophy is to run the same task through each one. Below is a basic chat completion — "Explain transformers in one sentence" — implemented in each SDK.

OpenAI — GPT-4o

from openai import OpenAI

client = OpenAI(api_key="sk-...")

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You are a concise data science tutor."},
        {"role": "user",   "content": "Explain transformers in one sentence."}
    ],
    max_tokens=100,
    temperature=0.7
)

print(response.choices[0].message.content)
# Usage stats
print(f"Tokens used: {response.usage.total_tokens}")

Anthropic — Claude

import anthropic

client = anthropic.Anthropic(api_key="sk-ant-...")

message = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=100,
    system="You are a concise data science tutor.",  # system is TOP-LEVEL, not in messages
    messages=[
        {"role": "user", "content": "Explain transformers in one sentence."}
    ]
)

print(message.content[0].text)
# Usage stats
print(f"Input tokens: {message.usage.input_tokens}")
print(f"Output tokens: {message.usage.output_tokens}")

Google Gemini

import google.generativeai as genai

genai.configure(api_key="AIza...")

model = genai.GenerativeModel(
    model_name="gemini-2.0-flash",
    system_instruction="You are a concise data science tutor."
)

response = model.generate_content("Explain transformers in one sentence.")

print(response.text)
# Token usage
print(f"Total tokens: {response.usage_metadata.total_token_count}")

Open Source via Together AI (Llama 3.3 70B — OpenAI-compatible)

from openai import OpenAI

# Together AI, Groq, Anyscale etc. all accept the OpenAI client
client = OpenAI(
    api_key="together-...",
    base_url="https://api.together.xyz/v1"   # only change needed
)

response = client.chat.completions.create(
    model="meta-llama/Llama-3.3-70B-Instruct-Turbo",
    messages=[
        {"role": "system", "content": "You are a concise data science tutor."},
        {"role": "user",   "content": "Explain transformers in one sentence."}
    ],
    max_tokens=100
)

print(response.choices[0].message.content)

Pricing — The Real Cost of Intelligence

Python: A Token Cost Estimator

import tiktoken

# Pricing per 1M tokens (input, output)
PRICING = {
    "gpt-4o":             (2.50,  10.00),
    "gpt-4o-mini":       (0.15,   0.60),
    "claude-sonnet-4-5": (3.00,  15.00),
    "claude-haiku-3-5":  (0.80,   4.00),
    "gemini-2.0-flash":  (0.10,   0.40),
    "llama-3.3-70b":     (0.88,   0.88),
}

def estimate_cost(prompt: str, expected_output_tokens: int, model: str) -> dict:
    enc = tiktoken.get_encoding("cl100k_base")
    input_tokens = len(enc.encode(prompt))
    price_in, price_out = PRICING[model]
    cost_in  = (input_tokens          / 1_000_000) * price_in
    cost_out = (expected_output_tokens / 1_000_000) * price_out
    return {
        "model":        model,
        "input_tokens": input_tokens,
        "cost_input":   round(cost_in,  6),
        "cost_output":  round(cost_out, 6),
        "total_usd":    round(cost_in + cost_out, 6),
    }

prompt = "Summarise the following 2000-word research paper in 5 bullet points: ..."

for model in PRICING:
    result = estimate_cost(prompt, expected_output_tokens=150, model=model)
    print(f"{result['model']:25s}  ${result['total_usd']:.6f}")

Context Windows — How Much Can It Remember?

The context window is how many tokens the model can "see" in one call — your prompt, chat history, retrieved documents, and the model's own response combined. A small context window forces chunking; a large one allows whole-book analysis in a single call.

Streaming — Making Your App Feel Instant

Without streaming, users stare at a loading spinner until the entire response is generated. With streaming, tokens appear word-by-word — like watching someone type. This is a UX requirement for any chat interface. All four providers support streaming.

OpenAI Streaming

from openai import OpenAI

client = OpenAI(api_key="sk-...")

with client.chat.completions.stream(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Write 3 fun facts about Python."}]
) as stream:
    for text in stream.text_stream():
        print(text, end="", flush=True)

Anthropic Streaming

import anthropic

client = anthropic.Anthropic(api_key="sk-ant-...")

with client.messages.stream(
    model="claude-sonnet-4-5",
    max_tokens=300,
    messages=[{"role": "user", "content": "Write 3 fun facts about Python."}]
) as stream:
    for text in stream.text_stream():
        print(text, end="", flush=True)

Gemini Streaming

import google.generativeai as genai

genai.configure(api_key="AIza...")
model = genai.GenerativeModel("gemini-2.0-flash")

for chunk in model.generate_content(
    "Write 3 fun facts about Python.",
    stream=True
):
    print(chunk.text, end="", flush=True)

Function Calling & Tool Use — Making LLMs Act

Function calling (OpenAI/Gemini) or tool use (Anthropic) lets the model decide when to call an external function and what arguments to pass. This is the backbone of AI agents — allowing models to search the web, query databases, call APIs, or run code.

OpenAI Function Calling

from openai import OpenAI
import json

client = OpenAI(api_key="sk-...")

tools = [{
    "type": "function",
    "function": {
        "name": "get_stock_price",
        "description": "Get the current stock price for a given ticker symbol.",
        "parameters": {
            "type": "object",
            "properties": {
                "ticker": {"type": "string", "description": "Stock ticker e.g. AAPL"}
            },
            "required": ["ticker"]
        }
    }
}]

messages = [{"role": "user", "content": "What is the current price of Apple stock?"}]
response = client.chat.completions.create(model="gpt-4o", tools=tools, messages=messages)

tool_call = response.choices[0].message.tool_calls[0]
args = json.loads(tool_call.function.arguments)
print(f"Model wants to call: {tool_call.function.name}({args})")
# → Model wants to call: get_stock_price({'ticker': 'AAPL'})

# Execute the real function and return result back to model
def get_stock_price(ticker): return {"ticker": ticker, "price": 213.49, "currency": "USD"}

messages.append(response.choices[0].message)
messages.append({
    "role": "tool",
    "tool_call_id": tool_call.id,
    "content": json.dumps(get_stock_price(**args))
})

final = client.chat.completions.create(model="gpt-4o", messages=messages)
print(final.choices[0].message.content)

Anthropic Tool Use

import anthropic, json

client = anthropic.Anthropic(api_key="sk-ant-...")

tools = [{
    "name": "get_stock_price",
    "description": "Get the current stock price for a ticker symbol.",
    "input_schema": {  # "input_schema" — Anthropic's term (not "parameters")
        "type": "object",
        "properties": {
            "ticker": {"type": "string", "description": "e.g. AAPL"}
        },
        "required": ["ticker"]
    }
}]

response = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=500,
    tools=tools,
    messages=[{"role": "user", "content": "What is Apple's stock price?"}]
)

tool_use_block = [b for b in response.content if b.type == "tool_use"][0]
print(f"Calling: {tool_use_block.name}({tool_use_block.input})")

Capability Matrix — What Can Each Do?

Safety & Alignment — Who Refuses What, and Why?

Structured JSON Output — Reliable Extraction

Production pipelines rarely need prose — they need structured data. Extracting a {"name": ..., "price": ..., "category": ...} object reliably from model output requires structured output features.

OpenAI — Strict JSON Mode with Pydantic

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI(api_key="sk-...")

class Product(BaseModel):
    name: str
    price_usd: float
    category: str
    in_stock: bool

response = client.beta.chat.completions.parse(
    model="gpt-4o",
    messages=[{
        "role": "user",
        "content": "Extract product info: Sony WH-1000XM5 headphones, $299.99, electronics, available."
    }],
    response_format=Product   # Guaranteed schema adherence
)

product = response.choices[0].message.parsed
print(f"Name:     {product.name}")
print(f"Price:    ${product.price_usd}")
print(f"Category: {product.category}")
print(f"In Stock: {product.in_stock}")

Anthropic — JSON via Prompt Engineering

import anthropic, json

client = anthropic.Anthropic(api_key="sk-ant-...")

message = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=300,
    system="""You are a data extraction assistant.
Always respond with valid JSON only — no prose, no markdown fences.
Schema: {"name": str, "price_usd": float, "category": str, "in_stock": bool}""",
    messages=[{
        "role": "user",
        "content": "Extract: Sony WH-1000XM5 headphones, $299.99, electronics, available."
    }]
)

product = json.loads(message.content[0].text)
print(product)

Batch API — Cheap Async Processing at Scale

For offline workloads — classifying 10,000 reviews, generating 5,000 product descriptions, embedding a corpus — the Batch APIs of OpenAI and Anthropic offer ~50% cost reduction with async processing (results in 24 hours).

Anthropic Message Batches

import anthropic

client = anthropic.Anthropic(api_key="sk-ant-...")

reviews = [
    "Great product, arrived quickly!",
    "Terrible quality, broke after one day.",
    "Average experience, nothing special.",
]

# Build a batch of requests
requests = [
    anthropic.types.message_create_params.MessageCreateParamsNonStreaming(
        custom_id=f"review-{i}",
        params={
            "model": "claude-haiku-3-5-20251001",
            "max_tokens": 10,
            "system": "Classify sentiment as POSITIVE, NEGATIVE, or NEUTRAL. One word only.",
            "messages": [{"role": "user", "content": review}]
        }
    )
    for i, review in enumerate(reviews)
]

batch = client.messages.batches.create(requests=requests)
print(f"Batch ID: {batch.id}  Status: {batch.processing_status}")
# Poll later — or use webhook for completion notification

Multimodal — Sending Images to the Model

OpenAI Vision

import base64, openai

client = openai.OpenAI(api_key="sk-...")

with open("chart.png", "rb") as f:
    b64 = base64.b64encode(f.read()).decode()

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{
        "role": "user",
        "content": [
            {"type": "text",      "text": "What trend does this chart show?"},
            {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}}
        ]
    }]
)
print(response.choices[0].message.content)

Anthropic Vision

import anthropic, base64

client = anthropic.Anthropic(api_key="sk-ant-...")

with open("chart.png", "rb") as f:
    b64 = base64.b64encode(f.read()).decode()

message = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=500,
    messages=[{
        "role": "user",
        "content": [
            {
                "type": "image",
                "source": {
                    "type": "base64",
                    "media_type": "image/png",
                    "data": b64
                }
            },
            {"type": "text", "text": "What trend does this chart show?"}
        ]
    }]
)
print(message.content[0].text)

Embeddings — Turning Text into Vectors

Embeddings are fixed-length numerical vectors that capture semantic meaning. They power semantic search, RAG (Retrieval-Augmented Generation), clustering, and recommendation systems. OpenAI and Gemini offer embedding endpoints directly; Anthropic does not — use open-source alternatives for embeddings when on Claude.

from openai import OpenAI
import numpy as np

client = OpenAI(api_key="sk-...")

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

texts = [
    "Machine learning automates pattern recognition.",
    "Deep learning uses neural networks with many layers.",
    "The Eiffel Tower is in Paris, France."
]

response = client.embeddings.create(
    model="text-embedding-3-small",
    input=texts
)

vectors = [item.embedding for item in response.data]

print("ML vs Deep Learning similarity:",
      round(cosine_similarity(vectors[0], vectors[1]), 4))
print("ML vs Eiffel Tower similarity:  ",
      round(cosine_similarity(vectors[0], vectors[2]), 4))

Open Source — Running Models Locally with Ollama

# Terminal: pull a model
# ollama pull llama3.3:70b
# ollama serve  (starts local server at localhost:11434)

from openai import OpenAI

# Ollama exposes an OpenAI-compatible endpoint
client = OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama"   # required but not checked
)

response = client.chat.completions.create(
    model="llama3.3:70b",
    messages=[
        {"role": "system", "content": "You are a helpful data science tutor."},
        {"role": "user",   "content": "What is gradient descent?"}
    ],
    stream=True
)

for chunk in response:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

Decision Framework — Which Provider For Which Task?

Provider-Agnostic Code — Write Once, Switch Freely

Avoid coupling your application code to a single provider. A thin abstraction layer lets you swap providers with a single config change — critical when a provider raises prices, has an outage, or releases a better model.

from dataclasses import dataclass
from typing import Literal
import openai, anthropic

@dataclass
class LLMConfig:
    provider: Literal["openai", "anthropic", "openai-compat"]
    model: str
    api_key: str
    base_url: str | None = None

def chat(config: LLMConfig, system: str, user: str, max_tokens: int = 500) -> str:
    if config.provider in ("openai", "openai-compat"):
        client = openai.OpenAI(
            api_key=config.api_key,
            base_url=config.base_url
        )
        resp = client.chat.completions.create(
            model=config.model,
            max_tokens=max_tokens,
            messages=[
                {"role": "system", "content": system},
                {"role": "user",   "content": user}
            ]
        )
        return resp.choices[0].message.content

    elif config.provider == "anthropic":
        client = anthropic.Anthropic(api_key=config.api_key)
        resp = client.messages.create(
            model=config.model,
            max_tokens=max_tokens,
            system=system,
            messages=[{"role": "user", "content": user}]
        )
        return resp.content[0].text

    raise ValueError(f"Unknown provider: {config.provider}")

# Swap providers by changing only the config:
gpt_config     = LLMConfig("openai",       "gpt-4o",             "sk-...")
claude_config  = LLMConfig("anthropic",    "claude-sonnet-4-5",  "sk-ant-...")
llama_config   = LLMConfig("openai-compat", "llama3.3:70b",       "ollama",
                           base_url="http://localhost:11434/v1")

prompt = ("You are a helpful tutor.", "What is overfitting in ML?")
for cfg in [gpt_config, claude_config, llama_config]:
    answer = chat(cfg, *prompt, max_tokens=80)
    print(f"\n[{cfg.model}]\n{answer}")

Golden Rules