Making Your First Request

The Request Lifecycle

When you send a chat completion request to ModelHive, it passes through several intelligent layers:

Each layer adds value:

1. Guardrails — Scans for prompt injection, toxicity, PII, and secrets
2. HiveCache — Checks for semantically similar cached responses (DIRECT/REUSE/TWEAK/MISS)
3. HiveState — Compresses long conversation histories into structured state snapshots
4. HiveRoute — Routes to the optimal model based on task difficulty (trivial/standard/complex)
5. LLM Router — Load balances across provider deployments with failover

Basic Chat Completion

Python

from openai import OpenAI

client = OpenAI(
    api_key="sk-your-modelhive-key",
    base_url="https://api.modelhive.ai/v1"
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "Explain quantum computing in 3 sentences."}
    ],
    temperature=0.7,
    max_tokens=200
)

print(response.choices[0].message.content)
print(f"Tokens used: {response.usage.total_tokens}")

JavaScript

import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: 'sk-your-modelhive-key',
  baseURL: 'https://api.modelhive.ai/v1',
});

const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    { role: 'user', content: 'Explain quantum computing in 3 sentences.' }
  ],
  temperature: 0.7,
  max_tokens: 200,
});

console.log(response.choices[0].message.content);

cURL

curl -X POST https://api.modelhive.ai/v1/chat/completions \
  -H "Authorization: Bearer sk-your-modelhive-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "user", "content": "Explain quantum computing in 3 sentences."}
    ],
    "temperature": 0.7,
    "max_tokens": 200
  }'

Streaming Responses

ModelHive fully supports Server-Sent Events (SSE) streaming:

Python

stream = client.chat.completions.create(
    model="claude-sonnet-4-20250514",
    messages=[
        {"role": "user", "content": "Write a haiku about AI caching."}
    ],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

JavaScript

const stream = await client.chat.completions.create({
  model: 'claude-sonnet-4-20250514',
  messages: [
    { role: 'user', content: 'Write a haiku about AI caching.' }
  ],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content || '');
}

info

When HiveCache returns a cached response with streaming enabled, it simulates the SSE stream from cache for a seamless experience — your client code doesn't need to change.

Multi-Provider Model Selection

Switch between providers by simply changing the model parameter:

# OpenAI
response = client.chat.completions.create(model="gpt-4o", messages=messages)

# Anthropic
response = client.chat.completions.create(model="claude-sonnet-4-20250514", messages=messages)

# Google Gemini
response = client.chat.completions.create(model="gemini/gemini-2.5-pro", messages=messages)

# Mistral
response = client.chat.completions.create(model="mistral/mistral-large-latest", messages=messages)

# Groq (ultra-fast inference)
response = client.chat.completions.create(model="groq/llama-3.1-70b-versatile", messages=messages)

Conversation with History

messages = [
    {"role": "system", "content": "You are a helpful coding assistant."},
    {"role": "user", "content": "How do I sort a list in Python?"},
    {"role": "assistant", "content": "You can use sorted() or list.sort()..."},
    {"role": "user", "content": "What about sorting by a custom key?"}
]

response = client.chat.completions.create(model="gpt-4o", messages=messages)

tip

For long conversations (2000+ tokens), ModelHive automatically activates HiveState — compressing conversation history into structured state snapshots, reducing tokens by up to 71% while preserving context. When active, HiveRoute also kicks in, automatically routing to the most cost-effective model based on task complexity.

Sending Images and Files

ModelHive supports multimodal requests — you can send images, documents, and other files alongside your text prompts. All vision-capable models (gpt-4o, claude-sonnet-4-20250514, gemini/gemini-2.5-pro, etc.) accept these formats.

Image via URL

The simplest way to analyze an image is by passing its URL:

Python

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Describe what you see in this image."},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://example.com/photo.jpg"
                    }
                }
            ]
        }
    ],
    max_tokens=500
)

print(response.choices[0].message.content)

JavaScript

const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'Describe what you see in this image.' },
        {
          type: 'image_url',
          image_url: {
            url: 'https://example.com/photo.jpg',
          },
        },
      ],
    },
  ],
  max_tokens: 500,
});

console.log(response.choices[0].message.content);

cURL

curl -X POST https://api.modelhive.ai/v1/chat/completions \
  -H "Authorization: Bearer sk-your-modelhive-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": [
          {"type": "text", "text": "Describe what you see in this image."},
          {
            "type": "image_url",
            "image_url": {
              "url": "https://example.com/photo.jpg"
            }
          }
        ]
      }
    ],
    "max_tokens": 500
  }'

Image via Base64

For local files, encode them as base64 and embed them directly in the request:

Python

import base64

# Read and encode a local image
with open("screenshot.png", "rb") as f:
    image_base64 = base64.b64encode(f.read()).decode("utf-8")

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What does this screenshot show?"},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"data:image/png;base64,{image_base64}"
                    }
                }
            ]
        }
    ],
    max_tokens=500
)

JavaScript

import fs from 'fs';

const imageBase64 = fs.readFileSync('screenshot.png').toString('base64');

const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'What does this screenshot show?' },
        {
          type: 'image_url',
          image_url: {
            url: `data:image/png;base64,${imageBase64}`,
          },
        },
      ],
    },
  ],
  max_tokens: 500,
});

Supported image formats

image/png, image/jpeg, image/gif, image/webp. Maximum file size depends on the provider, typically 20 MB.

Multiple Images in One Request

You can include multiple images to compare, analyze, or describe together:

Python

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Compare these two product designs. Which one is more user-friendly and why?"},
                {
                    "type": "image_url",
                    "image_url": {"url": "https://example.com/design-a.png"}
                },
                {
                    "type": "image_url",
                    "image_url": {"url": "https://example.com/design-b.png"}
                }
            ]
        }
    ],
    max_tokens=1000
)

cURL

curl -X POST https://api.modelhive.ai/v1/chat/completions \
  -H "Authorization: Bearer sk-your-modelhive-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "user",
        "content": [
          {"type": "text", "text": "Compare these two product designs. Which one is more user-friendly and why?"},
          {"type": "image_url", "image_url": {"url": "https://example.com/design-a.png"}},
          {"type": "image_url", "image_url": {"url": "https://example.com/design-b.png"}}
        ]
      }
    ],
    "max_tokens": 1000
  }'

PDF Document Analysis

Analyze PDFs by encoding them as base64. This works with models that support document input (e.g., gemini/gemini-2.5-pro, claude-sonnet-4-20250514):

Python

import base64

# Read and encode a PDF
with open("report.pdf", "rb") as f:
    pdf_base64 = base64.b64encode(f.read()).decode("utf-8")

response = client.chat.completions.create(
    model="gemini/gemini-2.5-pro",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Summarize the key findings from this report."},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": f"data:application/pdf;base64,{pdf_base64}"
                    }
                }
            ]
        }
    ],
    max_tokens=2000
)

print(response.choices[0].message.content)

JavaScript

import fs from 'fs';

const pdfBase64 = fs.readFileSync('report.pdf').toString('base64');

const response = await client.chat.completions.create({
  model: 'gemini/gemini-2.5-pro',
  messages: [
    {
      role: 'user',
      content: [
        { type: 'text', text: 'Summarize the key findings from this report.' },
        {
          type: 'image_url',
          image_url: {
            url: `data:application/pdf;base64,${pdfBase64}`,
          },
        },
      ],
    },
  ],
  max_tokens: 2000,
});

console.log(response.choices[0].message.content);

Mixed Content: Text + Images + Files

Combine different content types in a single conversation for complex workflows:

import base64

# Encode a local chart image
with open("sales_chart.png", "rb") as f:
    chart_b64 = base64.b64encode(f.read()).decode("utf-8")

# Encode a spreadsheet export as PDF
with open("financials.pdf", "rb") as f:
    pdf_b64 = base64.b64encode(f.read()).decode("utf-8")

response = client.chat.completions.create(
    model="gemini/gemini-2.5-pro",
    messages=[
        {
            "role": "system",
            "content": "You are a financial analyst. Analyze the provided data and give actionable insights."
        },
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Here is our Q4 sales chart and the full financial report. Identify the top 3 trends and suggest next quarter's priorities."},
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:image/png;base64,{chart_b64}"}
                },
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:application/pdf;base64,{pdf_b64}"}
                }
            ]
        }
    ],
    max_tokens=3000
)

Image Detail Level

Control how much detail the model uses when analyzing images with the detail parameter:

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Read all the text in this document."},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://example.com/document-scan.jpg",
                        "detail": "high"  # "low", "auto", or "high"
                    }
                }
            ]
        }
    ]
)

Detail Level	Description	Token Usage
low	Fast analysis, lower resolution (512×512)	~85 tokens per image
high	Detailed analysis, full resolution	~765+ tokens per image
auto	Model decides based on image size (default)	Varies

tip

Use "detail": "low" for simple classification or captioning tasks to save tokens. Use "detail": "high" for OCR, reading charts, or extracting fine details.

Bypass Cache (When Needed)

If you need a fresh response every time (e.g., for creative writing), bypass the cache:

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Write a creative poem."}],
    extra_body={
        "metadata": {
            "cache_bypass": True
        }
    }
)

Error Handling

from openai import OpenAI, APIError, RateLimitError, AuthenticationError

client = OpenAI(api_key="sk-your-key", base_url="https://api.modelhive.ai/v1")

try:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello!"}]
    )
except AuthenticationError:
    print("Invalid API key or insufficient budget")
except RateLimitError:
    print("Rate limit reached — try again shortly")
except APIError as e:
    if "guardrail" in str(e).lower():
        print("Request blocked by security guardrail")
    else:
        print(f"API error: {e}")

Supported Parameters

ModelHive passes through all standard OpenAI chat completion parameters:

Parameter	Type	Description
model	string	The model to use (required)
messages	array	Conversation messages (required) — each message content can be a string or an array of text, image_url parts for multimodal input
temperature	float	Sampling temperature (0-2)
max_tokens	integer	Maximum tokens to generate
top_p	float	Nucleus sampling
stream	boolean	Enable streaming
stop	string/array	Stop sequences
presence_penalty	float	Presence penalty (-2 to 2)
frequency_penalty	float	Frequency penalty (-2 to 2)
tools	array	Function/tool definitions
tool_choice	string/object	Tool selection strategy
response_format	object	Force JSON output