HiveRoute — Intelligent Model Routing

HiveRoute is ModelHive's zero-cost intelligent model routing system that automatically routes LLM requests to the most appropriate model based on task difficulty. Simple questions go to fast, cheap models; complex tasks go to powerful frontier models — all transparently.

Why HiveRoute?

Without HiveRoute	With HiveRoute
Same expensive model for every request	Cheap model for trivial tasks, frontier for complex
Manual routing logic in your code	Automatic classification — no code changes
Overpaying for simple queries	Up to 70% cost reduction on mixed workloads
No reasoning effort control	Automatic reasoning_effort / thinking.budget_tokens injection

How It Works

HiveRoute is a sub-feature of HiveState. When HiveState extracts the conversation state, it also classifies task difficulty and reasoning effort. HiveRoute then uses this classification to select the optimal model.

Processing Steps

1. Request arrives at /v1/chat/completions or /v1/messages
2. HiveState extracts conversation state (including difficulty and reasoning_effort)
3. HiveRoute maps difficulty → target model from configured levels
4. The request's model field is transparently rewritten to the target model
5. For reasoning models: reasoning_effort (OpenAI) or thinking.budget_tokens (Anthropic) is injected

Zero Extra Cost

HiveRoute piggybacks on the HiveState extraction that's already happening — there's no additional LLM call or latency for the routing decision.

Difficulty Levels

HiveRoute classifies every request into one of three difficulty levels:

Level	Examples	Typical Model
trivial	Simple questions, typo fixes, one-line changes, status checks	Llama 8B, GPT-4o-mini
standard	Normal coding tasks, feature implementation, explanations	GPT-4o, Claude Sonnet
complex	Architecture decisions, complex multi-file debugging, system design	Claude Opus, GPT-4o (high reasoning)

Request Headers

Header	Values	Behavior
x-hive-route	"false"	Opt-out of HiveRoute model routing (HiveState compression still applies)
x-hive-state	"false"	Opt-out of both HiveState and HiveRoute (since HiveRoute depends on HiveState)

Response Headers

Header	Example	Description
x-hiveroute-level	standard	Difficulty level classified (trivial, standard, complex)
x-hiveroute-model	gpt-oss	Target model the request was routed to
x-hiveroute-effort	medium	Reasoning effort injected (low, medium, high)

Usage Examples

Python

from openai import OpenAI

client = OpenAI(
    api_key="sk-your-modelhive-key",
    base_url="https://api.modelhive.ai/v1"
)

# Request any model — HiveRoute may transparently route to a better fit
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "What's 2+2?"}  # trivial → routed to cheap model
    ]
)

# Check routing decision via response headers
# x-hiveroute-level: trivial
# x-hiveroute-model: llama-8b
# x-hiveroute-effort: low

cURL

# HiveRoute activates automatically for long conversations
curl -X POST https://api.modelhive.ai/v1/chat/completions \
  -H "Authorization: Bearer sk-your-modelhive-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "system", "content": "You are a coding assistant."},
      {"role": "user", "content": "Refactor the auth module to use JWT"},
      {"role": "assistant", "content": "...long implementation..."},
      {"role": "user", "content": "Fix the typo on line 42"}
    ]
  }'

# Response headers:
# x-hiveroute-level: trivial
# x-hiveroute-model: llama-8b
# x-hiveroute-effort: low

Bypass Routing

# Keep HiveState compression but skip model routing
curl -X POST https://api.modelhive.ai/v1/chat/completions \
  -H "Authorization: Bearer sk-your-modelhive-key" \
  -H "Content-Type: application/json" \
  -H "x-hive-route: false" \
  -d '{
    "model": "gpt-4o",
    "messages": [...]
  }'

# Request stays on gpt-4o regardless of difficulty

Reasoning Effort Injection

HiveRoute doesn't just select the model — it also injects the appropriate reasoning effort level:

OpenAI Models

Sets the reasoning_effort field in the request body:

Effort	Value
Low	"low"
Medium	"medium"
High	"high"

Anthropic Models

Enables extended thinking with calibrated budget_tokens:

Effort	Default Budget
Low	1,024 tokens
Medium	4,096 tokens
High	16,384 tokens

// Automatically injected for Anthropic models
{
  "thinking": {
    "type": "enabled",
    "budget_tokens": 4096
  }
}

Load Balancing & Failover

When multiple deployments exist for the same model, the Router layer handles load balancing:

Routing Strategies

Strategy	Behavior
shuffle	Random order each request (default)
round-robin	Sequential cycling through deployments
latency	Prioritize lowest-latency deployments (EMA-based)

Circuit Breaker

Each deployment has automatic circuit breaker protection:

• Opens after 3 consecutive failures (configurable)
• Stays open for 60 seconds (configurable recovery period)
• On success: resets failure counter
• When all deployments broken: tries anyway (half-open mode)

Retry Logic

• Up to 3 retries with exponential backoff
• Retryable: 429, 500, 502, 503, 504, 408, network errors
• Non-retryable: 400, 401, 403 → fail immediately

Response Headers (Failover Info)

Header	Example	Description
X-Modelhive-Provider	azure-east	Deployment that served the request
X-Modelhive-Attempts	2	Number of attempts (only if >1)
X-Modelhive-Failed	azure-west,openai	Providers that failed before success
X-Modelhive-Cost	0.0012340000	USD cost of the request

Supported Endpoints

HiveRoute applies to all chat/completion endpoints:

Method	Endpoint	HiveRoute	Load Balancing
POST	/v1/chat/completions	✅	✅
POST	/v1/messages	✅	✅
POST	/v1/completions	❌	✅
POST	/v1/embeddings	❌	✅

Activation Condition

HiveRoute only activates when HiveState is triggered (conversation above the token threshold). For short conversations below threshold, the request goes to the originally specified model.

Cost Optimization

For maximum savings, combine HiveRoute with HiveCache. HiveCache handles repeated queries instantly, while HiveRoute ensures fresh queries use the cheapest adequate model.