HiveState — Adaptive State Compression

HiveState is ModelHive's adaptive state compression middleware that automatically compresses long conversation histories into structured JSON state representations. When token counts exceed a configurable threshold, HiveState replaces the full message history with a compact "state snapshot" that preserves the minimum sufficient information for the LLM to continue the conversation correctly.

Key insight: HiveState preserves executable workflow state, not history — extracting intent, identifiers, action outcomes, and pending requirements while eliminating redundant context.

Why HiveState?

Without Compression	With HiveState
Token costs grow linearly with conversation length	Long histories compressed to compact state
Context window limits restrict conversation depth	Conversations can run indefinitely
Manual summarization requires code changes	Zero-change integration via headers
Lost context when truncating	Structured extraction preserves critical state

Real-World Results

Domain	Activation Rate	Mean Token Reduction	Quality Score
Customer Service	85%	54.1%	4.74/5.0
Tool/Function Agents	87%	46.6%	4.50/5.0
Coding Agents	93%	71.5%	4.80/5.0
Web Agents	98%	60.1%	4.76/5.0

How It Works

Processing Pipeline

1. Threshold Gate — Below-threshold requests pass through with zero overhead
2. Dynamic Profiling — Classifies conversations as tool-agent, code-agent, or chat without LLM calls
3. Conversation Zoning — Splits messages into System (unchanged), History (compressed), Recent (preserved verbatim), Last (final user message)
4. Importance Scoring — Ranks messages by Recency, Error Signals, Decision Density, and Information Density
5. State Extraction — An extraction model produces a structured JSON state with intent, constraints, progress, and errors
6. CCR Injection — Cache-Compress-Retrieve: original history is cached, keywords overlap triggers selective reinjection on subsequent requests

Zero Configuration

HiveState works automatically when enabled. Just send requests to the ModelHive API and compression happens transparently for long conversations.

Request Headers

Header	Values	Behavior
x-hive-state	"false"	Opt-out of HiveState compression for this request

Response Headers

Every response from a HiveState-eligible request includes diagnostic headers:

Header	Example	Description
x-hivestate-mode	state	Processing mode: state (compressed) or none (pass-through)
x-hivestate-original-tokens	7000	Token count before compression
x-hivestate-tokens	220	Token count after compression
x-hivestate-ratio	0.97	Compression ratio (1 − result/original)
x-hivestate-latency-ms	1215	Extraction latency in milliseconds
x-hivestate-extraction-prompt-tokens	3200	Tokens consumed by the extraction prompt
x-hivestate-extraction-completion-tokens	180	Tokens generated by extraction
x-hivestate-intent	exchange_items	Extracted conversation intent
x-hivestate-constraints	{"identifiers":{...}}	JSON of active constraints
x-hivestate-fallback	below_threshold(350/4000)	Reason if compression was skipped

Usage Examples

Python

from openai import OpenAI

client = OpenAI(
    api_key="sk-your-modelhive-key",
    base_url="https://api.modelhive.ai/v1"
)

# Long conversation — HiveState compresses automatically
messages = [
    {"role": "system", "content": "You are a helpful coding assistant."},
    {"role": "user", "content": "Create a React todo app with TypeScript"},
    {"role": "assistant", "content": "I'll create a React TypeScript todo app..."},
    # ... many more turns ...
    {"role": "user", "content": "Now add drag-and-drop reordering"},
]

response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages
)

# HiveState transparently compresses the history if above threshold
# Check response headers for compression metrics

cURL

# Long conversation — HiveState activates automatically above threshold
curl -X POST https://api.modelhive.ai/v1/chat/completions \
  -H "Authorization: Bearer sk-your-modelhive-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "system", "content": "You are a coding assistant."},
      {"role": "user", "content": "Create a React todo app"},
      {"role": "assistant", "content": "...long response..."},
      {"role": "user", "content": "Add authentication"},
      {"role": "assistant", "content": "...long response..."},
      {"role": "user", "content": "Now add drag-and-drop"}
    ]
  }'

# Response headers:
# x-hivestate-mode: state
# x-hivestate-original-tokens: 7000
# x-hivestate-tokens: 220
# x-hivestate-ratio: 0.97
# x-hivestate-intent: add_feature

Bypass Compression

# Skip compression for a specific request
curl -X POST https://api.modelhive.ai/v1/chat/completions \
  -H "Authorization: Bearer sk-your-modelhive-key" \
  -H "Content-Type: application/json" \
  -H "x-hive-state: false" \
  -d '{
    "model": "gpt-4o",
    "messages": [...]
  }'

Extracted State Schema

When HiveState compresses a conversation, it produces a structured JSON state:

{
  "intent": "exchange_items",
  "difficulty": "standard",
  "reasoning_effort": "medium",
  "active_constraints": {
    "identifiers": { "order_id": "ORD-9842", "user": "sophia_h" },
    "values": { "item": "Blue Wool Sweater M", "reason": "wrong size" },
    "actions_taken": [
      {"tool": "lookup_order", "input": "ORD-9842", "result": "found, status: delivered", "success": true}
    ],
    "files_modified": [
      {"path": "src/app/page.tsx", "change": "created with shadcn Button import"}
    ],
    "progress": {
      "done": ["verified order", "confirmed eligibility"],
      "current": "processing exchange request",
      "next": ["generate return label", "send confirmation"]
    },
    "requirements": { "size": "L", "color": "same" },
    "errors_encountered": [
      {"error": "inventory_unavailable", "resolution": "suggested alternative color"}
    ]
  },
  "conversation_status": "in_progress"
}

Dynamic Conversation Profiling

HiveState automatically classifies conversations into profiles that tune compression parameters:

Profile	Detection Rule	Behavior
tool-agent	>30% messages are tool responses	Maximizes tool context preservation
code-agent	>20% messages contain code patterns	Higher budget for code-heavy conversations
chat	Default	Balanced compression for general chat

Conversation Zoning

Messages are split into zones with different treatment:

Zone	Treatment
System	Passed unchanged (system prompts preserved)
History	Compressed into structured state
Recent	Last N assistant steps preserved verbatim (configurable step_window)
Last	Final user message forwarded verbatim

Fallback Conditions

HiveState gracefully falls back to pass-through when compression wouldn't help:

Reason	Description
no_history	No conversation history to compress
below_threshold	Token count below configured threshold
no_model	No extraction model configured
insufficient_headroom	Recent + Last zones already use >85% of original tokens
extraction_failed	Extraction model returned an error
no_savings	Compressed result wouldn't save meaningful tokens

Supported Formats

HiveState works transparently with both request formats:

Endpoint	Format	Step Splitting
POST /v1/chat/completions	OpenAI	Turn-based (preserves tool_calls/tool in recent window)
POST /v1/messages	Anthropic	Step-based (preserves multi-turn sequences)

Context Recovery (CCR)

Compressing a history means the verbatim content — a file the agent read, the output of a tool — is no longer in the prompt. CCR keeps that content aside and re-injects it into a later turn when the conversation refers to it again, so compression does not have to choose between a small prompt and a complete one.

The scope is a security boundary

CCR holds original messages, so a retrieval that crossed callers would hand one tenant another tenant's source code. Everything is filed under a scope and is only ever retrievable under the identical scope:

scope = (team_id, key_hash, session_id)

There is no shared bucket and no global fallback. A scope needs both an owner — a team, or the key hash when the key has no team — and a session. A request that cannot produce both has CCR disabled: no memory at all is the safe failure, shared memory is not.

The partition key is a SHA-256 of the three components, each length-prefixed rather than delimited. That is deliberate: session_id comes straight from a client header, and with a delimiter — even NUL — a caller could pick a session name that hashes into someone else's partition. Length prefixes make the encoding injective for any input.

How the session is decided

Input	Session used
x-hive-session header	Taken verbatim
No header	Derived by hashing the system and developer messages plus the first user turn
Neither header nor any system/user message	None — CCR is off for that request

The derived anchor is closed by the first user turn, so it stays byte-stable as the conversation grows while differing between conversations. CCR therefore works without any client change; the header is there for callers that want to control the boundary themselves — for instance to keep two parallel threads of the same key apart, or to deliberately start fresh.

curl https://api.modelhive.ai/v1/chat/completions \
  -H "Authorization: Bearer sk-mh-..." \
  -H "x-hive-session: ticket-4471" \
  -d '{"model": "gpt-4o", "messages": [...]}'

Bounds

Storage is bounded twice, so one tenant cannot displace another:

• each scope holds at most 32 files and evicts its own oldest first;
• at most 512 scopes are live, evicting the least recently used.

Entries expire after 30 minutes. The decay counter is per scope — content leaves the working set after 5 turns without being mentioned — so unrelated traffic cannot age another scope's working set out.

Best Use Cases

HiveState provides the highest savings for long-running agent conversations — coding assistants, multi-step tool agents, and customer service bots with extended interaction histories.

Extraction Cost

The extraction call itself uses tokens (tracked as @hivestate provider spend). However, the net savings from reduced prompt tokens on the primary model typically exceed extraction costs by 3-10x.