Loops that die on one error
A single busy model mid-task and your agent throws away twenty steps of work. Hand-written retry logic never covers every case.
For agent builders
Agents that think fast and never stall
Planning, tool calls, summaries, retries. An agent makes dozens of model calls per task. Route each one by capability, fail over when a model is busy, and see exactly what every step cost.
One endpoint
for every step of every agent loop
Per-step cost
token counts and price on every call
Built-in fail-over
a busy model no longer stops the run
The problem
What is slowing you down today
Costs you only see at month end
Agents fan out unpredictably. Without per-call pricing you find out what a task cost long after it ran.
Overpaying for easy steps
Using your heaviest model to classify a tool result or format JSON burns budget on work a lighter model handles fine.
Capabilities
Everything you need, nothing you don't
Route by capability
Ask for reasoning on the planning step and chat on the formatting step. The router sends each to the lowest-priced enabled model that can do it.
Automatic fail-over
If a model is busy or errors, the request moves to another eligible model in the same call, so your loop keeps going.
Idempotent retries
Send an Idempotency-Key with each step. If your orchestrator retries, the usage is metered once, not twice.
Repeat prompts served from cache
Agents often ask the same question twice. Identical prompts inside the cache window come back instantly and are not billed.
Cost on every response
Each call returns input tokens, output tokens, latency and price, ready to attach to the step in your trace.
Rate limits per account
Set a requests-per-minute ceiling so a runaway loop hits a wall before it hits your budget.
Use cases
What teams run on it
01
Research agents
Plan with a reasoning model, read and summarise sources with a lighter one, and write the final brief with whichever you name.
02
Coding agents
Send code generation and review steps with capability code, and keep small planning chatter on cheaper chat models.
03
Support triage agents
Classify the ticket, draft a reply, check tone, escalate. Four calls, one key, a full cost trail.
04
Back-office automations
Agents that read inboxes, fill forms and reconcile records, with history you can audit step by step.
How it works
From sign-up to production
- 1
Create a key
Sign up, generate an API key in the console and copy it once. We store only a hash.
- 2
Point your agent at one URL
Send each step to POST /api/ainf/inference with a prompt and a capability, or a model_id when you need a specific model.
- 3
Watch the loop in the console
Request history and usage dashboards show every step, its tokens, latency and price.
Control
Built so you stay in charge
Keys you control
API keys are shown once at creation and stored only as a hash. Revoke any key instantly from Settings.
Spend you can see
Every request carries its own receipt: tokens in, tokens out, latency and price. Nothing is billed that you cannot trace.
Limits you set
Per-account rate limits and a free monthly allowance keep experiments safe before they become production.
Data you decide on
You choose exactly what you send. Repeat prompts can be served from your cache, and cache hits are never billed.
For developers
One endpoint. Every model.
Send a prompt and a capability, or name a model. One bearer key, one small JSON body, and a full receipt on every response.
curl https://YOUR-DOMAIN/api/ainf/inference \
-H "Authorization: Bearer $AI_INFERENCE_KEY" \
-H "Content-Type: application/json" \
-H "Idempotency-Key: order-1042" \
-d '{"prompt":"Summarise this ticket in one line","capability":"chat"}'Questions, answered
Can different steps use different models?
Yes. Pass a capability and the router picks the lowest-priced enabled model for it, or pass a model_id to pin a step to one model. Mix both in the same agent.
What happens if a model fails mid-task?
The router tries another eligible model in the same request. You only see an error if no eligible model can complete it.
Will retries double my bill?
Not if you send an Idempotency-Key. Usage for a given key is metered once, so a retried step is not counted twice.
Is there a free tier for prototyping?
Every account gets a free monthly token allowance, so you can build and test an agent before metered pricing applies.
Give your agents a sturdier spine
One key, every capability, fail-over built in. Start on the free allowance today.