Guessing where time goes
Without per-call latency, slow features are hard to diagnose.
Latency
Make every response feel instant
Users notice waiting. Choose responsive models, keep prompts and outputs lean, let repeats come from cache and measure latency on every call so you know what to fix.
Measured per call
latency_ms on every response
Cache-speed repeats
no generation needed
Lean requests
shorter prompts, tighter outputs
The problem
What is slowing you down today
Bloated prompts
Long boilerplate adds time on every single call.
Oversized outputs
Generating text nobody reads still costs time.
Capabilities
Everything you need, nothing you don't
latency_ms on every response
Each result reports how long the call took, so you can track it in your own monitoring.
Latency in the catalog
The model catalog shows typical latency figures for each model, so you can compare before you choose.
Cache hits
Identical prompts inside the cache window skip generation entirely.
Shorter outputs
A tighter max_tokens and a clear instruction on length reduce generation time.
Fail-over
A busy model is skipped for another eligible one rather than leaving you waiting on an error.
History for analysis
Request history shows latency per call, so outliers are easy to spot.
Use cases
What teams run on it
01
Interactive features
Pick responsive models and keep outputs short.
02
Background jobs
Trade latency for price where nobody is waiting.
03
Popular questions
Lean on the cache for instant answers.
04
Agent loops
Light models for quick intermediate steps.
How it works
From sign-up to production
- 1
Measure
Log latency_ms from every response.
- 2
Compare
Test candidate models in the playground and check catalog latency figures.
- 3
Trim
Shorten prompts, cap max_tokens and stabilise prompts for cache hits.
Control
Built so you stay in charge
Keys you control
API keys are shown once at creation and stored only as a hash. Revoke any key instantly from Settings.
Spend you can see
Every request carries its own receipt: tokens in, tokens out, latency and price. Nothing is billed that you cannot trace.
Limits you set
Per-account rate limits and a free monthly allowance keep experiments safe before they become production.
Data you decide on
You choose exactly what you send. Repeat prompts can be served from your cache, and cache hits are never billed.
For developers
One endpoint. Every model.
Send a prompt and a capability, or name a model. One bearer key, one small JSON body, and a full receipt on every response.
curl https://YOUR-DOMAIN/api/ainf/inference \
-H "Authorization: Bearer $AI_INFERENCE_KEY" \
-H "Content-Type: application/json" \
-H "Idempotency-Key: order-1042" \
-d '{"prompt":"Summarise this ticket in one line","capability":"chat"}'Questions, answered
What does latency_ms measure?
The time we spent completing the request, including any fail-over attempts.
Are cached responses faster?
Yes. They skip generation and return the stored answer.
Does the playground show latency?
Yes, alongside tokens and price for every run.
Can I get partial output while it generates?
No. Each request returns the complete response when it finishes, so keep interactive outputs short.
Find your slowest call today
Check latency in request history and start trimming.