Rate limits

Call, token, size and wait limits, with the error each returns.

Every limit returns an error code. Retry-After is in seconds. Sizes are in KB and MB: 1 KB is 1,024 bytes and 1 MB is 1,048,576 bytes. Each entry below gives the limit, its value and the error a request over the limit gets.

Limits

Keys

Account key (abk_) calls: 60 a minute and 5,000 per 24 hours by default, set per key; both windows roll. Error: 429 rate_limited.
LLM key (hlk_) chat completions: 200 a UTC day by default; resets at 00:00 UTC. Error: 429 daily_limit.

Language models

Tokens per member account: 300,000 a UTC day; resets at 00:00 UTC. Error: 429 daily_tokens.
Open chat completions per member account: 64; frees when a completion ends. Error: 429 too_many_streams.
Open chat completions per account on one model: The model's max_parallel; frees when a completion ends. Error: 400 too_many_parallel.
Chat request body: 32 MB. Error: 413 too_large.
Tokens per chat request: The model's context_length. Error: 400, with the model's own error type as code.

Audio and media

Media processing time per member account: 600 s a UTC day by default; resets at 00:00 UTC. Error: 429 budget-exhausted.
Open media jobs per key: 1,000 queued or running; frees when a job ends. Error: 429 too-many-open.
JSON request body: 4 MB on the audio and media routes. Error: 413 too-large.
Audio or image file: 20 MB in POST /v1/media/uploads and POST /v1/audio/transcriptions. Error: 413 too-large.
Saved voices per member account: 30 by default; frees when a voice is deleted. Error: 422 voice_quota.

Notes

A member account is an albastov.com account other than the owner's. The owner's account has no token, open-completion or media-processing limit.
Member account limits apply to every key of the account. max_parallel does not apply to hlk_ keys; a model with max_parallel 1 answers a second open request with 400 parallel_unsupported.
max_parallel and context_length per model are in GET /v1/models.
Open completions count toward the hlk_ daily limit; GET /v1/models does not.
A 429 daily_limit or 429 daily_tokens body carries limit and used; 429 budget-exhausted carries resets_at.

Events limits

Publish requests, per publish key: 120 a minute, bursts of 30. Error: 429 rate_limited with Retry-After.
Published events, per publish key: 300 a minute; each event of a batch counts. Error: 429 rate_limited with Retry-After.
Calls, per events key: 600 a minute and 200,000 a day. Error: 429 rate_limited with Retry-After: 60.
Publish body: 16 KB; a batch 256 KB and at most 100 events; data 8 KB. Error: 413 body_too_large for a body over its size.
idempotency_key window: 7 days, per source.
Webhook body: 2 MB. Error: 413 body_too_large.
Webhook deliveries, per source: 120 a minute, bursts of 30. Error: 429 rate_limited with Retry-After.
Refused webhook requests, per client address: 30, then 1 more every 2 s. Error: 429 rate_limited with Retry-After.
Open streams, per read key: 8. Error: 429 too_many_streams.
Stream catch-up: 1,000 events.
History: 90 days per source by default.

Waits

Model not loaded: 503 model_loading with Retry-After and retry_after_s; a streamed request without X-LLM-Wait stays open up to 1800 s while the model waits in the queue and loads.
Request waiting in the queue: 503 queued with position and Retry-After.
Speech or transcription not finished in time: 503 queued with job_id and Retry-After; the job stays in media jobs.
Speech or transcription job past its queue timeout: 503 queue-timeout with job_id and Retry-After; the job has expired.
Non-streamed request with no response within about 100 s: 524, plain text; stream the request instead.

Staying within limits

Set max_tokens on every chat completion.
Stream long chat completions with "stream": true.
A non-streamed request without X-LLM-Wait for a model that is not loaded waits up to 80 s, then answers 503 model_loading or 503 queued. state in GET /v1/models shows which models are loaded.

Retry rules per status: Errors.