Rate limits
Manage request volume and concurrency.
Per-key limits
Requests are subject to the limits on your API key, workspace, and upstream provider. The most restrictive active limit applies. The dashboard shows key limits you can configure.
Concurrent requests
A streaming request occupies a concurrency slot until it closes. Limit parallel requests in your client and close abandoned streams promptly.
Retries
For a 429 response, honor Retry-After when supplied. Use exponential backoff with jitter. Do not replay an interrupted generation blindly: the provider may already have processed part of it.
Model limits
Context and maximum output limits depend on the model and serving provider. A large advertised context does not guarantee arbitrary concurrency or unlimited output.
Availability depends on your connected provider. Check service status and keep x-request-id with your diagnostics.