Requests and concurrency
The backend checks request rate and concurrent in-flight inference at workspace level. Multiple keys share the same workspace limits. Creating another key does not create more capacity.
If
Retry-After is present, wait at least that many seconds. Otherwise, back off with jitter and a bounded retry count. Avoid automatically replaying a stream that already produced content.
Monthly spending cap
Owners and admins can select Edit cap on Rate and spending limits, enter an amount or choose no cap, and select Save spending cap. A$0 cap blocks new billable requests; requests already admitted may finish.
The accounting month follows UTC calendar boundaries. The check includes charges finalized in the current month and any holds from the previous reservation system, including unresolved holds from earlier months. A top-up increases the wallet balance but does not increase the spending cap. Organization and per-project caps can also be managed through the Administration API. A project cap is checked alongside the shared organization cap. API cap setters accept positive amounts; the console can additionally set a zero organization cap to block new requests.
New requests do not reserve a worst-case charge. They are admitted while the available balance is positive and the applicable spending caps have not been reached. Actual usage is charged after completion, so concurrent requests can finish after the balance or cap has been reached. Rate and concurrency limits still apply.
Output limits
Usemax_tokens or max_completion_tokens for Chat, max_output_tokens for Responses, and max_tokens for Messages. Messages requires this field. Chat defaults to 1,024 output tokens and Responses defaults to 32,768 for coding-tool compatibility, each capped at the model maximum. The maximum accepted bound comes from the model price configuration. Send only one output-bound field; conflicting fields are rejected.
See models for each model’s context and output ceilings. Input, reasoning and final answer share the context window. Request only the output budget you need.
Image-capable models accept request bodies up to 32 MiB; other models accept up to 8 MiB. Image upload limits also apply; see images.
Requests have a 60-second server execution deadline. Client and network timeouts can be shorter. Disable automatic retries when replaying an expensive request would be undesirable. A model’s output ceiling does not guarantee that a response will finish within the deadline.
Scope of protection
API and console playground inference share the workspace’s balance and admission controls. Catalog reads do not generate inference usage. Shared model capacity can still cause429 or temporary unavailability even when your workspace has capacity.
See errors for the response format and billing for charging behavior.