An API limit is not just about your task: what happens to the queue after a 429?
Plan shared API usage and queue behaviour after 429 responses without treating every cause alike.
Article language: English

In this article
At nine, the assistant stops answering. Every employee sent only one question, so nobody feels responsible for the disruption. In the background, however, a catalogue import is running while another team compares documents. All those tasks may use the same limit. A limit does not keep separate accounts for someone's good intentions.
A 429 response requires checking the cause and scope of the restriction. Sometimes the pace must slow and work wait; sometimes an exhausted quota needs a decision from the person managing the service. A queue should distinguish those cases, remember accepted tasks and divide available capacity among kinds of work. An automatic retry button alone cannot do that.
Consider a fictional team using one API, a service's programming interface, for employees' short questions, catalogue descriptions and periodic reviews of long documents. We examine shared-limit handling using artificial jobs. We give no account's limits and perform no test on a production service.
Whose limit are you using?
Map all processes using the relevant provider project, organisation and limit group. Include manually started tools and schedules an assistant user may not know about. The number of open chat windows alone does not describe all traffic. A list of process owners and normal start times helps.
OpenAI's rate-limit documentation describes limits at organisation and project levels and different units, including requests and tokens over time. Some model families share a limit. A separate user or a different model variant therefore need not mean a separate allocation.
For each restriction record the unit, scope and where to read current configuration. A request limit and a processed-text limit answer different questions. A burst of many small cases and several long documents may hit different boundaries. Do not convert all loads into one number of “AI questions” without explanation.
Distinguish an allowed maximum from the rate the solution actually sustains on your material. Checking AI timing under load (in Polish) helps assess the latter. Reading a limit in a dashboard does not prove every request within it will finish equally quickly.
In the fictional company, suppose a catalogue import can wait but an employee dealing with a client needs brief information during the interaction. The queue owner should know this difference in use. Do not assign priority solely by the requester's title or the loudness of their request; the rule should reflect an agreed work objective.
Check whether the limit really is shared. Separate keys can help attribute traffic without necessarily creating independent capacity. Do not propose multiplying keys to evade provider rules. First understand the relevant scope and organise work within available capacity.
Will a 429 go away after waiting?
Retain the code, description and relevant response information needed for diagnosis. The status number alone may not suffice. In an operator view, distinguish temporary rate restriction from an exhausted allocation or a condition requiring administrator action. Give an employee a simple message about their task's fate without copying confidential account details.
Anthropic's rate-limit documentation describes both rate restrictions and a 429 after reaching a specified monthly spending allocation. In the latter case, retries alone do not restore access. The documentation also distinguishes other configured limits; read the particular response instead of assuming one reason for the entire status number.
A temporarily deferred task should retain its identifier and known reason for waiting. For a restriction requiring human action, choose another path: stop automatic attempts, notify the responsible owner and decide how to proceed. Do not let an impossible task return all day as the newest urgent job.
If the provider recommends retrying the particular cause and supplies a valid Retry-After, treat that delay as the minimum wait. If the SDK version or settings cannot wait that long, defer the task rather than retrying sooner. Do not promise an exact result time: after the wait, execution may still take time or encounter another restriction. The header alone does not prove that a condition requiring human action will resolve through waiting.
Absence of that instruction also needs agreed behaviour. The integration owner should check the supported mechanism for increasing intervals, adding small random variation and bounding total waiting. Do not pick seconds merely to make an error disappear during a demonstration. Retain the configuration rationale and stopping method.
Inspect the client library's settings. Some retries may already occur below the application. The cited OpenAI documentation describes accounting for SDK retries and limiting attempts and total time. Two overlapping loops can send more requests than the operator's visible counter suggests.
Duplicate and retry control in automation (in Polish) develops recognition of the same operation. Here the pace of the whole group also matters. One attempt's reasonable behaviour may cease to be reasonable when dozens of tasks wake simultaneously and each independently tries to regain access.

Include two processes in acceptance that each stay within an agreed pace separately but together compete for one allocation. Check where sending is coordinated and how an operator knows the shared state. Two correct local counters may not suffice for the whole. First practise using controlled responses in a test environment without deliberately exhausting a real limit. Such a test checks queue logic, not yet provider-service performance or limits.
What does the queue promise someone waiting?
An accepted task needs a recognisable state. “Waiting for capacity” differs from “processing document”. The user needs to know whether they can leave, where to return for the result and what cancellation does. They should not increase the queue by resending the same request solely because the screen went silent.
Establish which data from submission stays attached to the task. A document can change while waiting. At execution, determine whether the agreed version is still in use or an update or new decision is needed. A long queue must not silently change the scope of previously accepted work.
Useful lifetime matters too. An answer needed during a morning conversation may be useless in the evening. Define expiry or transfer to another handling route rather than executing every backlog item at any cost. Retain the reason for ending so a counter does not present it as a completed result.
If a task already performed operations in other services, stopping before the next AI call does not undo them. Review partial execution by an agent across systems (in Polish) before planning to restart the entire run. The request queue is part of a wider case.
Test application recovery after restart too. Do accepted tasks still exist? Do they retain age and attempt count? Do they avoid all starting at once on resumption? These are acceptance scenarios for the chosen solution, not assumptions from the name of a queueing tool.
Put cases requiring a decision in an automation exception queue (in Polish). Ordinary controlled waiting for capacity need not immediately become a manual incident. But define when delay or cause transfers a case to the responsible person.
How do less urgent jobs avoid waiting forever?
In the fictional company, short questions take priority over catalogue import. If new questions arrive all day, however, import may never receive its share of capacity. The team answers promptly while using increasingly old descriptions. That is an effect of the ordering rule, not proof the batch process is lazy.
Microsoft's priority-queue pattern identifies the risk of continually delaying lower priorities and the need to account for their service. It discusses, among other measures, increasing the importance of older items. This is a possible mechanism whose fit must be assessed for the process.
Agree a rule giving important cases precedence while providing others a real route to completion. It may reserve an agreed share of capacity for import or provide a window for it. We give no universal proportion. The choice should follow deadlines, consequences of delay and actual arrival rates.
Accept-test the rule with two groups of artificial jobs. Sustain a flow of short questions and observe whether older import entries still advance to execution. Show the oldest task's age in each group and numbers completed and rejected. A total handled-request count can conceal a group that never moves.
Do not set priority solely by task size. A long document may concern an urgent agreement; a short question may be an experiment that can wait. Purpose and process rules matter. If every user can freely mark their task urgent, priority soon becomes default decoration.
Also compare behaviour after access is restored. Do not start the entire backlog at once without control if such a spike caused the restriction. Test gradual return within limits agreed with the environment owner. Backlogged cases retain context, but some may have become obsolete.
Assign the decision to resume after a restriction requiring intervention to the responsible person. An operator should know whether the specific cause was removed or merely time passed since the last attempt. Record the basis for return and the task groups covered. There is no need to reveal billing details to every employee here. Confirmed availability and an agreed continuation route suffice. This separates controlled resumption from everyone blindly checking whether something can be sent this time.
How can you tell restriction handling works?
A test should cover a temporary restriction, a condition requiring administrator decision, two competing work types and recovery after a break. Add task cancellation and material changing during the wait. Record the expected fate of each case, not merely the retry count.
Connect the result to the total cost of an AI process (in Polish). A longer queue may mean manual work, answers losing usefulness and handling explanations. The fee for successful calls alone does not show these costs. But do not convert a hypothetical backlog into a financial loss without appropriate data.
If the agreed scope of a review with Aeneia Audit includes these issues, the team can prepare a map of shared processes, configuration information without secrets and several controlled refusal cases. This material can help identify where waiting, a human decision or changes to work-sharing rules are needed. The provider still sets the limit, and the team agrees how to respond to 429s.









