API Rate Limit Calculator

RPM limits how many requests can start in one minute, while TPM limits the token volume processed in that minute. An AI API can remain under RPM and still exceed TPM when requests contain large prompts or outputs.

Concurrency measures requests in flight at the same time. Higher latency keeps requests open longer, so a workload can hit a concurrency ceiling even when its requests-per-minute rate appears acceptable.

Size the queue for short, expected bursts rather than sustained overload. The queue should have a maximum wait time and reject excess work with a clear retry response before latency becomes unacceptable.

Keeping 15% to 30% of upstream capacity in reserve is a common starting point. The reserve covers retries, monitoring calls, traffic spikes, and measurement error without immediately throttling customers.

API rate limits affect pricing because they define how much capacity a customer can consume in a short window, not just across a month. A plan with 1,000,000 included requests can still be expensive to operate if customers send all traffic during peak minutes. Price and limit together: monthly quota controls total cost, RPM and TPM control burst cost, and concurrency controls in-flight load. For example, a Growth plan might include more monthly usage plus a higher RPM limit, while overage handles sustained consumption. This keeps paid API pricing aligned with both volume and infrastructure pressure.