How to stay inside someone else’s rate limits without stalling your own pipeline — client-side throttling, retry ordering, Retry-After, and the multi-instance trap.
When you integrate with a marketplace, a courier or a payment provider, you inherit a constraint you cannot negotiate: their rate limits. Exceed them and you do not simply get slower — you get 429s, then temporary bans, then, on some platforms, a conversation with someone about your integration’s behaviour.
The instinct is to add a retry policy and move on. That instinct is wrong, and it is worth being precise about why.
Because a 429 is the receipt. By the time it arrives, the request has already been sent, counted and rejected. Retrying it consumes another slot in the next window.
Worse, under load this behaviour is self-reinforcing. Every rejected call comes back and competes with the fresh calls that were already queued, so the queue grows faster than it drains. A retry policy without a limiter in front of it turns a temporary overload into a sustained one.
Retry is still necessary — networks fail, servers restart. But it handles a different problem. Rate limiting is preventive; retry is corrective. You need both, in that order.
The mechanism is a counter with a time-based release. SemaphoreSlim gives you exactly that, and it is async-friendly, which matters because you want to wait, not block a thread pool thread:
public sealed class SlidingWindowLimiter : IDisposable
{
private readonly SemaphoreSlim _slots;
private readonly TimeSpan _window;
public SlidingWindowLimiter(int permitsPerWindow, TimeSpan window)
{
_slots = new SemaphoreSlim(permitsPerWindow, permitsPerWindow);
_window = window;
}
public async Task<IDisposable> AcquireAsync(CancellationToken ct)
{
await _slots.WaitAsync(ct);
// The slot returns exactly one window after it was taken — not after
// the request completes. A slow response must not shrink your budget.
return new Releaser(_slots, _window);
}
}The comment is the part that matters. The naive implementation releases the slot when the HTTP call finishes, which couples your throughput to the other side’s latency: their bad afternoon becomes your backlog. Releasing on a timer decouples them.
Two details to get right:
Task.Delay(...).ContinueWith(...) inside a scope that gets cancelled, slots leak and the limiter slowly strangles itself. Use a timer that is not tied to the request’s CancellationToken;SemaphoreSlim is not fair. It does not guarantee FIFO ordering. In practice this is fine for background sync; if you have a latency-sensitive path sharing the same limiter, it can starve. That is an argument for separate buckets, which brings us to the next point.Most real APIs do not publish a single number. They limit different endpoint families differently — order operations are usually tighter than catalogue reads, because they are more expensive on their side.
If you model that with one global limiter, you have to size it for the strictest limit, and your catalogue sync crawls for no reason. Two limiters, keyed by endpoint class, let each family run at its own ceiling:
private readonly SlidingWindowLimiter _orders = new(perWindow: 12, TimeSpan.FromSeconds(1));
private readonly SlidingWindowLimiter _standard = new(perWindow: 3, TimeSpan.FromSeconds(1));The numbers come from their documentation, with margin. Configure them, do not hard-code them — published limits change, and you do not want a redeploy to be the only way to react.
In a DelegatingHandler on the HttpClient, not in your service classes.
This is the same argument as global query filters for tenant isolation: if staying within the limits depends on a developer remembering to call the limiter, it is not a guarantee, it is a hope. A handler sits underneath every call made through that client, including the ones added next year by someone who has never read this article.
services.AddHttpClient<IMarketplaceClient, MarketplaceClient>()
.AddHttpMessageHandler<RateLimitingHandler>()
.AddResilienceHandler("marketplace", ConfigurePipeline);This is the question people get wrong most often, and Polly v8 makes the answer explicit: the first strategy added is the outermost one, and each subsequent strategy sits inside it.
| Layer | Position | Why there |
|---|---|---|
| Fallback | Outermost | It must see the final outcome, after everything else has given up |
| Retry | Outside the limiter | So a retried attempt goes back through the throttle |
| Circuit breaker | Inside retry | Otherwise the retry keeps hammering a service you have already declared dead |
| Rate limiter | Inside the breaker | Every attempt that reaches the wire pays the toll |
| Timeout (per attempt) | Innermost | It should bound one HTTP call, not the whole retry sequence |
The critical one is retry sitting outside the limiter. Invert those two and your retries bypass the throttle entirely — which is precisely the failure mode you built the limiter to prevent.
If you also want an overall deadline across all attempts, that is a second timeout at the very outside, not a replacement for the per-attempt one.
If the server tells you how long to wait, that number beats anything you compute. It is authoritative; your backoff curve is a guess.
Polly v8 lets you read it in the delay generator:
DelayGenerator = args =>
{
var retryAfter = args.Outcome.Result?.Headers.RetryAfter;
var delay = retryAfter?.Delta
?? (retryAfter?.Date - DateTimeOffset.UtcNow);
return ValueTask.FromResult(delay); // null → fall back to the configured backoff
}Note that Retry-After comes in two forms — a delay in seconds and an absolute HTTP date — and providers use both. Handling only Delta means silently ignoring half of them.
Otherwise: exponential backoff with jitter (UseJitter = true). Without jitter, every instance that got throttled at the same moment retries at the same moment, and you have rebuilt the spike you were trying to smooth.
This is the honest limitation, and it deserves stating plainly: SemaphoreSlim is per-process. Run three instances of your service and you are making three times the configured rate against an API that counts you once.
The options, in ascending order of effort:
| Approach | Cost | When it fits |
|---|---|---|
Partition the budget — each instance gets limit / N |
Trivial | Fixed instance count; wastes capacity when instances are idle |
| Pin the integration to a single worker | Small | Background sync that does not need to scale horizontally |
| Distributed limiter (Redis token bucket) | Real | Autoscaling, or the limit is tight enough that waste is expensive |
I have deliberately started with the first two on more than one project. An in-process limiter plus a single-writer topology is simpler to reason about than a distributed one, and if the integration is a background pipeline rather than a user-facing path, horizontal scaling was never the point. Take the Redis dependency when you actually need it — but know which of the three you are on, because “we scaled and started getting banned” is a bad way to find out.
POST that creates a shipment may well have succeeded before the timeout. Retry it only if the provider supports an idempotency key, or if you can check-then-act..NET 7 added System.Threading.RateLimiting, with FixedWindowRateLimiter, SlidingWindowRateLimiter, TokenBucketRateLimiter and ConcurrencyLimiter — and Polly v8 has a rate-limiter strategy built on it. For most cases, reach for those first.
One practical note, because it catches people: those types ship in the ASP.NET Core shared framework but not in the base one. In a console app or a worker service you need the System.Threading.RateLimiting package explicitly, and the compiler error you get first (“the namespace does not exist”) reads like a version problem rather than a missing reference.
The hand-rolled version earns its place when your constraint does not match the standard shapes: a limit expressed per endpoint family, a budget shared between a real-time path and a batch path with different priorities, or a provider whose effective limit is discovered empirically rather than documented. That was the situation on the marketplace integration — two endpoint families with different ceilings, one of them stricter than the published figure. Start with the built-in limiter; write your own when you can name the reason.
If you are integrating with an API that fights back and want a second opinion on the resilience layer before it reaches production, that is a conversation I am glad to have.
If you are building in this space, let us talk for 30 minutes.
Book a call