Обмеження частоти запитів
Механізм, що обмежує кількість запитів від одного клієнта або IP за інтервал часу; при перевищенні повертає помилки.
Rate limiting - це обмеження сайту на кількість запитів, яку один клієнт може зробити за відведений час, наприклад 60 запитів на хвилину на IP. При перевищенні сервер відповідає помилкою (зазвичай HTTP 429 Too Many Requests) замість контенту, іноді із заголовком Retry-After, який повідомляє, скільки чекати.
Сайти використовують ліміти, щоб захистити інфраструктуру від перевантаження й уповільнити автоматизований збір. Ліміт зазвичай застосовується на IP-адресу, на API-ключ або на акаунт. Для скрапера найчастіше заважає ліміт на IP: надсилаєте занадто швидко з одного IP - його тротлять або тимчасово банять.
Ротація проксі - стандартне рішення. Розподіляючи запити по великому пулу IP, кожен окремий IP залишається значно нижче порога на IP, тож сумарна пропускна здатність може бути високою, але жодна адреса не впирається в ліміт. Саме для цього створені ротаційні резидентні проксі.
Однієї ротації замало, якщо патерн запитів усе одно має роботизований вигляд. Поєднуйте ротацію IP з людиноподібним темпом (випадкові затримки), реалістичними заголовками й повагою до відповідей Retry-After. Довбання навіть із ротацією може ввімкнути захист рівня всього сайту, що виходить за межі лімітів на IP.
Counting requests per something
Rate limiting caps how many requests a client may make in a period. The interesting part is what the site counts against: usually an address, sometimes an account, sometimes a fingerprint or a cookie, and often a combination.
Implementations differ in how they refuse. A well-behaved API returns 429 with a Retry-After header. A defensive site may return 403, serve a challenge, quietly slow your responses, or return reduced data with a success code.
The counter is what determines your options. If it counts per address, more addresses solve it. If it counts per account, more addresses do nothing and you need more accounts or less speed. Diagnosing which one you face is the whole task.
Spreading load properly
On our products the practical fix is almost always more exits rather than a better exit:
Same volume, different shape
Wrong 200 threads on login_c_US_s_1_ttl_15m -> 429
Right 200 threads across 50 session ids -> spread
Also drop the session suffix entirely -> new exit per connection
Static: cycle across the addresses you own and rest each one.- We do not cap concurrent connections. A limit you meet is the destination's, and more parallelism on one address makes it worse.
- Respect Retry-After when it is sent. Ignoring it turns a temporary limit into a longer one on many platforms.
- Randomise pacing. Perfectly even intervals are a signature in themselves.
- Watch for silent limiting: responses that get slower or thinner without an error code.
Rate limit confusions
A 429 is not a ban
It is a temporary refusal. Slowing down usually restores service; retrying hard usually does not.
Better addresses do not raise limits
Mobile at $5.76 a day counts the same as datacenter at $1.15 when the counter is per address.
Account limits ignore your proxies
If the counter is on the account, rotating addresses changes nothing.
Silent throttling is common
Not every limit announces itself, which is why response times are worth monitoring.
How a rate limit shows up, and what each signal means
Most people meet this term at the moment it happens to them. The signal you get tells you which response is correct, and guessing wrong usually makes it worse.
| What you see | What it usually means | What actually helps |
|---|---|---|
| 429 with a Retry-After header | The server is telling you exactly how long to wait | Wait that long. It is the cheapest correct answer and the one most clients ignore |
| 429 with no Retry-After | The same refusal without the courtesy | Back off and grow the gap each time, starting around a minute |
| 403 after a burst | The limiter stopped slowing you and started refusing you | Slow down and change nothing else for a while. New addresses tend to extend the block |
| A CAPTCHA where there was none | A score crossed a threshold, often per session rather than per request | Cut concurrency, keep sessions alive longer instead of opening new ones |
| Everything works but slowly | Silent throttling, no status code involved | Measure response times, not just status codes, or you will not notice for days |
| Partial or empty results | Some APIs degrade the answer instead of refusing it | Compare against a known-good baseline, otherwise you record the degraded data as real |
Two of these have nothing to do with which addresses you use. If the limit counts per account or per API key, every request carries that key no matter where it came from, and adding proxies changes nothing except your bill.
Questions people actually type
What does rate limited mean?
A service counted how often you asked for something and decided you crossed the line it allows. It then either slows you down, makes you wait, or refuses until the counter resets. It is a traffic rule, not a judgement about you.
Why am I rate limited on a website?
Usually because too many requests arrived too quickly from the thing the site counts, and the thing it counts is often not what you assume. It may be your address, your account, your API key, your session, or a combination. Working out which one is being counted is the whole problem.
How long does a rate limit last?
As long as the window it counts over. Some reset every minute, some every hour, some every day, and a few extend the wait each time you knock during it. A Retry-After header gives you the answer directly. Without one, assume minutes rather than seconds and stop retrying in a tight loop, which is what usually turns a short limit into a long one.
Will more proxies fix being rate limited?
Only if the limit counts per address. If it counts per account or per key, more addresses change nothing, because the key travels with every request. This is the most expensive mistake in the area: people buy traffic to solve a problem the traffic cannot reach.
Is being rate limited the same as being banned?
No. A rate limit is temporary and expects you back, which is why it usually tells you when to return. A ban is a decision about you rather than about your pace. Treating a 429 as a ban leads people to rotate everything and lose the session that would have worked a minute later.
What is HTTP 429?
The status code for too many requests. It is the polite version of a rate limit: the server refuses this request, stays willing to serve the next one, and often includes a Retry-After header saying when.
Пов'язані терміни
Дивись на практиці
Готовий використовувати обмеження частоти запитів?
SotaProxy надає доступ до ротуючих резидентських, мобільних, дата-центр та ISP проксі. Без мінімальних платежів.
Почати