Scrape any site. No blocks, no CAPTCHAs.
Websites ban scrapers by IP. Rotate through millions of residential IPs and they never see the same address twice.
Why proxies solve this
Every web scraper hits IP bans eventually. Sites track request frequency per IP and block anything that looks automated. A residential proxy pool gives you a fresh IP from a real ISP for each request - the target server sees a regular user, not a bot.
The problems you hit without proxies
IP bans after 50–100 requests
Most e-commerce and news sites block a single IP after detecting scraping patterns. Rotating proxies spread requests across thousands of IPs so no single one hits the threshold.
CAPTCHAs on high-value pages
Datacenter IPs trigger CAPTCHAs because their ASNs are known. Residential IPs from real ISPs have higher trust scores and rarely see CAPTCHA challenges.
JavaScript rendering blocks
Some sites only serve full content to browsers. Pair proxies with headless Chrome (Puppeteer/Playwright) - configure the proxy at the browser level, not the request level.
Geo-locked data
Price data, search results, and content vary by country. Use city-level targeting to pull data from any specific market.
How SotaProxy handles it
SotaProxy residential proxies rotate automatically on each request or hold a sticky session for up to 30 minutes when you need to scrape paginated content. The pool covers 220+ countries with city-level targeting. Success rates stay above 99% for most targets.
Setup in 4 steps
Pick proxy type
Residential for sites that aggressively block scrapers. Datacenter for high-volume, low-risk targets where speed matters more than stealth.
Set rotation mode
Rotating session: new IP per request. Sticky session: same IP for 1–30 minutes. Use sticky for multi-step scrapes (login → navigate → extract).
Configure your scraper
Set the proxy endpoint in your HTTP client (requests, httpx, Scrapy, Playwright). Use HTTP CONNECT for HTTPS targets.
Add geo-targeting
Append country or city code to the username parameter. Pull data specific to any market without a VPN.
What a million pages actually costs
Proxy choice is an arithmetic problem before it is a technical one. Here is the same job priced both ways, using our list prices and a page that weighs 80 KB as gzipped HTML:
Datacenter wins on price whenever the target lets it through, and the gap grows with volume. Residential earns its price on sites that reject hosting ranges outright, and on those the choice is not between $27 and $76, it is between $76 and no data at all.
A configuration that survives contact with a real site
Start on datacenter, measure your block rate, and move only the targets that fail. Rotating residential needs no session id at all, which is what makes it cheap to run:
Python: datacenter pool with a residential fallback
import requests
from itertools import cycle
# Datacenter pool: rotate yourself
pool = cycle([
"http://login:password@198.51.100.20:50100",
"http://login:password@198.51.100.21:50100",
])
# Residential fallback: no session id means a new IP per request
RESI = "http://login_c_US:password@proxy.sotaproxy.com:10000"
headers = {"Accept-Encoding": "gzip"}
def fetch(url, use_resi=False):
proxy = RESI if use_resi else next(pool)
r = requests.get(url, proxies={"http": proxy, "https": proxy},
headers=headers, timeout=30)
if r.status_code == 403 and not use_resi:
return fetch(url, use_resi=True)
return r- Ask for gzip. It cuts an HTML page three to four times, and on residential that is the difference between $76 and $250 for the same run.
- Block images, fonts and video in your headless browser. They carry most of the weight and none of the data.
- Thirty second timeout. A residential route takes longer to build than a datacenter one, and a five second timeout throws away requests that would have succeeded.
- Spread load across addresses instead of speeding up one. One IP behaving like fifty users is the exact pattern rate limiters catch.
Four ways people burn money here
Retrying a 403 forever on datacenter
If the site rejects hosting ranges, every retry costs traffic and changes nothing. Detect the pattern once and route that domain to residential.
Running the whole crawl on residential
Most sites in a typical crawl never check IP reputation. Paying per gigabyte for those pages is a donation.
Downloading assets you never parse
A product page is 80 KB of HTML and 300 KB of images. Both are billed the same.
One address, maximum threads
You will hit 429 long before you hit our limits. There is no cap on concurrent connections, so use more addresses.
Frequently asked questions
Which proxy type is best for web scraping?
Residential proxies for sites with anti-bot protection (e-commerce, social media, travel). Datacenter proxies for high-volume scraping of targets that do not actively block cloud IPs.
Do rotating proxies work with Scrapy?
Yes. Use the rotating proxy middleware or set the proxy directly in DOWNLOADER_MIDDLEWARES. Set the endpoint once - rotation happens server-side.
How do I scrape JavaScript-rendered pages?
Use Playwright or Puppeteer with proxy configuration at the browser launch level. Set --proxy-server=your-proxy-endpoint when launching the browser.
What success rate can I expect?
For most e-commerce and news sites: 95–99%. For Google, Amazon, and LinkedIn: 90–95% with residential proxies. Lower with datacenter IPs against those specific targets.
Is web scraping legal?
Scraping publicly available data is generally legal in most jurisdictions. Always check a site's robots.txt and Terms of Service for restrictions.
Related use cases
Ready to start?
Create an account, top up, and get proxy credentials in minutes. No sales call. No monthly minimum.
Create account