Request Headers
Key-value metadata sent with every HTTP request, describing the client, accepted content, and context of the request.
Request headers are the metadata a client sends with every HTTP request, as key-value pairs preceding the body. They tell the server about the client and the request: User-Agent (the browser), Accept and Accept-Language (what content and language are wanted), Referer (the previous page), Cookie (session state), and many more. The server uses them to decide how to respond.
For scraping, headers are a major detection surface. Real browsers send a specific, consistent set of headers in a particular order. Default HTTP libraries send a sparse, unusual set - often just a bot user agent and little else. Anti-bot systems compare your headers against what a real browser would send and flag mismatches.
Getting headers right means more than setting a browser User-Agent. The full set must be coherent: Accept, Accept-Language, Accept-Encoding, and Sec-* headers should match the browser you claim to be, in the order that browser sends them. A Chrome user agent with headers Chrome would never send is an obvious tell.
Anonymity also lives in headers. A transparent proxy adds X-Forwarded-For revealing your real IP; an elite proxy sends none of these. When you check a proxy's anonymity, the checker inspects exactly which headers the proxy forwards to the destination - the difference between elite and transparent is written in the request headers.
The metadata that arrives with every request
Headers accompany each HTTP request and describe the client and what it wants: which content types it accepts, which languages, which encodings, whether it is continuing a session, where it came from.
Real browsers send a specific set in a specific order, and that order is remarkably stable per browser and version. HTTP libraries send fewer headers in a different order, which is a cheap and reliable signal for anyone looking.
Headers also have to agree with everything else. A German exit address sending Accept-Language: ru-RU while the TLS handshake says Python is not a visitor from Germany, and no single one of those signals had to be wrong for the combination to be.
Getting headers right alongside the address
The proxy chooses where you appear to be. Headers have to tell the same story:
A coherent set
Accept-Language: en-US,en;q=0.9 matches login_c_US
Accept-Encoding: gzip saves money on metered products
User-Agent: generated by the browser, not typed by hand
Referer: present when a person would have arrived from somewhere- Send Accept-Encoding: gzip on residential. Compression is the largest single saving available on a metered product.
- Match Accept-Language to the proxy country. Sites use it as much as the address to decide what to serve.
- Do not hand-assemble browser headers. Use a real browser or an impersonation library that also gets the order right.
- Verify what actually arrives: fetch a header echo service through the proxy and read the result.
Header misconceptions
Editing headers does not disguise a library
Order and TLS fingerprint still identify it, and now they disagree with your headers.
More headers is not more human
Real browsers send a specific set. Extra ones stand out.
A proxy does not add or remove headers here
Ours add none. What arrives is what your client sent.
Referer is not always harmless
Sending one that could not exist is a contradiction like any other.
See this in practice
Ready to use request headers?
SotaProxy gives you access to rotating residential, mobile, datacenter, and ISP proxies. No minimum commitment.
Get started