How to Build an Amazon Review Scraper That Actually Works

If you're staring at a login wall, a few featured reviews, and a proxy pool that starts burning the moment you scale past test traffic, you already know the problem. An Amazon review scraper isn't a simple HTML parser anymore. It's a pipeline that has to survive Amazon's anti-bot stack, handle partial visibility, and still give analysts something clean enough to use for sentiment reads, competitor monitoring, and product research.

Table of Contents

• What an Amazon Review Scraper Is Up Against in 2026
• Building the Discovery and Extraction Pipeline
  • The flow that survives real workloads
  • Where retries belong
• Choosing the Right Proxy Class for Review Scraping
  • Match the proxy to the job
• Anti-Blocking Tactics That Hold Up Under Load
  • The parts that actually matter
  • Where Sota Proxy fits
• Pagination, Parsing, and Cross-Marketplace Normalization
  • Pagination patterns you'll actually see
  • What belongs in the canonical row
• Legal and Ethical Lines You Cannot Cross
  • The line operators actually cross
• Monitoring, Storage, and Scaling the Pipeline
  • The operating stack
  • Scaling checklist

What an Amazon Review Scraper Is Up Against in 2026

You can still get a quick read on a product page, but the old “grab the reviews and move on” workflow is gone. On November 5, 2024, Amazon moved nearly all reviews behind a login wall, and the /product-reviews/ endpoint now redirects unauthenticated traffic to a sign-in page. What remains public is usually only 3 to 8 featured reviews on the product detail page, plus the aggregate rating, total count, and the 5-star to 1-star distribution. That shift changed review scraping from broad extraction to constrained data access, especially for teams doing sentiment analysis, competitor monitoring, or product research across markets, as described in the guide on Amazon's review access changes. Amazon review access changes

The practical result is ugly. Your first run looks fine, then the same ASIN starts returning partial data, blank review lists, or a sign-in page your parser wasn't built to recognize. That's why the workflow has to account for login walls, throttle behavior, and fallback logic from the start instead of treating them as edge cases. If you're already dealing with product-page review disputes, the resource on how to remove an Amazon review gives useful context on how Amazon's review surface behaves from the seller side too.

Practical rule: if a review page looks easy on the first URL, assume the harder path shows up as soon as you paginate, change locale, or reuse the session.

The bigger trap is cross-marketplace drift. A pipeline that looks healthy on one domain can still fail the moment you compare amazon.com against amazon.co.uk, amazon.fr, or amazon.de. Amazon's defenses aren't just about blocking requests, they also reshape what data is visible, where it appears, and how much work it takes to normalize it into one usable dataset.

If you need a quick reference for bot screening behavior, the internal guide on anti-bot systems is worth keeping open while you debug your scraper. The point isn't just to fetch reviews. It's to fetch the right slice, in the right market, before the session gets flagged.

Building the Discovery and Extraction Pipeline

A production-grade Amazon review scraper should not start with parsing. It should start with discovery. ScrapeOps' playbook recommends a two-step flow, first resolve the product's actual review URL from the ASIN or search results, then paginate through the review section and parse stable fields such as rating, author, date, body, and verified-purchase flag. Amazon review pages also expose these fields in structured outputs that can land cleanly in CSV or JSON. ScrapeOps Amazon reviews scraper workflow

The flow that survives real workloads

Treat the ASIN as the entry point, not the target. The search result or product page gives you the route to the review endpoint, and that route can vary by marketplace, locale, and page state. Once you have the review URL, the scraper should loop through review pages, collect stable fields, and stop cleanly when pagination ends or the page shape changes.

import requests
from bs4 import BeautifulSoup

def get_review_page(review_url, headers=None, cookies=None):
    resp = requests.get(review_url, headers=headers, cookies=cookies, timeout=30)
    resp.raise_for_status()
    return resp.text

def parse_reviews(html):
    soup = BeautifulSoup(html, "html.parser")
    rows = []
    for card in soup.select("[data-hook='review']"):
        rows.append({
            "rating": card.select_one("[data-hook='review-star-rating']").get_text(strip=True) if card.select_one("[data-hook='review-star-rating']") else "",
            "author": card.select_one(".a-profile-name").get_text(strip=True) if card.select_one(".a-profile-name") else "",
            "date": card.select_one("[data-hook='review-date']").get_text(strip=True) if card.select_one("[data-hook='review-date']") else "",
            "body": card.select_one("[data-hook='review-body']").get_text(" ", strip=True) if card.select_one("[data-hook='review-body']") else "",
            "verified_purchase": bool(card.select_one("[data-hook='avp-badge']"))
        })
    return rows

Use raw HTML parsing first. Switch to rendered DOM only when the page body or review container stops appearing in the server response.

Where retries belong

Retries should wrap the fetch, not the parser. If a response comes back malformed, blocked, or redirected, the fetch layer decides whether to rotate session state, change proxy class, or back off. The parser should only see a page that you already decided is worth processing.

Field	Source on page	Storage shape
Rating	Review card star element	Integer or string
Author	Profile name block	Text
Date	Review date line	Text
Body	Review body container	Text
Verified purchase	Badge presence	Boolean

For teams that prefer browser automation, a Playwright fallback makes sense when the DOM needs JS rendering or cookie persistence. The internal Python crawling guide on browser-based crawling patterns is a useful companion if you're wiring requests and browser fallback into the same worker pool. The important part is separation of concerns, discovery first, extraction second, retries around transport, not around business logic.

Choosing the Right Proxy Class for Review Scraping

Amazon doesn't punish every proxy class the same way. Datacenter and IPv6 ranges are cheap and fast, but they get burned quickly on bot screens. Residential IPs last longer because they look like normal household traffic, although the rotation can get noisy if you overuse a subnet. Mobile IPs inherit carrier trust and usually survive the hardest pages better, which matters when your scraper lands on login-protected review flows that keep tripping friction. The proxy taxonomy overview on proxy types maps cleanly to that reality.

Match the proxy to the job

If the task is ASIN discovery, datacenter can still make sense because the block risk is lower and the page cost is cheap. If you're pulling review pages at scale, residential should be the workhorse. If a product family keeps bouncing into CAPTCHA loops, mobile is the rescue layer. IPv6 can still help in controlled internal tooling, but Amazon's defenses usually make it the least forgiving option for review collection.

Proxy Class	Best Review Workload	Typical Block Behavior	Speed
Residential	Main review collection	Survives longer, still flagged if patterns repeat	Moderate
Mobile	Stubborn ASINs and login-adjacent pages	Highest trust, least friction in practice	Moderate
Datacenter	ASIN discovery and low-risk fetches	Burned fastest on bot screens	Fast
IPv6	Limited controlled testing	Often unstable against aggressive defenses	Fast

Practical rule: don't use your best proxy class for every stage. Spend the cheaper ranges on discovery, then reserve stronger traffic for review fetches that actually need it.

The right stack also depends on how you run sessions. Sticky assignment helps when a single ASIN needs continuity across multiple pages. Rotation helps when Amazon starts correlating repeated patterns. For operators already running AdsPower, Dolphin Anty, GoLogin, Multilogin, or Hidemyacc, the proxy layer has to fit the same session model as the browser profile, or you'll lose trust before the scraper reaches page two.

Anti-Blocking Tactics That Hold Up Under Load

The run usually fails where teams get careless. Amazon does not need to block every request. It only needs to add enough friction that your session starts looking scripted. The operating pattern that survives best under load uses paced requests, proxy rotation on a schedule, backoff when CAPTCHA behavior rises, residential proxies for the main fetch path, stealth browser headers, and cookies that stay consistent within each session. Amazon review scraping tactics

The parts that actually matter

User-agent strings matter less than many operators expect. Header fingerprints and TLS fingerprints usually expose the scraper first, especially when the browser says one thing and the connection stack behaves like something else. If you already run Facebook and TikTok ad accounts, manage account farming, or use cloaking for geo-targeted campaigns, you know how fast a mismatched fingerprint can break trust across an entire profile.

Session cookies carry the rest of the load. Amazon links behavior to continuity, so a stable cookie jar can keep a session coherent long enough to move through review pages without forcing a fresh trust check on every request. That is why proxy choice and browser choice have to be designed together, not treated as separate problems.

Keep session identity boring. Most blocks show up when the scraper changes too much, too fast.

Where Sota Proxy fits

Sota Proxy is useful for rotation control, sticky session control, and location targeting that matches the browser profile already in use. That matters for media buyers running geo-targeted campaigns, and it matters just as much for agencies that need a review pipeline attached to a multi-profile setup without burning the same IP reputation across every worker. For teams that want a tighter handle on session turnover, the proxy IP rotation guidance is the practical reference point for deciding when to keep an IP sticky and when to move on. If you are monetizing the workflow or passing the stack to peers, Sota Proxy's referral program, with up to 40% commission, is the kind of detail teams discuss after the technical controls are already in place.

The anti-blocking rule is simple. Do not fight Amazon with raw request volume. Spread the load, keep cookies stable, move proxies on a schedule, and back off the moment CAPTCHA behavior starts rising.

Pagination, Parsing, and Cross-Marketplace Normalization

The under-discussed problem isn't just pagination. It's that review data doesn't overlap across marketplaces such as amazon.com, amazon.co.uk, amazon.fr, and amazon.de, so scraping Amazon reviews is really multiple market-specific datasets. The actual work is market segmentation and normalization, not just extraction speed. Cross-marketplace review scraper context

Pagination patterns you'll actually see

Amazon review pages tend to expose a page-number flow, a next-page token, or a limited featured block that behaves like a small, fixed list. Your crawler should detect which pattern it has, then move through the list without assuming the same structure exists across every marketplace. That matters because locale switching can change not just the language, but the page shape itself.

A clean normalization layer should map every review into one canonical record. Keep the locale raw field, the marketplace domain, and the original date string, then normalize downstream for analysis. That prevents silent corruption when one market formats review dates or rating strings differently from another.

Market segmentation beats raw speed when the end goal is geo-targeted ad creative testing or international pricing research. A faster scrape of the wrong locale still gives you the wrong answer.

What belongs in the canonical row

• Marketplace domain: Keep amazon.com, amazon.co.uk, amazon.fr, and amazon.de separate at ingest.
• Raw review text: Store the source text before translation or truncation.
• Locale metadata: Preserve language and region so analysts can compare like with like.
• Rating and dates: Normalize these after ingestion, not during fetch.
• Product identifier: Keep the ASIN tied to the marketplace, not as a global unique key.

The engineering decision that saves the most pain is to treat locale as a first-class dimension. That lets one pipeline feed competitor research, ad testing, and brand monitoring without mixing markets that shouldn't be mixed. If the reporting team wants a single view, give them one, but build it from market-aware rows, not from a merged blur.

Legal and Ethical Lines You Cannot Cross

Amazon's terms are explicit. The Conditions of Use prohibit using “any robot, spider, scraper, or other automated means” to access Amazon Services without prior written permission, and Amazon's scraping-policy language also bans “data mining, robots, screen scraping, or similar data gathering and extraction tools.” Public product data, prices, ratings, search results, and limited review excerpts sit in the lower-risk bucket, while login-protected content, buyer order history, and Seller Central move into much riskier territory under the CFAA. Amazon scraping policy breakdown

The line operators actually cross

The hard boundary isn't whether a page looks visible in a browser. The boundary is whether your scraper needs a login wall, CAPTCHA bypass, or a privileged session to keep going. Once it does, the compliance posture gets worse fast, especially if you're running the workflow through multiple buyer accounts or trying to use antidetect browsers to hide that the same team controls the session set.

That's the point where teams should slow down and reassess. If the data you need is available as public snippets, use the public snippets. If you need logged-in content, the risk and the operational burden both climb sharply.

For validation work, the guide on how to vet Amazon review authenticity is helpful because it keeps attention on signal quality, not just collection volume. Review ingestion is only useful if the downstream team can trust what they're seeing.

If the scraper needs a login to do its job, the legal and contractual risk changes immediately.

The internal note on whether web scraping is legal belongs in every review team's runbook. Not because it gives permission, but because it forces the right question early, before the pipeline is already in production and tied to a campaign budget.

Monitoring, Storage, and Scaling the Pipeline

A review pipeline breaks in boring places. Markup changes, success rates sag, CAPTCHA frequency jumps, and the team notices only after the dataset is already stale. One 2026 benchmark summary reported the highest success rate across tested providers reached 96% for Amazon review scraping, while Decodo recorded an 11% success rate on Amazon with an average completion time of 10 seconds on the URLs it processed. That spread is a reminder that anti-blocking and retry logic matter more than raw HTML parsing. Amazon review scraping benchmark summary

The operating stack

Use Postgres for normalized review rows, S3 for raw HTML snapshots, and a queue between workers so retries don't block the whole run. Alert on sudden drops in successful fetches, CAPTCHA spikes, and shifts in page structure. If one marketplace starts drifting, degrade gracefully instead of forcing the same parser across every locale.

Scaling checklist

• Per-ASIN worker pools: Keep failure domains small so one bad product doesn't poison the whole batch.
• Regional proxy buckets: Split traffic by marketplace and locale instead of mixing everything in one pool.
• Structural diffing: Compare the current page shape against the last known good snapshot before you trust the parse.
• Retry budgets: Cap retries per ASIN so a blocked route doesn't consume the entire job window.
• Fallback storage: Save the raw page even when parsing fails, because the parser can catch up later.

If you want an example of what controlled scale looks like in a data-heavy environment, the Xr Voyage FalkorDB case study shows how teams think about growth without losing manageability. The same discipline applies here. Keep the collection layer narrow, the storage layer durable, and the monitoring layer noisy enough that you spot breakage before clients do.

---

If you need a proxy stack built for review pipelines, geo-targeted workloads, and session-sensitive scraping, Sota Proxy gives you residential, mobile, ISP, and datacenter coverage with sticky and rotating options that fit this use case. Set it up for the marketplace and browser profile you're already running, then use it to keep your Amazon review collection stable instead of chasing blocks all day.