Track every competitor price change. In real time.

The arithmetic on a 50,000 SKU catalogue

Price monitoring is a daily job on a fixed list, so volume is predictable. A product page is around 50 KB gzipped once you block assets:

Retail sites split cleanly into two groups. Small shops and price aggregators serve datacenter addresses without complaint, so run those on a datacenter pool. Large marketplaces fingerprint hosting ranges and quietly serve stale or default prices, which is worse than being blocked because the data looks fine. Those go on residential.

Sticky per store, rotating per catalogue

Marketplaces tie currency, delivery estimates and sometimes price to the session. Hold one address for the duration of a store pass instead of rotating mid-catalogue:

Python: one sticky address per store pass

def store_proxy(store_index, country="US", city=None):
    login = f"login_c_{country}"
    if city:
        login += f"_city_{city}"
    login += f"_s_{store_index}_ttl_1h"
    return f"http://{login}:password@proxy.sotaproxy.com:10000"

# One address per store for the whole pass
for idx, store in enumerate(stores, start=1):
    proxy = store_proxy(idx, country=store.country, city=store.city)
    for sku in store.skus:
        fetch(sku, proxy)

• Prices differ by city on marketplaces with local warehouses. If you compare a New York price against a Warsaw address, the difference you are measuring is shipping, not competition.
• An hour of sticky covers a full pass over a mid-size catalogue. Rotating inside it resets the currency and the delivery estimate.
• Store the address you used next to every price you collect. Without it you cannot explain outliers later.
• Spread the run across the day. Fifty thousand requests in ten minutes looks like an attack, the same run over six hours looks like traffic.

How price data goes wrong without anyone noticing