How to track competitor prices and stock every day

The short answer: keep a list of product URLs. Once a day, fetch each page as JSON with a few extract rules (price, stock, SKU), save the result as a snapshot, and compare it with the last one. Print or send only what changed. On Shopify and WooCommerce stores, try the free JSON routes first. A product page costs about 3 credits on the datacenter route, from our price table (that route was off in our test), so 100 products a day is about 300 credits. On residential, a page was billed 24 credits.

The whole tracker is four steps:

1. A URL list: one product page per line, in urls.txt.
2. One request per URL: response_types: "json" with extract rules, so you get fields, not HTML.
3. A snapshot and a diff: today’s fields saved to a file, compared with the last file.
4. An alert: the changed lines, printed or sent where you read them.

The free routes first

Many stores give you this data for free. Try these before you pay anyone, us included:

• Shopify: /products.json and /products/<handle>.json. Each variant has price, compare_at_price, sku and available. More in How to scrape any Shopify store.
• WooCommerce: the Store API at /wp-json/wc/store/v1/products, with prices, sku and is_in_stock. More in How to scrape any WooCommerce store.

curl -s "https://www.allbirds.com/products.json?limit=250&page=1"
curl -s "https://porterandyork.com/wp-json/wc/store/v1/products?per_page=1"

On 2 Oct 2026 these free routes answered on all five stores in this post. They are not enough when the store turned them off, blocks your script with a 403 or a bot check, answers 429 after a few calls, or when you need the price the shopper sees on the page. Then fetch the page itself.

Check a store first

Before you track a store, ask about the host. It is free. Over MCP, your agent calls check_site:

{"name": "check_site", "arguments": {"host": "porterandyork.com"}}

From cURL, the same answer is two calls: /v1/tier for the price, /v1/stats/domain for how our routes have done there lately.

curl -s -H "Authorization: Bearer YOUR_KEY" \
  "https://get.megalania.io/v1/tier?host=porterandyork.com"
curl -s -H "Authorization: Bearer YOUR_KEY" \
  "https://get.megalania.io/v1/stats/domain?host=porterandyork.com&days=30"

This is what /v1/tier answered us:

{"host": "porterandyork.com", "tier": null, "temporary": null, "price": null, "pending": null,
 "quotes": {"residential": 15}, "capabilities": {"http": ["residential"], "browser": []}}

tier: null means we have not priced this host yet, so a page pays the price of the route that delivers it, and quotes shows that number. Our test key had the datacenter routes off, so it shows residential only. Yours also shows datacenter, which is cheaper. All five stores in this post answered tier: null, and /v1/stats/domain had no requests for them yet: we had no history there. Check a site first explains every field.

One product, one request (cURL)

The body is the exact one we ran, on a Shopify store. Send it with a heredoc, so the quotes in the rules need no shell escaping:

curl -sS https://get.megalania.io/v1 \
  -H "Authorization: Bearer YOUR_KEY" \
  -H "Content-Type: application/json" \
  -D headers.txt \
  --data @- <<'JSON'
{
  "url": "https://www.allbirds.com/products/mens-dasher-nz",
  "response_types": "json",
  "extract": "title:meta[property=\"og:title\"]@content|price:meta[property=\"og:price:amount\"]@content|currency:meta[property=\"og:price:currency\"]@content|availability:script[type=\"application/ld+json\"]@json(offers.availability)|availability_variant:script[type=\"application/ld+json\"]@json(offers.0.availability)|sku:script[type=\"application/ld+json\"]@json(sku)",
  "max_credits": "64"
}
JSON

On WooCommerce stores we sent the same body with WooCommerce’s own class names:

name:h1.product_title|price:p.price ins .amount, p.price > .amount|regular_price:p.price del .amount|stock:p.stock|add_to_cart:form.cart button.single_add_to_cart_button|sku:.sku_wrapper .sku

Why these rules work on every theme is in the Shopify and WooCommerce posts. A rule that matches nothing comes back as null. max_credits is the most this page may cost; we skip any route above it.

Python: snapshot, diff, alert

One file. It reads urls.txt, picks the Shopify or WooCommerce rules by the URL path (/products/ or /product/), saves today’s fields to snapshots/, and prints every field that changed since the last snapshot.

# pip install requests
# Run once a day: MEG_API_KEY=YOUR_KEY python track.py urls.txt
import datetime
import json
import os
import pathlib
import sys
import uuid

import requests

LD = 'script[type="application/ld+json"]'
SHOPIFY = "|".join([
    'title:meta[property="og:title"]@content',
    'price:meta[property="og:price:amount"]@content',
    'currency:meta[property="og:price:currency"]@content',
    f"availability:{LD}@json(offers.availability)",
    f"availability_variant:{LD}@json(offers.0.availability)",
    f"sku:{LD}@json(sku)",
])
WOO = "|".join([
    "name:h1.product_title",
    "price:p.price ins .amount, p.price > .amount",
    "regular_price:p.price del .amount",
    "stock:p.stock",
    "add_to_cart:form.cart button.single_add_to_cart_button",
    "sku:.sku_wrapper .sku",
])
WATCH = ["price", "regular_price", "availability", "availability_variant", "stock", "add_to_cart", "sku"]


def fetch(url):
    rules = WOO if "/product/" in url else SHOPIFY
    resp = requests.post(
        "https://get.megalania.io/v1",
        headers={
            "Authorization": f"Bearer {os.environ['MEG_API_KEY']}",
            "Idempotency-Key": str(uuid.uuid4()),
        },
        json={"url": url, "response_types": "json", "extract": rules, "max_credits": "64"},
        timeout=90,
    )
    if resp.status_code != 200:
        err = resp.json()["error"]
        print(f"{url}: {resp.status_code} {err['code']}: {err['message']}")
        return None
        return None
    return resp.json()


snapshots = pathlib.Path("snapshots")
snapshots.mkdir(exist_ok=True)
previous = sorted(snapshots.glob("*.json"))
old = json.loads(previous[-1].read_text()) if previous else {}

today = {}
for url in pathlib.Path(sys.argv[1]).read_text().split():
    item = fetch(url)
    if item is not None:
        today[url] = item

stamp = datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%dT%H%M")
(snapshots / f"{stamp}.json").write_text(json.dumps(today, indent=1))

compared = [url for url in today if url in old]
changes = [
    (url, key, old[url].get(key), today[url].get(key))
    for url in compared
    for key in WATCH
    if old[url].get(key) != today[url].get(key)
]
for url, key, before, after in changes:
    print(f"CHANGED {url} {key}: {before!r} -> {after!r}")
print(f"{len(today)} fetched, {len(compared)} compared, {len(changes)} changes")

A block is HTTP 422 with error.code blocked and X-Billing: blocked, and it costs 0 credits, so the status check catches it. X-Hops on that answer lists every route we tried. A page that fails today is left out of today’s snapshot, so it is not compared; it comes back the next day. The snapshot name has the minute, not the second: two runs in the same minute write the same file, and the second run overwrites the first. Once a day, that never happens.

Run it from cron once a day:

0 6 * * * cd /srv/tracker && MEG_API_KEY=YOUR_KEY python track.py urls.txt >> track.log

For the alert, send the CHANGED lines where your team reads them: a chat webhook, an email, or a row in your database.

What our two runs found

We fetched the same 20 product pages twice, about 28 hours apart: 3 Oct 2026 at 00:07 UTC and 4 Oct 2026 at 04:31 UTC. Then we compared the same seven fields the script watches.

Nothing changed. That is a normal day for most products, and it is the honest result. It also tells you something useful: once a day is enough for most stores. A tracker that fetches every hour pays 24 times as much to learn the same thing. Fetch more often only when you know prices move, for example during a sale week.

regular_price is a WooCommerce field: it holds the old price during a sale. On a Shopify page the old (compare-at) price is not in the meta tags or the JSON-LD, so take it from /products.json if you need it.

What it costs per day

We fetch on the cheapest route first: plain HTTP on a datacenter exit (h:dc). If the store blocks it, we climb to a residential exit (h:res). The response headers say what happened:

x-credits-charged: 24
x-engine: http
x-hops: h:res:chrome=3653
x-identity: chrome
x-route: http/residential
x-target-status: 200

Measured on our dev cluster with the datacenter routes off, so this climb starts at h:res.

A page like this costs about 2-3 credits on h:dc (45-126 KiB, our five stores), worked out from our price table on 4 Oct 2026, because that route was off in our test; run check_site (free) to see the price for your site.

A page like this was billed 16-24 credits on h:res (45-126 KiB, our five stores) in our test on 3 and 4 Oct 2026; run check_site (free) to see the price for your site.

How the 3 adds up: 1 for the page, 1 for JSON output, and 1 for the started 128 KiB above the 64 KiB included. On h:res it is 15 on a host we have not priced yet, plus 1 for JSON, plus 8 for the extra block. One store had smaller pages (45-47 KiB), so its pages cost 2 on h:dc and 16 on h:res. Our second run of 20 pages was billed 448 credits in total, all on h:res.

The 76,000-credit beta grant covers 100 products a day for 30 days, even if every page needs a residential exit. The size is the bytes on the wire, which is what the price uses.

How we measured: 20 product pages, 4 each on www.allbirds.com, colourpop.com and www.deathwishcoffee.com (Shopify), and barefootbuttons.com and porterandyork.com (WooCommerce). Run 1 on 3 Oct 2026, 00:06 to 00:07 UTC; run 2 on 4 Oct 2026, 04:31 UTC. One request per page with the bodies above, not a batch.

More than a few hundred products

One request per product is fine for a few hundred pages. For more, send the same body as one batch with a urls list: Batches. Nothing is charged at submit; each URL is charged when we fetch it.

Being polite: rate and terms

• Blocks we detect are free. The answer carries X-Billing: blocked and costs 0 credits. Each try shows in X-Hops.
• Errors are free. Every error has a code and a fix in Error codes.
• Keep your rate low. Once a day, a few seconds between pages on the same store. Our runs left 2.5 seconds between pages on one host.

This is not legal advice. Track public product pages only. Do not log in, and do not get past a password. Read each store’s terms and robots.txt, and follow them. Do not collect personal data, such as reviewer names.