How to scrape it. With code that runs.
Each guide answers one question, shows the request, and says what a page costs.
Make a request leave from Germany, Japan or Brazil with one parameter, check where it left from, and see what each costs. Measured on our test store.
Turn a product list into a CSV file. One request per page with CSS rules returns JSON, a short Python loop follows the next link and writes the rows.
Keep your Scrapy spider and its selectors. Send the pages that need JavaScript through one API call and get rendered HTML back. Measured on our test store.
Give a Vercel AI SDK agent tools that read web pages as Markdown or JSON, with a credit cap the model cannot raise. Measured on our test store.
Scrape pages, list a site's URLs and run a batch to CSV from a shell, with curl or one small CLI and jq. No script to write. Measured on our test store.
Read pages from a Node.js or TypeScript service with built-in fetch. Markdown, JSON fields and a batch of 8 pages, with no browser to run.
Keep the requests library you know. Send the same call to one API and get Markdown, JSON fields and JavaScript pages. Measured on our test store.
Stop polling. Send a batch with a webhook URL, check the signed call in 40 lines of Python, and read the results when it arrives. Retries measured on dev.
A small Python server of your own that answers with a site's products as JSON. Rules from inspect, fields from extract, and max_age for a cheap cache.
Get every product of a WooCommerce store as JSON from its public Store API, page by page. Free first, then through our API for 1 credit a call.
axios in Node sends an API client's headers, one of them a format no browser offers. What it sends, how it throws on a 403, and the fix. axios 1.21.1.
curl sends three headers and exits 0 on a 403. What it sends, how to make it fail, which flags fix it, and a call that climbs. Tested with curl 8.14.1.