The short answer: httpx raises a timeout when a server does not answer in time, and its limit is short: 5 seconds by default, for connecting, for each read, for each write and for a free connection from the pool. That is unlike requests, which waits forever unless you pass timeout. Catch httpx.TimeoutException, the parent of ReadTimeout, ConnectTimeout, WriteTimeout and PoolTimeout. Set the limit per request or per client, and retry read timeouts yourself: the transport's retries setting only retries failed connections. When a page is slow because it needs a browser, post it to our API instead, and an answer that runs out of time costs nothing.
What httpx does on a slow server
Start a listener in one terminal with nc -lk 8799. It takes each connection and never answers, like a server that hangs. Run this in a second terminal:
import time
import httpx
t = time.monotonic()
try:
httpx.get("http://127.0.0.1:8799/")
except httpx.TimeoutException as err:
print(type(err).__name__, "|", err, "| after", round(time.monotonic() - t, 1), "s")ReadTimeout | timed out | after 5.1 s
- 5 seconds, then
ReadTimeout. We passed notimeout, so httpx used its default of 5 seconds. The connection opened at once; the read never got a byte. ncprinted what httpx sent:User-Agent: python-httpx/0.28.1,Accept: */*andAccept-Encoding: gzip, deflate, br, over HTTP/1.1.- The default is a
httpx.Timeout(5.0): 5 seconds each for connect, read, write and pool. Passtimeout=Noneto wait forever, which is rarely what a scraper wants.
How httpx reports a timeout
Our test store answers fast, so give it a limit too short to meet. The error is the same one a slow site gives you:
import httpx
try:
httpx.get("https://lab.megalania.io/returns/", timeout=0.05)
except httpx.TimeoutException as err:
print(type(err).__name__, "|", err)
print("is a TimeoutException:", isinstance(err, httpx.TimeoutException))ReadTimeout | The read operation timed out is a TimeoutException: True
- A 50 ms limit on a real page gave the same class,
ReadTimeout, with the messageThe read operation timed out. The connection was made in time; the answer was not. - Catch
httpx.TimeoutExceptionto get all four kinds at once. Each one is also anhttpx.TransportError.
The free fix: timeouts and retries
Set a limit on every request, and retry a few times with a pause. This fixes servers that are slow now and then:
import time
import httpx
client = httpx.Client(timeout=httpx.Timeout(20.0, connect=5.0), transport=httpx.HTTPTransport(retries=2))
def get(url, tries=3):
for n in range(1, tries + 1):
try:
return client.get(url)
except httpx.TimeoutException as err:
print("try", n, type(err).__name__)
if n == tries:
raise
time.sleep(2 * n)
res = get("https://lab.megalania.io/returns/")
print(res.status_code, res.http_version, len(res.text), "characters")200 HTTP/1.1 55477 characters
httpx.Timeout(20.0, connect=5.0)gives a slow page 20 seconds per read and fails fast on a host that does not take connections.HTTPTransport(retries=2)retries only connection errors, notReadTimeout. The loop inget()retries read timeouts too, with a pause of 2, then 4 seconds.- One client for all requests keeps connections open between them, so later pages skip the TLS handshake.
The fix that climbs: our API
Send the page to our API instead. We fetch it, run a real Chrome when you ask, and an answer that runs out of time costs nothing:
# Run: MEG_API_KEY=YOUR_KEY python fetch.py https://lab.megalania.io/returns/
import os
import sys
import httpx
res = httpx.post(
"https://get.megalania.io/v1",
headers={"Authorization": f"Bearer {os.environ['MEG_API_KEY']}"},
json={"url": sys.argv[1], "response_types": "markdown", "max_credits": "64", **dict(a.split("=", 1) for a in sys.argv[2:])},
timeout=100,
)
print("status:", res.status_code, "target:", res.headers.get("x-target-status"))
print("route:", res.headers.get("x-route"), "credits:", res.headers.get("x-credits-charged"), "billing:", res.headers.get("x-billing"))
print(res.text[:160])status: 200 target: 200 route: http/datacenter credits: 1 billing: None # Returns Changed your mind? Send it back within 60 days for a full refund, as long as it is unused and has its tags. ## How to return 1. Write to
- Give the call to us a limit longer than ours: we allow up to 90 seconds per page, so
timeout=100on the httpx side. - Extra arguments go into the body as they are:
engine=browser,actions=...andtimeout=20in the next run.
The same call on a page whose list never finishes loading, with a 20-second limit:
MEG_API_KEY=YOUR_KEY python fetch.py https://lab.megalania.io/new-arrivals/ engine=browser actions=wait_for:#item-99 timeout=20
status: 422 target: None
route: None credits: None billing: None
{"error":{"code":"action_failed","field":"actions","message":"actions entry 0 could not run: no visible element matched its selector in time, or the t- The list on this page has 12 items, so
#item-99never appears. We waited inside a real Chrome until the time for page actions ran out, then answered422action_failedafter about 14 seconds. - No
x-credits-chargedheader: you are not charged. Raisetimeout, up to 90, or fix the selector.
One honest note: our test store is fast, so these runs show a short limit or a page that never gets ready, not a slow site. On a slow site, the call is the same; the price depends on the route that worked, and our MCP tool check_site tells you that price for free.
What it costs
| What we ran | Route | Credits |
|---|---|---|
| The returns page as Markdown | h:dc | 1 |
| A page that never got ready, in the browser | c:dc | 0 |
sends.py, seen.py and fix.py, straight from httpx | none | 0 |
Every rule is in Credits. Plans are on the pricing page.
How we measured: httpx 0.28.1 on Python 3.13.5, one Linux server, 16 Nov 2026. nc -lk 8799 (OpenBSD netcat) as a server that never answers. Our test store for the short limit and for fix.py. Our API on our dev cluster for fetch.py, against the same store on our dev domain. The outputs are copied from those runs.
Limits
- Public pages only. We do not log in, and we do not solve CAPTCHAs. When a page needs either, we stop and say so, for free.
- A page built by JavaScript needs
engine=browser. It costs more. See How to scrape JavaScript-rendered pages. - Slow can mean slow down. A site that answers slowly may be under load. Keep your rate low, fetch public pages only, and follow the site's terms and robots.txt.
What to do when a site answers 429 instead is in 429 Too Many Requests when scraping.