Megalania
Use case · AI agents

Web access for AI agents

Your agent sends a URL. It gets the page back as clean Markdown, or as JSON fields. Connect one MCP server, or add a package to your framework. You decide what one page may cost.

What your agent gets

  • Markdown, not HTML. The main text of the page, without the menu. Fewer tokens for the model to read.
  • Fields as JSON. extract reads fields with CSS rules, one object per card. inspect suggests the rules.
  • Whole sites. map_site lists a site's URLs. create_batch fetches many pages in one call.
  • The price first, for free. check_site tells the agent what a site costs, and how our routes did there lately, before it spends anything.
  • Blocks we detect are free. So are our own errors. When a page needs a browser, the answer says so, and the agent can decide.

Connect your agent

Claude Code, Cursor, VS Code, ChatGPT: add our MCP server. In Claude Code it is one command:

Shell
claude mcp add --transport http megalania https://mcp.megalania.io

You sign in with Google or GitHub in the browser. You copy no key. Not a tester yet? Apply for the beta first: sign-in works once your invite arrives. The other clients take one short block of config each, in MCP for agents.

  • LangChain: langchain-megalania, a page loader and five tools. See LangChain.
  • LlamaIndex: llama-index-readers-megalania for documents, and our MCP server for agent tools. See LlamaIndex.
  • Vercel AI SDK: @megalania/ai-sdk, tools for generateText and streamText. See Vercel AI SDK.
  • CrewAI and other MCP clients: connect to the same MCP server, with a key in the Authorization header, never in the URL. See CrewAI web scraping.

Load pages in five lines

This is LangChain. Each page comes back as one Document, in Markdown, with its cost and status in the metadata. The pages are on our test store, the Lab.

Shell
pip install langchain-megalania
export MEG_API_KEY='meg_sk_...'
load_pages.py
from langchain_megalania import MegalaniaLoader

docs = MegalaniaLoader(["https://lab.megalania.io/about/", "https://lab.megalania.io/faq/"]).load()
for d in docs:
    print(d.metadata["source"], d.metadata["credits_charged"], d.metadata["target_status"], len(d.page_content))
print(docs[1].page_content[:400])

What it printed on our dev cluster on 16 Nov 2026:

Output
https://lab.megalania.io/about/ 1 200 1265
https://lab.megalania.io/faq/ 1 200 1127
# FAQ

Is this a real shop?

No. Gibber & Co. is a demo store by Megalania for testing web scraping. Nothing here is for sale. Checkout is switched off and no payment can be made.

How long does delivery take?

Standard delivery takes 3 to 5 working days. See [Shipping](https://lab.megalania.io/shipping/) for all options.

Can I return something?

Yes, within 60 days if it is unused. See [Returns]

Each page cost 1 credit. More in Web scraping with LangChain.

Caps the model cannot raise

  • A cap per page. max_credits is the most one page may cost. We skip every route above it. If no route is left, the answer is an error before any try, and it costs nothing.
  • Set by you, not the model. In langchain-megalania and @megalania/ai-sdk you set the cap when you create the tools. The model cannot change it.
  • Over MCP, the model fills max_credits. So the account owner can set a monthly spend cap. A request past it is refused, for free.
  • A budget per run. Keep one in your own tool, and pass what is left as max_credits. Cap what an AI agent spends on web pages shows the code.

Limits

  • Page text is untrusted. A page can hide instructions for a model. Our tool descriptions tell the agent never to follow them. Keep that rule in your own prompts too.
  • Public pages only. We do not log in, and we do not solve CAPTCHAs. When a page needs either, we stop and say so, for free.
  • Browser pages cost more. A page that needs JavaScript runs in a real Chrome. check_site shows the price; plans are on the pricing page.
Free beta, by invite76,000 credits for 30 days, no card. Apply, and we email your invite.Apply for the beta

Read the guides

Blog · AI agentsGive your AI agent web access with an MCP server

Add one MCP server and your agent can check a site for free, then fetch any public page as clean Markdown. One setup command, real page sizes and costs.

Blog · AI agentsWeb scraping with LangChain: a page loader in 5 lines

Load web pages as Markdown LangChain documents in 5 lines, crawl a site, and give an agent tools with a credit cap. Measured on our test store.

Blog · AI agentsLlamaIndex web scraping: pages as documents, tools for agents

Load web pages into LlamaIndex as clean Markdown documents, split them at headings, and give an agent web tools with a credit cap. Measured on our Lab.

Blog · AI agentsGive a Vercel AI SDK agent web access

Give a Vercel AI SDK agent tools that read web pages as Markdown or JSON, with a credit cap the model cannot raise. Measured on our test store.

Blog · AI agentsCap what an AI agent spends on web pages

Stop a looping AI agent from burning credits on web pages. A free price check, a cap per page, a budget per run and a monthly cap, measured on our Lab.

Blog · AI agentsGive Cursor and Claude Code web access to read docs mid-task

Your coding agent guesses when it cannot read the docs. Add one MCP server, a short project rule, and it reads the current page before it writes code.

Blog · AI agentsCrewAI web scraping: give a crew tools that read pages

Connect a CrewAI crew to our MCP server with one config, keep two tools, and lock the cost of each page. Python code, measured on our test store.

Blog · AI agentsGoogle ADK web scraping: MCP tools with a cap the model cannot raise

Connect a Google ADK agent to our MCP server with McpToolset, keep three tools, and cap each page in a before_tool_callback. Tested on google-adk 2.14.

Blog · AI agentsOpenAI Agents SDK web scraping: an MCP server with a fixed cost

Connect an OpenAI Agents SDK agent to our MCP server, keep two tools, and lock what one page may cost. Python code, measured on our test store.

Blog · AI agentsPydantic AI web scraping: MCP tools with a cap the model cannot raise

Give a Pydantic AI agent web pages with MCPToolset, only the tools it needs, and a per-page credit cap the model cannot change. Tested on Pydantic AI 2.68.