AgentReader

The problem

Agents fetch web pages all the time, and the result is often useless. These are the cases AgentReader is built for:

What it can't do: some sites (large news and shopping sites, for example) block automated readers with bot protection. AgentReader doesn't try to get around that. You get a blocked error within seconds and are not charged, so your agent can move on.

What you get

A real example: the React docs page react.dev/learn, measured on 18 Oct 2026 with the cl100k_base tokenizer.

<!DOCTYPE html><html lang="en" dir="ltr"><head><meta charSet="utf-8" data-next-head=""/><meta name="viewport" content="width=device-width, initial-scale=1" data-next-head=""/><script async="" src="https://www.googletagmanager.com/gtag/js?id=G-B1E83PJ3RT"></script><title data-next-head="">Quick Start – React</title><link rel="canonical" href="https://react.dev/learn" data-next-head=""/><link rel=…
(15 script tags, inline styles, navigation for the whole site…)

# Quick Start

Welcome to the React documentation! This page will give you an introduction to 80% of the React concepts that you will use on a daily basis.

### You will learn

-   How to create and nest components
-   How to add markup and styles
-   How to display data
-   How to render conditions and lists
…

Same content, about 22 times fewer tokens. Savings vary by page: simple pages shrink less, heavy pages more. Some agent tools already convert HTML to text, but they usually can't render JavaScript, get past consent walls or read PDFs.

With format=json you also get structured fields: title, author, date, siteName, lang, wordCount, links[], tables[] and meta (for example whether a browser was used).

Connect

AgentReader is a remote MCP server at https://agentreader.xyz/mcp (streamable HTTP) with the tools read_url, search and get_balance, and a plain REST API. No key is needed for the 5 free reads a day.

Claude Code

claude mcp add --transport http agentreader https://agentreader.xyz/mcp

# with an API key for prepaid credits:
claude mcp add --transport http agentreader https://agentreader.xyz/mcp --header "Authorization: Bearer ar_..."

Cursor (~/.cursor/mcp.json) and other clients with an mcpServers config

{
  "mcpServers": {
    "agentreader": {
      "url": "https://agentreader.xyz/mcp",
      "headers": { "Authorization": "Bearer ar_..." }
    }
  }
}

Leave out headers if you don't have a key.

Claude Desktop and claude.ai

Settings → Connectors → Add custom connector, then paste https://agentreader.xyz/mcp. Custom connectors can't send an API key header, so this uses the free daily reads.

opencode (opencode.json)

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "agentreader": {
      "type": "remote",
      "url": "https://agentreader.xyz/mcp",
      "enabled": true,
      "headers": { "Authorization": "Bearer ar_..." }
    }
  }
}

REST (curl or any HTTP client)

curl "https://agentreader.xyz/v1/read?url=https://react.dev/learn"
curl "https://agentreader.xyz/v1/read?url=https://react.dev/learn" -H "Authorization: Bearer ar_..."   # use prepaid credits

Keys and payment

• No key: 5 free reads a day per IP address.
• API key (Authorization: Bearer ar_..., or x-api-key): uses the prepaid credits on that key. You get a key when you buy a bundle or card pack.
• x402, no key: when free reads run out, the server answers HTTP 402 with a payment request. An agent with an x402 client pays $0.005 in USDC on Base and retries. https://agentreader.xyz/v1/x402/read?url=… always uses x402.

Options

For GET /v1/read. The MCP tool read_url takes the same url, format, mode, render and extract arguments. Unknown values return a 400 error that explains the valid ones.

POST /v1/read with the same fields in a JSON body also works.

Pricing

You only pay for successful reads. Blocked sites, timeouts and errors are free.

A search costs the same as a read. Buying without a key gives you a new API key that holds the credits. Keys don't include the free daily calls. Pay by card at agentreader.xyz/buy; the card provider handles verification, and credits arrive once the payment confirms on Base.

curl -X POST "https://agentreader.xyz/v1/credits/buy?bundle=500" -H "Authorization: Bearer ar_..."   # 402 with x402 payment request; pay and retry
curl "https://agentreader.xyz/v1/balance" -H "Authorization: Bearer ar_..."                                  # free calls left and credits

Also included

• Web search: /v1/search?q=… and the MCP search tool return results (title, url, snippet) for your agent to read next. Example: /v1/search?q=x402+protocol&limit=5
• Feeds and homepages: RSS/Atom feeds return items with title, link, date and summary. Example: /v1/read?url=https://feeds.bbci.co.uk/news/world/rss.xml
• Official APIs: StackOverflow/StackExchange, npm and PyPI pages are read through their official APIs.

FAQ and limits

Does it work on every site?

No. Sites with strong bot protection (Cloudflare challenges, DataDome, PerimeterX) usually block it. You get a blocked error quickly and pay nothing. Pages behind a login aren't supported.

How fast is it?

Static pages usually take under a second or two. Pages that need a browser take longer, often several seconds. Results are cached for 10 minutes.

Do I need an account or a crypto wallet?

No account. For the free tier you need nothing. Beyond that, an agent can pay per call with x402, or you can buy a credit pack by card and use the API key.

What is blocked?

Private and internal network addresses, and non-http(s) URLs. Requests are rate limited.

Where are the machine-readable docs?

llms.txt · OpenAPI · MCP server card · x402 resources · status