Cloudflare web_fetch: pagination refetches pages, so set request limits first

It reads pages and documents; it does not provide a full browser

Cloudflare's current announcement, dated October 12, introduces Agents SDK web_fetch. A model supplies a URL and receives content, the final URL, HTTP status, content type and continuation information. The announcement URL contains October 11, but the page displays October 12; the URL date does not establish an exact launch time.

The tool requests Markdown first. Markdown, JSON and plain text skip Workers AI conversion; HTML, PDFs and supported Office documents are converted into Markdown through Workers AI. Conversion does not establish accuracy: check that the content answers the task.

Requests use GET without browser cookies, credentials or custom headers. Choose a suitable browser tool for JavaScript rendering, an existing login session, screenshots or clicks. Converting a challenge page does not grant access to the underlying site.

Check the SDK, AI binding and registration order

Check the project's actual agents version: it must be at least 0.28.0, with an existing harness. The Wrangler example configures a Workers AI binding named AI with remote set to true. HTML and document conversions during local development still run remotely against your Cloudflare account. The reviewed sources do not specify exact charges, free allowances or availability by region; local development does not imply free conversion.

For Pi Durable, import webFetchTool from agents/webfetch/pi, install webFetchTool({ binding: this.env.AI }) in the registry, then call Harness.open(). Registering first makes the tool available to that harness; it does not automatically add it to every harness.

AI SDK and TanStack AI have their own adapters. Another harness can use fetchWeb(), windowWebFetchPage() and renderWebFetchPage() in its tool handler. These lower-level functions do not require a Durable Object, browser or Worker Loader binding. Your application still supplies its model and execution environment.

Five reading windows mean five GETs by default

Each result contains up to 20,000 content characters by default. Continue with the suggested offset instead of adding 20,000 yourself: a window may end at a line break. In structured output, nextOffset is null once the end is reached.

The built-in source fetches the URL again for each tool call. Hypothetically, reading a report in five windows means five requests, and the source may change between them. This example has not been executed. Pagination does not cache a downloaded document or guarantee that all five sections share one version.

To reuse one fetch, retain the WebFetchPage returned by fetchWeb(), then call windowWebFetchPage() with subsequent offsets. Pi session recovery is separate: completed results remain stored, while an interrupted fetch reruns after eviction. That does not make ordinary pagination a single fetch.

Limit hosts, then separate characters, bytes and time

Your application sets policy when creating the tool; the model chooses only url, offset and format. allowedHosts defaults to public hosts. For a task limited to approved documentation, list the required hosts. blockedHosts is checked first, and *.example.com includes subdomains but excludes the example.com root.

pageChars limits the content window sent to the model. maxBytes defaults to 5,000,000 and caps response body bytes. timeoutMs defaults to 30,000 milliseconds, including redirects and conversion; maxRedirects defaults to 5 and accepts 0–20. Narrow these settings to the task.

The built-in source checks each URL and redirect target. It rejects non-HTTP schemes, URL credentials, URLs over 2,000 characters and HTTPS-to-HTTP redirects. Local hostnames and private IP literals are blocked by default, but it does not resolve public hostnames to check their underlying DNS addresses. This is not complete private-network isolation.

Read the structured error before choosing a response

Readable HTTP error pages are returned with their status. Fetch failures have WebFetchError codes such as web_fetch_timeout, web_fetch_too_large and web_fetch_unsupported_content_type, plus a retryable field. Separate timeouts, oversized responses, unsupported formats and the page's own errors; an HTTP response alone is not a useful answer.

Pi reports failures in details with ok: false. AI SDK and TanStack AI adapters throw WebFetchError. Cancellation propagates as an abort, rather than a fetch failure. A detected bot challenge produces web_fetch_blocked and guidance to use a browser or another source; the site's access rules still apply.

Start with one document you are authorized to read

An unexecuted integration check would confirm the version and AI binding, register the matching adapter, then supply an approved public-document URL. Inspect the final URL, status, type, body and nextOffset. For consistent long-document reading, also check WebFetchPage reuse. No SDK installation, Worker deployment or model call was performed for this article.

The result wrapper labels external content as data; it does not guarantee protection against prompt injection. If you convert AI SDK UI messages yourself, pass the same tools to convertToModelMessages(messages, { tools }) to preserve earlier results' formatting and wrapper. Website text cannot authorize new actions on a user's behalf.