Cloudflare AI Search D1 data source beta: check the query before indexing a help library

Confirm the beta status and connection requirements

On November 12, 2026, Cloudflare announced the AI Search D1 data source in beta. You provide a read-only query, and AI Search indexes each returned row as a separate document; scheduled syncs pick up added, changed, and removed rows. The announcement describes the stage and basic behavior, but does not show that any particular account has enabled or validated the feature. Availability for a specific account, country or region, plan eligibility, and the final bill still need checking against the account and current billing rules.

When creating an instance in the dashboard, select a D1 database, enter a query, choose a document template, then select Test query to inspect one result row and its columns. That check happens before creation; it is not a full sync acceptance run. With the REST API or Workers binding, the D1 database must be in the same Cloudflare account as AI Search, the query must return an id column, and you must provide a service API token's token_id. The token_id is always required; the token's AI Search Index Engine permission includes D1 Read.

Before creation, AI Search checks the database, token, query, and result columns. If a check fails, the request returns an error and no instance is created.

Write a query that meets the D1 source rules

The query must be one read-only statement beginning with SELECT or WITH, with balanced parentheses. It cannot contain a semicolon, --, or /* comment marker, or bound parameters such as ?, :name, @name, or $name. Keywords such as INSERT, UPDATE, DELETE, REPLACE, and PRAGMA are rejected, as are names beginning with sqlite_, _cf_, or pragma_. These checks also scan string literals, so replace() is rejected too.

The result must contain at least one and no more than 100,000 rows, with a unique, non-NULL, non-empty id that is either all text or all integers from -9,007,199,254,740,991 to 9,007,199,254,740,991. Do not use ORDER BY or LIMIT; AI Search reads pages ordered by id. If an articles table has these columns, a query could be SELECT id, title, body AS content FROM articles WHERE published = 1. This illustrates the query shape only; this article has not run it. Queries are limited to 10,000 characters.

Treat column names as a document-field contract

Every result column needs a unique name. id identifies a row across syncs; title becomes the heading, content becomes the body, and meta_<name> becomes custom metadata available for retrieval filters. Other columns appear as labeled fields according to the chosen template. Prefer a stable, indexed primary key for id; changing a row's id deletes the old item and creates a new one.

An id can be up to 512 UTF-8 bytes, and each rendered document is limited to 64 KiB. Return no more than five meta_ columns. Column names are lowercased first. The name after meta_ must start with a letter, contain only letters, digits, or underscores, be at most 64 characters, and avoid reserved names such as timestamp, folder, filename, file, and row_id. A metadata value is limited to 500 characters and longer values are truncated; the JSON metadata for a row, including id, is limited to 8,000 bytes.

Understand when sync keeps or deletes items

A D1 instance runs jobs on its sync interval: new rows are indexed, changed rows are re-indexed, and unchanged rows are skipped. Rows no longer returned by the query are deleted only after AI Search has read the full result, so a partially read result should not be treated as a completed cleanup.

If the query fails or returns no rows, the sync job ends with an error and existing indexed items are kept. Do not use an accidental empty result to clear a help library, and do not assume that a scheduled sync makes content searchable immediately. Changing the query, template, or path rules starts a sync, but you cannot change the source database through an update.

Filter with WHERE and preserve path rules in updates

include_items and exclude_items match the item key generated for each row, such as 42.md. They do not reduce D1 reads: Cloudflare says AI Search still reads excluded rows from D1. To reduce the source set, it is usually simpler to use WHERE in the SQL query to select only the records you want indexed.

When updating an instance, omitting query or template keeps its current value. However, sending a source_params object replaces include_items and exclude_items. Include those rules again in the update request if you want to keep them. Review the query and filter configuration before submitting because these changes start a sync.

Budget for reads and file limits in a small acceptance run

Each sync reads the full query result at least twice with the service API token: once to check id values, then again in pages. Those reads count toward D1 rows read. A stable, indexed id helps D1 read pages efficiently; a computed id can make every page scan the full result. Return only needed fields and leave out frequently changing counters that could trigger unnecessary re-indexing.

D1 data sources are also subject to the per-instance file limit; the effective cap is the lower applicable limit. When a sync reaches that cap, rows are indexed in id order up to the limit, and that sync does not delete items. Start with a small set of authorized help records, then inspect the dashboard's one-row query test, the sync job result, and the rendered documents before expanding. This article has not connected to a real D1 database or tested an account configuration.