Brave Search Help

Brave Search Crawler

Like any other search engine, Brave search has a crawler to discover new pages and index their content.

The Brave Search crawler does not advertise a differentiated user agent because we must avoid discrimination from websites that allow only Google to crawl them. However, if a domain or page is not crawlable by Googlebot, then Brave Search’s bot will not crawl it either.

Note that robots.txt is not used to prevent a page from being indexed. A site owner can delist a page by using the robots noindex directive.

Brave Search’s crawler is partially powered by information provided by users enrolled in the Web Discovery Project (WDP) option in Brave browser’s search settings, which is an off-by-default (aka opt-in), privacy-preserving system. WDP has multiple mechanisms to prevent Brave from knowing who is contributing what (WDP is also open-source for auditing and inspection by anyone).

In case Brave Search is returning a page which does not exist anymore and you would like to get it delisted, you can submit your request to