Crawler info

BreezaroBot

The web crawler operated by Breezaro — an AI chatbot platform that lets website owners turn their own content into a self-service support assistant.

What this bot does

BreezaroBotfetches pages from a website only when that site's owner has explicitly added the URL to their Breezaro knowledge base. The fetched content is converted to text, indexed in a vector store, and used exclusively to answer questions from visitors of that same site through the site owner's chatbot widget.

The bot does not crawl the open web at large, does not train foundation models, and does not share content across tenants. Each Breezaro customer's index is namespace-isolated.

How often it visits

  • Initial setup: when a site owner adds a source, we crawl the listed pages once.
  • Re-crawls:only on explicit re-sync from the owner's dashboard, or via a nightly delta scheduler that visits a sitemap to detect changes (no full re-crawl).
  • Per-host concurrency:capped at a small fixed number of parallel requests so a Breezaro crawl cannot saturate the site's servers.

How to identify our traffic

Every request our crawler makes carries this User-Agent:

BreezaroBot/1.0 (+https://breezaro.com/bot)

The token (BreezaroBot) is the stable identifier — version bumps in the suffix do not change the identity rules below.

Verifying our identity

Every BreezaroBot request comes from a single static IPv4 address:

167.235.18.41

Its reverse DNS (PTR) resolves to crawler.breezaro.com, which forward-resolves back to the same address. This forward-confirmed reverse DNS (FCrDNS) lets you verify a request claiming to be BreezaroBot really is ours, no matter what the User-Agent says (anyone can spoof a User-Agent):

host 167.235.18.41      # -> crawler.breezaro.com
host crawler.breezaro.com   # -> 167.235.18.41

A request whose User-Agent says BreezaroBot but whose source IP is not 167.235.18.41 (or whose reverse DNS does not resolve to crawler.breezaro.com) is not us, and is safe to block.

Allowing our crawler behind a firewall or CDN

If your site sits behind Cloudflare, a WAF, or bot protection, a challenge page can stop us from reading your content even after the owner added the URL. The import then finishes with nothing indexed. To let BreezaroBot through, allow it by any of:

  • User-Agent containing BreezaroBot
  • source IP 167.235.18.41
  • reverse DNS crawler.breezaro.com (FCrDNS, the most robust option: it cannot be spoofed)

On Cloudflare, add a WAF Skip (allow) rule such as:

(http.user_agent contains "BreezaroBot") or (ip.src eq 167.235.18.41)

We are also enrolling BreezaroBotin Cloudflare's Verified Bots program, so allowlisting will work without a custom rule.

How to opt out

We honor robots.txt on every request. To block BreezaroBot from your site, add the following to your robots.txt:

User-agent: BreezaroBot
Disallow: /

Or, to allow indexing of public pages but exclude a specific path:

User-agent: BreezaroBot
Disallow: /admin/
Disallow: /private/

Changes to robots.txt are picked up within minutes by our per-host cache.

Reporting an issue

If our crawler is misbehaving (hitting your server too hard, ignoring robots.txt, fetching content you removed), please email us at support@breezaro.com. Include the source IP, a timestamp, and the affected URLs from your access logs and we'll investigate the same day.