Who is crawling your site — and which ones should you block?
The Bot Observatory watches real traffic on live websites around the clock and tells you which
crawlers and bot networks are out there this week: AI scrapers, SEO tools, search engines, and the
rented-server fleets that pretend to be browsers. Get the full picture, check whether a “Googlebot” is
really Google, hear about new bots the day they appear, and copy ready-made firewall rules.
33.1M requests, 18k networks and 398 named crawlers seen across 20 live websites in the last 7 days. Updated hourly — this page: 2026-10-27T22:25:03Z.
This week, in public
Busiest networks(how much of their traffic sites refused)
ASN
network
requests*
refused
AS150436
BYTEPLUS-AS-AP Byteplus Pte. Ltd.
11.7M
100%
AS4229
ZEN-NET
3.9M
100%
AS45102
ALIBABA-CN-NET Alibaba US Technology Co., Ltd.
242k
98.9%
AS22773
ASN-CXA-ALL-CCI-22773-RDC
158k
0.9%
AS16509
AMAZON-02
139k
0.8%
AS8075
MICROSOFT-CORP-MSN-AS-BLOCK
120k
0.2%
AI crawlers
crawler
requests*
networks
ClaudeBot
116k
4
Amazonbot
20k
3
PerplexityBot
14k
4
rednote-websearch-bot
55k
1
OAI-SearchBot
5k
4
Bytespider (ByteDance)
5k
11
* sampled requests over 7 days. The free public view is also at
dns.pizza/bots. Plans add the full feed — every network, crawler and TLS fingerprint, hour by hour — plus checks, alerts and rules.
For a site owner or SEO who wants to know what's out there.
The full feed: every network, named crawler and JA4 TLS fingerprint, 7 days hourly (normally $0.05 a pull)
5,000 “is this really Googlebot?” checks a month
Email the day a new crawler or bot network appears
A Monday summary of the week
Most useful
Team
$29 /month
For whoever runs the firewall.
Everything in Watch
Firewall rules, ready to paste: Vercel WAF, Cloudflare, robots.txt, JSON
50,000 verification checks a month
Alerts for new TLS fingerprints too
Signed webhook alerts for Slack or your own tools
Site
$49 /month
Your own traffic, from your own logs.
Everything in Team
Point your Vercel log drain at us: a report on your site's bots
“A new crawler hit your site” and “fake Googlebot on your site” alerts
Billed monthly by card through Stripe; local sales tax or VAT is added at
checkout where it applies. Cancel any time below — your plan keeps working until the end of the month you paid for.
You're subscribed. Your API key:
Save it somewhere safe — it is your account. We also email it to the address you paid with, and this browser remembers it.
Your plan
Alerts
Firewall rules
Cancel
Stops the monthly charge. Everything keeps working until the end of the period you paid for.
Site tier waitlist
The Site tier reads your own Vercel log drain (or a ten-line edge reporter) and
reports the bots on your site. Tell us where to reach you and we'll invite you when it opens.
Questions
Where does the data come from?
From the request logs of 20 live websites we run, including AboutUs.com and dns.pizza, rolled up every
hour. We publish what was seen and what was refused — never anyone's IP address.
What is an API key, and do I need to code?
The key is your account: paste it into “Your plan” above to see your plan, set alerts, copy firewall rules or cancel.
No code needed for any of that. If you do write code, send it as Authorization: Bearer <key>
to https://how.is/v1/bots.
Is it safe to paste the firewall rules?
They default to challenge, which lets real people through after a quick check, and they never include
search engines, link previews or Cloudflare. Start there, watch your logs for a day, then tighten.
What does “is this really Googlebot?” mean?
Anything can call itself Googlebot. The check looks up the address the request came from and tells you whether
it really belongs to Google (or Bing, Apple, OpenAI, and the rest).
How do I cancel?
Paste your key above and press Cancel. No phone calls, no forms. Lost your key? It is in the welcome email we sent when you subscribed.