AI crawler hits outnumbered human pageviews 10.5x on 8 small sites
From 23 September to 19 October 2026, across eight small sites, AI crawlers generated 10,329 hits against 984 human pageviews — 10.5×.
We did not go looking for this. We found it because one of our own sites reported 3,088 visitors from the United States and 22 from Spain, in a directory of Spanish businesses written in Spanish. That is not an audience. That is a crawler.
The numbers
Eight sites, 23 September – 19 October 2026, 27 complete days. Every hit below reached a JavaScript tracker and was classified by the user agent the visitor declared.
| What hit the sites | Hits | Share of bot traffic |
|---|---|---|
| AI crawlers — named below | 10,329 | 94.4% |
| Search engines | 515 | 4.7% |
| Everything else — headless browsers (52), self-declared crawlers we could not name (26), unidentified user agents (20), monitors (1) | 99 | 0.9% |
| Human pageviews, same eight sites, same days | 984 | — |
Which crawlers, by name
| Crawler | Run by | Hits | Share of AI traffic |
|---|---|---|---|
meta-externalagent | Meta | 8,869 | 85.9% |
OAI-SearchBot | OpenAI | 1,221 | 11.8% |
GPTBot | OpenAI | 64 | 0.6% |
ClaudeBot | Anthropic | 17 | 0.2% |
Bytespider | ByteDance | 10 | 0.1% |
| Not named (first day of name recording, 23 September) | — | 148 | 1.4% |
| All AI crawlers | — | 10,329 | 100% |
Meta alone is 86% of it. OpenAI runs two separate agents, and the search-oriented one
(OAI-SearchBot) outnumbers its training crawler (GPTBot) about 19 to 1.
Anthropic and ByteDance show up, but rarely.
And the search engines? Baidu, and only Baidu. Of the 515 search-engine hits,
507 were Baiduspider; the other 8 came on the first day, before names were recorded.
Not one Googlebot hit reached the tracker on any of the eight sites in 27 days. That does
not mean Google never visits; it means Googlebot rarely runs page scripts.
It is the clearest illustration of the first caveat below.
Every AI crawler above executed JavaScript. That is the part worth pausing on. The
standard advice in this field is that AI bots do not run JavaScript, so client-side analytics cannot
see them and only server logs will do. These hits were counted by a script in the page. Whatever the
general rule may be, meta-externalagent, OAI-SearchBot, GPTBot
and ClaudeBot all reached one. Server logs would still show more — a crawler that never
runs the script is invisible here — so treat this as a floor, not a census.
It is not evenly spread, and that matters
The average hides the interesting part. Two sites carry 96% of it, and one site shows the exact opposite pattern:
| Site | AI crawlers | Search engines |
|---|---|---|
| A local business directory (tens of thousands of listings) | 7,084 | 0 |
| An e-commerce blog | 2,875 | 134 |
| A free online-tools site | 230 | 281 |
| A customer-support SaaS site | 86 | 40 |
| A developer-tool landing page | 34 | 9 |
| Three other small sites, combined | 20 | 51 |
A big directory of structured, factual content is exactly what a model wants to ingest, and it got scraped thousands of times while receiving zero measurable search-engine crawling. A site of small interactive tools — nothing to extract, everything to use — is the only one where search engines outnumber AI crawlers. If you run a content site, the first pattern is likelier to be yours than the average is.
What is wrong with this data
Quite a lot, and anyone quoting it should quote this part too.
It measures bots that execute JavaScript. These hits were counted by a tracker script in the page, not by reading server logs. A crawler that fetches your HTML and never runs the script is invisible here — Googlebot above is the proof — so the search-engine column is a floor, not a crawl count. The honest version of the headline is: among bots that run JavaScript, AI crawlers dominate — and more of them run JavaScript than this field generally assumes.
Eight sites is not a study. They are small, they all belong to one portfolio (ours), and one of them produces most of the AI traffic. Twenty-seven days is still a snapshot. Treat this as a reading from a thermometer we happened to be holding, not as a market-wide figure.
It trusts the user agent. These crawlers announce themselves — GPTBot says it is GPTBot. A scraper that disguises itself as Chrome is not in the AI column; it is somewhere in the human numbers, which is a good reason to distrust anybody's human numbers, including ours.
Earlier readings
The first two readings were shorter and included two sites run by Logly users. They point the same way:
- 17–21 September (nine sites, bot type only): 2,878 AI-crawler hits, 124 search-engine hits.
- 23–29 September (nine sites, first week with names): 2,395 AI-crawler hits —
meta-externalagent81.6% — 115 search-engine hits.
Why almost nobody can tell you this about their own site
Every analytics tool filters bots. Almost none of them show you what they filtered:
- Google Analytics 4 states it plainly: you "cannot… see how much known bot traffic was excluded".
- Plausible, Simple Analytics, Pirsch and Umami filter bots and publish no report of what was filtered.
- GoatCounter stores it in a table its own changelog describes as "never accessed".
- Matomo excludes bots by default; a separate report needs a third-party plugin.
- Fathom does show how much it blocked and by which detection method — datacenter IP, user agent, headers — but not which bots they were.
- Logly counts them apart and labels them by type, so "AI crawlers" is a line you can read instead of a number that quietly vanished.
See the full breakdown on the comparison pages.
Want this number for your own site? Add the Logly script, wait a day and read the bot line on your dashboard. Free plan, no card, under 1 KB, no cookie banner — and if the answer turns out to be "almost none", you will have learned that for free too.
Method, in full
Traffic is classified at the moment the tracker fires, from the User-Agent header, against
a published list of self-declaring crawlers grouped by type: search engines, AI crawlers, social link
previews, SEO tools, monitors, headless browsers and scripted clients. Anything that does not match a
pattern and does not look like a browser is counted as unidentified and reported separately rather than
guessed at — the numbers above put that bucket at 20 hits in 27 days. Bots are excluded from visitor,
session and pageview counts; they are never quietly folded into them.
A hit is one page load on which the tracker ran: the script file itself is not counted, the
request that records the hit is built by the script at run time, and there is no no-script fallback
pixel. Only the first signal of each page load counts; the keep-alive pings the tracker sends every
60 seconds do not. Classification trusts the declared user agent and does not verify IP addresses or
reverse DNS (Logly does not store IP addresses), so a headless scraper that claims to be
meta-externalagent would be counted as Meta, just as one claiming to be Chrome is counted
as human.
Period: 23 September – 19 October 2026 inclusive (UTC). Sites: eight, all ours, all tracked by Logly. Figures are raw hits, not deduplicated by session, because a crawler does not have sessions. Human pageviews count page loads, not keep-alive pings, and include our own visits to these sites; nothing was removed.
Questions, or a data set of your own you want compared against this? Use the chat widget in the bottom-right corner of this page and we will answer with the query we ran.