Counting Agentic Traffic

Writing · 7 Jul 2026

Counting Agentic Traffic

Why your analytics does not see the AI agents reading your site

This month I brought Lemmid Count back: the website statistics of my platform, rebuilt from scratch without AWStats and without a database. Its party piece is the dimension every other analytics tool throws away: the bots. In the agentic era the question "what did AI agents read on my site" matters as much as "how many humans visited". This post is about counting what pretends not to be there, and about putting the answer in your customer's pocket.

Google Analytics is blind here

Google Analytics works by placing a piece of JavaScript on your pages. A browser runs it, the script reports home, and that report is your statistics. I explained the difference with log-based tools back in 2018. In 2026 that difference has become the whole story. GPTBot, ChatGPT-User, ClaudeBot, PerplexityBot and their friends fetch your HTML and read it. They do not run your JavaScript. They never appear in Analytics. If the agentic web is where your next customers come from, your dashboard is showing you a shrinking world.

The web server sees everything. Every request, every user agent, every status code, JavaScript or not. In July my site hartvaat.nl served 114,883 pageviews. 2,187 of them were humans. Analytics would have shown you the 2,187 and called it a quiet month.

hartvaat.nl in July 2026: 114,883 pageviews, 2,187 by humans, 112,696 by bots; Analytics would have shown the small number
hartvaat.nl in July 2026: 114,883 pageviews, 2,187 by humans, 112,696 by bots; Analytics would have shown the small number

Logs, not scripts

Lemmid Count v3 reads the per-site logs of the web server directly, every hour, and folds them into small JSON files per site per day. No database anywhere, no tracking script on the pages, no third party at runtime: the visitor is never touched. That also fixes what killed the previous version: AWStats pumping a near-raw copy of the logs into MariaDB forever. Now the rotated logs are the raw store and the summaries are the product.

Two principles. Pages, never people: no visitor counts, no uniqueness tricks, no IP addresses on disk, the browser language instead of geo. And auto configuration: a site with a log file has statistics, nobody has to register anything. Every site on the platform runs along automatically.

Classify, do not filter

Every request becomes either human or bot, and a bot gets a family and a group: search, ai, social, seo, monitor, feed, lib, scraper. Declared bots are named by their user agent. Instead of filtering them out, Count shows them as first class citizens: a "read by AI agents" card with the agents by name, and a toggle between all, humans and bots. That is new, and it makes people look up. On a restaurant site, in the first days of July, ChatGPT had read 75 pages and Claude 51, next to 185 human pageviews.

All, humans or bots: one restaurant site in the first days of July, 185 human pageviews next to 1,171 by bots
All, humans or bots: one restaurant site in the first days of July, 185 human pageviews next to 1,171 by bots
The humans view: top pages of a cafe site, filtered in one tap
The humans view: top pages of a cafe site, filtered in one tap

The pretenders

Labeling is difficult, because some bots pretend to be human. A current Chrome user agent, HTTP/2, a plausible language header, one page per IP address, thousands of addresses from cloud providers. The user agent is worthless there. Behaviour is not. Real browsers load images. Real visitors arrive from search and links, and go to the same popular pages. A crowd that visits thousands of distinct pages, each exactly once, is not a crowd, it is a harvest.

Hosting more than a hundred websites is the structural advantage: a signature that shows up on many sites within one hour is not a person. Add honey pots, pages no human would ask for, and address space from datacenters as a hint, never as a verdict. The classifier is versioned, and after every upgrade the last two weeks of history are recomputed. That matters, because the other side moves. One fleet answered a new rule within a day by fetching a single CSS file as an alibi. The next version asked for images. I will not list every signal here, that is the point of a honey pot.

Read by AI agents, by name: chatgpt-user, claudebot and the stealth group of pretenders on the same site
Read by AI agents, by name: chatgpt-user, claudebot and the stealth group of pretenders on the same site

Full circle: in your pocket

Here is the thing: detailed statistics are not unique. Anyone with logs and patience can count. The other half of the job is translating it back into something friendly, pocketable and actionable. Count is an app my customers carry in their pocket, next to the other Lemmid apps: one screen per site, the month in one chart, the top pages, the sources, and the agents by name. No login rituals, no setup, it simply appears for every site you own.

The feedback is the best part. A customer showing the stats during the coffee round, exactly the use I had in mind years ago. A cafe owner seeing that ChatGPT read the menu seventy times this month, and drawing the obvious conclusion: put the menu where the agents read it. That is a statistic turned into an action, on a phone, in ten seconds.

The pocket app: Manager, Count, Handler, User and Bill, one home screen for every site you own
The pocket app: Manager, Count, Handler, User and Bill, one home screen for every site you own

Conclusion

The share of humans on the web will keep falling, and a tool that only counts humans will keep telling you the wrong story. Count the agents, name them, and hand the answer to the person who can act on it. High tech in the log parser, low tech on the screen.

The bots read. You had better make sure they read the right thing, ha!

Midnight on 6 July, the night Count came back, terminal on the iPad
Midnight on 6 July, the night Count came back, terminal on the iPad

← More writing