AI Crawl Report

Blog

Notes on AI crawlers and AI referrals: which bots read websites, how to tell the real ones, and what they send back.

RSS feed

GPTBot in your logs might not be GPTBot

Any request can call itself GPTBot or ClaudeBot. Here's how to tell the real AI crawlers from the ones borrowing their names, and what to do about the ones you can't check.