AI crawlability is the ability of AI crawlers and answer engines (GPTBot, ClaudeBot, PerplexityBot, Google-Extended and the rest) to find, access, understand and revisit your content. The logic is brutally simple: if an answer engine can’t crawl your page, it can’t cite it. And if it can’t cite you, you don’t exist in AI search, no matter how strong your classic rankings are.
What evidence is there that search systems are developing an understanding of my website?
Most analytics measure visitor activity. System Flow Analysis measures search system behaviour.
Search systems do not build an understanding of a website by reading isolated pages. They construct a probabilistic model by repeatedly moving between connected pages. Every movement reinforces some parts of the website while ignoring others. Over time these transitions determine where attention naturally concentrates and which pages become central to the websites knowledge structure. This report measures those observed movement probabilities to help explain how search systems are building an understanding of your website.
Report date: Wednesday, 12 August 2026
|
Total Visits
645
|
Core Hits
17
|
Core Capture Rate
2.64%
|
Core → Core
27.59%
|
Google: 0
Bing: 0
SEO tool: 3
AI crawler: 73
Exploit scanner: 31
Unknown: 393
During this reporting period the following identified AI systems requested pages from your website. These requests may originate from AI training crawlers, AI retrieval systems, or AI-assisted user browsing. Repeated requests generally indicate continued reinforcement of your website within that AI ecosystem.
| AI System | Role | Requests |
|---|---|---|
| AmazonBot | AI assistant | 67 (39.0%) |
| ChatGPT-User | AI-assisted browsing | 34 (19.8%) |
| PerplexityBot | AI answer engine | 23 (13.4%) |
| Google Extended | AI training crawler | 19 (11.0%) |
| OpenAI GPTBot | Foundation model crawler | 9 (5.2%) |
| ClaudeBot | Foundation model crawler | 9 (5.2%) |
| OpenAI SearchBot | AI search retrieval | 9 (5.2%) |
| ByteSpider | Foundation model crawler | 2 (1.2%) |
Background Internet Automation (Non-Human) traffic does not imply human visitors. In most cases, it reflects automated infrastructure activity rather than real user behaviour.
| Metric | Value |
|---|---|
| Core Capture Rate | 2.64% |
| Core → Core Transitions | 27.59% |
| Core Page Hits | 17 |
| Total Visits | 645 |
Measures identified AI crawler requests across valid website pages during the last 30 days, showing the breadth and repetition of observed crawler activity. Learn how repeated AI observations contribute to a website’s evolving knowledge footprint.
| Metric | Value |
|---|---|
| Identified AI Crawler Requests | 64 |
| Pages Observed by AI Crawlers | 15 |
| Average Reinforcement per Page | 4.27 |
| Repeated Reinforcement Requests | 64 |
| Change vs Previous Day | |
These are the pages most frequently revisited by identified AI crawlers during the reporting period. Learn how repeated AI visits reinforce website understanding.
Reporting Period:
14 Jul 2026 – 12 Aug 2026
This matrix represents the cumulative movement behaviour observed across all recorded search system visits since measurement began. Unlike a daily snapshot, it reflects the long-term structural behaviour of search systems as they repeatedly explore your website. As additional observations are collected, the transition probabilities gradually stabilise, providing an increasingly reliable representation of how search systems navigate, interpret and reinforce your websites knowledge structure. Learn how Markov Chains model search system movement.
| From / To | E Entry Pages |
B Supporting Content |
X External / Exit |
H High-Value Pages |
S Structural Pages |
C Commercial Pages |
A Authority Core |
F Future Pages |
|---|---|---|---|---|---|---|---|---|
| E | 3.05% | 36.53% | 48.33% | 9.2% | 0.95% | 0.39% | 1.38% | 0.18% |
| B | 0.46% | 93.96% | 0% | 1.6% | 1.37% | 1.25% | 1.25% | 0.11% |
| X | 26.69% | 55.49% | 8.25% | 4.65% | 1.49% | 0.53% | 2.37% | 0.53% |
| H | 2.82% | 31.34% | 12.32% | 48.59% | 1.41% | 0% | 3.17% | 0.35% |
| S | 0% | 38.3% | 0% | 0% | 59.57% | 2.13% | 0% | 0% |
| C | 0% | 32.14% | 0% | 0% | 0% | 67.86% | 0% | 0% |
| A | 0% | 23.53% | 0% | 5.88% | 0% | 0% | 69.61% | 0.98% |
| F | 0% | 0% | 0% | 18.18% | 0% | 0% | 0% | 81.82% |
This analysis estimates the long-run equilibrium of observed search system movement through your website. Higher probabilities indicate areas that naturally attract and retain search system attention after repeated exploration. Learn how probability influences search visibility over time.
| State | Probability |
|---|---|
| E Entry Pages |
3.37% |
| B Supporting Content |
59.2% |
| X External / Exit |
4.37% |
| H High-Value Pages |
7.66% |
| S Structural Pages |
5.55% |
| C Commercial Pages |
6.23% |
| A Authority Core |
6.95% |
| F Future Pages |
6.69% |
Limited Core Discovery.
Few Entry Pages lead search systems into your Authority Core. Review internal links from important landing pages.
Supporting Content Isolation.
Supporting Content is receiving attention but is not effectively reinforcing the Authority Core. Consider strengthening contextual links back to your principal knowledge pages.
Healthy Internal Exploration.
Search systems continue exploring beyond your high-value pages, helping redistribute authority across the website.
Weak Authority Core.
Search systems are not naturally concentrating on your Authority Core. Review the internal structure and links leading to your principal knowledge pages.
Supporting Content Attraction.
Supporting Content is attracting sustained search system attention.
New Entrants
The website contains a highly dominant Supporting Content network and a smaller but strongly self-reinforcing Authority Core. The principal structural difference between them is accessibility: Supporting Content attracts movement from across the website, while the Authority Core demonstrates strong persistence primarily after it has already been reached.
Taken together, the report shows that automated interaction with the website is not evenly distributed.
Supporting Content demonstrates the strongest cumulative internal persistence, while the Authority Core also shows substantial reinforcement once reached. At the same time, the daily Core Capture Rate of 2.64% shows that only a small proportion of the day’s total recorded requests reached Authority Core pages.
The AI crawler data provides a second perspective. Identified AI systems are repeatedly accessing multiple pages, with some pages being revisited across numerous separate days rather than receiving only isolated crawler requests.
The report therefore measures three related aspects of machine interaction with the website: which systems are present, which pages and structural areas they access, and how recorded activity moves between those areas over time.
These results describe observed crawler and system behaviour. Repeated access can demonstrate observation and persistence, but it does not by itself prove that an AI model has learned, retained or incorporated the content into its internal knowledge.
Search System Behaviour Analysis makes the invisible activity of search engines and AI systems visible, revealing what they access, what they revisit, and how they move through the website.
