Crawlglass.

What it is

Crawlglass fetches your web page the same way an AI assistant's crawler does — plain HTTP, no JavaScript run — and shows you, side by side, what a person sees versus what the bot actually gets. A free scan checks extractable text, which named AI crawlers your robots.txt allows or blocks, structured data, headings, and any JavaScript-only content that never reaches the bot. Pro adds whole-site crawls, a white-label report, and generated llms.txt and schema.

Who it's for

Freelance web developers and small agencies who want a client-ready report on how AI-readable a site is, and any site owner who wants to know whether ChatGPT, Claude, or Perplexity can actually read their pages instead of assuming they can't.

Why I built it

AI assistants are becoming a real source of traffic and citations, and most site owners have no way to check whether their site is even reachable by the crawlers behind them. Existing SEO tools weren't built to answer that question. I wanted a tool that reports verifiable technical facts a developer can act on, rather than promising that a site will get cited or ranked by anything — that guardrail is the whole point of the product.

How it works

Paste a URL and Crawlglass fetches it under the same constraints an AI crawler has: no JavaScript execution, redirects followed the way a bot follows them. It reports which named crawlers — GPTBot, ClaudeBot, PerplexityBot, and others — are allowed or blocked in your robots.txt, whether you publish an llms.txt, what structured data is present, and what text is actually extractable. Pro crawls up to 25 pages of a site and turns the results into a dated, white-labeled PDF plus a CSV export you can hand to a client.

What's next

It's live and the scanner works end to end against real sites, with Pro subscriptions now purchasable. The next step is a proper launch post and finding the freelancers and agencies who'd actually buy the whole-site report.