AI crawlers saw 82 words of our website. Here's how we fixed it.
Until this week, an AI crawler visiting betcollab.com got 82 words: a page title, a meta description and a small block of structured data. Not one word of the copy you see in a browser. The site was a client-rendered React app, so every sentence was assembled by JavaScript after the page loaded, and the crawlers behind ChatGPT, Claude and Perplexity don't run JavaScript. Anthropic's documentation says its web fetch tool “does not support websites dynamically rendered with JavaScript”.
We fixed it by prerendering every page to static HTML at build time. The design didn't change. The homepage now serves more than 700 words of real copy to a crawler, unknown URLs return a real 404, and our build fails if any of that regresses.
If your sportsbook is a single-page app, there's a good chance you have the same problem. Finding out takes one command.
The five-minute test
Run this against your homepage. It fetches the page without running JavaScript, strips the tags and counts what's left:
curl -sL https://www.your-sportsbook.com/ | sed 's/<[^>]*>//g' | wc -wA few dozen words means crawlers that don't execute JavaScript are seeing an empty page. Googlebot does run JavaScript, so Search Console can look perfectly healthy while ChatGPT has nothing to read. That's exactly where we were.
Before and after
| Check | Before | After |
|---|---|---|
| Words a crawler sees on the homepage | 82 | 750+ |
| A URL that doesn't exist | 200 OK | 404 Not Found |
| /llms.txt | The HTML app shell | Plain text |
| Pages in the sitemap | 1 | Every page |
| Canonical URL on inner pages | The homepage, on a host that redirects | The page itself |
| AI crawlers named in robots.txt | None | 11 |
Three things we didn't expect
Our headline read as one word
“Design Retainers for iGaming Operators” was built from one element per word so each could animate in, with the gaps added by CSS margin. Strip the tags and it becomes DesignRetainersforiGamingOperators. The same happened to our stats, nav links and buttons. The fix is a real space character that the browser collapses visually. If you split text for animation, check it.
Every page said it was the homepage
One canonical tag sat in the shared HTML template, so the design system page and the work page both told search engines they were duplicates of the homepage. The URL in that tag also redirected to the www version of the domain. Each page now declares its own address.
Our contact form was hard for an agent to fill in
The labels weren't connected to their inputs, and our “Book a call” buttons were scripts that scrolled the page rather than links. Google's guidance on agent-friendly websites asks for real buttons and links, labels tied to fields with the for attribute, and interactive elements larger than 8 square pixels. An automated accessibility check now finds no violations on any of our pages.
What we didn't do
We didn't rebuild the site in a new framework. A prerender step on top of the existing React app was enough, and it took a day.
We didn't treat llms.txt as the answer either. Google says you don't need new machine-readable files, AI text files or special schema to appear in its AI features. We added one because it costs nothing, not because it moves anything.
And being readable isn't the same as being recommended. Whether an assistant names you depends on whether other sources agree about who you are and what you do. Readable HTML is the entry ticket. It doesn't win the game.
Why sportsbooks should care more than most
Odds, markets, promotions and the bet slip are almost always rendered in the browser. On a typical sportsbook, the content an assistant would need to describe you is exactly the content it can't see. Bot protection makes it worse: OpenAI says sites that opt out of OAI-SearchBot won't appear in ChatGPT search answers, and a challenge page can have the same effect without anyone deciding it should. We're running this test across UK sportsbooks next.
The checks we now run on every build
- Every content page serves at least 300 words of real text without JavaScript
- Unknown URLs return 404, and
llms.txtis served as plain text - Every page has its own title, description and self-referencing canonical
- Structured data parses, and every price in it appears on the page
- Every page links to every other top-level page, so nothing is orphaned
- Every form field has a label, and no words run together when tags are stripped
- robots.txt names the AI crawlers we want, and the sitemap lists every page
If you'd like us to run the same checks on your sportsbook or casino, get in touch. We'll send you the numbers.