Make sure the important content and links are in the HTML that the server sends, using server rendering or static generation.
For teams on React or Next apps worried search engines and AI bots see an empty page. You will confirm what is seen and know what to ask developers for.
Modern sites built with React, Vue or similar frameworks often show a blank or near-empty page until JavaScript runs. Google can render JavaScript, but rendering is slower and less reliable than reading plain HTML, and many other crawlers, including a number of AI bots, do not run scripts at all. The safest position is to make sure the content that matters is in the first response.
1Compare what the server sends with what the browser shows
There are two versions of every page. The raw HTML is what the server sends. The rendered HTML is what exists after scripts have run. Compare them for a handful of key pages.
curl -s https://example.com/features | grep -i "<h1"
curl -s https://example.com/features | grep -c "<a href"In your browser, use View Source for the raw HTML and the Elements panel for the rendered DOM. List everything that exists in the second but not the first: headings, body text, links, structured data, metadata.
2Move the important content to the server
The most dependable fix is to produce the finished HTML on the server, or at build time, so the first response already contains the content. Most modern frameworks support this.
| Approach | What it means | Crawler view |
|---|---|---|
| Client-side rendering | The browser builds the page with scripts | Empty until scripts run |
| Server-side rendering | The server builds the HTML on each request | Complete first response |
| Static generation | Pages are built ahead of time | Complete first response |
| Hydration | Scripts add interactivity to ready HTML | Complete, then interactive |
In the first response
- Headings and body copy
- Links to other pages
- Title, description and canonical
- Structured data
Fine to load later
- Chat widgets
- Animations
- Personalisation
- Interactive demos
3Use real links
Crawlers discover pages by following links in the HTML. A link that exists only as a click handler is invisible to them.
Not crawlable
<span onclick="go('/pricing')">Pricing</span> or <a href="#" onclick=...>
Crawlable
<a href="/pricing">Pricing</a>
Frameworks can still intercept the click for a smooth transition. The real href is what a crawler follows.
4Give every URL its own metadata
Each indexable page needs its own title, description and canonical in the server response. A single-page app that changes these only in the browser may present identical metadata to crawlers for every URL.
5Test as a crawler, and again after each release
- Run URL Inspection in Search Console and read the rendered HTML and screenshot
- Fetch with curl and check the content is in the raw response
- Check that blocked scripts or styles in robots.txt are not stopping rendering
- Recheck after each major release, since new components can quietly move content to the client
Worked example
A feature page that looked empty (illustrative)
A made-up product site renders its feature pages entirely in the browser. Search Console shows them as crawled but with thin content, and AI assistants know nothing about the features.
Developers switch content pages to static generation, keep the demo widget client-side, and replace button-based navigation with real links. A curl fetch now returns the full headings, copy and links, and the pages begin to be indexed with their full content.
Say it in 30 seconds
The 30-second version
Compare the raw and rendered HTML of key pages. Ask for server rendering or static generation so headings, copy, links and metadata are in the first response, use real anchor links for navigation, and test with a plain HTML fetch after each release.
Worksheet
JavaScript crawlability checklist
The full guide and a worksheet to fill in, as a PDF you can print or share with your team. The worksheet covers:
- Pick five key pages and view source for each
- View the rendered DOM for the same pages and compare
- List content, links and tags present only after scripts run
- Share the list with developers and ask for server rendering or static generation on content pages
- and 7 more
One email with the PDF. No list, no follow-ups.
