Next.js & React for AI: Making JavaScript Frameworks Visible to AI Crawlers
React, Next.js, and similar frameworks are excellent — but they can produce a page that looks perfect to a human and is completely blank to an AI crawler. The difference comes down to where your content is rendered. This guide explains the one question that decides it, and how to land on the right side of it.
The one question that decides everything
When an AI crawler (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) fetches your page, it reads the HTML your server returns. It usually does not run JavaScript the way a browser does. So the only question that matters is:
Is your text already in the HTML the server sends — or does it only appear after JavaScript runs in the browser?
If your main content is built in the browser, the crawler sees an almost-empty shell. → JavaScript and AI crawlers
Next.js: usually fine, if you let it render on the server
Next.js can be very AI-friendly, because it’s built to render on the server:
- App Router (Server Components) renders your components to HTML on the server by default. Your text ends up in the initial HTML — good.
- Pages Router is fine when you load your main content with
getServerSideProps(SSR) orgetStaticProps(static generation). That data is baked into the HTML.
The trap is data you fetch only in the browser — for example inside a useEffect, or with a client-side data library, where the page ships empty and fills in after load. A "use client" component is fine for interactivity, but if the words that matter arrive only after a browser fetch, they’re not in the HTML the crawler reads. Put your main content on the server path.
Plain React SPAs: the hard case
A client-only React app (Create React App, or Vite without server rendering) typically ships an empty <div id="root"></div> and draws everything in the browser. To a human it’s a full page; to an AI crawler it’s blank. Your options:
- Move to a framework that renders on the server — Next.js, Remix, or Astro.
- Add static prerendering for your content routes so crawlers get real HTML.
Don’t forget metadata and structured data
Rendering text on the server is most of the battle, but finish the job:
- Title, description, canonical, Open Graph — use the Next.js Metadata API (App Router) or
next/head(Pages Router) so they’re in the server HTML, not set later by JavaScript. - Structured data — output a server-rendered
<script type="application/ld+json">so AI gets explicit facts. → structured data · sitemaps & canonical
The honest part
You don’t have to abandon React. Hydration and rich interactivity are completely fine — AI reads the initial HTML, and your app can stay as dynamic as you like on top of that. The fix isn’t “use less JavaScript,” it’s “make sure the first HTML response already contains your words.” Server rendering, static generation, and React Server Components all achieve that.
How to check
Open “View Source” (not the inspector — that shows the live DOM after JS) or run curl https://yoursite.com and look for your actual paragraph text. If you see your headline and body copy, crawlers will too. If you see an empty <div id="root"> and a pile of script tags, your content is browser-only. Or run your URL through a free AI-visibility check to confirm at a glance.
The bottom line
React and Next.js are not the enemy — browser-only rendering is. Keep your main content on the server (SSR, SSG, or Server Components), put metadata and structured data in the server HTML, and verify with “View Source.” Start with the GEO overview and JavaScript and AI crawlers, and if AI still isn’t citing you, see why isn’t my site in AI.