Technical SEO for AI Search: What Google Actually Documents
If search is increasingly generating answers instead of listing links, does technical SEO still do anything? For Google Search's generative AI features, yes, with a qualification worth sitting with: information has to be accessible before it can become a candidate at all. Access is not the same as selection, and most advice on this topic blurs the two together, which is where it starts to mislead people.
Google is the clearest case study here, mainly because it publishes documentation on the point rather than leaving practitioners to guess. In its guide to optimising for generative AI features, Google states that the SEO fundamentals that applied before AI Overviews and AI Mode existed still apply, and that from its perspective, optimising for generative AI search is still SEO. There is no parallel technical standard next to the familiar one, so the work of getting a page through that gate hasn't changed. What's changed is what happens on the other side of it. Other AI systems, built on different indexes and crawlers, may weigh accessibility differently — this piece is about what Google documents for its own generative features, not a claim about how every AI product works.
What Google Actually Requires
The requirement, as Google states it, is narrow. To be eligible to appear in AI Overviews or AI Mode, a page must be indexed and eligible to be shown in Google Search with a normal snippet, fulfilling the same Search technical requirements as standard results. Google's guide adds one more condition: a site also has to remain included in Search generative AI features inside Search Console, since site owners can opt out. Beyond that, nothing further is asked for: no AI-specific markup, no dedicated submission path, no special file format.
Google is equally direct that meeting these requirements guarantees nothing. Satisfying every requirement, best practice, and policy doesn't mean Google will crawl, index, or serve a given page. Technical compliance buys a ticket to be considered, not a ticket to be shown.
Google's guide also pushes back on tactics circulating under the AEO and GEO labels, in a section it labels "mythbusting": no need to "chunk" content for AI systems, no need to rewrite copy in some AI-specific register, and no special schema.org markup required for generative features. Structured data is still worth using for the rich-result benefits it provides, but it isn't a generative-AI requirement.
How Crawling, Rendering and Indexing Affect Whether a Page Can Be Considered
Crawling, rendering, and indexing get used almost interchangeably in casual conversation, but they're different stages, and a failure at any one can knock a page out before it has a chance to be considered.
Crawling is a crawler's ability to fetch a page at all. A robots.txt disallow rule, a login wall, or persistent server errors stop this before anything else happens. But robots.txt controls crawling, not indexing directly. Google's own documentation on robots.txt notes that it "is not a mechanism for keeping a web page out of Google," and that a page blocked from crawling can still appear in search results — typically without a description — if Google learns about the URL some other way, such as an external link. A page that can't be fetched can't be processed as content in the normal way, but blocked crawling and blocked indexing are not the same failure, and conflating them leads site owners to the wrong fix. Where a page truly needs to be kept out of Google entirely, the documented mechanism is a noindex directive or authentication, not robots.txt alone. Authentication barriers and persistent server failures, by contrast, do prevent successful access outright, since there's no content for Google to fetch in the first place.
Rendering happens after fetching, when Google processes the page roughly the way a browser would. JavaScript itself isn't necessarily the problem; Google's systems are able to process JavaScript-driven content as long as it isn't blocked. The real risk is narrower: content or links dependent on rendering that times out or fails, leaving Google with an incomplete version of the page even though the crawl succeeded. That's a risk to check for on JavaScript-heavy sites, not a default assumption that any such site has a problem.
Indexing is the decision to store a processed version of the page at all. A noindex directive, once processed, tells Google not to index the page. Canonicalization is a separate mechanism: it signals which URL among several similar ones should generally be treated as the preferred version, rather than removing anything from consideration outright. Google's own canonicalization documentation frames this as a signal it weighs alongside others, not an absolute instruction it always follows. The two get confused because both can result in a page not showing up, but one is closer to an instruction and the other a preference signal, and a conflicting set of canonical signals can leave that preference ambiguous.
Crawler access can also matter in ways specific to a given AI product rather than to Search overall. A SIGIR 2026 study by Grossman and colleagues compared the sources behind Google Search, AI Overviews, and Gemini across an 11,500-query benchmark and found that websites blocking Google's dedicated AI crawler were significantly less likely to be retrieved by AI Overviews, despite the pages being fully accessible otherwise. That's a published research finding rather than something Google documents as a rule, and it measures retrieval rather than citation specifically. It's part of a broader pattern the same study reports: source overlap between the three systems was consistently low, even though each returned a similar number of sources per query — the researchers attribute the divergence to differing retrieval methodologies rather than one system simply pulling from a shallower pool. The narrower, practical point for a site owner: which crawler a site permits can matter separately from whether the page is indexed for ordinary Search, and that distinction is worth checking independently.
A sitemap helps Google discover a URL. It's part of normal technical discovery, not a special AI-visibility mechanism — it doesn't tell Google to index a page, and Google doesn't document sitemap submission as a factor in AI selection specifically.
How Site Architecture Affects What Gets Discovered
An orphan page, one with no internal links pointing to it, can still be discovered and indexed through other paths, such as an external link or a sitemap entry. What it lacks is context: the internal relationships that tie it to the rest of the site and help Google understand how the page relates to what else is there. Internal links help discovery and connect a page into the site's broader structure; their absence doesn't automatically mean a page can't rank or appear, but it means the page is relying more heavily on other discovery paths.
What Structured Data Actually Does
Structured data, implemented in JSON-LD, Microdata, or RDFa, helps Google parse specific facts on a page: that a set of steps is a recipe, a number is a price, a date belongs to an event. Google's general guidelines for structured data state what this buys: markup that follows the guidelines becomes eligible for rich-result display. Eligible, not guaranteed.
Structured data can describe a fact precisely. It can't make an AI system cite the page containing it. There's no special schema.org markup Google requires for generative features, and no documented mechanism by which adding schema causes a page to be selected as a source for a generated answer. Its value is real but narrower than the "AI visibility" framing implies: it reduces ambiguity about the specific properties it marks up, for the systems that choose to parse them.
Why Technical Accessibility Does Not Guarantee an AI Citation
This is the distinction most AI-SEO advice collapses, and the rest of this piece rests on it.
Crawlable doesn't mean indexed. Indexed doesn't mean visible for the queries that matter. Visible doesn't mean retrieved as a candidate for a specific generated response. Retrieved doesn't mean cited. Cited doesn't mean clicked. Each link in that chain runs on different mechanics, and none follows automatically from the one before it.
Google hasn't published a stage-by-stage architecture matching this exact sequence, so it helps to treat it as an editorial framework for discussing where technical SEO's influence ends, rather than Google's documented internal pipeline: access, crawl, render, index, understand, retrieve, select, cite. Technical SEO does real work in the earlier stages, from access through indexing. From "understand" onward, a different set of factors takes over: relevance, comparison against other candidate pages, and for generative features, a selection process Google hasn't documented in detail. Its own guide gestures at part of that process through retrieval-augmented generation and query fan-out, where the model issues related queries to gather material across a topic, but stops well short of explaining how one retrieved page gets chosen over another. That helps explain why different AI search systems can produce different sources for the same query, even when they are drawing on the same open web. Other AI systems may structure this process differently again, or not at all in any way Google's documentation would predict.
A page can be fast, indexed, and well-structured, and still not be the one selected. That isn't necessarily a technical failure waiting to be fixed. Eligibility and selection are answering two different questions.
What Content Quality Still Has to Do With It
Technical accessibility gets a page into consideration. It doesn't give a system a reason to prefer that page once several candidates clear the bar. Google's guide is explicit that, of everything it recommends, creating content people find unique and useful is likely to matter more for long-run visibility in generative AI search than any of its technical suggestions. It draws a specific distinction between a firsthand account built on direct experience and a summary that restates what's already available elsewhere, and between what it calls commodity content — generic advice that could have come from anyone — and non-commodity content built on a specific, expert, or lived vantage point.
That's not a disclosed ranking formula, and it isn't a technical fix. It's the other half of the argument this article has been making from a different angle: technical SEO can make information accessible and processable. It can't manufacture a reason for a system to prefer one accessible, well-structured page over another equally accessible one that says nothing distinctive.
What llms.txt Does and Does Not Do
llms.txt is a proposed Markdown convention intended to give AI systems a curated overview of a site's important content. It's separate from robots.txt, and its adoption and role across AI systems remain unsettled.
For Google Search, the position is documented directly, under the heading Google itself uses for this kind of claim: mythbusting. Google's guidance states that llms.txt and other AI-specific files, markup, or Markdown summaries aren't needed to appear in Google Search's generative features, because Google Search doesn't use them for that purpose. Creating one for other tools won't hurt a site's Google visibility, since Google ignores it either way. That's a claim about Google Search specifically. It says nothing about how other AI platforms handle the file, and current public evidence doesn't establish how widely, or how consequentially, it's used elsewhere. It isn't a Google Search requirement, and beyond that, its role for other systems is a separate and still-open question, not one this piece resolves.
Indexing gets treated as a finish line. It isn't one. A page being indexed means Google has stored a processed version of it and can potentially serve it, nothing more specific than that. It doesn't guarantee ranking, retrieval as a candidate for a generated response, or citation if retrieved. Indexing is entry into consideration, not a guarantee of anything downstream.
How to Audit a Site for AI Search
Because AI eligibility runs through the same fundamentals as ordinary search eligibility, an audit doesn't need a separate AI checklist, just the standard questions asked with "is this page even a candidate" in mind.
Does the page return a normal response, free of robots.txt rules that would block crawling? Is it indexed, and if not, is that a noindex directive, a canonical pointing elsewhere, or a quality judgment Google made on its own? Does the rendered version match what was intended, particularly where content or links depend on JavaScript? Search Console's URL Inspection tool shows the rendered HTML as Google processed it, the most direct way to check this. Do internal links connect the page into a relevant cluster of content? Does structured data validate, and does it match what's visible on the page? Does the page say something a summary of the same topic elsewhere wouldn't? And is the site still opted into Search generative AI features in Search Console?
None of this needs tools built specifically for AI search. It needs existing technical SEO practice applied carefully to the pages that matter most.
When Search Console's Generative AI Report Is Useful
Google provides reporting for visibility in its generative AI features, separate from standard performance reporting, covering AI Overviews and AI Mode. This kind of reporting can show impressions grouped by dimensions such as page, country, device, and date, though the specific fields on offer are an implementation detail Google continues to adjust.
What a report like this can do is confirm that a page appeared inside an AI-generated response. The dedicated report can't explain why Google's systems chose that page over a comparable one, and it doesn't provide click data or the specific queries behind an AI impression. A rising impression count confirms visibility exists somewhere in Google's AI features. It says nothing about why that source was picked, or what would need to change for a different page to be picked instead. Treat it as a visibility signal, not a selection explanation.
What Technical SEO Cannot Control
Technical SEO removes barriers. It doesn't choose the winner. It can make a page a viable candidate for crawling, indexing, and eventual retrieval. It can't decide whether a search or AI system selects that page over a competing one, how a generative system paraphrases or attributes it once selected, or whether an AI-generated summary reduces the click-through a well-ranked page might once have earned.
It also has no reach into how other AI systems operate. A chatbot or AI search tool built on a different index, crawler, or retrieval process than Google's makes selection decisions that technical SEO practice, built around Google's documented requirements, simply can't see into. Different systems can produce different answers from similar underlying web content for reasons that often have little to do with any one site's technical setup, from differences in retrieval infrastructure to how each product's crawler is allowed to access the web in the first place.
That's the boundary this argument rests on. Technical SEO makes information accessible to the systems that consume it and gives them something processable once they arrive. It cannot compensate for content that offers nothing distinctive once it's there. What happens after accessibility is established — whether the information is selected, cited, or clicked — depends on systems, quality judgments, and decisions a site owner doesn't get a vote on.
