Meta Robots Tag
In one line
The meta robots tag sits in the page <head> and tells crawlers whether to index it and follow its links — the per-page lever for indexing policy.
Going deeper
The meta robots tag goes in the page head as <meta name="robots" content="...">. The directives you actually use are mostly noindex, nofollow, none, noarchive and nosnippet, combined to control per page whether to index it, follow its links and show snippets.
Two mistakes show up a lot. First, leaving 'noindex, follow' on for long stretches — Google has said that over time noindex pages effectively stop passing link signals, so the combo is less useful than it looks. Second, a page launched with a temporary noindex that quietly gets stuck that way. A simple pre-launch checklist line — 'verify robots tag' — kills a surprising amount of damage.
In the AI search era, nosnippet and max-snippet are getting fresh attention. They give you a partial lever over how AI Overviews and ChatGPT-style answers quote your content, which is increasingly part of the on-page strategy conversation.
Related terms
X-Robots-Tag
X-Robots-Tag is the HTTP response header version of the meta robots directive — useful when you need to control indexing for non-HTML assets like PDFs, images and other binaries.
SEOIndexability
Indexability is whether a crawled page is actually eligible to be stored in the search index — a page can be crawlable yet still not indexable.
SEOIndex Bloat
Index bloat is the state where too many low-value or duplicate URLs are sitting in the search index, dragging down the site's average quality and indirectly hurting rankings.
SEOJavaScript SEO
JavaScript SEO is the discipline of making sure JS-rendered pages get crawled and indexed correctly — usually via some combination of SSR, prerendering and hydration.
GEO·AEOAI Overview
Google AI Overviews is the AI-generated summary that appears above the standard results in Google Search — one of the most prominent zero-click surfaces today.
How does your brand show up in AI answers?
Villion measures how your brand appears across ChatGPT, Perplexity and AI Overviews, then automates the work that lifts citation rate and share of voice.
Get a free audit