noindex vs other signals
- noindex, follow — page stays out of the index, but the bot may follow links (most common variant).
- noindex, nofollow — no indexing and no link equity passed onward.
- robots.txt Disallow — blocks crawling; does not guarantee removal of URLs already indexed.
- Do not combine noindex with a canonical pointing elsewhere — conflicting signals for the bot.
Best practices
- Use noindex on pages that should not rank but may be linked (e.g. thank-you pages, tracking parameters).
- Remove noindexed URLs from the XML sitemap — consistent signals make crawling easier.
- On staging, apply noindex globally (meta or X-Robots-Tag) before the environment goes public.
- After removing noindex in production, check GSC for re-indexing — it does not happen instantly.
Common mistakes
Accidental noindex on a template (e.g. after a staging deploy with forgotten meta) is one of the most common SEO regressions. Technical monitoring and meta robots audits catch it before organic traffic drops. noindex alone does not instantly remove a page from the index — the bot must revisit and read the directive.
Example
A post-form thank-you page — reachable for users, but excluded from the index while still passing link signals.
HTML
<meta name="robots" content="noindex, follow">HTTP
X-Robots-Tag: noindex, nofollowFAQ
Does noindex stop bots from visiting the page?
No. The bot can still fetch the URL and read the directive. To limit crawling, use robots.txt or a non-200 HTTP response — a separate mechanism.
Can a noindexed page be in the sitemap?
Technically yes, but it should not be. A sitemap suggests URLs to index; noindex says the opposite. Keep those signals aligned.