robots.txt
User-agent: *
Disallow: /robots.txt
Whether a rule stops Googlebot reading your site or particular pages, using the same rule order Google follows.
What we look at
Page and header
noindexNoindex
In the page's meta tags and in the X-Robots-Tag header, on your home page and a sample from your sitemap.
sitemap.xml
<loc>…/pricing</loc>Your sitemap
Whether it exists, reads correctly and is listed in robots.txt, and whether its pages load.
Canonical tag
rel="canonical" href="other-site.com"Canonical tags
Whether a page tells search engines that its main address is somewhere else.
Common questions
Why is my page marked noindex when I can't see the tag?
It may be in the X-Robots-Tag response header, which never shows in the page source. SEO plugins, staging settings and proxies can all add it. This check reads both.
Google says "blocked by robots.txt". What do I change?
Find the Disallow rule that matches the page and remove or narrow it. If a rule names Googlebot, Google ignores the general rules, so check that group first.
Does a clean result mean Google has indexed my site?
No. It means nothing obvious is in the way, based on a sample of pages. Whether Google actually indexed a page is only shown in Search Console.
How quickly will pages come back after a fix?
That's up to Google and varies from days to weeks. You can ask for a page to be recrawled in Search Console.