What is “robots.txt Test” in GWT?

Sunday, December 13th, 2015

If you click on the Crawl tab on the side bar of GWT, you’ll see a link for robots.txt Tester.

Robots.txt Tester page displays whether you have errors or warnings in your robots.txt file. Sometimes a webmaster may add something to the robots.txt that may disallow Googlebot from accessing your site so your entire website may get de-indexed. It is good to check this page every once in a while.

Filed under: DocumentationWebsite Performance
Tags: , , , ,

Disclaimer: This information is intended for informational purposes. Some information may not be accurate or current and are of the author's opinion (not 10KHits). It's not meant as financial, investment advice, a recommendation regarding a course of action or legal advice.

Leave a Reply

Your email address will not be published. Required fields are marked *