What is “robots.txt Test” in GWT?

Sunday, December 13th, 2015

If you click on the Crawl tab on the side bar of GWT, you’ll see a link for robots.txt Tester.
image063

Robots.txt Tester page displays whether you have errors or warnings in your robots.txt file. Sometimes a webmaster may add something to the robots.txt that may disallow Googlebot from accessing your site so your entire website may get de-indexed. It is good to check this page every once in a while.

Filed under: DocumentationWebsite Performance
Tags: , , , ,

Disclaimer Notice


Exclusive Offer!

1,500,000 Points for $110 $68. If you're running low on points — don't miss out

Buy Now — Save $42!

limited time