Noted July 4, 2019

Great to see Google has expanded its guide to robots.txt

Remember robots.txt will not prevent a site from being indexed, only a noindex meta tag/HTTP header will do that.

By Peter Wilson

Peter has worked on the web for twenty years on everything from table based layouts in the 90s to enterprise grade CMS development.

Peter’s a big fan of musical theatre and often encourages his industry colleagues to join him for a show or two in New York or in the West End.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.