The Google Blog explains the robots exclusion protocol:
“Usually when the Googlebot finds a page, it reads all the links on that page and then fetches those pages and indexes them.”
But, what if you don’t want Google to index a page, or follow the links on a page, or archive a page?
Here is a list of META tags you can use:
- NOINDEX tells Google not to index the page
- NOFOLLOW tells Google not to follow the links on the page
- NOARCHIVE tells Google not to store a cached copy of the page
- NOSNIPPET tells Google not to show a snippet (description) under your Google listing, it will automatically get NOARCHIVE too
September 5th, 2007 at 12:30 am
Thanks for these very useful tags!