The Google Blog explains the robots exclusion protocol:

“Usually when the Googlebot finds a page, it reads all the links on that page and then fetches those pages and indexes them.”

But, what if you don’t want Google to index a page, or follow the links on a page, or archive a page?

Here is a list of META tags you can use:

• NOINDEX tells Google not to index the page
• NOFOLLOW tells Google not to follow the links on the page
• NOARCHIVE tells Google not to store a cached copy of the page
• NOSNIPPET tells Google not to show a snippet (description) under your Google listing, it will automatically get NOARCHIVE too