In a previous post I showed how you can hide individual pages from your WordPress page navigation menus. I also warned that they can still be indexed by search engines and therefore seen by unintended people through search results.
Why Do Search Engines Still Index My Hidden Pages?
Because we haven’t told them not to…yet. The solution for blocking search engines from indexing individual pages lies in using what are known as robots meta tags. In a nutshell, the robots meta tags deliver instructions to search engines on what to do when they get to the content of an individual web page. The optional tags you can specify are:
"index"
"noindex"
"follow"
"nofollow"
The “index” and “noindex” tell search engines to read the page or not. The “follow” and “nofollow” tell search engines to either follow the links on your pages or not. You can use any combination of these robots meta tags, but only the following combinations make sense logically.
meta name="robots" content="noindex", "follow"
meta name="robots" content="index", "nofollow"
meta name="robots" content="noindex", "nofollow"
How To Specify Robots Meta Tags on Your Pages and Posts
If you were writing individual HTML pages, you would need to insert these robots meta tags in the <HEAD> section of each of your pages. Because WordPress handles the creation of our pages dynamically it would be very difficult to do on our own…[private_free] Luckily, there are many WordPress plugins to help us accomplish this easily. Not only are there plugins to help us insert these tags on a per page/post basis, but at least one cutting edge theme company (StudioPress) has already built this functionality into their themes.
What Plugins Can I Use?
If you’re not using a theme that has robots meta tags built-in, you’ll want to grab a plugin to help you along. The plugin I prefer most is by Joost de Valk and is named Robots Meta. Once installed, it adds a “Robots Meta” box to the right side of your “Post/Page Edit” screens. It looks like this.
While creating your post or page, simply choose what robots meta tags you would like to include on that particular post or page. Whenever I hide a page from my navigation menus, chances are that I don’t want it seen by anyone except the intended party, so I simply choose “noindex, nofollow”. However, there are situations where you might want to hide a page and still have it indexed…but more on that in a later post. You can explore more robots meta plugins here.
Robots Meta Options in Themes
There are a few themes out there that can help, but if you happen to be using any themes from StudioPress (and I highly recommend that you consider it), you will see the same robots meta options on your editing screens like in the image below.
StudioPress themes also have other built-in Search Engine Optimization settings available. You can learn more here.
