Meta tag myths

The "Robots" meta tag

Here are a couple of examples of meta tags SEO (search engine optimisation) companies may put into your site to justify their bill:

<meta name="robots" content="index, follow">
<meta name="googlebot" content="index, follow">

Robots tags are used by search engines, not to find out which pages to index, but to find out which ones the website owner does not want to be indexed by Google, Yahoo, MSN etc..

The first example is a generic "robots" tag aimed at all search engines. The second example is a "robots" tag targetted just at Google.

By default a search engine will index the whole of your website by going in at the home page and following all the links it finds, thus, if your web site is built properly (plug for 123Live here!) the search engine will find all your pages and index them (let people find it when they search for terms included in your website). There is no need to tell them to do this. It's their job to search web pages by following links and indexing the pages so that people like you and me can find them.

The danger in putting these tags into your web pages, apart from bloating the page a little, is that the search engines are looking for this tag but they are looking for "noindex" or "nofollow". If they find "noindex" they will not index this page, and if they find "nofollow" they will not follow links from this page to other pages.

Search engines have a potential copyright issue if they do index a web page who's owner does not want it to be indexed. This applies particularly to those search engines, like Google, that keep copies of your pages in their cache. So, if there is a typo in your robots tag, the search engines may just err on the side of caution and not index the page, or follow links on the page, just in case.

In other words, one slip and your website might not get listed in the search engine at all.

I short, don't put robots tags in your web pages. Don't worry, we've made sure you can't do this in 123Live.