We all know that Search Engine Optimization is built on the basis of outperforming your competitors in search engine results and maintaining that competitive level throughout the life of your site. But while we can all agree on what SEO actually is, there are many SEO myths that can cause even the best SEO expert to make mistakes that may not just cost their client’s search rankings, but may cost them in the way of resources as well.
SEO MYTHS: USING THE META ROBOTS TAG
Everyone knows the importance of Meta information in Search Engine Optimization, or at least they should. Those who don’t understand Meta Tags and their importance immediately needs to stop what they’re doing and learn the basics of SEO. One of these Meta Tags that you’ll see occasionally is the “ROBOTS” tag, but you don’t see these in the code of a main page. This “ROBOTS” tag is found in a custom document, usually manually created, called ‘robots.txt’ specifically designed to give you customized options regarding search indexes and comment spam, among a lot of other options. This file is read by Google, Bing, Yahoo, and other search engines so that they know which pages and what content is to be indexed by the “spiders” that crawl your site. By indicating a ‘noindex’ description following your tag, this tells the search engine (obviously) not to index particular pages. A ‘nofollow’ description following the tag will tell the crawlers not to follow your links for indexing (such as for the purposes of Backlinking). By default, as long as you have ‘robots.txt’ file, even if it is blank, Google will index and follow everything in a full-on site crawl. Why this is classified as a ‘myth’ is because Google will index and follow everything by default… it’s just considered a ‘best-practice’ to include a ‘robots.txt’ file for the benefit of your site’s crawl and indexation time
There are many things you can ultimately do using the ROBOTS Meta Tags. To see some of the other types of controls this tool gives you, check out this guide from Yoast here.
Image Source: Wikimedia Commons.