What is the difference between "index, follow" or "follow"?

Question

What are the differences in the lists below:

<meta name="robots" content="index, follow">

<meta name="robots" content="follow">

<meta name="robots" content="noindex, follow">

<meta name="robots" content="index, nofollow">

<meta name="robots" content="noindex, nofollow">

Eugeniaeugenics

It is spelled out in the documentation (e.g. developer.mozilla.org/en-US/docs/Web/HTML/Element/… , under robots.) What exactly do you find unclear about it? – Glaze

Answer 1

First of all, you may know about meta robots tags.

Meta robots tags are to tell the spider or crawler which page to crawl or index or not.

Follow means: page will be crawled.

No Follow means: page will not be crawled.

Index means: your page show in search Results.

No Index means: your page does not show in search Results.

---

<meta name="robots" content="index, follow">

This tag tells the web crawler all pages will be indexed and to crawl them.

---

<meta name="robots" content="follow">

This tag is used to tell the web crawler all pages will be crawled and indexed.

---

<meta name="robots" content="noindex, follow">

This tag is used to tell the crawler to follow but not index the page in your database.

---

<meta name="robots" content="index, nofollow">

To tell the web crawler to not crawl the page, but index them.

---

<meta name="robots" content="noindex, nofollow">

To tell the web crawler to neither index nor crawl.

Dulcimer

"follow" has nothing to do with the current page getting crawled. The crawler has to crawl the page to see the tag. It has to do with whether the crawler should follow the links on the page to crawl other pages. – Mikes

Answer 2

The spider will index your whole website. The spider will not only index the first web-page of your website but also all your other web-pages as it crawls through links from this page.

Tells the search engines that it may follow links on the page, to discover other pages (both of these are default behaviour).

<meta name="robots" content="index, follow">

<meta name="robots" content="follow">

The spider will now index your whole website. The spider will not index the web-page but it may follow links on the page, to discover other pages.

<meta name="robots" content="noindex, follow">

The spider will index this page but it won't follow links on this page to new pages.

<meta name="robots" content="index, nofollow">

The spider won't index this page at all and won't follow links on this page to any other pages.

<meta name="robots" content="noindex, nofollow">

Sources:

• Meta name robots tag

• The ultimate guide to the meta robots tag

• <meta>: The metadata element

Tuchun

Answer 3

The main difference between these tags is:

noindex: The page is not supposed to be shown in the result list of sites like Google, Yahoo and so on.

index: Means the opposite and allows it.

nofollow: Disallows crawlers/robots to call (follow) links that are embedded in (found on) pages that carry this flag in their meta data.

follow: Means the opposite and allows it.

Wrung