How to add html robot meta tags to customize indexing?

These tags in the HTML page specify whether SearchBlox can or cannot index the page, and can or cannot spider the entire website.

The following robot meta tag specifies that the page can be indexed and the page can be crawled for other links.

<meta name="robots" content="index, follow">

 

 The following robot meta tag specifies that the page can be indexed and the page cannot be crawled for other links.

<meta name="robots" content="index, nofollow">

 

 The following robot meta tag specifies that the page cannot be indexed, but the page can be crawled for other links.

<meta name="robots" content="noindex, follow">

 

 The following robot meta tag specifies that the page cannot be indexed and the page cannot be crawled for other links.

<meta name="robots" content="noindex, nofollow">
Have more questions? Submit a request

Comments