Robots meta tags are HTML elements that provide instructions to web crawlers (like search engine bots) on how to handle the indexing and crawling of a web page or website. These tags are placed in the <head> section of an HTML document and help control the behavior of web robots.
Here are some common robots meta tags with examples:
- robots=”noindex”: Instructs search engines not to index the page.
Example:<meta name="robots" content="noindex"> - robots=“nofollow”: Instructs search engines to not follow links on the page.
Example:<meta name="robots" content="nofollow"> - robots=“noarchive”: Instructs search engines not to cache or archive the page.
Example:<meta name="robots" content="noarchive"> - robots=“nosnippet”: Instructs search engines not to display a snippet or description for the page in search results.
Example:<meta name="robots" content="nosnippet"> - robots=“all”: Instructs search engines to index and follow all links on the page (default behavior).
Example:<meta name="robots" content="all"> - robots=“none”: Instructs search engines not to index, follow links, or cache the page.
Example:<meta name="robots" content="none">
These tags can be combined by separating them with commas. For example:<meta name="robots" content="noindex, nofollow">
Robots meta tags are useful for controlling how search engines interact with specific pages or sections of a website, allowing webmasters to optimize their site's visibility and crawling behavior.
Related: