/tags/noindex/ Recent bookmarks tagged #noindex Mon, 02 Nov 2026 08:24:54 +0000 http://www.intelligentpositioning.com/blog/2009/08/x-robots-tag-control-google-indexing-via-http-headers/ We’ve all got used to being able to control how the major search engines index our sites using a combination of robots.txt and the robots meta tag to add attributes like ‘noindex’ to individual pages. While this works great for the pages themselves, it’s not so good for non-HTML, indexable content such as PDFs or embedded media, as we have no HTML <meta> tag in which to insert the meta-information. In this article we take a look at a potential solution to this problem: the X-Robots-Tag HTTP Header.<br/> <p><strong>Tags:</strong> #google #imported:pinboard #noindex #webdev</p> Wed, 08 Aug 2012 16:32:15 +0000 linkding-ssg:bookmark:32711 google imported:pinboard noindex webdev http://www.intelligentpositioning.com/blog/2009/08/x-robots-tag-control-google-indexing-via-http-headers We’ve all got used to being able to control how the major search engines index our sites using a combination of robots.txt and the robots meta tag to add attributes like ‘noindex’ to individual pages. While this works great for the pages themselves, it’s not so good for non-HTML, indexable content such as PDFs or embedded media, as we have no HTML tag in which to insert the meta-information. In this article we take a look at a potential solution to this problem: the X-Robots-Tag HTTP Header.<br/> <p><strong>Tags:</strong> #google #noindex #webdev</p> Wed, 08 Aug 2012 16:32:15 +0000 linkding-ssg:bookmark:22406 google noindex webdev