HTTP headers archives

HTTP status codes, like 404, 301, and 500, might not mean much to a regular visitor, but they are incredibly important for SEO. Not only that, search engine spiders, like Googlebot, use these to determine the health of a site. These status codes offer a way of seeing what happens between the browser and the server. …

Traditionally, you will use a robots.txt file on your server to manage what pages, folders, subdomains, or other content search engines can crawl. But did you know there’s also such a thing as the X-Robots-Tag HTTP header? Here, we’ll discuss the possibilities and how this might be a better option for your blog. Quick recap: …

Sometimes, your site will need some downtime, so you can fix things or update plugins. Most of the time, this tends to be a relatively short period in which Google will most likely not attempt to crawl your website. However, in the case that you need more time to get things fixed, chances are much …

We’ve said it way back when, but we’ll repeat it: it keeps amazing us that there are still people using just a robots.txt files to prevent indexing of their site in Google or Bing. As a result, their site shows up in the search engines anyway. Do you know why it keeps amazing us? Because …

HTTP/2 is a newish protocol for transporting data that will drastically speed up the web and can help your SEO. The ‘old’ HTTP1.1 protocol only allows web servers to send files down a single line one at a time, and that line has to open and close after each file has been sent – a …

In 2016, a new HTTP status code saw the light. This status code, HTTP 451, is intended to be shown specifically when content has been blocked for legal reasons. If you’ve received a take down request, or are ordered by a judge to delete content, this is the status code that allows you to indicate …