You can do a million and one different things to optimize your site’s visibility in the search results pages. Every last one of them requires one thing to happen first, and that one thing is indexation. It doesn’t matter how good your content is, how well-used your keywords are, or how nicely formatted your post is; if Google can’t find it, it might as well not exist.
There are many ways you can try to get Google to index your site, but one of the fastest, easiest, and most consistent is uploading a sitemap.
Generating a sitemap is easy. Many blogging platforms even create them automatically. Your site might even have one by default; you can check by checking your domain with /sitemap.xml at the end. If you have one, you’ll see something that looks like this. There are two quirks of ours that you probably won’t see, though:
Either way, the result is the same; either you have a file there or you don’t. It’s easy enough to check.
Uploading a sitemap to Google is also easy. All you need is your site verified in the Google Search Console and then to visit the “upload sitemap” link and give Google a link to the file.
End of the story, Google has access to an index of all your pages, and you’re good to go.
Sometimes, when you give google the link to your sitemap, Google will throw an error. They can’t fetch it, which means they can’t use it to index your site.
Now, before you panic, this doesn’t mean Google can’t index your site. They can (probably) still see your site and index your pages. They don’t have a nice and easy list of all of the pages and posts on your site for them to index. They might miss a page, mainly if it isn’t linked to from elsewhere.
If your site has indexation issues, that’s another story and worth writing a separate post on the subject. We might cover that another time. For now, let’s talk about the sitemap.
If you can verify that the sitemap exists, you’re in a bit of a pickle. The problem is that Google won’t find the sitemap, and there’s not much you can do about it. You can try a few potential solutions, though, so let’s go through them.
The first option you should try is checking to see if legitimate problems are preventing Google from indexing your sitemap. Verifying that you have a sitemap and that it’s publicly visible is just one part of the puzzle.
One potential issue is that a directive, either in your site robots.txt file or your .htaccess file, is blocking Google from seeing your sitemap. Both of these methods are generally used to prevent indexation of system pages (like an admin login page, a WordPress attachment page, or a file directory) so they won’t show up in the search results and can’t damage your site’s overall quality levels by being thin, useless pages.
Unfortunately, sometimes useful system pages, like sitemaps, get added to the blocklists. Usually, Google will have a specific “this sitemap is blocked by robots.txt” error if this is the case, but in rare instances in which Google gets confused, they can mix up the errors.
You can read more about robots.txt blocking here.
Another possible source of blocking, if your web host uses Apache, is the .htaccess file. This file allows you to deliver commands to visiting users, including bots and can do everything from forcing www or non-www versions of URLs, adding redirects, and restricting access from bots. You can read more about how it works here. Check that file and look for anything indicating that your sitemap is being blocked or redirected.
This XML Sitemap Validator Tool is a third-party tool that checks your XML sitemap to make sure it’s well-formed and not broken somehow.
Again, this is a case of “Google will tell you a specific error about this” if it’s the case, but if the sitemap is broken oddly, Google might not know how to parse it and say it’s inaccessible. By validating your sitemap with this tool, you can see if any errors need fixing.
This one might seem odd, but it’s a clever troubleshooting tip. Bing is one of Google’s biggest competitors. That’s not saying much – they still have a pretty low percentage of the market share – but they have many of the same kinds of tools that Google offers. Including, in this case, the search console equivalent, Bing Webmaster Tools.
If you’ve never used Bing before, you’ll need to set up your account. You can choose to have them pull information directly from Google Analytics, or you can have them index data themselves. Once you’re up and running, you can find the Submit Sitemap button under the Sitemaps section on the left-hand side of the home screen.
There are two reasons to do this. This first is to check if Bing also has trouble indexing your sitemap. If they do, it indicates your problem has to do with your site or your web host. If they don’t, it’s more likely to be on Google’s end.
The second reason is to get Bing to index and rank your site more effectively. Bing might be a small percentage of the search market share, but it’s better than zero, so it’s beneficial to check in with them from time to time.
Sometimes, bugs and errors creep into massively complex mechanisms like Google. You may be stuck and need to remove your existing sitemap and resubmit it. In the Search Console, you need to click the Remove Sitemap button, wait a few minutes, and then submit your sitemap URL again. If you’re lucky, it will work the second time (or third, or fifth) around, and Google will successfully crawl and index the pages in your sitemap.
If it doesn’t work after four or five tries, don’t keep hammering at it. Either come back later and try again or proceed to other possible solutions.
Your second option for solving this issue is, unfortunately, just waiting. Sometimes, Google uses the “couldn’t fetch sitemap” error to indicate that they can’t do it right now. Maybe they have a lot of work queued up for the Google bots to handle, and yours will take a few hours or a few days to get to.
Maybe something broke, but it’s more like a micro-downtime from your web host or a strange, temporary configuration issue.
Waiting isn’t always ideal. If you’re trying to get certain pages indexed faster, you can use techniques like submitting them to the URL inspector or performing enough outreach to earn high-value links to those pages. But luckily, if waiting is all it takes, you don’t need to work yourself up over the issue.
If you’ve tried waiting and the issue persists even after a week or two, you probably have a more persistent problem. It may be on Google’s end, and it may not be something you can solve, but you can potentially “trick” Google into getting it to work. That’s what the other options on this list are all about.
This scenario is most likely an issue with sites that have been around for quite a while.
You probably had to do a lot of redirecting, many link changes within your internal links, and a lot of waiting while Google shuffled around your site in their rankings because technically, all those URLs are new, and they need to figure out what’s going on.
It’s also possible that when you linked your website property to the Google search console, you linked the HTTP version (because that’s what you had at the time), and since then, you have added the HTTPS version. You probably (hopefully) only use the HTTPS version by now since HTTPS is a search ranking, security, and trust factor.
Unfortunately, the presence of the HTTP version might be causing problems. Some people have reported that removing the HTTP property and re-adding it as HTTPS is enough to fix these continuity issues.
This one is bizarre, but it works for quite a few people. All you do is add another / to your URL.
Specifically, say your website is https://www.yourdomainname.com. This domain makes your sitemap URL:
What you do is add another / in front of the sitemap. So, it would look like this:
Strangely, Google doesn’t have an issue with this. It seems like, somewhere in their vast mechanism, they have a simple check that looks for URL problems like that and removes the extra slash before proceeding. They probably assume it’s just a typo or something.
But, according to a few sources, this extra slash – and triggering whatever URL rewrite Google has in their software – is enough to get it to parse a sitemap.
This hack isn’t super consistent. Plenty of people in the comments of posts discussing this problem have tried this method and found it doesn’t work for them. But, if it works for you, who’s to argue? You might as well give it a try.
This scenario is pretty much all we have for fixing the “couldn’t fetch sitemap” error in the Google search console. There are some other errors, but they generally all have much more straightforward solutions.
And, of course, there are other errors for very niche situations, all specified by Google here. Of all of them, the “couldn’t fetch sitemap” error is the one that isn’t clear and easy to fix, which is why this whole post focuses on it. Everything else should be easy.
If you’re struggling with a sitemap error and nothing seems to work, let us know; we’ll see if we can help. Also, if you have a solution for the fetching error we didn’t mention, leave it in the comments.