That being said, there’s more to the story. Just because Google and other search engines have gotten more advanced in their ability to deal with JS doesn’t mean that your pages are necessarily being indexed as they should be.
Back in the old days, search engines would be able to see the content of most pages through a downloaded HTML response.
Googlebot (aka Google’s web crawler) will queue pages for both crawling and rendering and then crawl every URL that is in the queue. After the Google bot sends a GET request to the server, the server will respond with the HTML document. At this point, the web crawler determines which resources are required in order for the content of the page to be rendered.
What this means is that it will crawl the HTML of the page and not the CSS or JS files. The reason for this is that rendering JS takes a ton of resources.
Are your product pages on Shopify not being indexed by Google? If so, take a look at this post that explains why your product pages aren’t being indexed and what you can do to fix the problem.
It’s worth noting that there are separate queues for regular crawling and rendering, which means that sites that require rendering take longer than those that don’t.
Googlebot goes through the queue the first time to request and receive the server-supplied HTML and deals with the rendering later. Even though Google has stated that the delay between the first crawl and rendering is now only a matter of seconds, it often takes days or weeks more to index pages that need to be rendered versus those that don’t.
Are there pages on your site that aren’t linked to anywhere else on your website? These are known as orphan pages, which can have both a negative impact on your SEO and your user experience. This guide looks at why you don’t want to leave orphan pages unattended and how to find and fix them.
Here are some of the common problems as well as the best practices you can use in order to avoid them:
Are you wondering how you can boost the performance of your webpage without sacrificing your valuable JS resources? Check out this post about how to delay loading JS assets until the user’s about to interact with them.
Just because your site launches just fine on Google Chrome doesn’t mean that Googlebot will be able to render its content. In order to find out if your pages can be rendered by Googlebot, you’ll want to go to the Google Search Console and use the URL Inspection Tool. Once you get here, you’ll want to enter the URL of the page you are testing into the bar at the top of the page and hit enter.
Once you’ve done that, select the button on the far right of the screen that says “Test Live URL.”
This process takes a minute or two, but soon enough, you’ll see a “Live Test” tab appear. You’ll want to select “View Tested Page” so that you can see a screenshot of the page’s code.
In order to look for missing content or any discrepancies, take a look at the “More Info” tab.
One of the more common culprits for your JS pages not being rendered by Google is your site’s robots.txt file accidentally blocking the rendering process. You can make sure that no essential resources are being blocked from crawling by adding this code to your robot.txt file:
If you block these resources, it can mean that your content doesn’t get rendered and indexed.
Now that you have full confidence that your pages are rendering as they should, it’s time to make sure that they are being properly indexed. You can either continue to use Google Search Console, or you can just head to the regular old Google search engine.
Using Google Search Console, you’ll want to head to the URL Inspection Tool and select “View Crawled Page” to take a look at the HTML source code of the page. Look for snippets of JS content as you scan the code.
There are a number of reasons you might not see your JS content here– it could be that the page is timing out when the content is being indexed, that the JS is generating internal links that cause the URL to not be discovered, or that the content can’t be rendered by Googlebot.
To use the search engine, you’ll want to use the “site:” command.
For example, if your site was blue-pig-media.com and you wanted to check the “Hire Us” page to make sure it’s being indexed, you would enter the following into the search bar:
Your page will show up as a result if the page is indexed. Your next step will be checking to see if the JS-generated content is indexed, which you can do using the site command with a snippet of the JS content from the page in question. Using the above example, this would look like:
site:blue-pig-media.com/hire-us/ “snippet of JS content”
If this specific part of your JS content has been indexed, it will show up in the snippet.
If your page doesn’t appear at all after doing the first basic site command, it means that your page isn’t indexed.
There are a number of site audit tools you can use out there to crawl your JS in the same way that Googlebot does. This can help you find any issues as well as provide potential solutions for remedying them.
If you’re not interested in purchasing SEO software just to climb the learning curve, though, you can also hire a professional SEO agency to perform a site audit and fix any issues that are keeping your JS content from rendering.
Are you due for a full site audit? Use this checklist to make sure that your on-page SEO is in tip-top shape.
If you’re dealing with issues like whether or not your JS files are being indexed by Google, you might be wondering if you would be better off hiring professionals to take care of your SEO needs. After all, while it can be truly fascinating and rewarding to manage your own SEO, it also requires that you stay constantly tapped into the ever-changing landscape of SEO and search engines.
If you’ve been on the lookout for the right digital marketing agency to help drive traffic to your site and grow your business, you’re in the right place! Blue Pig Media is a full-service digital marketing and SEO agency that can help you with absolutely every aspect of your site– from designing and building your site to optimizing your site for search engines.
No matter what your business goals are, we’re here to help. If you’re ready to get started, reach out and contact us today.