Product sitemaps failing to be read in Google Search Console

[AshJa]

Every now and then our product sitemaps have been not read in Google Search Console. We just discovered this but have experienced the results of it several times, not knowing what had happened. See our screenshots. 

Does anyone know why this happens and/or what we can do to make sure they get read? Because we notice drops in traffic and sales as soon as the green bars in the second screenshot drop. 

I have been in touch with Shopify support and they suggested looking for help here.  Greatful for any help!

[Shanthi_Vutukur]

One common cause for Google Search Console not reading your product sitemaps is improper configurations in the Robots.txt file.

Here’s how to fix it:

• Ensure that the robots.txt file is correctly configured and does not contain any disallow directives.
• Verify that the sitemap's location is correctly specified within the robots.txt file.
• Utilize Google's robots.txt Tester within Google Search Console to check for syntax errors or misconfigurations.
• Confirm that the robots.txt file has appropriate permissions.

Here’s how to access your robots.txt file:

Log in to your Google Search Console account >> go to the robots.txt report >> click the file you want to open in the list of robots.txt files.

[akshay_bhatt]

Hello @AshJa 
It looks like Google Search Console sometimes fails to read your product sitemaps, which could be causing drops in traffic and sales.

A. Why does this happen?

There are a few possible reasons why Google isn’t reading your product sitemaps:

1. Google is trying to read an old sitemap URL

-Shopify automatically updates sitemaps when you add/remove products.
-If Google tries to access an old sitemap that no longer exists, it may fail.
Fix: Go to Google Search Console -Sitemaps, remove any failed sitemaps, and re-submit your main sitemap.

2. Shopify’s servers may be overloaded

-If Google tries to read your sitemap when Shopify’s servers are busy, the request may time out.
-This can happen when many bots (including Googlebot) crawl your site at the same time.
-Fix: Go to Google Search Console - URL Inspection, enter a failed sitemap URL, and click Test Live URL to check if it’s accessible.

3. Googlebot might be temporarily blocked or delayed

-Shopify has a built-in robots.txt that tells search engines how to crawl your site.
-In rare cases, Shopify’s system may temporarily block Google’s crawler.
-Fix: You can check robots.txt.liquid in your Shopify theme. If needed, Shopify Support can help adjust it.

4. How to fix it
- In Google Search Console - Sitemaps, remove any failed sitemaps.
- Re-add sitemap to ensure Google has the latest version.
- In Google Search Console - URL Inspection, test a failed sitemap URL to see if it loads properly.
- Check Indexing- Pages in Google Search Console to see which pages Google is indexing and if any products are missing.

Feel free to reach out if you need any assistance. Thanks
Best Regards,
Akshay

[AshJa]

Thanks for your reply. One of the failed sitemaps was read overnight so I am assuming it's either 2 or 3 on your list.  I checked both the failed product sitemaps and the URL is available to Google and can be loaded properly.  So still unsure of what more we can do.