I am using Google Search Console for manually submitting changes to products. This is working well and new products/changes are generally being indexed within 24 hours.
However, I have noticed that many products are found in Google Search Console but they are reported as not being present in the Shopify generated sitemap.xml file.
I have reported this to Shopify but I get the brush-off and they don't seem interested.
I now do not trust the Shopify sitemap.xml generation. Does anybody have a similar experience, and how have they resolved the issue?
If you view your sitemap.xml file then copy the "sitemap_products_1.xml" URL that's in there (it's important you do this as it's unique to yours as a result of product IDs in the query parameters shaping what is output on the page), are the products there? That's what I'd want to know for myself: do I see the product in my product sitemaps?
If you have a large SKU store, you will see multiple product sitemaps in there.
Have you submitted a products sitemap previously? Or are you submitting just the index(sitemap.xml) to GSC?
If you have submitted a products sitemap (and it was some time ago) your current product sitemap url is most likely different as this changes whenever you add a product. This could be causing GSC to think x amount of products are not in your sitemap because any products that were added to your store after you submitted the sitemap will not be in the sitemap you submitted. We don't submit a products sitemap we just submit the index.
We have a large store (3 products sitemaps) and we intermittently see entire sitemaps unrecognized by google search console. 5001 products indexed but not submitted in sitemap...
This is most likely the way shopify produces product sitemaps and google becoming confused when our sitemaps urls change almost daily.
I have noticed that google still crawls these "older" sitemaps every so often and if a product is deleted and that products id is used in either the start or end of the range for an old sitemap, it will return a 400 error and those also end up in crawl stats in GSC.
Shopify really needs change the way they produce product sitemaps, move away from query prams in the url and have stable sitemap urls like with collection pages etc.