Bing it is showing 403 (Forbidden)
is shopify or cloudflare blocking bing bots by mistakes?
It appears it is something related to the DNS so I suspect it is a Cloudfare/Shopify/Bing specific error but can't pinpoint which one.
Try following bing tools to get the exact solution of your query
1- Fetch as Bingbot
The DNS resolution for the host of the URL (or the redirected url) could not be resolved." this status can be because of the wrong url, your server is not responding, your URL is expired, etc
Contact your server team to resolve this issue
rest refer this https://www.bing.com/webmaster/help/why-is-my-site-not-in-the-index-2141dfab
This issue is most likely Shopify or Cloudflare blocking IPs used by Bing Webmaster tools.
Google Search Console uses IP's for it's search console which show as Google Cloud customer IP's. When many people are submitting URL's via GSC or Bing Webmaster tools, using many IP's for load balancing it is easy for Cloudflare or Shopify to block those IP's assuming they are malicious.
This error is not Shopify customer error, it is Cloudflare or Shopify. Sadly both companies refuse to acknowledge this issue let alone fix it.
I flipped my dns over to cloudflare from godaddy and only using dns I am able to register my txt record now. Still won't crawl bing https
No it won't, use http . Shopify should resolve this but they don't seem bothered at all which I find rather unusual. I suppose help reps with no technically knowledge won't have a clue, it should be passed up the food chain though.
403 is a client server-side message from shopify servers telling Bing that bingbot has no access to that sitemap url via SSL connection. As it was suggested by another poster, if you use non secure protocol, the message will go away and bingbot will be able to crawl your sitemap. instead of https://yourdomain/sitemap.xml change it to http://yourdomain/sitemap.xml
All, I want to share the email exchanges that I'm in the middle with both Bing support team and Shopify team.
1st response from Bing:
Thank you for the patience during our investigation, This is to inform you that when Bingbot is trying to crawl the site it is receiving 403 error and the server is blocking the Bingbot from crawling the site ( https://altcookinghub.com/sitemap.xml ). Here is the error message what we are receiving while making an attempt to crawl the site "403 HTTP Status " this status code indicates that the server is refusing access. I would request you to check with your Web Host/Web Service Provider, .htaccess files, virus detection, and data center for any unintentional blocks placed on our User Agents. Once the issue on the server-side is resolved, Our Bingbot will crawl the site and it will get site verified in Bing webmaster tool successfully. Please see the below screenshots for your reference.
Response from Shopify:
Michael here with Shopify Technical Support; it's a pleasure to meet you.
Kelly brought me up to speed on the issues you're having with submitting your sitemap to Bing. This issue would be on Bing's end and the way they handle HTTPS Server Name Indication (SNI). The sitemap is readily available for them to crawl, however in some situations we have notice Bing reporting issues due to a 403 error. This comes down to Bing needing to whitelist HTTPS SNI so they can correctly crawl through the sitemap and index everything for the storefront. We wouldn't be able to make any adjustments on our end for this as the issue is entirely coming from their end and they should be able to go through further steps for getting that whitelisted.
It is still finger pointing at this time. I'm forwarding Shopify response to Bing and awaiting for Bing's response back. I will update you guys if I hear from them back.
Comments are welcome!
|3 hours ago|
|7 hours ago|
|8 hours ago|
|8 hours ago|