Why is every page of my store giving 429 errors in semrush.com?

New Member
2 0 2

On my site health report in semrush.com I am getting 429 errors for virtually every nav link on every page of my site. The links are being reported as broken, but all of the links I've tested are working fine.

For example, please see this screenshot: http://www.screencast.com/t/jDmYT9MliNE4

2 Likes
Shopify Staff
Shopify Staff
11 0 3

Hi Carisa,

Typically these 429 errors come about when you're going over Shopify's API call limit. This document from our Shopify manual will explain in detail what this means and how to avoid it. It just means that SEMrush is sending Shopify too many requests at one time, and Shopify is sending a message back saying "whoa, that's way too many requests."

If you go into your SEMrush.com settings, you can manually set a delay for the bot in SEMrush. When you do that it reduces the amount of 429 errors that will appear.

Don't worry though, these errors don't indicate that there is anything wrong with your links or your site.

1 Like
New Member
2 0 2

Thanks so much for your reply. I will try to adjust some settings in SEMRush.

0 Likes
New Member
1 0 0

I added the delay to mine and still getting them. I also noticed when re-auditing the url, http code changes to 200. The link shows as broken even though is live. Most of the problem is the url query that gets added to the product.

0 Likes
New Member
1 0 0

Lisa P, doesn't that only work if you can edit your robots.txt in order to indicate to SEMRush what the delay rate is? 

0 Likes

Hey Lisa P, unless I'm missing something in my SEMrush Pro account, the setting you're referring to doesn't exist. That being the case, and since we can't modify robots.txt, we're kinda left stranded by both parties. I've requested that SEMrush add that option, but what can Shopify do for us? In the 429 Errors and Crawl Delay post, someone said "Would be great if Shopify limited their crawl rate in their own robots.txt. Without that setting, we have zero control over whether Shopify blocks a legitimate tool or not, since we often can't set the crawl limit at the robot's side." Have you considered that? 

2 Likes
Highlighted
Shopify Expert
67 0 0

Waiting for a fix on this too. 

0 Likes