Surge in 430 status code inspite of using Buy Identity header

[varun_agarwal]

I build custom apps for 100+ merchants. A lot of this functionality is through server to server (s2s) APIs using admin rest and storefront graphql. For the last 2 weeks I am getting a lot of 430 status codes from Shopify for both of these endpoints. Requests from my server will all come from a common pool of IP addresses so we use a delegate access token and Shopify-Storefront-Buyer-IP header in our outgoing requests. As per docs this tells shopify the originating IP which should help in reducing 429 and 430 status codes. But this has not helped at all.

Requests are CRUD operations to admin endpoint, queries and mutations to storefront graphql endpoints. Has anyone else faced this issue, what is the solution for this? We, and our Shopify plus merchants are losing a lot of money on a daily basis due to this.

[Liam]

Hi Varun_agarwal,

Based on this description, it seems like you're hitting the rate limits set by Shopify's API, which results in 430 status codes - Shopify Security Rejection. This typically occurs when Shopify deems your requests as potentially malicious.

You can tackle this issue in several ways:

1. Optimize your requests: Ensure your requests only fetch the data you need. Reducing the complexity of your requests can help you stay within the rate limits.

2. Use Caching: If your app uses data frequently, consider caching it to reduce the number of requests to the Shopify API.

3. Proper Error Handling: Implement error handling in your code to gracefully handle rate limit errors. If you keep making requests without addressing these errors, your app may not recover smoothly.

4. Queuing and Throttling: Implement a queuing system where you can line up the API calls and process them at a rate that is within the Shopify API rate limits. This distribution can help avoid hitting the rate limits.

5. Respecting the Retry-After Header: When Shopify returns a 429 or 430 status code, it includes a `Retry-After` header indicating how many seconds to wait before making another request. Respect this delay to avoid further rejections.

Hope this helps!

[varun_agarwal]

Hi Liam, thanks for the reply. Can you address Jjwarren's answer below? The Aug release which effectively removes rate limit on a single storefront token seems to have tightened up the anti spam/ botting software being employed. The anti-botting measures have even been mentioned in his link. Partners like us who have several Shopify+ merchants using our server side rendered storefronts are not happy with getting our API requests randomly blocked. We can try and optimize our code but is there any other suggestions you can make?

[bhr]

Thanks @Liam ! We've been getting more error status codes as well. To some extent, it's caused by this issue in the Shopify Admin Python SDK. Our workaround is to fetch only a subset of orders with each API call. 

[jjwarren]

You're not alone - we've also been experiencing the same issue since 01/08/2023. This appears to have coincided with Shopify "removing rate limits" from their Storefront API (https://www.shopify.com/editions/summer2023?product=storefront-api). The documentation now states "There are no rate limits applied on the number of requests that can be made to the API." however it seems in reality the new "bot protection" applies a much stricter rate limit than before.

This has practically made their Storefront API un-usable for large-scale server-side rendered headless storefronts, which inherently make a reasonable number of requests from the server-side.

We've been in contact with Shopify Plus support and unfortunately they're not aware of a solution either at this time, besides the Shopify-Storefront-Buyer-IP header which hasn't helped us much. Use the API less appears to be the primary solution, admittedly not ideal.

[varun_agarwal]

Thanks for bringing this up. If most of the API calls to the SSR storefront across all our merchants is for managing cart and checkout sessions, this becomes very difficult to optimize.

Shopify support asked for several request IDs but haven't gotten back either.

The whole point for adding a Buyer IP header is to allow SSR storefronts to handle larger traffic. We were (sadly) doing significantly better before they made this change on 1st Aug. While caching catalogs and some APIs are possible, we cannot cache mutations to create a storefront checkout or cart or applying coupons. Any suggestions on how you are managing that?

I was looking at frequent IP rotation as one alternative for my server clusters, have you tried something like that?