Upcoming change in API limit calculations

Highlighted
Shopify Staff
Shopify Staff
154 0 34

Edit: After hearing your complaints we have chosen a different strategy to handle the API calls limit. I've edited this post to reflect the new techniques.

The number of awesome apps that communicate via Shopify's API is steadily growing, and so the number of API calls Shopify must process is growing too.
To balance the load, apps are currently limited to making 500 API calls per shop every 5 minutes. However, rather than making calls regularly during this period, many apps function by making API calls as fast as possible until they hit their limit (receiving a 429 status code), then blindly retry every so often until the limit is reset. It's easy to see how this behaviour causes load spikes on Shopify, affecting the performance of _all_ apps during these short bursts of activity.

To address and improve this, we are changing the way we enforce the API usage limit on Shopify. On November 18th, instead of having a hard limit of 500 calls that resets every 5 minutes, we are going to use a flexible limit that focuses more on the average number of calls per second. This means you don't have to wait long after hitting the limit to continue work, but still reduces the number of calls you can do in a short period.

To be more specific, we will be using a leaky bucket algorithm with a bucket size of 40 calls, and a leak rate of 2 per second.

How does that work:

  • Each call you make will increase the level of a virtual bucket level by one.
  • Every half a second the bucket leaks, decreasing the current level by one.
  • Your calls are processed immediately as long as the bucket has room, otherwise you get a 429 error
  • The X-Shopify-Shop-Api-Call-Limit header will be indicating the current level and the bucket size as 'x/40'

What does that mean for your app:

  • You can do bursts of up to 40 calls in a second. They will all be processed immediately
  • Once you hit the limit it will take 500 ms before another call will be allowed through
  • As long as you stay under an average of 2 calls per second you won't hit the limit
  • This actually maxes out to around 600 calls per shop per 5 minutes, 20% more than before!

See these graphs as examples of how the limit rises and falls over time with certain call behaviour.

First some random calls, not hitting the limit (first) and hitting it (second). See how the bucket level decreases over time when you don't do any calls, this makes room for new calls. 

Next some repeating larger bursts of the same size. Doing them infrequently enough makes sure you don't hit the limit (first). If you don't wait long enough between bursts for the bucket level to decrease, your will be dropped (second).

Now if you do a stead stream of calls with an average of 2 seconds (first), you won't hit the limit. If it is higher, you will creep up to the limit and get 429 eventually (second).

If you do average at 2 per second, remember that any call extra will accumulate which brings you closer and closer to the limit (first). If you average lower than 2, this won't happen and gives you room to do little bursts (second).

Most apps won't have to do anything for this change. For example, apps that only do infrequent or small bursts, or are spreading their calls over time already will be unaffected. If your app currently relies on bursts bigger than 40 calls, or if you need to perform hundreds of calls in succession, you will have to adapt your app. Those apps need to make sure they handle 429s or spread their load so their calls are not dropped.

If you want to try out the new limiting system on a specific test shop of yours, just let us know which shop! We can switch you over to new system before the mentioned deadline so you can make sure it responds as you expect.

1 Like
Shopify Expert
2000 13 313

Wow. Thanks for the advanced warning. Please reconsider this November 4th limit as that is a crushing deadline for those of us with a non-trivial number of Apps in production use. 

Since you're asking us to take on the burden of smoothing our API call bursts to better accommodate your network usage, we deserve better in terms of notification. It is NOT as trivial as you make it out to be. Seriously, you think adding in a simple "divide by 10" is the solution here?

Custom Shopify Apps built just for you! hunkybill@gmail.com http://www.resistorsoftware.com
2 Likes
Shopify Expert
2000 82 1452

I don't have the pain that some will in terms of sheer quantity of apps but even I can see that 14 days is really soon. The tasks that could be done quickly in a burst are going to feel mighty slow now too - wonder how many users will notice. 

★ Winning Partner of the Build a Business competition. ★ http://freakdesign.com.au
1 Like
Explorer
127 0 15

Sigh. HunkyBill's right, you guys really need to work on giving us a bit more notice before doing stuff like this.

0 Likes
Shopify Partner
20 0 2

That's less than ideal! Reducing the API call to 10 per 6 seconds will give a pretty poor user experience. Did (or would) you consider rate limiting each _minute_? 

When someone signs up, we get their Shop. That's one. Then we set up 5 webhooks. That's 6. Then we get their customers lists. That's ~10. Then we want the customer's in those lists. That's 1 per 250 customers. Granted it's asynchronous, but for some of our customers it might take quite a while to sync.

It's good that we now don't have to wait 5 mintues for the limit to reset, but making it every 6 seconds is a bit of overkill.

2 Likes
Explorer
127 0 15

I agree wholeheartedly with Nickm. A finer interval is okay, but 6 seconds is crazily small, as there are totally legitimate individual transactions which do take over 10 calls. One minute would be much better; 30 seconds would be acceptable.

0 Likes
Shopify Expert
2000 13 313

No explanation as to why 10calls/6secs is so great for Shopify (from 500calls/300secs) and in looking at my new imposed workload I am now despairing. 

This little imposition is going to cost me a cool grand in my billable time and that is a very lowball estimate. This change by Shopify was POORLY communicated, and for that I am bitter. A decent technical explanation as to the proof that this change is good for everyone would be nice along with a more compassionate deadline date for the switch over would be nice. 

In the past Shopify has imposed changes like this on a deadline and then failed to follow through, so I see no reason they cannot push issue this to some date in the near future that is more palatable. Six weeks anyone?

That or I send all my clients that will almost certainly be screaming at me directly to Shopify where they can chew out support for a few hours/days/weeks. 

 

Custom Shopify Apps built just for you! hunkybill@gmail.com http://www.resistorsoftware.com
1 Like
Shopify Partner
46 0 1

I'm glad to be browsing the forums and just found this.

I never got any email notification or any other type of communication about this change. This is too much of a short notice. Very poorly communicated and timing. 10 orders/6 second is not ideal. Many merchants that will hit easily.

Better Fraud Detection for Shopify | Check out our Shopify App: http://apps.shopify.com/subuno
0 Likes
Shopify Staff
Shopify Staff
154 0 34

In response to the things mentioned in this topic we are internally discussing alternate ways of limiting API calls. We're working on a way that will be a compromise between being able to do reasonable sized but infrequent bursts of calls, and at the same time reducing the peak load of certain heavier apps. 

For now I can tell you that both the time of the planned change will be moved a bit, and that this new solution will affect a lot fewer apps.

0 Likes
Shopify Partner
9 0 1

Pro tip: if you are processing a lot of calls for different shops at the same time, perform them concurrently instead of one shop after the other!

Just as a heads up - the shopify_api ruby gem isn't thread safe.

0 Likes