We're syncing our Shopify shop to our ERP system and our variant pricing dynamically changes. This means we have to clear our variants and add new ones every night.
It seems this is only possible using a single DELETE on a variant but this is tediously slow for 100 variants per product.
Is there a way to bulk delete variants?
oh that's cool!
How slow is doing 100 DELETE operations on average? Is it blocking or can you just run it in a background thread or process?
Also curious, what platform are you creating the custom connector with, AWS / Azure or something like that?
Trying to update large number of Shopify resources is a common challenge with Shopify's API rate limiting. A few questions I have in terms of scale here:
1. How frequent do the dynamic pricing happen (it seems to be just once a day where you update at night)
2. How many total products and average number of variants do you have?
3. What are the risk of "delayed syncing"
You can try using Shopify graphQL for bulk deleting. The rate limit would be 10 delete operations/second vs the 2 request/second for the REST API. To make it faster, you can try creating custom apps for the store to acquire more API keys.
And we are also curious what system you are using to build the custom integration and if you plan to host/maintain them yourself?
To both replies above. We're running a few services on digital ocean that handles it. It's doing nightly updates on products that changes. It results in around 700ms-1400ms per product depending on the variants that we need to create.
We have around 25 variants per product.
We're running each variant delete action in a job, so each product update in a seperate job.
@samcThanks for the GraphQL suggestion. Yeah we're just hosting them ourselves, it's just a simple Node application.
Got it! That makes sense.
So we released a Shopify automation platform called Shypyard (https://apps.shopify.com/shypyard) that is developer friendly. Essentially you can use the platform to write nodejs code that interacts with Shopify APIs, and the platform handles all the infrastructure issues like scaling, error retries, monitoring and logging. Essentially, you only have to think about your automation business logic, and we take care of the rest. While there are a lot of flexibility of hosting it yourself, we understand it comes with many other cost on maintaining.
If you are interested to learn more, please reach us at email@example.com and we can schedule a short demo to learn more about your use cases.