bulkOperationRunMutation Speed Estimates for Creating or Updating Many Variants

[ndrtek]

Hello.

Does anyone have experience using bulkOperationRunMutation to create and/or update large volumes of Variants... like in the 10's or 100's of thousands?

I am seeking some guide rails on how long processes might take to, for instance:

1) Create 100,000 Variants

2) Update Price & Availability of 100,000 Variants.

Thank you!

Gabe

[dogowner]

I didn't even know this existed.  I'd also be curious how fast this is.  Seems like only running a single mutation operation at a time might be kind of limiting though.

Right now I just bulk download the products and take a "diff" with my current system and then run single mutations to make things match.  The same thing with inventory.  I might have 1000 products but I think it is less and there are not many changes.

I think everything depends on whatever plan you have.

[ShopifyDevSup]

Hi @ndrtek,

That's a great question. We don't have any public documentation on how long to expect that kind of operation to take for different numbers of variants. 

Stores on some plans could see a throttle if attempting to create too many variants with a an API client over a certain threshold, but for updating existing resources, bulk operations are performed as asynchronous background jobs that run on the platform rather than linked directly to an API client's rate limits so I suspect that the overall time bulk mutations run would be related to both the complexity of the update for each variant, as well as overall load running on the platform rather than the plan that the store is on.

Hope that helps!

- James

[ndrtek]

Hello @ShopifyDevSup .

Thank you for your response!

Could I ask you to dig a bit deeper than the already published documentation?

Yes, I understand from the existing documentation that the bulk operations are background jobs and don't count against API rate limits.

However, I cannot judge the feasibility of using Shopify's bulk operations API for various use cases without any guidance about possible throughput for common reasons one would be wanting to use this API, for instance, inventory & pricing updates.

I have a potential plus merchant on the hook, and I have to steer him either to Shopify or to a custom solution, and the direction I steer is directly related to how quickly Shopify would be able to make price and inventory updates to 10's or 100's of thousands of SKUs.

This merchant has daily updates to both and we would need price and inventory updates to post within a few hours on potentially 100,000 variants.

Surely there are people on Shopify's API development team who built this endpoint, who architected it, who tested it, and who supervised it. And surely someone tested this endpoint at various levels of scale.

Another thought... is that someone perhaps is measuring average throughput on this bulk operation API endpoint systemwide... and an average count of updates per second would also be sufficient for users to judge feasibility for their use cases.

Could you please inquire within and see what guidance or anecdotes you can dig up to help the community understand if we're talking hours or days... for large scale price and inventory udpates?

TY!

Gabe

[mwpcard]

HI!

In the end, were you able to find out how long it takes for bulkOperationRunMutation to update 10-100k variants?
I have to approach the problem with a similar amount, 20-30k of variants, and I can't find any information on the execution speed of bulk operations.

[ndrtek]

No, sorry, we ended up choosing a different platform for this project where
we have direct access to the database.