Does a Bulk Operation have a limit on how much data can be fetched?

[doabledanny]

I can't find any info in the Shopify documentation on limits of fetching data with a Bulk Operation. Are there any limits in how much data can be fetched? For example, if I needed to fetch 100,000s of orders, can I rely on a bulk operation to do this?

[Blair]

Hi @doabledanny,

Thank you for reaching out to the Community! I'm happy to provide some assistance with bulk operations, to ensure this gets resolved. To confirm I'm fully understanding, are you looking to export orders along with their transaction histories? 

If so, you can look into using the Order CSV File to achieve this. This feature does not have any limitations and allows for you to export all or selected orders. To achieve this you can: 

• Head to Orders, from the Shopify admin.

• Select Export, in the top right hand corner of the orders page. 

• In the Export orders window:

  • Select the option for the orders that you want to export. For example, if you want to export your orders by date, then click Export orders by date and set the start and end dates for the orders that you want to export.

  • Under Export as, select a file format.

• If you want to download all information about your orders, then click Export orders. If you want to download your transaction information only, then click Export transaction histories.

If however, you're looking to use the Bulk Operations feature to edit product information, this would have limitations. As it currently stands, the bulk editor can load up to 250 records and has infinite scrolling to allow for changes to be completed. For more information on this you can check out our Bulk Editing documentation. 

Please let me know if you have any other questions or concerns. 

[doabledanny]

Thanks for the response, however, I'm looking to perform a Bulk Operation in a Shopify app, not from a store dashboard. Could you provide me with the limitations for fetching data into an app via a Bulk Operation?

[Ablestar_Daniel]

Hi @doabledanny,

We use the bulk operation API a lot and haven't run into any hard limits regarding the number of rows returned. Our calls are more on product objects rather than orders but looking through our logs I see a recent successful bulk operation that returned over 1.8mn records.

The bulk operations do occasionally fail though the reliability does seem to have improved recently.

One other thing, when you make the requests, I'd be sure to log the complete request body for the API call to create the bulk operation, as well as the complete response with headers. If you have an issue with the API, having this data available will help if you need to contact Shopify support. This is especially useful for infrequent errors that are hard to reproduce on your end.

Best,

Daniel

[doabledanny]

Thanks Daniel, good to know it can handle a couple of million records. I'll also take your advice on logging the complete request body & headers when creating the bulk operation.

[R-Daneel]

I've just come across an interesting limit: "Bulk queries cannot contain more than 5 connections." The six connections (edges/node combination) that I was calling are as follows:

1. companies
   1. metafields
   2. contacts
2. company locations
   1. metafields
   2. catalogs

While I worked around this issue by calling companies and company locations in two separate calls, I think it's worth noting this limitation here.