Best way to retrieve ALL orders from a store to perform Machine Learning tasks

Solved
Highlighted
Shopify Partner
6 0 0

I am working on a Shopify App that performs some machine learning (forecasts) on orders data.

To perform this, and fit my models, I need access to all the orders a store has got. I know that I need to ask for permission to retrieve orders older than 60 days, and I asked for it.

At the same time, I started to create an app using Node.js and React with the Shopify CLI following this tutorial: https://www.youtube.com/watch?v=PIXN032XJJ8&ab_channel=ShopifyDevs

 

Now I am trying to understand what could be the best way to export all orders a store has and store it somewhere else (S3, DynamoDB). I know I can use the graphql API, which already comes in the App template created by the Shopify CLI.

However, my question is what is the best way to do this bulk export from a store to another resource?

I also thought I could prompt the user to upload the .csv of the export, and then using webhooks updated when a new order is made.

However, it is not clear to me what is the best practice when dealing with apps that implement some kind of data science/machine learning, since I believe it is becoming more common every day, and those kind of apps are definitely data-consuming.

0 Likes
Highlighted
Shopify Staff
Shopify Staff
8 1 2

This is an accepted solution.

Hi @robsanna!

It sounds like your use case is a good candidate for GraphQL Admin API bulk queries, which will allow you to access all of the data you need in a single asynchronous operation.

You can see if our tutorial for performing bulk operations works for you.

Hope this helps!

1 Like
Highlighted
Shopify Partner
6 0 0

Thanks, @paulomarg, that seems exactly what I was looking for!

0 Likes