Get Data Order - How to loop through more than 100 orders

[mVanguard]

Good day everyone!

I am trying to write a query for GetOrderData that would allow me to go through all my orders since January 1st, I am using a  schedule trigger.

I have (way) more than 100 orders, and so far I can't figure out how to write my query so every time my script runs, it goes to the next 100 orders. If you have some suggestions, please send it my way 🙂

Thanks!

[paul_n]

I think you need to post what you are using for your query. 

[Sanket_Patel]

Hi,

I have same question, In Shopify Flow, how can we iterate more than 100 items, For loop has limitation of 100. 
 
I like Shopify Flow to schedule run every day for syncing compare_at_price to metafield for all products/variants. 
How can we loop in this case ? I have set flow but it updates only 100.

[zaidkhan]

hello Sanket, did you ever find a solution here?

[Sanket_Patel]

Not possible to do in Shopify Flow.
Therefore, Completed task using custom app using REST/Graphql api. 

[zaidkhan]

@Sanket_Patel 
can you please give me any hints on the app you developed or how did you approach the problem in code?

[paul_n]

The answer here (in Flow) is to run it more frequently so that the 100 thing isn't a problem. If you run every 10 minutes, and it's still hitting the 100 max, then it won't work in Flow. This should be pretty rare.

[zaidkhan]

@paul 
The update flow will work just fine, I am trying to conditionally set a value in  a metafiled for all products.
So for the starter data, I need to traverse all items(>1500).
Once the metafields are set for the whole store, the product update scheduled query will work just fine.
I cannot bulk update the metafields since the metafields will be calculated based on the check 'an item or any of the variant' is on sale or not.

[paul_n]

You can traverse all those items by setting a tag after you have processed them and excluding any products with that tag from your query (tag_not:processed)

[zaidkhan]

The issue with that is I cannot add metafields and add tags in the same flow.
Please let me know if I should provide more details

[paul_n]

You can loop over the same thing twice (in separate loops). When doing that we recommend decreasing the maximum records pulled back to ~50. 

[zaidkhan]

Thanks Paul for getting back so quickly.
How can i use two loops in the same flow that I have depicted above?

[paul_n]

put the second loop in the then path of the first for each loop

[zaidkhan]

@paul_n I got this working by following your advise. So I have mass updated the metafileds for all the products.
Now I need a practical solution so that very product when updated, the flow pics it up and adjust the metafields accordingly.
the following flow is picking the same hundred products everytime even though we updated all the product pricing just today with Bulk editor

[paul_n]

I don't see anything off in the query, but if it's truly getting the same 100 every time either (1) the query is off or (2) the product are being repeatedly updated. I think what is happening is that you are setting the Product metafield in the same workflow, which is considered an "update". So you need to build something into this so that it doesn't do an infinite loop. Like check if that metafield is already set to the value, and only set it when they are not equal. 

[zaidkhan]

@paul_n 
I think what's happening is its considering the 'scheduled at' the time the flow has started and not the repeat flow time.
Given that its a very usual product update query, what would you suggest given that we mass update all our products?

[zaidkhan]

and also that i am setting the metafiled to a Boolean which will help me create a fullprice dyanmic collection that stays true all the time. I need that collection to limit our coupon code to only fullprice items.

[paul_n]

Not sure what that means, but it's running on the scheduledAt date, which changes for each run. You can verify put putting the same query in a Log output action and seeing how it changes. 

The issue is you have an infinite loop. You are getting product updates and then updating a product. I put a way to resolve that in my previous answer

[zaidkhan]

@paul_n 
i understand that but the problem is for 1500 products, I want to loop through all the updated products(in our case today all of them were updated at once). While right now I am only getting the first 100 updated products in ascending order every time the flow runs.

[paul_n]

If you have 1500 simultaneous changes, that's it own problem to solve, true. But if you don't solve the loop problem you won't ever get past 100. 

I'm not sure the 1500 once is solvable via Flow. But I think you would need to find a way to filter down that 1000 and also you might need to change the query to get more than the 10 minute window (since it won't pull in all updates). This would be far easier if you could query by metafield values. But since you can't you might be able to get created with tags. Like for example set a tag once you update a metafield for something like "processed_hour_12" and then check only for products that don't have that tag. This would be pretty tough to build though.

[zaidkhan]

@paul_n 
Hello, we have been able to resolve the issue above. i have another related query. Previously I wanted to loop through the whole catalog(products) and we went with the approach to apply tag_not in query and at the end of the flow run added the tag to the processed item.
Now i want to loop through all variants of my store to bulk update variant metafield called availability(in stock, pre order, out of stock) based on the inventory of the variant.
How can I loop through all variants around 5000 for a mass update.