API request for orders does not return all the records

[Vinh]

I used to be able to request all of our orders (about 40k) from Shopify API. But since the beginning of this week, it only brings back less than 3000 records. In the header of the 10 or 11 page, there is no hash for the next page. I I use the latest API version 2021-01. This is my quasi-code in Python. 

I have 5 seconds pause between each call, so it could not exceed the API rate limit  

import shopify
# Request the first page
limit = 250
page = shopify.Order.find(limit=limit, created_at_min=created_at_min) # let's say created_at_min = '2020-01-01'

# Loop from the second page: find the hash of the next page to jump to next
# First get the header info
resp_header = shopify.shopify.Order.connection.response.headers.get('Link')
# Second read the hash for the next page from the header
page_info_next = {parsing logic here for the hash}

# Jump to the next page
page = shopify.Order.find(limit=limit, page_info=page_info_next)

Does anyone experience similar problems? Or anyone understand why does it suddenly not work for requesting more than around 3000 records? Really appreciate any information here. 

[Jason]

How are you getting the following pages?
That's not clear in your pseudo code but is likely where the problem is.

[Vinh]

The header of the response contain the hash-id of the next page, if it's available. This how a header looks like, I simplified the hashes for readability

<https://lockenbox-com.myshopify.com/admin/api/2021-01/orders.json?limit=250&page_info=hash-of-previous-page>; rel="previous", <https://lockenbox-com.myshopify.com/admin/api/2021-01/orders.json?limit=250&page_info=hash-of-next-page>; rel="next"

I parse the hash-of-next-page and put it in page_info_next, then go to the next page, get the data records, find the hash-of-next-page and the loop repeats

Usually this repeats until I get all the orders. But now after page 10 or 11, the header does not have rel="next" and the hash-of-next-page any more.

[Jason]

Ok. That's good info so we're onto the next check now.

• What is the order status you are asking for? Default or have you specified any?
• Does your app have permissions to pull in orders older than 60 days?

https://shopify.dev/docs/admin-api/rest/reference/orders/order

[Vinh]

Thanks for your help Jason 

• I don't specify any status. Just the limit and created_at_min = '2020-01-06'

page = shopify.Order.find(limit=limit, created_at_min=created_at_min)​

• My app has the permission to pull orders later than 60 days. This is the dataframe of the orders I just request with created_at_min='2020-01-06'. You can see the record in the bottom has created_at 2020-05-10.
  

[Jason · Accepted solution]

That status is likely where you're getting stuck. It's not as well documented as it could be. If I remember next week, I'll look to get that changed.
From the docs:

status

Filter orders by their status. (default: open) open: Show only open orders. closed: Show only closed orders. cancelled: Show only canceled orders. any: Show orders of any status, including archived orders.

eg: /admin/api/2021-01/orders.json?status=any

Without that parameter you'll only see open orders. I am guessing that is what you're seeing and the rest of the orders are closed or cancelled.

[Vinh]

I just give it a quick try but status is not a parameter for the find() function.

page = shopifyObject.find(status='any', limit=limit, page_info=page_info_next)

This documentation confirms it Make paginated requests to the REST Admin API. This is my method of making request using Python. I will give my updates if I make it works again.

[Vinh]

So it turns out that someone in my company manually change the status of the orders to fulfilled and that's why I don't get as many records as usual. The default status=open its the right info. So basically Shopify API and my code worked as it's supposed to.  

I still don't know how to request the fulfilled orders using the python library of ShopifyAPI though. If anyone has some work done here, I would love to know it.

I actually can add status='any' as parameter. In my last try, I added it to the wrong request. It must be in the first request where you put all your filters in. After that, following requests just use the hash for the next page. So it looks like this

page = shopify.Order.find(limit=250, created_at_min=created_at_min, status='any')

[yosmos]

Hi, I have the same problem, but the solution is not working for me,

https://xx/admin/api/2021-10/orders.json?status=any

This endpoint always return 4 orders, it should return more than that, I do not know what is the problem. I have 15 orders on my management ui. It was working without a problem before.

I even tried adding query like this, but no luck,

status=any&limit=250&financial_status=any

[pramodmna]

Hey! I had the same issue. Any updates/solutions? 

Good Luck!

Pramod