So we've noticed a problem with Shopify's cursors on one our clients' stores. A client was complaining about certain orders being omitted from our app. We investigated, and sure enough, there were indeed orders missing.
We've traced the problem to an issue with Shopify's cursors. For example, when we make the following call:
We get a list of 250 orders, as expected. It gives us the following as the next link in the response headers:
This next page (the last page), returns only 170 orders, which is fine; being the terminal page.
However, between these two pages, there are 5 orders that are being completely omitted, all with the same created_at timestamp. So there's an issue here where Shopify is literally just omitting orders between cursors. The cursors look to be constructed correctly (their last_id and last_value values are correct), but they're still omitting information. This is consistently repeatable.
The orders do show up if a different created_at_min and created_at_max period is chosen, so the orders are not consistently omitted in all queries; it appears to be entirely cursor dependent.
Any Shopify devs out there able to help?
Solved! Go to the solution
If you search through this forum you will see a few similar threads --> https://community.shopify.com/c/Shopify-APIs-SDKs/REST-API-Cursor-based-pagination-Infinite-Loop-quo....
I think the Shopify API cursor-based pagination is just looking at timestamps and not taking unique record ID's into account in order to provide the full result set. Either they need to drop the timestamp down to milliseconds to pull unique records or else also link record ID's to provide good referential integrity. So far I haven't seen any Shopify dev staff respond to the other threads.