Cursor based pagination is great if you want to iterate over an entire dataset. But for many app use cases accessing just the nth to mth values is desired.
Currently I need this functionality and am forced to make up to 40 requests to Shopify Servers just to get the 10,000th product. Which used to be possible in a single request. This creates pain for the developer and more importantly wasteful api requests just to jump around pages.
It should be straight forward to support both methodologies and allow the developer to decide which tool is best for the job.
Please reconsider adding page-based pagination, I think many of your developers would greatly appreciate it. And it'll save Shopify money on wasted server / bandwidth resources.
If you ask for the nth record, with a cursor returned for first and last records (assuming you want more than 1 record), you now have the cursors to grab as many records as you want right? So if you know you only need records till the mth, you can simply page using cursors till you are satisfied your set of records contains the mth. How is that a problem? It is virtually the same as paging in the old school sense, with the exception of you have to setup for cursors. Is there a problem with that I don't see?
Hi HunkyBill, thanks for your response.
Maybe I'm the one missing things, how can I access the nth record without iterating cursors until I reach the nth record?... Once I have the nth cursor it's trivial to get all products up until the mth. But the initial work of finding the nth record requires making multiple requests since I'm limited to 250 results per request and cannot specify an `offset` parameter.
Is there a simpler way to directly access the nth record that I'm unaware of?
You seem stuck on this idea that you are entitled to ask for records from N to M without writing some kind of encapsulating code to page/iterate.
There is no choice. Write a generic little loop asking for records. As I stated. Start at a known point in the records N. Now in your little tiny simple loop, keep asking for records till you get M. When you do, stop asking for records. You now have your set of records including N to M.
The old system of page, limit, offset, borrowed from SQL style computing, is no better, and it was proven by Shopify to be a poor performer on a large scale. The newer method of cursors is more efficient, and really, hard to distinguish the difference for the most part.
TL:DR; NO. There is no simpler way of accessing a group of records between arbitrary points in a set from N to M.
I didn't mean to come off entitled. Only desire was for the devs to consider the benefits of keeping page based pagination alongside cursor pagination.
My observation was if one doesn't already know the Nth product, then looping over records up until N is the only way to get there.
Which means making multiple API requests to accomplish a fairly common task.
Shopify made it pretty clear, the old paging mechanism was slow, inefficient, and cause for headaches. While it is consistent with 30 years of database patterns, that does not mean it works on large distributed datasets, so they dropped it for that reason. As for API calls, that is less of a concern as that is simple plumbing and plays little role in the end result, giving you the data you want and need to do work! You had to make API calls before, you have to make them after. The more things change, the more they stay the same!
@HunkyBill is on point. Regardless of the underpinnings, usually requesting a group of records requires some sort of an iteration loop. It's not terribly involved in terms of coding, and usually most API providers observe greater throughput with GET'ing as opposed to POST'ing and PUT'ing. So it's not usually going to consume a crazy amount of time to retrieve those record groups. If the data consistency and integrity is improved with cursor-based paging (which isn't in dispute amongst the majority of API consumers and providers) then in the big picture that's a big win!