cursor-based pagination

Solved
Highlighted
Shopify Partner
15 0 1

Got the email as others have about page no longer being supported via a rest API. I suspect prior to Shopify adding cusor-based pagination most people would get the count of records divide that count into page blocks and pass the page as parameter.

 

I've looked at the documentation but it's not clear to me given the above what is the execution flow. For example I have 350 products. In the past I would get the count, then determine the number of pages using the 250 limit. So I would just loop through the number of pages (2 in my example) calling get products specifying the page and limit

 

I'm not understanding why Shopify would do it this way. It seems now I have to parse the header to be able to extract a URL to determine where I am and if I'm done or not. Am I missing something?

 

0 Likes

Success.

Shopify Staff
Shopify Staff
1008 67 133

Hey @JohnCole , 

 

It sounds like you have the right idea here - you submit your first GET request with your limit parameter (up to 250) and then use the value of the 'Link' header from the response to follow the rel=next link to the next page until there is no rel=next value left. Once that happens you know you've paged through all of a resource. 

 

We're making the change from the page parameter because page is known to have some issues such as the potential to miss data, and the database queries it generates create a lot of extra load while also being extremely slow and error-prone. With cursor-based pagination, many of the issues with page are resolved and it's also far, far more efficient, relieving load on Shopify and enabling app developers to have faster apps at the same time. 

 

I can understand changing from current behaviour isn't ideal when you've gotten used to how something works, but in my experience so far cursor-based pagination does create a fairly substantial speed boost.

0 Likes
Shopify Partner
15 0 1

Just confirming it works as proposed. In my case performance on my end was never an issue, but I get that Shopify is always looking for ways to optimize just not a fan of how it got implemented. Would have preferred the Link info just be included in the response body or if it has to be in the header that it was JSON payload where I didn't have to parse a string. I know I'm being nit picky as the parsing isn't difficult just sharing my thoughts.

 

Thanks for the help Josh

0 Likes
Shopify Staff
Shopify Staff
1008 67 133

Hey again @JohnCole , 

 

Thanks for getting back to me - and that's a fair assessment. 

 

I'm not sure why the decision to use a header was made in all honesty, but I would imagine it's because it is a bit less intrusive and helps avoid issues with apps that are parsing resources a specific way. I know in the past that making changes to the body of webhooks, for example, has caused a lot of headaches when people are consuming them and expecting the body to be formatted a specific way.

 

I do see where you're coming from as well though, and I'll pass the feedback along to the team behind cursors here in the event this becomes feedback we're hearing frequently. 

0 Likes