Paginate the 2019-07 REST Product API

Retired Boards Appdev Customers Discounts And Orders
1

I switched to the 19/07 version of the product API and noticed that passing the page parameter with the endpoint will generate an error:

{“errors”:{“page”:“page cannot be passed.”}}

Is the pagination functionality changed in this new version?

Using the 2019-04 works sweetly.

Thank you

2

Hi @Giulio1 ,

We replaced page-based pagination for a number of endpoints in the 2019-07 version. This was done in favour of a much more performant method of pagination we’ve also released for these endpoints called cursor-based pagintion. We’ve released this guide on how to make calls using cursor-based pagination. The TLDR is that you need to look for the “Link” response header in your requests. The value of this header will be a link to both the next and previous page of results in the following format.

#...
Link: "<https://{shop}.myshopify.com/admin/api/{version}/products.json?page_info={page_info}&limit={limit}>; rel={next}, 
<https://{shop}.myshopify.com/admin/api/{version}/products.json?page_info={page_info}&limit={limit}>; rel={previous}"
#...

That being said, you can continue making requests with the page param using the 2019-04 version until it is no longer supported with the release of 2020-04.

Let me know if you have any lingering questions or concerns.

Cheers,

3

Thank you @Busfox

That is pretty clear

4

HI

We are not getting Link header in response headers returned by shopify while calling orders api below are the headers we are getting

1.We are calling orders api orders.json which returns orders all fine but we dont have any way to get second page of orders and documentation says we will get link header that will mention page_info param but api is not returning any link header or page_info as you can see below in response headers

HTTP/1.1 200 OK
Date: Mon, 02 Sep 2019 06:04:06 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: __cfduid=daaa7a5d79a531ae7a3dfa4f59a13dc5d1567404245; expires=Tue, 01-Sep-20 06:04:05 GMT; path=/; domain=.myshopify.com; HttpOnly
X-Sorting-Hat-PodId: 84
X-Sorting-Hat-ShopId: 7382728789
Vary: Accept-Encoding
Referrer-Policy: origin-when-cross-origin
X-Frame-Options: DENY
X-ShopId: 7382728789
X-ShardId: 84
X-Stats-UserId: 
X-Stats-ApiClientId: 2972013
X-Stats-ApiPermissionId: 115969425493
X-Shopify-API-Terms: By accessing or using the Shopify API you agree to the Shopify API License and Terms of Use at https://www.shopify.com/legal/api-terms
HTTP_X_SHOPIFY_SHOP_API_CALL_LIMIT: 1/40
X-Shopify-Shop-Api-Call-Limit: 1/40
X-Shopify-API-Version: 2019-10
Strict-Transport-Security: max-age=7889238
X-Request-Id: c69c7392-e281-4733-9884-775df38d0825
X-Shopify-Stage: production
Content-Security-Policy: default-src 'self' data: blob: 'unsafe-inline' 'unsafe-eval' https://* shopify-pos://*; block-all-mixed-content; child-src 'self' https://* shopify-pos://*; connect-src 'self' wss://* https://*; frame-ancestors 'none'; img-src 'self' data: blob: https:; script-src https://cdn.shopify.com https://checkout.shopifycs.com https://js-agent.newrelic.com https://bam.nr-data.net https://dme0ih8comzn4.cloudfront.net https://api.stripe.com https://mpsnare.iesnare.com https://appcenter.intuit.com https://www.paypal.com https://js.braintreegateway.com https://c.paypal.com https://maps.googleapis.com https://www.google-analytics.com https://v.shopify.com https://widget.intercom.io https://js.intercomcdn.com 'self' 'unsafe-inline' 'unsafe-eval'; upgrade-insecure-requests; report-uri /csp-report?source%5Baction%5D=index&source%5Bapp%5D=Shopify&source%5Bcontroller%5D=admin%2Forders&source%5Bsection%5D=admin_api&source%5Buuid%5D=c69c7392-e281-4733-9884-775df38d0825
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
X-XSS-Protection: 1; mode=block; report=/xss-report?source%5Baction%5D=index&source%5Bapp%5D=Shopify&source%5Bcontroller%5D=admin%2Forders&source%5Bsection%5D=admin_api&source%5Buuid%5D=c69c7392-e281-4733-9884-775df38d0825
X-Dc: gcp-us-east1,gcp-us-east1
NEL: {"report_to":"network-errors","max_age":2592000,"failure_fraction":0.01,"success_fraction":0.0001}
Report-To: {"group":"network-errors","max_age":2592000,"endpoints":[{"url":"https://monorail-edge.shopifycloud.com/v1/reports/nel/20190325/shopify"}]}
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
CF-RAY: 50fd48d78f90d5f8-BOM
5

It looks like you are using the 2019-10 version of the API which is unstable and shouldn’t be used for production.

Maybe that is causing the problem.

@jeffBlum_ffa try to use the 2019-07 version.

6

Tried that as well here is the result @Giulio1 @Busfox

HTTP/1.1 200 OK
Date: Mon, 02 Sep 2019 09:22:58 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: __cfduid=d350db241396c03c9acc0fecb395410491567416177; expires=Tue, 01-Sep-20 09:22:57 GMT; path=/; domain=.myshopify.com; HttpOnly
X-Sorting-Hat-PodId: 84
X-Sorting-Hat-ShopId: 7382728789
Vary: Accept-Encoding
Referrer-Policy: origin-when-cross-origin
X-Frame-Options: DENY
X-ShopId: 7382728789
X-ShardId: 84
X-Stats-UserId: 
X-Stats-ApiClientId: 2972013
X-Stats-ApiPermissionId: 115969425493
X-Shopify-API-Terms: By accessing or using the Shopify API you agree to the Shopify API License and Terms of Use at https://www.shopify.com/legal/api-terms
HTTP_X_SHOPIFY_SHOP_API_CALL_LIMIT: 1/40
X-Shopify-Shop-Api-Call-Limit: 1/40
X-Shopify-API-Version: 2019-07
Strict-Transport-Security: max-age=7889238
X-Shopify-Stage: production
Content-Security-Policy: default-src 'self' data: blob: 'unsafe-inline' 'unsafe-eval' https://* shopify-pos://*; block-all-mixed-content; child-src 'self' https://* shopify-pos://*; connect-src 'self' wss://* https://*; frame-ancestors 'none'; img-src 'self' data: blob: https:; script-src https://cdn.shopify.com https://checkout.shopifycs.com https://js-agent.newrelic.com https://bam.nr-data.net https://dme0ih8comzn4.cloudfront.net https://api.stripe.com https://mpsnare.iesnare.com https://appcenter.intuit.com https://www.paypal.com https://js.braintreegateway.com https://c.paypal.com https://maps.googleapis.com https://www.google-analytics.com https://v.shopify.com https://widget.intercom.io https://js.intercomcdn.com 'self' 'unsafe-inline' 'unsafe-eval'; upgrade-insecure-requests; report-uri /csp-report?source%5Baction%5D=index&source%5Bapp%5D=Shopify&source%5Bcontroller%5D=admin%2Forders&source%5Bsection%5D=admin_api&source%5Buuid%5D=d3cd8feb-bf9f-482a-9ee2-1d7fa0b5b987
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
X-XSS-Protection: 1; mode=block; report=/xss-report?source%5Baction%5D=index&source%5Bapp%5D=Shopify&source%5Bcontroller%5D=admin%2Forders&source%5Bsection%5D=admin_api&source%5Buuid%5D=d3cd8feb-bf9f-482a-9ee2-1d7fa0b5b987
X-Dc: gcp-us-central1,gcp-us-east1,gcp-us-east1
NEL: {"report_to":"network-errors","max_age":2592000,"failure_fraction":0.01,"success_fraction":0.0001}
Report-To: {"group":"network-errors","max_age":2592000,"endpoints":[{"url":"https://monorail-edge.shopifycloud.com/v1/reports/nel/20190325/shopify"}]}
X-Request-ID: d3cd8feb-bf9f-482a-9ee2-1d7fa0b5b987
NEL: {"report_to":"network-errors","max_age":2592000,"failure_fraction":0.01,"success_fraction":0.0001}
Report-To: {"group":"network-errors","max_age":2592000,"endpoints":[{"url":"https://monorail-edge.shopifycloud.com/v1/reports/nel/20190325/shopify"}]}
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
CF-RAY: 50fe6c26ffcad9b8-SIN

@g

7

@Giulio1

Tried 2019-07 version as well but no luck :frowning:

HTTP/1.1 200 OK
Date: Mon, 02 Sep 2019 09:22:58 GMT
Content-Type: application/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: __cfduid=d350db241396c03c9acc0fecb395410491567416177; expires=Tue, 01-Sep-20 09:22:57 GMT; path=/; domain=.myshopify.com; HttpOnly
X-Sorting-Hat-PodId: 84
X-Sorting-Hat-ShopId: 7382728789
Vary: Accept-Encoding
Referrer-Policy: origin-when-cross-origin
X-Frame-Options: DENY
X-ShopId: 7382728789
X-ShardId: 84
X-Stats-UserId: 
X-Stats-ApiClientId: 2972013
X-Stats-ApiPermissionId: 115969425493
X-Shopify-API-Terms: By accessing or using the Shopify API you agree to the Shopify API License and Terms of Use at https://www.shopify.com/legal/api-terms
HTTP_X_SHOPIFY_SHOP_API_CALL_LIMIT: 1/40
X-Shopify-Shop-Api-Call-Limit: 1/40
X-Shopify-API-Version: 2019-07
Strict-Transport-Security: max-age=7889238
X-Shopify-Stage: production
Content-Security-Policy: default-src 'self' data: blob: 'unsafe-inline' 'unsafe-eval' https://* shopify-pos://*; block-all-mixed-content; child-src 'self' https://* shopify-pos://*; connect-src 'self' wss://* https://*; frame-ancestors 'none'; img-src 'self' data: blob: https:; script-src https://cdn.shopify.com https://checkout.shopifycs.com https://js-agent.newrelic.com https://bam.nr-data.net https://dme0ih8comzn4.cloudfront.net https://api.stripe.com https://mpsnare.iesnare.com https://appcenter.intuit.com https://www.paypal.com https://js.braintreegateway.com https://c.paypal.com https://maps.googleapis.com https://www.google-analytics.com https://v.shopify.com https://widget.intercom.io https://js.intercomcdn.com 'self' 'unsafe-inline' 'unsafe-eval'; upgrade-insecure-requests; report-uri /csp-report?source%5Baction%5D=index&source%5Bapp%5D=Shopify&source%5Bcontroller%5D=admin%2Forders&source%5Bsection%5D=admin_api&source%5Buuid%5D=d3cd8feb-bf9f-482a-9ee2-1d7fa0b5b987
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
X-XSS-Protection: 1; mode=block; report=/xss-report?source%5Baction%5D=index&source%5Bapp%5D=Shopify&source%5Bcontroller%5D=admin%2Forders&source%5Bsection%5D=admin_api&source%5Buuid%5D=d3cd8feb-bf9f-482a-9ee2-1d7fa0b5b987
X-Dc: gcp-us-central1,gcp-us-east1,gcp-us-east1
NEL: {"report_to":"network-errors","max_age":2592000,"failure_fraction":0.01,"success_fraction":0.0001}
Report-To: {"group":"network-errors","max_age":2592000,"endpoints":[{"url":"https://monorail-edge.shopifycloud.com/v1/reports/nel/20190325/shopify"}]}
X-Request-ID: d3cd8feb-bf9f-482a-9ee2-1d7fa0b5b987
NEL: {"report_to":"network-errors","max_age":2592000,"failure_fraction":0.01,"success_fraction":0.0001}
Report-To: {"group":"network-errors","max_age":2592000,"endpoints":[{"url":"https://monorail-edge.shopifycloud.com/v1/reports/nel/20190325/shopify"}]}
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
CF-RAY: 50fe6c26ffcad9b8-SIN
8

@jeffBlum_ffa

I had a look at the docs and the 2019-07 Order API still uses the pagination by page parameter.

page
DEPRECATED

The page of results to show.

Removed in version 2019-10. Use cursor-based pagination instead.

So, for now just pass the good old page param: ?page=N

I understand that you may want to uniform all your systems to the new cursor-based pagination, but the 19-07 is supported till 07/2020 so no rush.

9

Hi @jeffBlum_ffa ,

If you’re targeting the 2019-10 version of the order index endpoint, you should definitely be seeing a link header assuming there is a next page to be linked to. Does your response set have a second page of results?

10

@jeffBlum_ffa how to use cursor based pagination?

can you please explain?

Did not understand how to use Link?

Thanks

11

Hi @jeffBlum_ffa

Did the older version on the API worked?

Do you have more than 50 orders in your store?

12

No i am using 2019-07 only.

not for order. i want to use for products.

pagination want to use. i have 100 products i dont want to load 100 products in 1 request.

thanks

13

Hi,
Anyone know how to parse link data in PHP? so I can get proper next and previous page links?

14

I am using this function to navigate to the next link

It’s in VBA but should be easy to translate it.

Function getNextPageURL(ByRef linkHeader As String) As String
    Dim linkArray() As String: linkArray = Split(linkHeader, ",")
    For Each link In linkArray
        link = Trim(link)
        Dim linkfields() As String: linkfields = Split(link, ">;")
        If linkfields(1) Like "*next*" Then
            getNextPageURL = Right(linkfields(0), Len(linkfields(0)) - 1)
            Exit Function
        Else
            getNextPageURL = ""
        End If
    Next link
End Function

It only finds the next link as I never need to go back

Also I wrote this specifically for Shopify APIs so there are good chances that it won’t work with other applications.

15

@Jivan_Suhagiya I answered a similar question elsewhere today, try preg_match():

preg_match("/<(.*)>; rel={next}/", $link_str, $matches);

$matches[1] will in the above case be the next link. There’s probably a better expression to use, but that works for me in PHP.

16

I also wasn’t getting the link header when performing a get request and figured out the solution.

The solution was to state the API Version in the get request URL ex:

setting the API version to 2019-10 in my private app settings then using this URL for get request did not produce the link header: https://key:pass@my-store.myshopify.com/admin/products.json

All I had to do to get the link header was include the API version in the URL ex: https://key:pass@my-store.myshopify.com/admin/**api/2019-10/**products.json

17

To sum up:

The page parameter was dropped from all the REST Admin API endpoints (Some in the 07/2019 and the rest in the 10/2019 release)

Now in order to navigate the API response you will need to parse the HTTP LINK response header, which will contain if they exists, the URLs to the NEXT and PREVIOUS page. You can also set the LIMIT parameter in the HTTP LINK header.

Check:

Page-based pagination replaced by cursor-based pagination across multiple REST endpoints (Shopify)

Making requests to paginated REST Admin API endpoints (Shopify)

The HTTP Link entity-header field (MDN)

RFC 5988, section 5: The Link Header Field (IETF)

Addendum:

This has basically killed async requesting as now you have to wait a request’s response before you can send another one.

18

Previously, I can use the page & limit parameters to visit the X-th page I want. I cannot do so with the new API?

19

Try to export all the products

Order them by product id

and then use the ?since_id=

This kinda recreates the old pagination

20

We found this variation of preg_match worked better in PHP

preg_match('/<([^<]*)>; rel="next"/', $header, $matches);
$next = $matches[1];