I'm developing an app that stores content along side various store entities (Products, Pages, Articles) and have hit an issue with the rate limit when querying multiple entities on the REST API.
Can anyone point me in the right direction to produce a list of Pages (and Articles) that contain a certain metafield for use as a dashboard / index.
For Products I'm able to use the GraphQL API but it doesn't seem possible to use this for Pages or Articles.
Currently the application requests all Pages and then checks each one individually using the REST API but with a rate limit of 2-4 requests/second this doesn't seem feasible for production.
I'm not sure why metafields aren't exposed on OnlineStorePage or OnlineStoreArticle. I'll follow up with our product teams about that.
In the meantime, you can try increasing the upper limit of your REST calls which should allow you to get more data at once. The leaky bucket algorithm used by the REST throttle will allow up to 40 calls in quick succession, and those will replenish at a rate of 2/sec. In this case, you can try making a call to
GET /admin/api/2021-01/pages.json?limit=150 to get 150 records in the response, instead of the default 50. You can also consider using an additional filter param for fields, if you're only interested in certain fields in the response. For example, adding the param
&fields=id,title,metafields will return just those fields in the response.
Hi _JB, and thank you very much for your response.
It does seem very strange that Shopify would push the GraphQL API so hard whilst not allowing some fundamental data to be retrieved by using it - I was hoping I'd just missed something and there'd be a magic bullet but that doesn't seem to be the case.
In regards the increasing the upper limit of the REST calls, I don't think that's actually going to help my situation as the REST API doesn't seem to return metafields, even when using your example below:
Reading the Page Rest API Documentation metafield is listed as a page property but never shown in any responses - it seems to me that you can add metafields using a page PUT or POST but never actually query them (without querying the metafield resource itself as per: Metafield Rest API Documentation).
If it was possible to retrieve page metafields in bulk using your example above of pages.json then this would go someway to mitigate the issue (assuming I could do the same for articles) but unless I'm still missing something I don't think that's the case.
In short, I still can't find a solution to query 𝑥 pages worth of metafields without making 𝑥 individual requests.