Our Partner & Developer boards on the community are moving to a brand new home: the .dev community forums! While you can still access past discussions here, for all your future app and storefront building questions, head over to the new forums.

How should I bulk fetch Page (and Article) Metafields?

How should I bulk fetch Page (and Article) Metafields?

jamiewscott
Shopify Partner
3 0 0

Hi all,

I'm developing an app that stores content along side various store entities (Products, Pages, Articles) and have hit an issue with the rate limit when querying multiple entities on the REST API.

Can anyone point me in the right direction to produce a list of Pages (and Articles) that contain a certain metafield for use as a dashboard / index.

For Products I'm able to use the GraphQL API but it doesn't seem possible to use this for Pages or Articles.

Currently the application requests all Pages and then checks each one individually using the REST API but with a rate limit of 2-4 requests/second this doesn't seem feasible for production.

 

Replies 3 (3)

_JB
Shopify Staff (Retired)
836 100 223

Hey @jamiewscott,

I'm not sure why metafields aren't exposed on OnlineStorePage or OnlineStoreArticle. I'll follow up with our product teams about that.

In the meantime, you can try increasing the upper limit of your REST calls which should allow you to get more data at once. The leaky bucket algorithm used by the REST throttle will allow up to 40 calls in quick succession, and those will replenish at a rate of 2/sec. In this case, you can try making a call to  GET /admin/api/2021-01/pages.json?limit=150 to get 150 records in the response, instead of the default 50. You can also consider using an additional filter param for fields, if you're only interested in certain fields in the response. For example, adding the param &fields=id,title,metafields will return just those fields in the response.

JB | Solutions Engineer @ Shopify 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit Shopify.dev or the Shopify Web Design and Development Blog

jamiewscott
Shopify Partner
3 0 0

Hi _JB, and thank you very much for your response.

It does seem very strange that Shopify would push the GraphQL API so hard whilst not allowing some fundamental data to be retrieved by using it - I was hoping I'd just missed something and there'd be a magic bullet but that doesn't seem to be the case.

In regards the increasing the upper limit of the REST calls, I don't think that's actually going to help my situation as the REST API doesn't seem to return metafields, even when using your example below: 

GET /admin/api/2021-01/pages.json?limit=150&fields=id,title,metafields

Reading the Page Rest API Documentation metafield is listed as a page property but never shown in any responses - it seems to me that you can add metafields using a page PUT or POST but never actually query them (without querying the metafield resource itself as per: Metafield Rest API Documentation).

If it was possible to retrieve page metafields in bulk using your example above of pages.json then this would go someway to mitigate the issue (assuming I could do the same for articles) but unless I'm still missing something I don't think that's the case.

In short, I still can't find a solution to query 𝑥 pages worth of metafields without making 𝑥 individual requests.

jam_chan
Shopify Partner
925 23 188

I'm looking for the same thing. Currently, it's only possible to bulk fetch products and collections with metafields. But pages and articles are not possible.

After Shopify Unite, I still don't see it's possible despite the huge update on metafields API. Am I missing anything?

BYOB - Build Your Own Bundles, SPO - SEO App to research keywords & edit social link preview