Get all metafields for products

New Member
2 0 0

Hello,

We are running into the same exact problem. For several of our customers we would like to retrieve and process the meta datafields every day. However, with the current restrictions this is just not an option because this could result in 100k+ requests every day. 

We hope there will be found a solution to this problem in the future.

0 Likes
Shopify Expert
9807 91 1557

^ What are you actually doing that would require you to have 100k API calls daily? I'd like to understand context as there may be smarter ways to approach things.

★ Winning Partner of the Build a Business competition. ★ http://freakdesign.com.au
0 Likes
New Member
2 0 0

As a datafeed management company we currently download all the product data from Shopify, process this (e.g remove products which are out of stock, add the color of the product to the title etc) and send the modified data to "export channels" such as Amazon and Google Shopping. We don't push any data to Shopify, only retrieve it every day.

Some of our Shopify customers store data in the meta data fields, for example color, sku's etc. and our customers want to use this information as to better optimize their ads on the "export channels".

We streamline this process as much as possible as to make it as easy as possible for the customer. When a customer changes an item's description (or stock changes after a sell), this  will automatically be changed on the export channels. We want to do something similar for the meta data fields.

Some customers have a lot of products in their stores and thus getting all the meta_fields for all the products every day would be a huge amount of api calls.

We are aware that the amount of api calls could be reduced by giving a customer an option to fetch the meta_data fields in Channable instead of doing this every day. However, we would like to keep our strucuture the same for all ecommerce shops and this is not required for other shops. Furthermore, this would require a big change in our backend structure.

I hope this clears things up.

0 Likes
Shopify Expert
3943 16 330

This one made me giggle. Please Shopify, could you alter your systems to match mine so I don't have to work anymore? Would be swell!! Thanks and tata... have a nice day.

Custom Shopify Apps built just for you! hunkybill@gmail.com http://www.resistorsoftware.com
0 Likes
Shopify Partner
7 0 0

+1 for this

One call per product to get the metafields is way too much when you have to deal with thousands of products.

Awesome Store Apps for your Shopify store - https://awesomestoreapps.com/
0 Likes
Shopify Expert
3943 16 330

Soon enough we'll get GraphQL calls and then we can all get the associated metafield resources needed, and this whole silly thread can go away forever.

Custom Shopify Apps built just for you! hunkybill@gmail.com http://www.resistorsoftware.com
0 Likes
Shopify Partner
660 0 109

Do you have non public knowledge about this?

As far as I understand GraphQL is currently limited to "customer facing" entities.

It's not clear to me exactly where the boundary between customer facing and back office entities is, but I'd think that metafields are considered "back office"..

0 Likes
Shopify Expert
3943 16 330

No dude, I have no secret info. But that is almost certainly where things are going. Eventually we will have GraphQL access to objects and resources. Until then, we deal with the RESTful API and do 100,000 API calls in ~14 hours. YMMV. 

Therefore, whether we like it or not, we understand that using Metafields is non-performant when your call count is high. Use a different strategy. That is the secret. Knowing how to squeeze performance. 

Custom Shopify Apps built just for you! hunkybill@gmail.com http://www.resistorsoftware.com
0 Likes
Shopify Expert
3943 16 330

I get X calls per week from merchants that complain to me about Apps that bungle this performance issue, and there is zilch I can do for them, except to tell them they cannot use an App that is architected to fail on big product counts. 

Using a lowest fruit hanging from the tree approach (sometimes Metafields), is clearly easy, but has hazards. It is obvious from the get-go, so expecting magic and a sudden elimination of this performance is not likely. We could wake up tomorrow with metafields available, but then again, if it was easy, we would've had that 8 years ago. Clearly, it is best to wait for the internal wiring to catch up, and when it does, using Metafields will likely be suitable for a wide range of uses currently not practical.

Custom Shopify Apps built just for you! hunkybill@gmail.com http://www.resistorsoftware.com
0 Likes
Shopify Partner
108 2 13

+1

Pulling in 1,000 products with their metafields currently takes around 10 minutes to complete because of the API call limit.

As others have suggested, the ability the pull metafields based on owner_resource and namespace would significantly reduce the number of calls required.

Something simple such as 4 metafields returned per product could reduce the total number of API calls from 1,004 to 20.

Co-Founder / CTO @ Intuitive Shipping
Intuitive Shipping | Automate Shipping Profiles
0 Likes