Shopify Community AMA with Shopify Developers: The New GraphQL Product APIs

[Jacqui]

Join us Tuesday, June 18th from 11 am to 1 pm ET for a live AMA where our team will answer questions on the new GraphQL Product APIs announced in the 2024-04 stable API release.  Hit "Reply" below to ask your questions before June 18th, 2024 and we’ll answer them live, or bring your questions to be answered on the day of.

What to know:

The new GraphQL product APIs that support up to 2,000 variants per product were recently launched in the 2024-04 stable API release, enabling developers to build support for larger and more complex catalogs. Alongside the release of the new GraphQL product APIs, we also announced deprecations of the REST product APIs, as well as several fields within the existing GraphQL product APIs. 

This AMA is an opportunity to interact with the developers who built these APIs to get any questions on both the new APIs and associated deprecations answered.  

In advance of the session, please review the API documentation on the new GraphQL product APIs, our FAQ and our relevant community discussions.

Rules of engagement:

We’ll do our best to follow up on every question during the event.

• You must be logged into your account on Community to participate.
• Assume positive intent with one another and be kind. Some users may be new and learning how to engage with the Community. 
• Be authentic and honest with your questions and answers.
• Do not ask for or share store admin login details or passwords.
• Like posts and threads that are of interest or helpful to you.
• Be constructive with criticism, not offensive.
• Share your experiences and insights whenever you can to support others. 
• Abuse, harassment, and spam content won’t be tolerated.

[AsafGitai]

Hi @tejasvyas 

Please see our response to your question here 🙂 
https://community.shopify.com/c/new-graphql-product-apis/presentment-price-support-in-new-graphql-pr...

[AlexisBreuil]

Hi!

We can update a metafield with a key and a namespace, why is the metafield id mandatory on the productSet mutation?

thanks

[kestutisbalt]

This is particularly true, especially when considering that each key must be unique. To update a metafield now requires querying it first and then performing the update—two operations instead of one. If there are multiple updates, this doubles the number of requests and increases waiting time accordingly.

[EfstathiosS]

Thanks for this feedback, we will pass this along to our team.

[ClementBR]

We need to sync all products and variants data in our database.

I see that moving forward, the product webhook will only contain data for the first 200 variants. I guess we are expected issue one or more queries to get the data for all the remaining variants.

That sounds very inefficient, especially considering that only a single variant might have changed.

To make things more efficient, can you introduce a new set of webhook topics for variants specifically, so that we can know when a specific variant was changed? Such as variants/created , variants/updated, variants/deleted.

While on this topic, it has been a long standing issue that updating a variant metafield (via the API, no the admin) does NOT trigger any products/update. Maybe this can be fixed at the same time, by firing variants/updated whenever a metafield is updated?

[AsafGitai]

Hi @ClementBR 

We recently added an updated_at timestamp & sorting by it to the GID. You should be able to use that info to decide if and what variants you would need to query to update your app.

[ClementBR]

1) I'm not sure if I'm supposed to know how to decipher what you mean. GID is for global ID? What do you mean that you added "updated_at" & "sorting" by "it"? Incredibly confusing.

I looked at the variants sorting key and UPDATE_AT is not in the list.

2) Otherwise, what about my actual questions? Polling is incredibly wasteful (both for Shopify and for apps). Webhooks are designed exactly to avoid having to poll. If we have to poll, it feels like a big step backwards for such core functionality.

[AsafGitai]

Hi @ClementBR, hope this helps explain:

The description should mention these are sorted by updated_at desc, will make sure we fix that.
Let me know if you have further questions

[ClementBR]

1) @AsafGitai thanks but still a bit unclear. Does the array you highlighted include only variants that have changed? Or does it always include all variants? We'd still prefer you would add new topics for variants... It's strange to introduce a completely new pattern when there is already a clear and simple pattern for all other webhooks.

2) Could you please also respond to my other question above regarding variant metafield not triggering a webhook?

[ClementBR]

1) @AsafGitai thanks but still a bit unclear. Does the array you highlighted include only variants that have changed? Or does it always include all variants? We'd still prefer you would add new topics for variants... It's strange to introduce a completely new pattern when there is already a clear and simple pattern for all other webhooks.

2) Could you please also respond to my other question above regarding variant metafield not triggering a webhook?

[AsafGitai]

we have updated the documentation to make it a bit clearer

[JustinKillian]

Hello there,

We are an app that requires to sync an external database to Shopify. We support syncing media, products, variants, inventory items and available inventory quantities (through inventory levels). In previous versions of the GraphQL API we could sync complete products (with all attributes previously mentioned) in a single bulk mutation using `productCreate` or `productUpdate`.

With recent changes to the product model we need to refactor our workflow to be backward compatible. We are thinking about doing something like this:
1. Create new media using bulk `createFile`
2. Create/update products and variants using bulk `productSet`
3. Publish products using `publishablePublish`

4. Update inventory items using bulk `productVariantUpdate` (with nested `inventoryItem`) or use `inventoryItemUpdate`
5. Update inventory quantities using `inventoryActivate`. We could potentially use `inventoryAdjustQuantity` too
6. Delete, rearrange, update media

It would require us to perform at least 6 API calls instead if 1 to continue to support what we already support in our app. Some of the API calls previously mentioned support bulk mutations but some others don't.

I am worried about the complexity that's getting introduced for use cases like ours, as well as rate limits and the app performance of our integration (timeouts from Shopify) when we get to sync customer's shops for thousands of products with hundreds (and now potentially thousands) of variants each. 

Two questions:
1. Are the plans to extend `productSet` so it would work as `productCreate` and `productUpdate` used to work? It was very useful for use cases where a complete database sync should happen
2. Is the algorithm described above accurate to achieve what we need to achieve or there are better/easier ways to achieve our goal?

Thanks!

[DavidT]

This has been my biggest headache with all these changes. My current app workflow just wouldn't support having multiple bulk mutations for each different field. Also as you said some of these don't even support bulk mutations.

[tedtate]

👋 Hi Justin,

1. For the initial release we limited the scope of the mutation to try and keep performance reasonable, especially with higher variant counts. Conceptually this is the exact type of operation we want to work really well with productSet - when you have a database of info that you would like to sync into Shopify. Adding additional fields is something that achieves that goal and we are considering for future releases.
2. Yup, as of today thats the correct workflow. The one adjustment you could make is to use the new inventorySetQuantities mutation (added in 24-07) to set instead of adjusting quantities - https://shopify.dev/docs/api/admin-graphql/unstable/mutations/inventorySetQuantities

[kestutisbalt]

Also, are there any plans to introduce change history for product updates?

Currently, it's challenging to troubleshoot issues when multiple apps update the same product. Customer support often tells merchants that it's the app's fault without trying to understand where the issue is coming from (I guess they have no technical background). As app developers, we then have to spend long hours as outsiders gathering timestamps from the API and proving to clients that the app did not update the product and that it was likely another third-party app. By this point, the merchant doesn't believe us because Shopify customer support stated that it was the app's fault, and 9 out of 10 times, it isn't. Having a change history showing who made changes would solve this issue and save time for app developers and Shopify customer support.

[EfstathiosS]

Thank you for your feedback. We will make sure that both the change history request and the customer support observations are passed along to the relevant teams.

[ShashankAgrawal]

Customer support misdirection is a real issue. In the lack of information they point it to any random app that may or may not have permission to make those changes. 

[jam_chan]

Hi,

I have some questions:

1. How can I delete all variants with an API call? Specifically, I want to clear the variants and keep the 1st variant only.
2. How can I search through the variants with a keyword? I want to look for variants with a specific option value like all yellow tees. My use case is to create a variant picker to pick some variants in a product. My app users can select the variants available in a customized bundle.
3. Liquid
   1. I used to print the variants of products in a collection with the JSON filter. Will this implementation become slow if I want to print the product variants in liquid? Something like variants: {{ product.variants | json }},
   2. Besides the Graphql API, Is the liquid updated to support the upcoming changes? I don't see any update about liquid. Did I miss anything?

Thanks for your attention

[Mete]

An API call returns 250 products and all of their variants. In the standard plan, we have 2 API calls per second, equating to 500 products and all of their variants.

Can you compare this with GraphQL? With a corresponding 100 points/second, how many products and variants can you get per second?

According to this documentation https://shopify.engineering/rate-limiting-graphql-apis-calculating-query-complexity, each object costs 1 point. However, when I look at the actual values, the computation methods seem different for the product resource. Could you explain the new cost calculation method?

"Running a query is the best way to find out its true cost" isn't a good explanation for people developing applications.

[EfstathiosS]

The easiest way to understand the cost is to rebuild your REST request as an equivalent GraphQL query and run the query. Due to the dynamic nature of queries, costing them is best done practically, rather than theoretically.

Thank you for your feedback on the rate limits, we are actively looking into this area and appreciate all feedback. We will be providing more information at the time that some merchants will start receiving early access to 2K variants.

[Mete]

The costs are not working logical that's why I'm asking. 

[EfstathiosS]

May you clarify your use case, please?

After rewriting your equivalent REST query in GraphQL you can use the `X-GraphQL-Cost-Include-Fields: true` header on your request to receive a breakdown of the query cost. Have you tried this and found inconsistent results?

[hmpws]

Hello, we have tested the new product API briefly. At the moment, we ran into workflow issues when we try to create many products.

With the previous version, we were able to use productCreate with bulk mutation. With the new version, product creation is now separated into productCreate and then productVariantsCreate/productVariantsBulkCreate but the latter operation isn't enabled with bulk mutation. We found that productSet is not a solution because we can't set inventory items (e.g. costs) or add media like some people have pointed out. Please enable bulk mutation with productVariantsBulkCreate or advice on how best to proceed when creating many products.

Related to this, some of our clients have run into daily variant creation limit when they have many product variants on the store (https://shopify.dev/docs/api/usage/rate-limits#resource-based-rate-limits). Is this limit going change with the new API seeing as products now support many variants?

Thanks.

[SKULabs]

That "bulk bulk" updates being mandatory to meet the REST API capabilities may result in large slow downs. If we can't find parity with the REST API performance-wise, it won't be the partners politely highlighting these concerns once this migration is forced on all of the sellers.

The Shopify REST API holds the Shopify team to a high standard I suppose. It's been very efficient and reliable.

Update: In another comment, they said they expect performance parity by EOY.

[tedtate]

👋 Hi Hmpws,

We are working on making bulkMutation support better across a bunch of mutations. Currently we are looking at making it work with the productVariantsBulk* mutations. We also are considering adding additional features to productSet so that it could be a fit usable in this situation.

With respect to the daily variant creation limit, we are currently doing a holistic evaluation of our general constraints and resource limits. This is one of the areas that we're actively evaluating and will post an update on changes here in the next couple of months.

[ravneet-kaur]

Hello,

Since the new Product GraphQL API allows creating up to 2000 variants, what is the best way to sync/pull all products along with their variants of a store from Shopify? I experienced that even if a store has 100 variants mapped to products, the GraphQL API breaks with 500 (Internal server error) when we try to fetch 250 products along with variants in a single request. The requested cost of such a query is less than 1000 points (which is the allowed requested cost for a single query) but API breaks. What is the recommended way to do that?

[SKULabs]

We can vouch for the instability of both the bulkQuery and standard graphQL endpoints being a large hurdle. We're never sure if we should retry or if we will have to reach out to Shopify support or if we've somehow improperly designed a query?

Given these limitations, it is currently not possible to build to the standard of reliability our customers or Shopify partner program agreements expect. We either significantly (think 5-20x slower) increase how long it takes to sync or we face intermittent faults. It's lose-lose currently until significant strides are made.

[John_Hardin]

Can you provide more details about the Bulk Query endpoint being unstable?

[SKULabs]

Can I get you in touch with one of our two developers working on this issue full time right now?

The issues we're encountering remind me of the arbitrary 500 errors when accessing certain pages of products via REST from certain stores. If I remember correctly, there was some kind of indexing fault in that case but some stores are just hitting some kind of wall also. e.g. we have to restrict our page size to well below the advertised maximum which reduces efficiency in the eyes of the seller because they have to wait longer.

In one case, we see unpredictable 50+ minute delays for a simple handful of orders. This prevents us from running *any* product bulkQueries during that window. With 3-4 subsystems all depending on bulkQueries we can't arbitrarily cancel any bulkQuery taking longer than 10 minutes or some processes may never complete.

In another case, we just get arbitrary timeouts which force us to retry bulk and regular queries with half the request size e.g. first = first / 2 then retry or for bulk, half of the chunk of IDs we tried last time (again *well* under the advertised 250 limit - think 10-20 at a time 😬)

Specifically on the bulk side, only getting partial results or seeing frequent bulk query failures forcing robust retry logic.

On the plain graphql side, this would involve complex multi-tiered nested queries to ensure we retrieve 100% of the connections under a product/order which is high-risk for making mistakes.

Bulk would be preferable but the performance/reliability is just not there.

[John_Hardin]

Ah, ok. I neglected to consider the complexity of the bulk query, as we're only retrieving a fairly small subset of critical data for all products that way.

Thanks!

[EfstathiosS]

Please reach out to Shopify support, platform stability and performance is important to us.

[John_Hardin]

For pulling all of the products, a bulk (background) query is also an option. It's more reliable but the delay is less predictable.

Perform bulk operations with the GraphQL Admin API (shopify.dev)
Shopify is also strongly suggesting this as the preferred way to get all products.

[JustinKillian]

+1 on this.

We have also faced this issue (500 with timeout) while polling a bulk querying status. Our query fetches a bunch of data for each product (images, variants, inventory items, inventory levels) and my understanding is that either reducing the amount of data we ask for or ask for less products at a time is a reasonable workaround. However, I do think that it would help (us and our customers) a lot if this gets fixed on the Shopify side.

[EfstathiosS]

Hello!

Your approach sounds correct, paginated calls to the product or productVariants queries is the recommended way.

You shouldn’t experience 500 errors but, in the event that you do, please report the issue to support and try again with a smaller query. For example, try fetching a smaller number of products.

[initjsk]

1. Will the GraphQL Product API support marking a product for exclusion from all discount codes and automatic discounts?

2. Will there be an equivalent of the REST product count endpoint?

[itsdarrylnorris]

There is a graphql query for /products/count.json- https://shopify.dev/docs/api/admin-graphql/2024-04/queries/productsCount

[AsafGitai]

Yes, you can use productsCount for this. It’s currently limited to 10,000 products, we are looking into the best way to lift the limit sooner than later

[ShashankAgrawal]

How do we retrieve 2000 variants for multiple products using the defer directive in a single StoreFront API call? 

Doing multiple calls for each product isn't very efficient. 

[AsafGitai]

Hi @ShashankAgrawal ,

We are working on adding new functionality to the storefront graphql api that should enable you to query for a more targeted set of variants.

[ShashankAgrawal]

The GraphQL product count API is limited to 10,000 count only. Can we expect a fix for this soon?

[AsafGitai]

Hi @shashankagarwal 

The initial limit was tied into performance concerns, we are looking into the best way to lift the limit sooner than later

[Harelk1015]

link to join ?

[Harelk1015]

anyone konws ?

[Jacqui]

Hey @Harelk1015, 

This AMA is not via video. You can post your questions here. Our developer team is working on answering as many questions as possible during the time limit.

[Harelk1015]

ooo thanks!

[ShashankAgrawal]

When can we expect support for product data for B2B users through GraphQL API. Currently GraphQL API doesn't return price, availability or publish status for a B2B user. 

[AsafGitai]

Hi @ShashankAgrawal 

This can be achieved via the Admin API and Storefront API. 

Storefront API

Via the @inContext directive’s new buyer argument. 

query Products (
   $buyer: BuyerInput
) @inContext(buyer: $buyer) {
   products(first: 5) {
     nodes {
       id
       variants(first: 5) {
        nodes {
          id
          quantityRule {
            maximum
            minimum
            increment
          }
          quantityPriceBreaks(first: 5) {
            nodes {
              minimumQuantity
              price {
                amount
                currencyCode
              }
            }
          }
        }
      }
    }
  }
}

// variables

"buyer": {
   "customerAccessToken": "shpsb_eyJh123456789",
   "companyLocationId": "gid://shopify/CompanyLocation/10079785100",
}

Admin API

By passing the companyLocationId to the publishedInContext or contextualPricing fields. You’ll need to ensure your app has the correct access scopes for products and publications.

query Products {
  products(first: 10) {
    nodes {
      id
      publishedInContext(context: {
        companyLocationId: "gid://shopify/CompanyLocation/1"
      })
      contextualPricing(context: {
        companyLocationId: "gid://shopify/CompanyLocation/1"
      }) {
        minVariantPricing {
          price { amount currencyCode }
        }
        maxVariantPricing {
          price { amount currencyCode }
        }
      }
    }
  }
}

query ProductVariants {
  productVariants(first: 10) {
    nodes {
      id
      contextualPricing(context: {
        companyLocationId: "gid://shopify/CompanyLocation/1"
      }) {
        price { amount currencyCode }
        compareAtPrice { amount currencyCode }
      }
    }
  }
}

[ShashankAgrawal]

Thanks @AsafGitai, we'll try this and let you know if it solves our case.

[Addrevenue]

Is there a faster way to retrieve products and its variations?

Our use case is that we fetch products from hundreds of stores (our customers, that has authorized our app), creating product feeds for them to use in external systems.

We are using the GraphQL endpoint, and I’m running a bulkOperationRunQuery mutation on productVariants, which is also fetching its product. I’m then polling the BulkOperationId for a total of 30 times (somewhere around 6 minutes).

But this operation is very slow! At best 5-10 seconds, most often 20-30 seconds and sometimes it even times out!

And even worse, during the time it processes in the background, I’m not allowed to start other processes, as it says ”A bulk query operation for this app and shop is already in progress”. Isn’t this sort of the point with background operations? That you can run multiple?

For our use case, this is very annoying. Image fetching hundreds of feeds, where some feeds takes minutes, and we are not allowed to make any other calls during this whole period! And we do this every night.

Isn’t there a faster method to do this? Ideally synchronously, if it is < 10 seconds.

We have similar functionality for WooCommerce for example, and it takes a second or two to fetch even big sets of data.

[tedtate]

👋 Hi @Addrevenue,

Do you have an example of a mutation that you are trying to run that is timing out?

From reading more about your use case I think designing a set of mutations that would allow you to avoid using bulkOperation would be ideal. If you have to do many calls to bulkOperation in sequence it kind of kills the usefulness of that operation.