Why GraphQL is so inefficient?

Seller_Panda
Shopify Partner
45 0 9

It seems that GraphQL was built for complex queries that needs a lot of data about something specific, but when it comes to basic simple queries that require a lot of edges the rate limit is so small that it doesn't make sense using GraphQL, here is an example:

 

I would like to show a list of products for the user to choose from (he can choose one product), I would like to show him the product image and the title.

with the REST rate limit you can get up to 10,000 products in 40 calls immediately,  but with GraphQL, you can only get up to 500 products.

 

So for this specific task I would go with the REST api without a doubt, the issue starts when you want to filter the products by title.

You can no longer filter products by title in REST, only get products with the exact title, Shopify removed it from REST and implemented it in GraphQL, but In GraphQL I can only get 500 products before reaching the rate limit, and then I have to wait 5 seconds before getting another 250 products.

 

And there is also the batch operations of the GraphQL, but I don't like making calls every second just to see if the operation finished, seems a bit over kill for such simple thing to do.

 

As far as I can see I only have 2 choices:

1. Do it all with REST api and do the filter on my own, which is a big issue, because if someone have 100,000 products, and the products he was searching for, is the last in the list, I will have to do 400 calls just to get it.

2.Use REST for calls without a title filter (when the user first enter the page), and if someone do a title filter, I will use GraphQL call, which seems a little bit too much for such a simple use case of the API.

 

Did I miss anything? I'm sure others have done the same, can you share here what worked best for you?

0 Likes
Greg_Kujawa
Shopify Partner
951 79 203

So I'm trying to wrap my head around this scenario. You are building a custom app that presents the user with all products to scroll through, search through, etc. correct? Does this have to be a near-realtime presentation of the products? If not, then you could export all products to an intermediary database that you control. This would allow you to make basic SQL-type queries and get results a lot more efficiently.

If this custom app requires more realtime results then there are a couple of options. If it were me I'd utilize GraphQL and look into:

  • Present the user with an initial listing a page at a time. For example, the first 25 items are on page 1. Based on the cursor-paginated API response, where you store the cursor ID. If the user clicks the page advance button then you reference the cursor ID to present the next 25 items. Likewise if the user clicks the page back button, you reference that particular cursor ID to present those 25 items.
  • If the user clicks into a search control and provides lookup values, then you utilize the query syntax to narrow down the result set. The cursor-based pagination would likewise come into play to present the user with pages of up to 25 matches at a time.
  • When we are talking about 25 products per page we aren't talking about terrible response times. Likely appearing in a second or so. I'd imagine it would take longer to display the content on the web page or app form.
0 Likes
Seller_Panda
Shopify Partner
45 0 9

Hey @Greg_Kujawa

Thank you for your reply,

 

About creating my own DB of products, its seems way too costly for this task, thinking about users with a lot of products and making sure all the products are updated (new products, deleted products, and correct up to date titles), it can get messy very quickly.

 

About your other solution, it is exactly what I had in mind, the problem is that the rate limit seems to be very low for me to "risk" and use GraphQL all the way.

If I would use 100 products per page, after the first page, I will be left with 800 credits for other stuff, so 1/5 of the credit was 'wasted' just to show a single page of products (just the featured image and the title), while doing almost the same in REST you would only "waste" 1/40 calls.

 

Not to mention that some stuff are simply not possible with GraphQL, for example, filter products of a specific collection:

https://community.shopify.com/c/Shopify-APIs-SDKs/Getting-quot-access-denied-quot-when-querying-with...

https://community.shopify.com/c/Shopify-APIs-SDKs/GraphQl-filter-products-in-a-collection/td-p/76664...

 

So it seems like app developers need to jump between using GraphQL and REST, and always search the best way to do something, and if its even possible.

Using both APIs in a single app seems messy, I just wonder if im missing anything or this is really the case?

0 Likes
Greg_Kujawa
Shopify Partner
951 79 203

You can definitely retrieve a set of products that belong to a particular collection. Here's a GraphQL example, where I am retrieving a set of products that belong to the Jewelry collection. By defining the numProducts variable I can define how many records I want in my result page. And by storing and redefining the cursor variable I can pull the next or preceding page. 

query ($numProducts: Int!, $cursor: String) {
  collections(first: 1, query: "title:Jewelry") {
    edges {
      node {
        id
        title
        products(first: $numProducts, after: $cursor) {
          pageInfo {
            hasNextPage
            hasPreviousPage
          }
          edges {
            cursor
            node {
              id
              title
            }
          }
        }
      }
    }
  }
}

 

0 Likes
Seller_Panda
Shopify Partner
45 0 9

In your example, you are filtering collections, you do not filter the products inside a specific collection 

 

Here is what I would to get:

get all products that include 'copy' in their title, that are in collection id '5'.

0 Likes
Greg_Kujawa
Shopify Partner
951 79 203

If you look, I am first pulling a specific collection --- Jewelry. In my example I am pulling the collection based on the title, although I could likewise pull by ID or whatnot. Once I have that collection I am then pulling products that belong to that collection. Unless I'm missing something I don't see any difference between intent and outcome. And in the product pull I can query with copy* as the query parameter.

query ($numProducts: Int!, $cursor: String) {
  collections(first: 1, query: "title:Jewelry") {
    edges {
      node {
        id
        title
        products(first: $numProducts, after: $cursor, query: "title:copy*") {
          pageInfo {
            hasNextPage
            hasPreviousPage
          }
          edges {
            cursor
            node {
              id
              title
            }
          }
        }
      }
    }
  }
}
0 Likes
Seller_Panda
Shopify Partner
45 0 9

Thats the whole point, you can't do that, you can't query on products inside collection, it will give you 'access denied' message, but will work if you emit the query on products.

{
  collections(first: 10, query: "title:all*") {
    edges {
      node {
        id
        title
        products(first: 10, query: "title:copy*") {
          pageInfo {
            hasNextPage
            hasPreviousPage
          }
          edges {
            cursor
            node {
              id
              title
            }
          }
        }
      }
    }
  }
}

 

The above will result in this :

 

"errors": [
    {
      "message": "access denied",
      "locations": [
        {
          "line": 7,
          "column": 9
        }
      ],
      "path": [
        "collections",
        "edges",
        0,
        "node",
        "products"
      ]
    }
  ],

 

While the below will work as expected:

{
  collections(first: 10, query: "title:all*") {
    edges {
      node {
        id
        title
        products(first: 10) {
          pageInfo {
            hasNextPage
            hasPreviousPage
          }
          edges {
            cursor
            node {
              id
              title
            }
          }
        }
      }
    }
  }
}

 

You can see Shopify response in this link: https://community.shopify.com/c/Shopify-APIs-SDKs/Getting-quot-access-denied-quot-when-querying-with...

0 Likes
Greg_Kujawa
Shopify Partner
951 79 203

Good point. I ran my test just after posting and ran into this same error response. Definitely a less than optimal solution. So in actuality this would require pulling a larger result set than you want (i.e. - all products within a specific collection). And then you'd have to iterate through the result set, only keeping those records that match the user-supplied search criteria. Like I said, not optimal, but at least a path forward.

0 Likes
Seller_Panda
Shopify Partner
45 0 9

Exactly, so I return to my first point, If I want to only filter by product title, my best go is GraphQL, but if I would like to filter products by collection and product title, my best bet would be to use REST api, get all products in a specific collection, and match titles manually with a loop.

 

Not even close to ideal or logical to be honest, so if had something that was already built with GraphQL, lets assume I build the filter without collection, only by product title, I would use the GraphQL api and do it easily, but if I would like to update this filter to include collections later on, I would have to delete the GraphQL code and rebuilt it in REST because its simply not possible to do what I want with GraphQL and the rate limit is not realistic to do it on a large (or even small) scale.

All of this instead of editing one line of code.

 

I hope anyone from Shopify will be able to answer my question, did I miss anything? is it really built that way by design?

On one hand Shopify is removing functionality from the REST api and moving it to GraphQL, but at this point of time, its simply not possible to use GraphQL without defaulting back to the REST API for different tasks that are simply not possible on GraphQL, or reach the rate limit so fast that its not practical to use in a real world situation.   

0 Likes