Shopify API Speed on Orders endpoint

[SergioFilho]

Is there a way we can retrieve a page of 250 orders from one store with less than 10 seconds? Because I’ve been testing and most of the times the page returns on 15 to 20 seconds. I’ve also tried GraphQL to get only the fields I need, but I can’t get a page of 250, instead, only a page of 50 to 100 (because I need to request 30 fields including line items) and I can’t get the line items properly.

If I try to reduce the page size to half, the time is cut in the same proportion, but we will need to run double requests. Also, is there a way we can get more than 1 page at a time using the cursor based pagination?

[SBD_]

Hey @SergioFilho,

The timing will depend on the amount of information you're requesting and the connection between your server and Shopify.

If you're experiencing these delays locally, try running the code from another server (say, AWS) to see if there are any improvements.

I’ve also tried GraphQL to get only the fields I need, but I can’t get a page of 250, instead, only a page of 50

You can also filter fields via the REST API using the fields parameter:

GET /admin/api/2020-01/orders.json?fields=created_at,id,name,total-price

[SergioFilho]

Hi @SBD_ ,

Thanks for replying!

I have an application running on production already, which runs on AWS (us-east-1). Yes, We're using the field query param.

Here are some facts:

• We're using AWS (us-east-1) to perform requests to Shopify
• We're listing only the fields we use
• We're requesting 2020-04 version
• We're using the cursor based pagination
• We're listing 250 rows per page
• We're using the created_at_max query param

Most pages take from 5 seconds to up to 20 seconds to fulfill

[SBD_]

Thanks @SergioFilho, are you able to provide the a request ID from the response headers please?

[SergioFilho]

Hi @SBD_ , thanks again for replying.

Sure, this request took 29.3 seconds to fulfill. 

60fd2fe3-5408-4bbe-aded-a11bb4d13c9b

[SBD_]

Thanks @SergioFilho

That's a lot of data. Taking ~20 seconds from my end too. I'm not sure much can be done about this - try experimenting with different filters. Otherwise, lowering the limit is the best bet.

[NotAGuru]

I would love to re-open this one as well...

it takes forever to run the get Order or the financial transaction api's. I even reduced the requested fields to the barest minimum, and still running forever... I am talking about minutes to get 50 orders through as loop thanks to the datamodel... The only thing i need is the order name, yet this is hidden in the core getOrder... and to get there, i need to jump through hoops and loops...

has anyone solved this yet? (or do you all dump data every night in  your DB's (AWS or wherever) and run quick analysis there? - Back to the 90s 🙂

Thanks,

Christian 

[Gregarican]

How about a GraphQL API BulkOperations query (https://shopify.dev/tutorials/perform-bulk-operations-with-admin-api)? I know this model has queued progress monitoring that's required by the client, but it does present a jsonl file with all of the records that can be fetched. This versus iterating through paginated results a chunk at a time via the REST API with its associated server- and client-side bottlenecks. 

When I ran a test, pulling 49,581 products via the GraphQL API the bulk operation took 11 minutes (see below). Based on the REST API limits (https://shopify.dev/concepts/about-apis/rate-limits) this same operation would've taken about almost 7 hours. Not counting any server- or client-side bottlenecks.

{
  "data": {
    "currentBulkOperation": {
      "id": "gid://shopify/BulkOperation/53544747170",
      "status": "COMPLETED",
      "errorCode": null,
      "createdAt": "2020-08-06T16:10:38Z",
      "completedAt": "2020-08-06T16:21:12Z",
      "objectCount": "49581",
      "fileSize": "29183334",
      "url": "{download_url}",
      "partialDataUrl": null
    }
  },
  "extensions": {
    "cost": {
      "requestedQueryCost": 1,
      "actualQueryCost": 1,
      "throttleStatus": {
        "maximumAvailable": 1000,
        "currentlyAvailable": 999,
        "restoreRate": 50
      }
    }
  }
}

[Gregarican]

...side note. That estimate of 7 hours to retrieve the results by _not_ using the GraphQL bulk operations query was incorrect. Since a REST API query can fetch up to 250 records per response page. But even then, if you were seeing up to 20 seconds per response page pulling the orders, you'd be looking at around a half hour's time using my 49K record example. I'd take 11 minutes versus about half an hour anytime!