500 {"errors":"Internal Server Error"} from simple /admin/orders.json requests

Highlighted
Shopify Partner
11 0 2

My request:

GET /admin/orders.json?fields=id,created_at,updated_at,status,financial_status,fulfillment_status&updated_at_min=2017-11-06T10:33:59.4800000Z&status=any&limit=250&page=1

Yields a 500 response code, with a body of:

{"errors":"Internal Server Error"}

This is after a number of successful requests, but happens each time I try to sync orders.  How can I get help with such an error please?

1 Like
Highlighted
Tourist
9 0 1

Ran in to this today too. Grabbed a load of orders successfully with the same endpoint then suddenly 500 Internal Server Error. Respecting the API call limits too so I can only assume something is broken in the API.

0 Likes
Highlighted
Shopify Partner
3 0 0

I'm getting 500's all the time now which is super disruptive for our integration. Hopefully Shopify can clean this up. 

0 Likes
Highlighted
Shopify Staff
Shopify Staff
297 32 71

Hey,

Zameer here from the Shopify Developer Experience team.

These 500 Internal Service Errors are caused by timeouts due to the size of the queries being made. There are a few things that can be done to improve the efficiency of the queries and reduce the overall number of errors:

  1. Including a 'since_id' parameter in your requests will dramatically help improve the efficiency of the queries, resulting in fewer timeouts. You can learn more about the 'since_id' parameter in our documentation, which includes an example of it in use. It basically requires you to store the id of orders received from previous requests while paginating.
  2. Attempting to make the same calls but with a lower limit value. I see that most of the calls that are currently being made include a value of 250 for the limit. Reducing this to even 100, and then making more iterations, should help decrease the possibility of a timeout.
    1. In your application logic, I would also include some sort of mechanism for 500 errors, which retries the request with a lower value for the limit parameter. ie: decrementing by 20 or 50 each time. This could be useful if there are a few orders in particular which are causing longer access times.
  3. Converting to a GraphQL implementation could help deal with a lot of these issues as the overall efficiency of GraphQL queries are far superior to that of REST. However, a full-on conversion to GraphQL is obviously a very developer intensive process, so this would be more of a long-term strategy. There are many online resources which Shopify provides that helps explain the implementation. The associated API limits for GraphQL calls are also greater than those of REST.
0 Likes
Highlighted
Shopify Partner
2 0 0

I've also just started getting 500 errors. Been working with the api all day with no problem until ~30 minutes ago (2:30 PST). I'm fetching from a dev shop that has very few products and even fewer mock orders. This is also with an auto limit of 2 calls every second with a max bucket size of 30.

Additionally I tried to limit the call to return a single product with no luck.

I can fetch collections just fine.

0 Likes
Highlighted
Shopify Staff
Shopify Staff
297 32 71

Hey Adam,

I can take a look to see if you are facing a separate issue. Can you please provide me with your request id found in the 'X-Request-Id' header sent in the response by Shopify?

Thanks

0 Likes
Highlighted
Shopify Partner
9 0 2

@Zameer you suggested the use of `since_id`.  How would we use that if we are trying to pull orders that have been updated_at since a particular day?  what criteria does `since_id` use?  Is it literally the ID that is > than that ID? If so, that doesn't help with trying to request by 'updated_at", but might be a substitute for created_at.

 

For instance,

  • if I were loading history of all orders, I believe I could use the since_id as it would just increment as orders are created, right? 
  • if I were loading incremental updates "since yesterday" I would need to use updated_at_min and updated_at_max. right?

Thoughts?

0 Likes
Highlighted
Shopify Staff
Shopify Staff
297 32 71

Hey Joerz,

 

It's actually still possible to use `since_id` in the case that you've described. You would just have to use it in conjunction with the `updated_at_min` filter.

 

For example, if you want to only retrieve all orders that have been updated in 2019, your original query would look like:

GET /admin/orders.json?since_id=0&updated_at_min=2019-01-01T00:00:00-05:00&limit=250&status=any

You would then retrieve the max order_id returned in that set and use it as the `since_id` value for your next request. In doing so, you will receive an exhaustive list of orders which have been updated in 2019. However, they will simply just not be ordered by their `updated_at` field, which you will then have to order by yourself, should that be important.

 

Let me know if that makes sense for what you're trying to accomplish!

0 Likes
Highlighted
Shopify Partner
9 0 2

@Zameer that does make sense.    So just to confirm, I will just use my max `order.id` in my next request vs. using `page`.  Right?  Also, to confirm... I'm using update_at_min *and* updated_at_max... If so, that's brilliant that I may use the `since_id` in conjunction with other params.

 

I will give this a shot, I think this will work.  And hopefully help me pull my data.  For example, we have one week with ~220,000+ orders and have not been able to successfully retrieve the data.  

 

Thank you.  Hey on another thread re: gift cards, I'd love a little help too... If you could or point any other Shopify experts to this Q: that'd be great...  https://community.shopify.com/c/Shopify-APIs-SDKs/Query-Gift-Card-API-for-updated-after/m-p/487469#M...

 

0 Likes
Highlighted
Shopify Partner
9 0 2

@Zameer thanks for the input.  My apologies for just asking only you...

 

I have gotten `since_id` to work, however, there are days where I am unable to retrieve order data.  Literally I cannot pull data for 1 day with a limit of ONE 1.  Params are:

{
  status: 'any',
  limit: 1,
  created_at_min: 2018-11-24T00:00:00Z,
  created_at_max: 2018-11-24T23:59:59Z,
  since_id: 0
}

 

  The url:

 

https://--shopname--.myshopify.com/admin/orders.json?status=any&limit=1&created_at_min=2018-11-24T00:00:00Z&created_at_max=2018-11-24T23:59:59Z&since_id=0

Response Headers:

 

 

Server → nginx
Date → Fri, 08 Mar 2019 01:16:11 GMT
Content-Type → application/json; charset=utf-8
Transfer-Encoding → chunked
Connection → keep-alive
X-Sorting-Hat-PodId → 85
X-Sorting-Hat-ShopId → 2976313
X-Request-Id → c4f09563-5425-4942-b6eb-6ac531f634a9
X-Shopify-Stage → canary
Content-Security-Policy → default-src 'self' data: blob: 'unsafe-inline' 'unsafe-eval' https://* shopify-pos://*; block-all-mixed-content; child-src 'self' https://* shopify-pos://*; connect-src 'self' wss://* https://*; frame-ancestors 'none'; img-src 'self' data: blob: https:; script-src https://cdn.shopify.com https://checkout.shopifycs.com https://js-agent.newrelic.com https://bam.nr-data.net https://dme0ih8comzn4.cloudfront.net https://api.stripe.com https://mpsnare.iesnare.com https://appcenter.intuit.com https://www.paypal.com https://maps.googleapis.com https://www.google-analytics.com https://v.shopify.com https://widget.intercom.io https://js.intercomcdn.com 'self' 'unsafe-inline' 'unsafe-eval'; upgrade-insecure-requests; report-uri /csp-report?source%5Baction%5D=index&source%5Bapp%5D=Shopify&source%5Bcontroller%5D=admin%2Forders&source%5Bsection%5D=admin_api&source%5Buuid%5D=c4f09563-5425-4942-b6eb-6ac531f634a9
X-Content-Type-Options → nosniff
X-Download-Options → noopen
X-Permitted-Cross-Domain-Policies → none
X-XSS-Protection → 1; mode=block; report=/xss-report?source%5Baction%5D=index&source%5Bapp%5D=Shopify&source%5Bcontroller%5D=admin%2Forders&source%5Bsection%5D=admin_api&source%5Buuid%5D=c4f09563-5425-4942-b6eb-6ac531f634a9
X-Dc → chi2,gcp-us-central1

This happens for multiple days  all these days are failures... 

11-21-2018 -- OK

 

11-22-2018 -- X-Request-Id -- 08db0e45-1cfc-4062-bf72-eadab2e52af7

11-23-2018 -- X-Request-Id -- b960fabb-5b2d-4134-97e5-5758b24cffa4

11-24-2018 -- X-Request-Id -- 7788d1a5-60ab-4403-a6cd-1c0cd5da609b

11-25-2018 -- X-Request-Id -- d1ec0d8f-8cd4-4fcc-a711-964e550957f6

11-26-2018 -- X-Request-Id -- b924ce33-d233-438b-b039-9d9d752743a5

11-27-2018 -- X-Request-Id -- 24dd5589-9f84-4dac-a0d2-2081509b932a

11-28-2018 -- X-Request-Id -- f59885f6-538e-4fb2-addc-4f6182a63e0b

11-29-2018 -- X-Request-Id -- 3eeaf5d2-9382-4cef-8bec-c5148940efc8

11-30-2018 -- OK

 

I REALLY need help to get this data loaded.  Help from anyone at Shopify please.  :-)

 

 

1 Like