FROM CACHE - en_header

500 {"errors":"Internal Server Error"} from simple /admin/orders.json requests

Kieren_Johnston
Shopify Partner
11 0 2

My request:

GET /admin/orders.json?fields=id,created_at,updated_at,status,financial_status,fulfillment_status&updated_at_min=2017-11-06T10:33:59.4800000Z&status=any&limit=250&page=1

Yields a 500 response code, with a body of:

{"errors":"Internal Server Error"}

This is after a number of successful requests, but happens each time I try to sync orders.  How can I get help with such an error please?

Replies 25 (25)
Jamie13
Tourist
9 0 1

Ran in to this today too. Grabbed a load of orders successfully with the same endpoint then suddenly 500 Internal Server Error. Respecting the API call limits too so I can only assume something is broken in the API.

Dan58
Shopify Partner
3 0 0

I'm getting 500's all the time now which is super disruptive for our integration. Hopefully Shopify can clean this up. 

Zameer
Shopify Staff
Shopify Staff
297 31 88

Hey,

Zameer here from the Shopify Developer Experience team.

These 500 Internal Service Errors are caused by timeouts due to the size of the queries being made. There are a few things that can be done to improve the efficiency of the queries and reduce the overall number of errors:

  1. Including a 'since_id' parameter in your requests will dramatically help improve the efficiency of the queries, resulting in fewer timeouts. You can learn more about the 'since_id' parameter in our documentation, which includes an example of it in use. It basically requires you to store the id of orders received from previous requests while paginating.
  2. Attempting to make the same calls but with a lower limit value. I see that most of the calls that are currently being made include a value of 250 for the limit. Reducing this to even 100, and then making more iterations, should help decrease the possibility of a timeout.
    1. In your application logic, I would also include some sort of mechanism for 500 errors, which retries the request with a lower value for the limit parameter. ie: decrementing by 20 or 50 each time. This could be useful if there are a few orders in particular which are causing longer access times.
  3. Converting to a GraphQL implementation could help deal with a lot of these issues as the overall efficiency of GraphQL queries are far superior to that of REST. However, a full-on conversion to GraphQL is obviously a very developer intensive process, so this would be more of a long-term strategy. There are many online resources which Shopify provides that helps explain the implementation. The associated API limits for GraphQL calls are also greater than those of REST.
abohannon
Shopify Partner
2 0 0

I've also just started getting 500 errors. Been working with the api all day with no problem until ~30 minutes ago (2:30 PST). I'm fetching from a dev shop that has very few products and even fewer mock orders. This is also with an auto limit of 2 calls every second with a max bucket size of 30.

Additionally I tried to limit the call to return a single product with no luck.

I can fetch collections just fine.

Zameer
Shopify Staff
Shopify Staff
297 31 88

Hey Adam,

I can take a look to see if you are facing a separate issue. Can you please provide me with your request id found in the 'X-Request-Id' header sent in the response by Shopify?

Thanks

joerz
Shopify Partner
9 0 2

@Zameer you suggested the use of `since_id`.  How would we use that if we are trying to pull orders that have been updated_at since a particular day?  what criteria does `since_id` use?  Is it literally the ID that is > than that ID? If so, that doesn't help with trying to request by 'updated_at", but might be a substitute for created_at.

 

For instance,

  • if I were loading history of all orders, I believe I could use the since_id as it would just increment as orders are created, right? 
  • if I were loading incremental updates "since yesterday" I would need to use updated_at_min and updated_at_max. right?

Thoughts?

Zameer
Shopify Staff
Shopify Staff
297 31 88

Hey Joerz,

 

It's actually still possible to use `since_id` in the case that you've described. You would just have to use it in conjunction with the `updated_at_min` filter.

 

For example, if you want to only retrieve all orders that have been updated in 2019, your original query would look like:

GET /admin/orders.json?since_id=0&updated_at_min=2019-01-01T00:00:00-05:00&limit=250&status=any

You would then retrieve the max order_id returned in that set and use it as the `since_id` value for your next request. In doing so, you will receive an exhaustive list of orders which have been updated in 2019. However, they will simply just not be ordered by their `updated_at` field, which you will then have to order by yourself, should that be important.

 

Let me know if that makes sense for what you're trying to accomplish!

joerz
Shopify Partner
9 0 2

@Zameer that does make sense.    So just to confirm, I will just use my max `order.id` in my next request vs. using `page`.  Right?  Also, to confirm... I'm using update_at_min *and* updated_at_max... If so, that's brilliant that I may use the `since_id` in conjunction with other params.

 

I will give this a shot, I think this will work.  And hopefully help me pull my data.  For example, we have one week with ~220,000+ orders and have not been able to successfully retrieve the data.  

 

Thank you.  Hey on another thread re: gift cards, I'd love a little help too... If you could or point any other Shopify experts to this Q: that'd be great...  https://community.shopify.com/c/Shopify-APIs-SDKs/Query-Gift-Card-API-for-updated-after/m-p/487469#M...

 

joerz
Shopify Partner
9 0 2

@Zameer thanks for the input.  My apologies for just asking only you...

 

I have gotten `since_id` to work, however, there are days where I am unable to retrieve order data.  Literally I cannot pull data for 1 day with a limit of ONE 1.  Params are:

{
  status: 'any',
  limit: 1,
  created_at_min: 2018-11-24T00:00:00Z,
  created_at_max: 2018-11-24T23:59:59Z,
  since_id: 0
}

 

  The url:

 

https://--shopname--.myshopify.com/admin/orders.json?status=any&limit=1&created_at_min=2018-11-24T00:00:00Z&created_at_max=2018-11-24T23:59:59Z&since_id=0

Response Headers:

 

 

Server → nginx
Date → Fri, 08 Mar 2019 01:16:11 GMT
Content-Type → application/json; charset=utf-8
Transfer-Encoding → chunked
Connection → keep-alive
X-Sorting-Hat-PodId → 85
X-Sorting-Hat-ShopId → 2976313
X-Request-Id → c4f09563-5425-4942-b6eb-6ac531f634a9
X-Shopify-Stage → canary
Content-Security-Policy → default-src 'self' data: blob: 'unsafe-inline' 'unsafe-eval' https://* shopify-pos://*; block-all-mixed-content; child-src 'self' https://* shopify-pos://*; connect-src 'self' wss://* https://*; frame-ancestors 'none'; img-src 'self' data: blob: https:; script-src https://cdn.shopify.com https://checkout.shopifycs.com https://js-agent.newrelic.com https://bam.nr-data.net https://dme0ih8comzn4.cloudfront.net https://api.stripe.com https://mpsnare.iesnare.com https://appcenter.intuit.com https://www.paypal.com https://maps.googleapis.com https://www.google-analytics.com https://v.shopify.com https://widget.intercom.io https://js.intercomcdn.com 'self' 'unsafe-inline' 'unsafe-eval'; upgrade-insecure-requests; report-uri /csp-report?source%5Baction%5D=index&source%5Bapp%5D=Shopify&source%5Bcontroller%5D=admin%2Forders&source%5Bsection%5D=admin_api&source%5Buuid%5D=c4f09563-5425-4942-b6eb-6ac531f634a9
X-Content-Type-Options → nosniff
X-Download-Options → noopen
X-Permitted-Cross-Domain-Policies → none
X-XSS-Protection → 1; mode=block; report=/xss-report?source%5Baction%5D=index&source%5Bapp%5D=Shopify&source%5Bcontroller%5D=admin%2Forders&source%5Bsection%5D=admin_api&source%5Buuid%5D=c4f09563-5425-4942-b6eb-6ac531f634a9
X-Dc → chi2,gcp-us-central1

This happens for multiple days  all these days are failures... 

11-21-2018 -- OK

 

11-22-2018 -- X-Request-Id -- 08db0e45-1cfc-4062-bf72-eadab2e52af7

11-23-2018 -- X-Request-Id -- b960fabb-5b2d-4134-97e5-5758b24cffa4

11-24-2018 -- X-Request-Id -- 7788d1a5-60ab-4403-a6cd-1c0cd5da609b

11-25-2018 -- X-Request-Id -- d1ec0d8f-8cd4-4fcc-a711-964e550957f6

11-26-2018 -- X-Request-Id -- b924ce33-d233-438b-b039-9d9d752743a5

11-27-2018 -- X-Request-Id -- 24dd5589-9f84-4dac-a0d2-2081509b932a

11-28-2018 -- X-Request-Id -- f59885f6-538e-4fb2-addc-4f6182a63e0b

11-29-2018 -- X-Request-Id -- 3eeaf5d2-9382-4cef-8bec-c5148940efc8

11-30-2018 -- OK

 

I REALLY need help to get this data loaded.  Help from anyone at Shopify please.  🙂

 

 

Zameer
Shopify Staff
Shopify Staff
297 31 88

Hey Joerz,

 

I've taken a look at the request id's which you've provided and I can see that these 500 errors are caused by exceptional cases when there are a lot of queries being made to the database across multiple API clients.

 

In these instances, the recommended approach is to implement some sort of time delay and then to retry your request. Increasing the time between each subsequent retry can also be a useful approach to limit the overall number of API calls made, as it's not possible to tell when the database associated with the requested shop will be back under normal operating load.

joerz
Shopify Partner
9 0 2

Hi @Zameer 

 

Wow that's unfortunate. I have already built in a bunch of retry / back off logic.  Could work that same logic for 500's - tough to distinguish real errors vs. this "false" error. 

 

Our current logic is fine for small Shopify stores, but on days with lots of volume, its not possible.  We have a client which is averaging 44,000+ orders per day.  At this point I have NO idea how to do a full historical data load from 2015..  As for daily incremental loads, this MIGHT work, but I felt I would have to use webhooks vs. polling APIs.

 

Would graphQL help in these scenarios?

Would finding a starting point (order number) and just query "since_id" from that point forward?  That helps w/ historical load (not daily "updated_at" incrementals)

 

Thank you so much for your guidance.

 

Heysenior
Shopify Partner
4 0 2

Hi Joerz,

 

I have the same error but extracting more than 50000 products, pagination from 100 in 100, Re: 500 {"errors": "Internal Server Error"}

 

We insert all the fields that the shopify api brings to our database, it's frustrating, when I remove the body_html it works correctly, apparently it is the large amount of data that each product brings, presumably body_html brings a large amount of data.

 

Some help?

 

Thank you,

JoeRzep
Tourist
7 0 1

Hi sagitario

 

There isn't a solid solution from Shopify; including webhooks and GraphQL.  Now, my issue was related to orders, not products, and was related to the number of orders within a date range. My scenario was trying to receive orders that were updated in a 24hr timeframe.  It is definitely related to the amount of data *and* how busy the servers are.

 

In my case I was date based, so the biggest thing I've done that helps is breaking the time frame into smaller timespans:  1 day turned into 24 hour blocks.  Only retrieve data for the 1 hour blocks.  Thus it was a smaller amount of data and worked without issue.  I even perform the blocks in parallel (up to 8 at a time).

 

Other things I've implemented:

* I use the `since_id` parameter to help pull in an optimized way (as noted in discussions -- but this really didn't help much)

* when retrieving a throttled status (http 429), I sleep and wait as oppose to bailing out

* changing page size didn't help much either - but *could*

 

I've found, you need to rework your algorithm to pull the data.

rickydazla
Shopify Expert
711 0 83

We're experiencing the same thing with Customer API calls... Trying to count Customers with a certain tag, borks after page 40 (which may or may not coincidentally be the 10,000th record?): `/admin/customers/search.json?query=tag:tag&fields=id,tags&limit=250&page=41`

 

@Zameer  any advice here?

I'm a million different people
milep
Tourist
6 0 2

We have a similar issue, but instead of 500 error we are getting a lot of 403 errors. It's not about permissions, because our integrations have been running a long time without issues and those requests succeed after retries(using Sidekiq).

Zameer
Shopify Staff
Shopify Staff
297 31 88

@rickydazla The issue is most likely being caused by the offset created by the query. I would recommend switching over to `since_id` pagination to help deal with the issue.

 

@milep Can you please provide me with a Request-Id for the 403 error you're seeing?

milep
Tourist
6 0 2
We are in process of adding response headers logging. Any other way you can help before we get that ready?
Zameer
Shopify Staff
Shopify Staff
297 31 88

What shop are you experiencing this on and what's the API key for the app?

milep
Tourist
6 0 2
Api key: 99adb7d80e116b3a9ae12010fd8e50d0
Error happens in every installed shop(at least in those that have any considerable amount of requests to api). I can provide more details in private messages.

- Miika