502 Response On Order API

[michaelm]

Hi, we're noticing 502 responses for the Order API. Strangely, an HTML paged with the title "Something went wrong" is returned when this happens. It seems to be only happening for one of our customers, and we've noticed that if we decrease the "limit" on the query we give (all the way down to 4), the query returns successfully.

An example request-id is 21514af3-9750-47c6-86b0-a474e8517fb2.

Is there any known reason this happens, and what is the best practice way to handle this?

[hassain]

HI @michaelm ,

Thank you for providing the example X-Request-ID value. It was extremely helpful in finding your request through our logs and determining what was going wrong.

From the logs, this is the GET Request you sent to our Shopify Admin REST API to get information about the orders for this particular shop: `GET /admin/orders.json?status=any&updated_at_max=2019-11-04T08%3A44%3A39%2B00%3A00&updated_at_min=1970-01-01T00%3A00%3A00%2B00%3A00&since_id=1&limit=5 HTTP/1.1`

The reason why this GET Request returns a 502 error is because the SQL Query created by this request takes too long to complete and eventually needs to time out. It seems the culprit here causing this is the "since_id=1" query string parameter you added to the GET Request. From my own tests, removing this query string parameter significantly improves the speed of the query and will constantly give your GET Request the successful 200 response with all the data you are looking for. So if this "since_id=1" query string parameter is not important to you, I would remove it to resolve this issue

[michaelm]

Thanks for the response @hassain. 

Unfortunately we need that since_id parameter because we need the Order records to be ordered by updated_at/ID. Out of curiosity, would this problem be solved if we made the same query using the Bulk API (since it may have a longer timeout), or the GraphQL API (I've seen that it's supposedly "faster")?

[hassain]

Hey @michaelm ,

Thanks for the additional context. As a follow up question, do you intend for the "since_id" query string parameter to be set as "1" instead of a real order_id for your store? It is intended and expected for a real order_id to be passed with this parameter, and if you did pass one in it would greatly improve the chances of this GET Request to work.

To answer your questions, the Bulk API is the recommended solution for querying and returning very large sets of data. This API is built on top of the GraphQL API. The GraphQL Admin API is generally considered faster than the REST Admin API, but thats because with GraphQL you can specify what data fields you want returned back to you (for example if you're querying orders you can specify that you only want the customer name and createdAt date for each order, and not get any data about line items or price). Since you can choose which data fields you want in return instead of just getting all data associated with an order, it is generally considered faster to use GraphQL than REST. 

[michaelm]

@hassain So when we initially start pulling data for a given time window (updated_at_min/updated_at_max), we use a since_id of "1" as the default value when we haven't pulled any orders yet. We need to put a value for since_id in order for it to order the results by ID. What would we put as this initial order ID for "since_id" when we don't know any order id's yet?

Makes sense that GraphQL would be faster because it fetches less data (and probably does less join's on your end). We'll explore that option.

[paul2019]

HI Hassain

We are also getting 502 randomly when call the orders api   following is the request url that I'm calling 

orders.json?limit=250&created_at_min=2019-09-14T14:50:53+00:00&status=any

after 1 or 2 pages I get 502 but not all the time its randomly happens I have a 1sec wait between calls 

Do you have any issue in shopify API ? 

What can I do in my side to prevent it 

Thanks

Ronny

[hassain]

Hey @michaelm ,

Sorry for the late response here. If you do not know which order ID to use for the "since_id", the expected value to use is "0" not "1". Hope that helps!

@paul2019 

You can always check the Shopify status page to see if our API has any issues:https://status.shopify.com/

However some suggestions that could help improve the speed of your requests and prevent timeouts:

1) Use relative cursor based pagination: https://www.shopify.com/partners/blog/relative-pagination 

2) Using GraphQL instead of REST, in order to pull less data and get faster results

3) Use the new Shopify GraphQL Admin API Bulk Operations functionality to pull large amount of data in one go, without pagination: https://help.shopify.com/en/api/guides/bulk-operations