API: 403 'Forbidden' response after a call to a Shopify 'canary' server

Retired Boards Appdev Fulfillment Api

Topic summary

Intermittent 403 Forbidden errors occur when fulfilling orders via the Shopify Fulfillment API; each order typically fails 4–5 times before a later attempt succeeds.

Successful flow (header: X-Shopify-Stage: production):

  • GET /admin/api/2023-04/locations.json
  • GET /admin/api/2023-04/orders/{order_id}/fulfillment_orders.json
  • POST /admin/api/2023-04/fulfillments.json

Failing flow (header: X-Shopify-Stage: canary):

  • GET /admin/api/2023-04/locations.json succeeds
  • Subsequent GET /admin/api/2023-04/orders/{order_id}/fulfillment_orders.json returns 403 Forbidden

Observations:

  • The issue appears correlated with requests routed to “canary” (a rollout/test stage used during deployments) rather than “production.”
  • Screenshots are provided showing the error, supporting the reported behavior.

Latest update:

  • Shopify support requested X-Request-ID values for recent failing requests.
  • Provided IDs: Successful (production) X-Request-ID: 063b31c7-8e00-4ebb-81d3-8178f19fab1e; Unsuccessful (canary) X-Request-ID: 12a8c001-4979-4a38-a944-df20d31ac0eb.

Status:

  • No resolution yet; awaiting further investigation from Shopify based on the supplied request IDs. Key question remains why canary-stage routing yields 403 on the fulfillment_orders endpoint.
Summarized with AI on January 21. AI used: gpt-5.
1

We have an in-house app that uses the Shopify API.
We use it to fulfil orders.

Lately, when we try to fulfil an order, we get an error response:
403 Forbidden

forbidden error.jpg
forbidden error.jpg344×122 4.55 KB

For every order, we get this response 4-5 times, and then it works!

Successful fulfilment

During a successful fulfilment, there are these calls:

  • GET /admin/api/2023-04/locations.json
  • GET /admin/api/2023-04/orders/xxxxxxxxxxxxx/fulfillment_orders.json
  • POST /admin/api/2023-04/fulfillments.json

A successful fulfilment also contains the header response:

X-Shopify-Stage: production

Unsuccessful fulfilment

An unsuccessful fulfilment only has the first call:

  • GET /admin/api/2023-04/locations.json

This call is successful, but during the next one, get the above ‘403 Forbidden’ error message

However the header response is different:

X-Shopify-Stage: canary

Observations

We can infer that the problem may be when Shopify returns a response from

X-Shopify-Stage: canary

instead of

X-Shopify-Stage: production

then the subsequent call fails.

We can also infer that the problem occurs in the call to

GET /admin/api/2023-04/orders/xxxxxxxxxxxxx/fulfillment_orders.json

API Error 1.jpg
API Error 1.jpg1896×898 361 KB

2

Hi there :waving_hand:

Would you have the X-request-id for any of these failing requests that have happened in the last 24 hours?

3

In the above ‘successful’ fulfilment with a call to

  • GET /admin/api/2023-04/locations.json

with a response from

X-Shopify-Stage: production

..had this X-Request-ID:
X-Request-ID: 063b31c7-8e00-4ebb-81d3-8178f19fab1e

In the above ‘unsuccessful’ fulfilment with a call to

  • GET /admin/api/2023-04/locations.json

with a response from

X-Shopify-Stage: canary

..had this X-Request-ID:

X-Request-ID: 12a8c001-4979-4a38-a944-df20d31ac0eb