FROM CACHE - en_header

Upcoming improvements to Webhooks

Ryan
Shopify Staff
Shopify Staff
499 42 120

Update Jan 12th below:

The de-bounce change is now live for all shops.  Please keep leaving your questions and concerns around this new improvement in this thread.  Feel free to share some stats as well.

As for other webhook related issues; feel free to keep bringing them up as well, just note that these improvements are solely based on eliminating duplicates and do not alter the contents of any existing webhooks.

 

Update Jan 4th below:

We are going to be starting a tiered rollout early next week for the webhook improvements.

The changes will be what is described in the original post, with the exception that we will exclude any webhook topics that are /add or /create.

As always we look forward to your feedback before and after this feature has rolled out.  Feel free to reply with any questions or concerns.

 

Hey All,

We are introducing a number of improvements to how Shopify delivers webhooks.  Our existing webhooks implementation offers a poor experience for app developers on a number of dimensions:

  1. Flooding.

    • A spike of requests on Shopify's side causes a spike of webhooks to be sent to apps (e.g., a flash sale). Apps have to design against this.

  2. Duplication.

    • Our webhook implementation often sends duplicate webhook payloads in a short period of time. For app developers, this translates to increased hardware requirements for processing as well as additional contention for the affected resources.

We are introducing changes to improve the above issues that will have a twofold impact; a net reduction in webhooks received from Shopify, as well as short delay between an action and the webhook being sent in order to ensure that only the most recent state is communicated to your app.

For example:  The following actions are performed in quick succession;

  1. A product is created

  2. A title is added to the product

  3. A description is added to the product

  4. A price is added to the product

Previously the behaviour would result in a products/create webhook, and then three subsequent products/update webhooks, not necessarily in the order of changes made.

After introducing these improvements, a products/create webhook would be sent, and then a single products/update webhook with the most up-to-date payload would be sent after a short delay.

If you have questions or concerns about these changes, please comment in the thread below so we can address them prior to launch.

 

All the best,

Ryan O

Developer Experience

Shopify

Ryan | Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

Replies 45 (45)
Felix2
Shopify Partner
98 0 17

Great improvements.

I experienced that for a single shop attribute change, the shop/update webhook gets fired four times (details in this post). Is this going to be improved, so that it gets fired only once, too?

Looga.io
Martin7
New Member
4 0 0

Awesome! We were looking for updates about address changes. I thought that customer.updated would be that -- but that was firing on every new order. 

Thanks!

Ryan
Shopify Staff
Shopify Staff
499 42 120

Hi Felix,

The issue you describe is indeed one of the scenarios this change is aiming to fix.  Thanks for your feedback!

Ryan

Ryan | Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

HSL
Shopify Expert
37 0 3

Hi Ryan,

That's a great update!

Do all webhook topics get the delays or only the ones that can have an issue like the one you described. I'm most interested in the the orders/create webhook, as a delay on that one would have a very negative impact on loads of apps. 🙂

And do you have more info on the timeline of the updates?

Cheers,

Harold

ClementG
Shopify Partner
660 0 142

Awesome news!

On a very related note, I'm still trying to figure out, Does the [orders/updated] webhook fire whenever any other [orders/*] webhook fires? No point subscribing to all other topics if that is the case...

Paul37
New Member
6 0 0

Hi there,
Wonderful!
From this morning we are heading some issues with APP deletion webhook. With some test account works correctly with others not.
It started more or less today, do you think is a conseguence of your webhooks restructuring process?

Ryan
Shopify Staff
Shopify Staff
499 42 120

Thanks for all the feedback, please keep leaving questions and comments.  We are reading and discussing all of the feedback we've gotten.

 

Ryan

Ryan | Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

Ryan
Shopify Staff
Shopify Staff
499 42 120

Hey all,

We are going to be starting a tiered rollout early next week for the webhook improvements.

The changes will be what is described in the original post, with the exception that we will exclude any webhook topics that are /add or /create.

As always we look forward to your feedback before and after this feature has rolled out.  Feel free to reply with any questions or concerns.

 

Ryan

 

Ryan | Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

atradef
New Member
1 0 0

Do you have information on how long the delay is after the events occur? 

Searchanise_Tea
Tourist
8 0 2

Previously, we’ve got the issue that products/update webhook didn’t come to us during the product purchase, and we didn’t have any ability to detect the product count changes in the store.

Are you going to fix this issue during your future webhook changes? Will the products/update webhook always come to us if something changed with the product data?

ClementG
Shopify Partner
660 0 142

And there is also this bug where the product webhook gets fired but doesn't have the latest data.

https://ecommerce.shopify.com/c/shopify-apis-and-technology/t/possible-bug-in-webhook-product-update...

Eric77
Tourist
15 0 1

Will /create webhooks eventually be de-duped? Those really are only ones we are about being de-duped.

Ryan
Shopify Staff
Shopify Staff
499 42 120

Hi Eric,

Will /create webhooks eventually be de-duped? Those really are only ones we are about being de-duped.

I think you missed a key word in this sentence! However, there is no plan to include changes to /add or /create webhooks in the future.

 

Ryan | Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

Eric77
Tourist
15 0 1

Why? Those are really the only ones that we care about de-duping.

Ryan
Shopify Staff
Shopify Staff
499 42 120

Why? Those are really the only ones that we care about de-duping.

Hi Eric,

Can you elaborate on this? Are you seeing duplicates for /create webhooks? 

 

Ryan

Ryan | Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

Eric77
Tourist
15 0 1

Yes. Dupe /create webhooks are a huge pain. Mainly for orders and checkouts.

HSL
Shopify Expert
37 0 3

I had a lot of dupes too, made a workaround that worked perfectly fine, but in the end I found a little bug in the queuing system that I use for webhooks.

Since I fixed that a couple of months ago I received 0% dupes on the orders/create webhook, so I don’t think there is a dupes issue on the API side. 🙂

ClementG
Shopify Partner
660 0 142

In the case where we do receive multiple webhooks for the same object, it would be useful to have an incrementing id in the headers so that we know which one is the latest.

It would come in handy in the case when the app is temporary down and Shopify may requeue many webhooks that failed but it is not obvious to the app which one represents the latest payload. Or is the best practice to consume the request with the highest update_at and discard the others?

Ryan
Shopify Staff
Shopify Staff
499 42 120

In the case where we do receive multiple webhooks for the same object, it would be useful to have an incrementing id in the headers so that we know which one is the latest.

It would come in handy in the case when the app is temporary down and Shopify may requeue many webhooks that failed but it is not obvious to the app which one represents the latest payload. Or is the best practice to consume the request with the highest update_at and discard the others?

Hi Clement,

The intended behaviour would be that for any webhook subject to the de-dupe, /update for example, would only have one retry per resource.  So when your app starts responding again after being down, there would be a single retry queued per resource updated while offline.

Cheers,

 

Ryan

Ryan | Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog