Webhook orders

Hi there,

 

When we subscribe multiple webhook topics, shopify sends the webhooks altogether inside a 10 sec time-frame based on the documentation here

 

Is there any information such as event timestamp inside webhook request so that we can check the order of events? Or is it guaranteed that an update event will come after a create event and a delete event will come after create/update event on a topic?

Co-Founder / Developer at: merchbees
Merchbees Low Stock Alert - Keep track of your low stock items by email and slack
Merchbees Inventory Value - Know your inventory value and quantity in real-time
Push Down & Hide Out of Stock - Move out of stock products to the bottom of the collection to improve SEO & hide/ unhide automatically
0 Likes
SBD_
Shopify Staff
Shopify Staff
1043 141 183

Hey @Mete,

 

Unfortunately there's no guarantee or timestamp - can you describe your use case?

 

When we subscribe multiple webhook topics, shopify sends the webhooks altogether inside a 10 sec time-frame based on the documentation here

Shopify won't send all notifications at once in a 10 second window, it will de-duplicate same notifications. E.g. If a product is updated 5 times in 10 seconds, only one update will fire, with the most up to date information. 

0 Likes

@SBD_ wrote:

Hey @Mete,

 

Unfortunately there's no guarantee or timestamp - can you describe your use case?

 

When we subscribe multiple webhook topics, shopify sends the webhooks altogether inside a 10 sec time-frame based on the documentation here

Shopify won't send all notifications at once in a 10 second window, it will de-duplicate same notifications. E.g. If a product is updated 5 times in 10 seconds, only one update will fire, with the most up to date information. 


Hi @SBD_,

 

Actually there could be so many use cases because it is much more related to application stability. De-duplicate only works for a specific webhook topic. But for certain actions we get different webhooks belongs to different topics at the same time.

 

For example: If you deactivate a location, you will get both inventory_levels/disconnect and inventory_levels/update webhooks at the same time. Normally we expect an update first because the stock at the location must be transferred to some other location which makes the inventory_level at this location 0 and then it should disconnect. But sometimes disconnect comes before update which causes issues on our end. See the example below:

inventory_levels/disconnect
{"inventory_item_id":34012039446660,"location_id":41167421572}


inventory_levels/update {"inventory_item_id":34012039446660,"location_id":41167421572,"available":0,"updated_at":"2020-02-28T13:46:45-05:00","admin_graphql_api_id":"gid://shopify/InventoryLevel/74966270084?inventory_item_id=34012039446660"}

Disconnect and delete webhooks doesn't come with a time information like updated_at so there is no way for us to understand which one comes first. We delete the inventory_levels then we get an update webhook for a deleted item. In some other cases, we see an update webhook comes before create webhook. How can we sync our db using webhooks in such cases?

 

If there would be an event timestamp along with webhook message, we could find a method to identify the order. Slack uses this approach as you can see here.

Co-Founder / Developer at: merchbees
Merchbees Low Stock Alert - Keep track of your low stock items by email and slack
Merchbees Inventory Value - Know your inventory value and quantity in real-time
Push Down & Hide Out of Stock - Move out of stock products to the bottom of the collection to improve SEO & hide/ unhide automatically
0 Likes
SBD_
Shopify Staff
Shopify Staff
1043 141 183

Hey @Mete 

 

But sometimes disconnect comes before update

This can happen and the application should be designed to handle these cases.

 

If there would be an event timestamp along with webhook message, we could find a method to identify the order.

Introducing a timestamp creates another layer of complexity since you'd need to create your own queue and there's no way to know if more webhooks are on the way. 

 

We delete the inventory_levels then we get an update webhook for a deleted item... ...we see an update webhook comes before create webhook

There are a few ways to handle these scenarios. Here's one approach:

 

  • To account for out-of-sync updates - keep a local copy of updated_at and check the value before processing an update. E.g. "If (webhook updated_at) > (local updated_at)..."
  • To account for update after delete - wrap updates in "if item exists (i.e. hasn't been deleted)..."
  • To account for update before create - wrap updates in "if item doesn't exist, check the API to see if it's already been created..."

 

0 Likes

Hi @SBD_,

 

Thanks for the quick reply.

 

We delete the inventory_levels then we get an update webhook for a deleted item... ...we see an update webhook comes before create webhook

There are a few ways to handle these scenarios. Here's one approach:

 

  • To account for out-of-sync updates - keep a local copy of updated_at and check the value before processing an update. E.g. "If (webhook updated_at) > (local updated_at)..."
  • To account for update after delete - wrap updates in "if item exists (i.e. hasn't been deleted)..."
  • To account for update before create - wrap updates in "if item doesn't exist, check the API to see if it's already been created..."

The last item here creates a dependency to the API. We don't want to waste our API tokens that's why we use webhooks. Could you suggest an approach that is not using API at all?

Additionally, how can we handle create after delete for the same resource? Is this something that could happen?

 

If there would be an event timestamp along with webhook message, we could find a method to identify the order.

Introducing a timestamp creates another layer of complexity since you'd need to create your own queue and there's no way to know if more webhooks are on the way. 

We don't need a queue implementation here. Assume shopify sends "deleted_at" field along with delete/disconnect webhooks. When we get delete webhook, we can update deleted_at field on our local database from null to a real value instead of removing the resource from db. If we need to check items that are not deleted, we can search the ones which deleted_at fields are null.

 

To handle update/create after delete => Compare resource's deleted_at field with webhook's created_at or updated_at fields.
To handle update before create => Upsert (Insert if not exists) the resource. Create will be ignored after time comparison.

 

A cron job can purge the deleted items periodically if necessary.

 

Using this approach, we don't need to check API to validate webhooks. Also this will allow us to make our db operations atomic (of course, it depends on the database). We just need a timestamp (with fractional seconds if possible) for each event sent via webhooks.

Co-Founder / Developer at: merchbees
Merchbees Low Stock Alert - Keep track of your low stock items by email and slack
Merchbees Inventory Value - Know your inventory value and quantity in real-time
Push Down & Hide Out of Stock - Move out of stock products to the bottom of the collection to improve SEO & hide/ unhide automatically
0 Likes

@Mete wrote:

Hi @SBD_,

 

Thanks for the quick reply.

 

We delete the inventory_levels then we get an update webhook for a deleted item... ...we see an update webhook comes before create webhook

There are a few ways to handle these scenarios. Here's one approach:

 

  • To account for out-of-sync updates - keep a local copy of updated_at and check the value before processing an update. E.g. "If (webhook updated_at) > (local updated_at)..."
  • To account for update after delete - wrap updates in "if item exists (i.e. hasn't been deleted)..."
  • To account for update before create - wrap updates in "if item doesn't exist, check the API to see if it's already been created..."

The last item here creates a dependency to the API. We don't want to waste our API tokens that's why we use webhooks. Could you suggest an approach that is not using API at all?

Additionally, how can we handle create after delete for the same resource? Is this something that could happen?

 

If there would be an event timestamp along with webhook message, we could find a method to identify the order.

Introducing a timestamp creates another layer of complexity since you'd need to create your own queue and there's no way to know if more webhooks are on the way. 

We don't need a queue implementation here. Assume shopify sends "deleted_at" field along with delete/disconnect webhooks. When we get delete webhook, we can update deleted_at field on our local database from null to a real value instead of removing the resource from db. If we need to check items that are not deleted, we can search the ones which deleted_at fields are null.

 

To handle update/create after delete => Compare resource's deleted_at field with webhook's created_at or updated_at fields.
To handle update before create => Upsert (Insert if not exists) the resource. Create will be ignored after time comparison.

 

A cron job can purge the deleted items periodically if necessary.

 

Using this approach, we don't need to check API to validate webhooks. Also this will allow us to make our db operations atomic (of course, it depends on the database). We just need a timestamp (with fractional seconds if possible) for each event sent via webhooks.


Hey,

Is there anyone from Shopify team to help on this issue. When I check the webhook logs, I see 50 minutes delayed create webhooks causing sync issues because it arrives after delete.

Thanks.

 

Co-Founder / Developer at: merchbees
Merchbees Low Stock Alert - Keep track of your low stock items by email and slack
Merchbees Inventory Value - Know your inventory value and quantity in real-time
Push Down & Hide Out of Stock - Move out of stock products to the bottom of the collection to improve SEO & hide/ unhide automatically
0 Likes
marcusradica
Shopify Staff
Shopify Staff
33 3 4

Hey there, could you please DM me so I can look into the logs for your delayed create webhook?

marcusradica | Developer @ Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit Shopify.dev or the Shopify Web Design and Development Blog

0 Likes