I am trying to implement HunkyBill’s webhook queue specified in this thread:
https://ecommerce.shopify.com/c/shopify-apis-and-technology/t/webhook-url-is-called-between-2-4-times-220007
I however ran into an unexpexted oddity- the webhook ID – suppsedly given in the header response with the key X-Request-Id – is simply not there. There seems to be a bit of confusion hinted at its existence near the end of that thread, but the end of this thread made my heart sink:
https://ecommerce.shopify.com/c/shopify-apis-and-technology/t/product-update-webhook-repeat-excessively-till-getting-deleted-206129
So, what’s the deal? The app I’m creating for my client depends on this.
Here are all the header values I’m getting back:
X-Shopify-Topic: products/update
X-Shopify-Shop-Domain: (omitted)
X-Shopify-Product-Id: 4060486342
X-Shopify-Hmac-Sha256: 5xL1YBVUGzhFMGPLKTvOeXHDGJs/gD/t78tDIyx/ClA=
Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
Accept: */*
User-Agent: Ruby
X-Newrelic-Id: VQQUUFNS
X-Newrelic-Transaction: PxQFVlRWD1YDBQNUB1UHAAIIFB8EBw8RVU4aUA8KAQVWVlgDB1FXUgJQDkNKQQsHBVdZUQAHFTs=
Connection: close
Host: (omitted)
Content-Length: 23900
X-Forwarded-Proto: https
X-Forwarded-For: (omitted)
The X-Newrelic-Id seems to be different every time, even if it’s for the same webhook, so that’s no good. I was thinking as a worst case scenario I could use the SHA256 hash of the body, but that could conceivably be the same for two distinct webhooks. Or am I wrong? Can they be guaranteed to be different?
Someone from Shopify please respond, I really don’t want to end up like the skeleton of Jason T.
The payload of the webhook itself is your saviour. If you POP the payload off a stack or queue, and your process doing that is single-threaded, you can certainly decide if the payload is unique or a dupe. Another approach is to let your persistance layer deal with dupes. For instance, create a unique index in your DB so that if any process tries to create something that already exists, it ignores it.
Well, so much for doing things the right way. I just discovered that the data that Shopify sends in the payload does not even reflect the new changes for the “products/update” webhook. I’ve resorted to re-querying shopify every time I get a webhook and checking for changes on the product manually. Sigh.
HunkyBill, are you getting the X-Request-Id header in your webhook payloads? Do you know when/why they stopped doing this? Is it only on my account?
I don’t think there is such a thing. Your best bet is to use the payload itself as a guide on what to do. It does suck though. For example if you get two product update webhooks to process and they represent the same change.
I know I get tons of webhooks from Shopify where the delivery time between them, is .2 second, or even .02 seconds… etc and they represent the same resource change. That truly bites.