Solved

Not receiving webhook notifications after "Your webhook is failing" emails

phuh
Shopify Partner
7 1 1

I had an issue with my webhook callback endpoint therefore I kept receiving "Your webhook is failing" emails.

I fixed the endpoint today and created new webhook subscriptions but Shopify does not seem to send me notifications anymore. I also tried uninstalling/reinstalling app.

 

Is it possible that the system automatically disabled webhook notifications due to earlier failures? Could someone  from Shopify team please have a look into my app (ID: 3747789) and re-enable it if the system blocked it? Thank you-

Accepted Solution (1)
phuh
Shopify Partner
7 1 1

This is an accepted solution.

@SBD_ Ah! I've just tried copying request headers & body of "carts/create" notification received at Hookbin and manually sending it to the endpoint. I got some clue from an error code specific to Cloudflare. The issue could be on their end not the Shopify servers after all. Thanks for looking into this, I will investigate this issue with Cloudflare and update here on the progress. Thank you!

View solution in original post

Replies 11 (11)

SBD_
Shopify Staff
1828 269 398

Is it possible that the system automatically disabled webhook notifications due to earlier failures?

Webhooks will be removed if they consistently fail over 48 hours. https://shopify.dev/tutorials/manage-webhooks#frequency

 

But they can be recreated - are they listed if you GET webhooks?

Scott | Developer Advocate @ Shopify 

phuh
Shopify Partner
7 1 1

But they can be recreated - are they listed if you GET webhooks?

Yes, there are active subscriptions listed in webhook. Here are sample webhook subscription IDs. Please help-

 

[841475293243] carts/create
[841475326011] collection_listings/remove
[841475358779] checkouts/create
[841475391547] app/uninstalled
[841475424315] checkouts/update

phuh
Shopify Partner
7 1 1

I used Hookbin (https://hookbin.com/) to test a different endpoint and interestingly this endpoint outside my domain worked. (Subscription ID: 841792356411)

This was tested on same app, same shop, same access token.

 

Shopify's webhook notification server probably blacklisted my domain name entirely after many failed attempts. It is fixed on my end but I need your team's help whitelisting it on your end. Thank you-

phuh
Shopify Partner
7 1 1

I've just noticed this note on https://shopify.dev/tutorials/manage-webhooks#frequency and it seems related to my issue since I'm serving this with Cloudflare Workers (https://workers.cloudflare.com/😞

 

If you are using Cloudflare on your domain, then you should disable it for the registered webhook URL.

I also received "Your webhook is failing" email again, so I confirmed the server did try to send out a webhook notification.

Is it the case that I am not able to use Cloudflare Workers as callback endpoint? I'm using Full SSL certificate on this domain and it is trusted by all major web browsers according to SSL checker. Is there any reason why Shopify blocks only Cloudflare?

SBD_
Shopify Staff
1828 269 398

Hey @phuh 

 

Cloudflare Workers should be fine.

 

Our logs show the webhooks are firing, but often receive a 500 response from the server. Are you able to set up additional logging on your end?

Scott | Developer Advocate @ Shopify 

phuh
Shopify Partner
7 1 1

Thanks @SBD_ for checking for me!

 

I've been logging every request hitting the Cloudflare Workers server, and I've been closely monitoring the logs.

All I've received recently was one request hitting at "/webhook/carts/update" sent from postman (AU IP) about 20 minutes ago, and it returned status 200.

 

I am pretty sure there is something weird going on at the Shopify servers. I think you're right I must have returned 500 about a week ago when I tried it, but it is no longer sending me anything since I started logging all requests.

 

I am curious why the doc says Cloudflare (reverse proxy) needs to be disabled in order to receive webhook. Assuming it is certificate related, I purchased a new dedicated SSL certificate at Cloudflare that shows my domain name as common name. However I am not sure how long your servers cache the old universal certificate (SNI-based) that I was using at the time webhook failed about a week ago.

SBD_
Shopify Staff
1828 269 398

That was me testing the endpoint from Postman.

 

I just added something to cart on your demo store which triggered carts/create. A request was sent to https://<your subdomain>/webhook/carts/create at 2020-05-19 02:11:11.083 and returned a 500 - do you see a log of this?

Scott | Developer Advocate @ Shopify 

phuh
Shopify Partner
7 1 1

@SBD_ Unfortunately no, I don't see any request logged since your postman test.

phuh
Shopify Partner
7 1 1

This is an accepted solution.

@SBD_ Ah! I've just tried copying request headers & body of "carts/create" notification received at Hookbin and manually sending it to the endpoint. I got some clue from an error code specific to Cloudflare. The issue could be on their end not the Shopify servers after all. Thanks for looking into this, I will investigate this issue with Cloudflare and update here on the progress. Thank you!

shingekinokyoji
Excursionist
20 1 2

How do you send  send a 200 OK response to Shopify using PHP? It says you should do that here https://shopify.dev/apps/webhooks/configuration/https I'm able to receive information via Shopify but in the same file I receive the information I try to respond to Shopify but the webhook keeps retrying/sending the same set of information.

yanteb
Shopify Partner
34 0 4

how did you fix the endpoint? im getting those emails now