Webhook crashes app

[OlchowskiV]

Hello. I've managed to build an app for shopify and now I'm making some testing. During the installation of the app, I'm creating an webhook, to get informed, when the customer uninstalls my app.

I'm running a dart script on an arch linux webserver. The script runs on port 4040 and ports 443 and 80 are redirecting to port 7070. Installation works fine, but when I uninstall the app, the webhook crashes my app. I'm receiving only this message:

SocketException: OS Error: Connection reset by peer, errno = 104, address = 0.0.0.0, port = 4040
#0      main (file:///home/bin/main.dart:47:3)
<asynchronous suspension>
#1      _startIsolate.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:303:32)
#2      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:172:12)

But, when I'm using ngrok, and forwarding the traffic trough this service to https://localhost:4040 everything works just fine.

Does someone has a hint, what leads to this behavior?

[adam_modd]

Could be that it's not handling the payload properly. I remember that the normal webhooks are missing a few headers you'd normally expect in a message, like Content-Length, and rely on simply closing the connection to denote message end. This is legal in the HTTP spec, but is sometimes not properly handled. Maybe that's it? Maybe ngrok is transparently adding the Content-Length header?

[OlchowskiV]

Thanks for your reply. I've done a test, by using webhook.site as address and this are the headers, which come with the request. Looks like there is nothing wrong with them.

x-forwarded-for 	35.239.136.64
host 	webhook.site
connection 	close
content-length 	1590
user-agent 	Ruby
accept 	*/*
accept-encoding 	gzip;q=1.0,deflate;q=0.6,identity;q=0.3
x-shopify-api-version 	2019-10
x-shopify-hmac-sha256 	xxxxxxxxxxxxxxxxxxxxxxxxxxxx
x-shopify-shop-domain 	xxxxx.myshopify.com
x-shopify-topic 	app/uninstalled
content-type 	application/json 

[adam_modd]

Huh. Are you sure that's correct? Usually the User-Agent is "Shopify-Captain-Hook", not "Ruby". Did you get the webhook in the exact same way that your app got it?

[OlchowskiV]

It looks like both user agents are possible.

[adam_modd]

Hrm. Don't know then. If both are possible; could it be possible that it's sometimes missing the Content-Length header? Maybe try submitting a faux-webhook yourself, without a Content-Length, and see if you get the same exception. Shopify's not exactly known for being super consistent on exactly how it sends things over.

[OlchowskiV]

I've just read this in the documentation for dart HttpServer "Incomplete requests, in which all or part of the header is missing, are ignored, and no exceptions or HttpRequest objects are generated for them."

So I think, that the headers are not the source of this.

[adam_modd]

It's not that the headers are missing, it's still a valid request. It's just that whatever you're piping it into may expect a Content-Length, and is treating the "connection reset by peer" as an exception, instead of the valid termination of the HTTP request parsing process. I only say this because I ran into this exact issue on a piece of software that was doing a bit of manual request parsing, and treated hangups as not the end of the request if it hadn't received a content-length.

EDIT: Actually; looks like connection reset by peer isn't just the connection closing; it's your end trying to send a packet after the connection has been closed. Maybe dart isn't properly seeing the connection closed header, and is still trying to communicate a response on the same socket, after an initial response was already sent? Honestly not sure at this point.

[adam_modd]

Actually; is it possible that you're taking a while to generate a response? For performance reasons, Shopify likely closes the connection pretty quick, so sending a response to an already closed connection would generate that error. Using a proxy may also get around this for you, because it'll likely wait a lot longer than the web hook connection. When it proxies your response out, it likely just ignores the error. Could that be it?

[OlchowskiV]

I don't think, that it is an issue with the response time. My server already crashes at this line

await for (HttpRequest request in server) {

which is at the beginning of the script. After this line and before the response there are some log printing, and they are not printed to console, when the app crashes.

For testing purpose, I've commented out the response and the app has crashed just like before.