Application Proxy - There was an error in the third-party application.

Thomas_Kwan
Shopify Partner
24 0 0

Hi there

 

Our app suddenly started getting this error since Friday (21 Jun, 2019). And the error can be seen from here - https://etechfocus-demo.myshopify.com/apps/simple-testimonials

 

That URL will hit our proxy URL ( https://myshopifyapps.com/simple-testimonials/proxy?shop=etechfocus-demo.myshopify.com) which is returning valid liquid codes. So we are not sure why our clients and our test app are getting this error.

Thanks for your help in advance.

Replies 33 (33)
Thomas_Kwan
Shopify Partner
24 0 0

Actually, we notice that our proxy was not even called. We setup our proxy as

 

Subpath prefix: apps

Subpath: simple-testimonials

Proxy URL: https://myshopifyapps.com/simple-testimonials/proxy

 

When we accessed our test store's https://etechfocus-demo.myshopify.com/apps/simple-testimonials, our proxy was not called. 

Saw the error code 500 from cloudflare

 

* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
< HTTP/2 500
< date: Mon, 24 Jun 2019 14:50:10 GMT
< content-type: text/html; charset=utf-8
< set-cookie: __cfduid=df665a7c97b070af8aed7c0543ae7f76f1561387809; expires=Tue, 23-Jun-20 14:50:09 GMT; path=/; domain=.myshopify.com; HttpOnly
< set-cookie: _shopify_y=62873d60-a98e-4222-9d6a-0f59242c3fa1; path=/; expires=Thu, 24 Jun 2021 02:28:34 -0000
< set-cookie: _orig_referrer=; Expires=Mon, 08-Jul-19 14:50:10 GMT; Path=/; HttpOnly
< set-cookie: _landing_page=%2Fapps%2Fsimple-testimonials; Expires=Mon, 08-Jul-19 14:50:10 GMT; Path=/; HttpOnly
< set-cookie: secure_customer_sig=; path=/; expires=Fri, 24 Jun 2039 14:50:10 -0000; secure; HttpOnly
< set-cookie: cart_sig=; path=/; expires=Mon, 08 Jul 2019 14:50:10 -0000; HttpOnly
< content-security-policy: block-all-mixed-content; frame-ancestors *; upgrade-insecure-requests; report-uri /csp-report?source%5Baction%5D=app_liquid&source%5Bapp%5D=Shopify&source%5Bcontroller%5D=storefront_section%2Fapp_proxy&source%5Bsection%5D=storefront&source%5Buuid%5D=7d0fe70a-cdd0-4cd2-8d06-6afae0b17d26
< vary: Accept-Encoding
< x-shopid: 21465161
< x-shopify-stage: production
< x-sorting-hat-podid: 52
< strict-transport-security: max-age=7889238
< x-dc: gcp-us-east1,gcp-us-east1,gcp-us-east1
< x-sorting-hat-shopid: 21465161
< x-shardid: 52
< x-content-type-options: nosniff
< x-request-id: 7d0fe70a-cdd0-4cd2-8d06-6afae0b17d26
< x-download-options: noopen
< x-xss-protection: 1; mode=block; report=/xss-report?source%5Baction%5D=app_liquid&source%5Bapp%5D=Shopify&source%5Bcontroller%5D=storefront_section%2Fapp_proxy&source%5Bsection%5D=storefront&source%5Buuid%5D=7d0fe70a-cdd0-4cd2-8d06-6afae0b17d26
< x-permitted-cross-domain-policies: none
< content-language: en
< expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< server: cloudflare
< cf-ray: 4ebf8333caa6a366-HKG

jem_punsalan
New Member
5 0 0

We are also getting this issue, when we tried to use proxy tunnel (ngrok) pointing to our local app, it is working fine. But when using a domain for the app proxy it is the same as what you are getting. Anything else we need to look into? Thanks

Thomas_Kwan
Shopify Partner
24 0 0

Hi jem_punsalan

 

Are you just recently getting this error? I started getting this error since around last Friday 06/21/2019.

 

Thomas

jem_punsalan
New Member
5 0 0

Hi @Thomas_Kwan we only get this issue yesterday, am not sure if theres new implementations related to App Proxy for Shopify.

Thomas_Kwan
Shopify Partner
24 0 0

Hi jem_punsalan

 

We just tried ngrok, and yeah, it worked for us. This will be a good short-term work around for us. Almost feel like there is cloudflare is calling some IPs (ngrok's ) but not all (our server for example).

So with ngrok.io domain in the proxy URL works, but not with our domain. Using ngrok introduces an extra layer of networking routing which is not desirable.

 

thomas

jem_punsalan
New Member
5 0 0

Hi @Thomas_Kwan we only use ngrok for local testing our app. As for the domain app proxy issue, still not clear of the root cause. We are still trying to find the solution for this. Let us know if you also find something related to the app proxy issue.

pixeledluke
Tourist
8 0 1

Hi Thomas,

 

We have been experiencing exactly the same issue at: https://unionsubscriptions.myshopify.com/apps/fulfilment which is supposed to be proxying a subdomain, but doesn't seem to be getting hit at our side. Getting the same 500 error you have reported.

 

If we visit the proxying url directly, it works, just as you are seeing. We first started noticing this on the 20/06/2019.

 

I've seen your thread with CloudFlare, and seen they mentioned that Shopify have just started working with them. Have you have any update from them, or have you got any further with the issue your side?

 

Cheers,

Luke.

Thomas_Kwan
Shopify Partner
24 0 0

Hi Luke

 

I did not get any further update from Shopify yet. Currently as a short-term fix, I have setup a ngrok channel and I have changed the proxy url to point to ngrok instead of my server directly.That seems to be working fine.

 

We have 2 apps and I have left one app pointing to our server so that I can tell if the problem has been resolved or not.


https://etechfocus-demo.myshopify.com/apps/simple-testimonials (With short-term fix, running thru ngrok)

https://etechfocus-demo.myshopify.com/apps/proxy (Pointing to our server, still broken)

 

I will update this thread when i have additional info.

 

thanks

thomas

Thomas_Kwan
Shopify Partner
24 0 0

Here is timeline with the Shopify Support so far

 

Jun 24, 03:55 (13350334)

- Reported the error and I was told the problem will be reported to the Partners team

"regarding to that I can ask the Partners team, I have your email and i will give it to the said team so that they can contact you"

 

Jun 25, 04:41 (13363587)

- Enquiring status and I was told that the ticket is in the system but the ticket was NOT assigned
"The partner specialists have the ticket in the system so you have done everything right, this is a technical query so a Guru like me would be of no help to you, but they will reach out as soon as the ticket arrives on their desk"
" Then I can tell you that at this time there is no partner manager assigned to your account"

 

Jun 26 17:35 (13383873)

- Asking about status and got help from support to raise the priority up
"I will set the ticket as more urgent priority so they review it sooner but typically it does take our escalated teams 24-72hrs to review tickets"
"And it was escalated on Monday, I will also include your 2nd support ticket with it so they can review both at once"

 

Jun 27 16:55 (13395784)
- Asking for status

"just reaching out to our partner team to what's going on with the ticket since I can see it was opened on Monday"
"It appears that we can escalate this to our technical team as well so I will do that, I'm just going to ask you a few more questions so that I can get all of the information the first time and save some time"

jem_punsalan
New Member
5 0 0

Thanks @Thomas_Kwan let us know for any updates. For now we use ngrok to tunnel our app as short term solution.

Thomas_Kwan
Shopify Partner
24 0 0

Jun 28 18;11 Got email from Partner Support

 

"Sorry to see the issue has been open for so long, I'm going to escalate to our Developer support team for further assistance. They should be in touch in a couple business days!"

Thomas_Kwan
Shopify Partner
24 0 0

July 1 4:15pm (13430540)

- sent 2 emails in last 2 days and got no response

- asking for update via support chat

"Okay, since this has already been sent to our Developer Support, technical has no control on when our Developer Support will reply. I see it is marked as a high priority, but I can put this one up to urgent for you so they can get onto it straight away. How does that sound?"

Alex
Shopify Staff
Shopify Staff
1561 81 321

Hey guys.

 

Are you absolutely certain you are not receiving these web requests? Going over our app proxy logic, it's pretty explicit in that there are two code paths leading to that error in third party application state.

 

1. Your response is too large. If you cannot replicate the error by sending a tiny response as a test, then that would be your issue.

2. If the response from your server is in the 500 range. The expression to check this is quite literally if response.status/100.floor == 5 (this is the most common case).

 

Let me know what you guys think. I do know cloudlfare has been having some issues lately, where do you guys host your apps?

Alex | Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

Thomas_Kwan
Shopify Partner
24 0 0

Hi Alex

 

Thanks for getting back. I am pretty sure I am not getting the web request. I have been monitoring the access log on the server when I tried to access the proxy link on the Shopify side. This is the normal flow (correct me if I am wrong)

 

  Shopify's proxy URL (for example  https://etechfocus-demo.myshopify.com/apps/proxy) -> Our server (https://myshopifyapps.com/simple-meta/proxy)

 

When I hit https://etechfocus-demo.myshopify.com/apps/proxy I don't see any new entry in our access log on our server. Yet, the browser returns a 500 error from Cloudflare. I believe Shopify calls Cloudflare which calls our server. You can try it yourself.

 

If I change the flow like the following, it works (without any server side changes on our end)

 

  Shopify's proxy URL -> ngrok (a free proxy service) -> Our server

 

The issue could be on Cloudflare' side so even the logic you showed works, it does not mean much. The Cloudflare integration is on Shopify side, so we have no way to check why Cloudflare is returning 500.

 

Alex
Shopify Staff
Shopify Staff
1561 81 321

Thanks for that information @Thomas_Kwan.

 

The 500 you're seeing in the browser from cloudflare is the status that we're resolving to after determining that the proxied app response is in the 500 range, so a 500 leading to a 500 from us. That said, I'm working on making certain that what I'm seeing in the logic is in fact not a 500 range response from some proxy or middleware which that code is hitting rather than your app host itself. Stay tuned on that one.

 

Cheers.

Alex | Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

pixeledluke
Tourist
8 0 1

Hi Alex,

 

To confirm from our side, we are also checking our access logs on our server, and are not getting anything coming through when using the proxy:

https://unionsubscriptions.myshopify.com/apps/fulfilment => https://checkout.unionroasted.com/

 

As per Thomas's issue, we are also seeing the 500 error from Cloudflare, but via ngrok, our server gets the request, however this cannot be used as a permanent fix in our case.

 

As for the size of the request, we have adjusted the response we are sending back to one line of text, which results in the same error.

Alex
Shopify Staff
Shopify Staff
1561 81 321

Thanks @pixeledluke. That helps.

 

I'll update this thread when I've got something.

Alex | Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

Alex
Shopify Staff
Shopify Staff
1561 81 321

Hey guys.

 

I spoke to the teams responsible for the app proxy logic and I can confirm that there is only two cases where the error you're seeing is surfaced:

 

1. Your response is too long, try shortening it first to make sure that's not the issue (you guys have ruled this out already I believe).

2. Your server is returning a status in the 500 range.

 

When specifically referring to 2, the code path is effectively if res.status/100.floor == 5 (then throw the error). res.status I have confirmed is absolutely the response from the server we're hitting, we don't hit any middleware of our own, the code calling this is the proxying mechanism. Is it possible you guys are behind a reverse proxy or load balancer which is where the 500 response is being sent back, before it hits your app server maybe?

 

Cheers.

Alex | Shopify 
 - Was my reply helpful? Click Like to let me know! 
 - Was your question answered? Mark it as an Accepted Solution
 - To learn more visit the Shopify Help Center or the Shopify Blog

Thomas_Kwan
Shopify Partner
24 0 0

Hi Alex

 

We are not 1 or 2. Things work when ngrok is being used in front of our service. We do not have any other proxy service running.

 

We have these configured on the proxy setting page

 

Subpath prefix: apps

Subpath: proxy

Proxy URL: https://myshopifyapps.com/simple-meta/proxy

 

Can you confirm with your team the following

 

a) When Shopify see 500 as error, can you capture the entire response and send it to us?

b) Or,  Is there a debug mode that we can turn on to get more info in the http response?

 

thanks