Can we set up a reverse proxy with a blogging platform for SEO?

We’d like to be able to setup our own blogging platform alongside our Shopify site, and serve it from a sub-directory for SEO purposes. This would require a reverse proxy (most likely Nginx) to accomplish. I’ve seen a handful of similar requests in this forum, and the response is frequently that it isn’t recommended, and would (may?) trigger the bot detection stuff.

Is this still the case? Or is there now a path to accomplish this? If not, is there any work underway / timeline for when it would be possible?

It seems that it would be fairly useful to be able to seamlessly leverage multiple platforms in addition to Shopify within a site, and it does seem to be something people have run into before.

Did you figure out any solution for this? We are also thinking of using reverse proxy for our site.

@kgurung we got answer from support this is not possible as their anti bot system will not let proxied traffic in. So we had to use subdomain, which works well for us.

@kgurung I was able to setup AWS CloudFront to work like a reverse proxy, similar to the guide here:

https://medium.com/@davidgurevich_11928/cloudfront-as-a-reverse-proxy-fb6a8dcbed96

The idea being that CloudFront will distribute the requests from a large enough pool of servers that it won’t trigger bot detection. If you’re not using AWS, I assume it might be possible to configure another CDN similarly… It did require a little massaging to get things working correctly, so allow a little time to troubleshoot if you go this route.

@DennisJ Thanks for pointing to that Medium article. Do you have any other details on how to get that configured? I don’t see how this strictly forwards traffic from one subfolder in the URL. Thanks in advance!

Sure!

Basically you setup a CloudFront distribution with 2 origins (content sources)- 1 for Shopify & 1 for your blog. Mine looks like this:

Origin domain name and path:

blog.{{mysite}}.com (this is setup as a DNS A record to my blog server’s IP)

shops.myshopify.com

Then I created behaviors in the distribution to route /blog and /blog/* to the blog origin, and the default behavior routes to the shopify origin. It’s almost identical to the first part of this answer: https://serverfault.com/questions/850574/nginx-reverse-proxy-ip-forwarding-for-shopify/987990#987990

This should route the /blog traffic to your blog. Configuring the blog to serve from a subfolder if it’s not already is a separate issue.

Hope that helps!

Thank you very much for elaborating!

First of all, sorry for bringing the topic back from graveyard.

I have the similar case and used CloudFront to setup a reverse proxy as described here.

Everything was working fine for a first couple of days, but then we started getting 403 Forbidden errors from cloudflare. Since cloudflare is only doing this if their origin (Shopify) is rejecting requests, we have no information why is this happening.

I tried to use the other shop for testing purposes, but this time only first request was successful and afterwards I got the same error from cloudflare.

I also tried to use shop’s custom domain (not myshopname.myshopify.com) as an origin, but then I was getting 502 Errors from CloudFront saying that origin doesn’t respond.

Did something change in the meantime?

We haven’t changed our setup since we originally set it up in July of last year, and it’s continued to work fine.

You may need to troubleshoot your CloudFront distribution further- I’m afraid I don’t know much beyond the articles above.

We did have to ensure our blogging platform served a valid SSL cert, but that affected our blog origin not the Shopify origin.

In case it’s useful, our distribution is configured to use:

Custom SSL Client Support Clients that Support Server Name Indication (SNI) - (Recommended)

And our Shopify origin:

Origin Domain Name shops.myshopify.com

Origin Protocol Policy Match Viewer

Hope that helps!

Your setup actually differs in one thing and that is Origin Domain Name. I’m using .myshopify.com for the domain name, so that Shopify actually knows which store I want to use. That’s bit weird because shops.myshopify.com is generally used domain for all the shops as a CNAME Record in case you have custom domain. The link from serverfault.com you provided earlier is also using the same approach as I do…

I was also turning on and off some settings in cache behaviours and realised that in some cases it works, but Shopify is redirecting me to the .myshopify.com after I access it with CloudFront. This is, of course, not what I would need, but it might point out that problem is in the DNS records.

How did you setup domain records, so that you’re using shops.myshopify.com and that Shopify in the same time knows which shop you want to use as origin?

Hi @AdnanC ,

Were you able to resolve your issues in the end?

We have a customer looking to run their Wordpress blog on the /blog URI, but I’m concerned by the reports of bot detection.

I’m not sure if being a Plus customer has any baring on this behaviour?

Thanks

Hi @AlexChapman

Our setup is a bit more complex, where we are replacing also Shopify’s homepage with our own. Because of that, some things like domain and SSL verification are affected and create problems for us. If you are just adding blog under the same domain as Shopify, it shouldn’t be a big deal.

However, because of the bot detection (which you cannot turn off), you have to use some CDN for this and this works pretty reliably. I “just” had to trick Shopify’s own system that verifies domains and also add some response headers. The process from above should work, but if you need more details, I’m in a process of writing a blog about it where everything will be explained.

However, if you are also replacing homepage, there is one problem that happens on regular basis. The SSL certificates that Shopify issues is expiring every few months and usually this is renewed automatically. Well, not anymore if you put a CDN in front of it. For some reason this renewal fails and Shopify renders your shop as “not verified” and it basically goes offline. If you know how, you can verify your shop in 2-3 minutes, but it is a fairly big pain (if you have a busy shop) because it happens randomly during the day when certificate should be generated (which is at least predictable).

We tried to get Shopify’s tech support on this problem, but we always get a non-tech support who copy/paste us instructions on how to setup your DNS records (which is not helpful at all). Shopify plus that we have didn’t help us either.

Hi @AdnanC . Thanks for your answers here, really very helpful. One of my clients also wants to use a third-party blog platform on the same domain as the shopify store. store.com - shopify, shop.com/blog - blog platform. Do I understand correctly that we need to disconnect the main domain from shopify in order to use the settings for proxying on store.myshopify.com? In this case, won’t we have problems with duplicating pages in the search engine for store.com and store.myshopify.com ?

Also, could you share a link to your blog?:slight_smile: I would like to read your article about it. Thank you

Hi @AdnanC ,

I thought you’d had problems using CloudFront in the past with it triggering the bot detection. Is that not the case?

Also with the SSL issue, curious to know why that only impacts when replacing the homepage and not if replacing Blog?

My primary concerns with this are:

  • Not knowing the boundaries of bot detection.

  • Additional latency.

Thanks,

I wonder whether this might help with the bot detection… Although highly likely they’re just looking at the IP header.

https://aws.amazon.com/about-aws/whats-new/2021/10/amazon-cloudfront-client-ip-address-connection-port-header/

Hi @Slava_Maksimov I haven’t finished the blog yet, but I’ll try to do it as soon as I get some free time. Everything will be explained there.

Regarding the duplicating pages in search engine, you won’t have those problems because the Shopify continues to behave the same as before (redirection to the main shop’s domain).

Hi @AlexChapman ,

CDN (in this case CloudFront) is working without bot detection problems. We even tried to send a lot of requests from the same IP and all of them landed without any issues. Our customers are also coming mainly from one country and never got any complaints from them.

The issue with SSL is another black box, because we couldn’t get any additional details or tech support on why is this happening. I assume that renewal of the certificate depends on the response from the homepage (headers, body…) and since it’s not from Shopify, automated process for that cannot finish the validation of the certificate.

You don’t have to worry about latency. If you setup your CDN correctly, most of the requests will be served by your CDN and not not go to Shopify to fetch the content.

Hi @AdnanC ,

What DNS/Origin setup did you end up using? Do you use shops.myshopify.com like Dennis or {storename}.myshopify.com?

I’m just setting this up now, but having the issue where it’s redirecting me to the {storename}.myshopify.com domain when I target the distribution. We don’t have the primary domain linked to the store yet so I’m thinking this might be why.

I can’t add the primary domain yet because it’s currently live on Wordpress, so will need to do this as a final step (not looking forward to that without being able to test first!)

My plan was to CNAME www to the CF distribution host and have the origin pointed at {storename}.myshopify.com.

Is that how you have it?

Thanks for all the help

I’m wondering how to get around the domain verification on Shopify as well, which checks that the [email removed] A record and www CNAME point at the required Shopify records.

See below when trying to add a new domain to the store:

You might be able to point your DNS directly to Shopify, add the domain in Shopify, then update your DNS to point to the proxy after the domain is validated…

I added a proxy sometime after I’d already set up my store, so I’m not sure this would work, but may be worth a try :slightly_smiling_face: