A/B Testing different themes

Jessica_Lafleur
Tourist
5 0 3

Hi all,

I recently switched to a new theme, and think that sales have dropped as a result.
It's silly to be guessing about this, when I have Optimizely installed.

Is there an easy way to A/B test a completely different theme?

What I am investigating now is: Shopify has a "?preview_theme_id=1234567" query parameter... I am wondering if I can do a Optimizely redirect from http://www.starkskincare.com/   to  http://www.starkskincare.com/?preview_theme_id=1234567  as an A/B test.

Has anyone done this?

Thanks in advance for any ideas & help!
Adriano 

Replies 6 (6)
BrennanM
New Member
10 0 0

Hey Jessica/Adriano

 

From what I've heard you can't do A/B testing on shopify because everything is live :(. You can take a look at your new theme though and see if you've lost a bit of SEO goodness in the change. Sometimes the change can erase a few things. Just double check everything and if that's not the problem, maybe ask a few friends if they find it as appealing as the old theme. That's probably the best way to find out what the problem is. 

I really like the look of your store by the way, it looks really professional and the pictures you're using are great! 

 

Brennan

Shopify_Craig
Shopify Staff (Retired)
Shopify Staff (Retired)
359 0 27

Brennan,

That isn't actually the case.  You can use tools like Optimizely to do a/b tests on Shopify.  That said, I haven't heard of anyone A/B testing different themes, but I really do like the idea.  The only caveat is that you often need at least 100 conversions on both A and B before you can see any significance.  That means you'll need to wait for over 200 orders before you can figure out which theme works best.

 

Craig

 

BrennanM
New Member
10 0 0

Hello Craig

Ah, ok, good to know! Yeah testing different themes would be a huge benefit for stores, knowing the "click through" and "conversion rates" would be great advertising for different themes too, (our theme converts more customers etc.) hmmm.... 

 

Brennan

robedwithlove
New Member
1 0 1

Craig, just picking this thread up...

Is it now possible to a/b test a product page from a 2nd theme?

We want to determine how a redesigned product page performs with FB traffic vs. our current theme.

Thanks,

Lindsay

Josh_Uebergang
Shopify Expert
853 30 214

I wrote about a theme spit-test strategy many years ago for Shopify. For the most reliable theme split-test site-wide, the best strategy is to do it on two domains with redirects.

Run Google Shopping ads? Get the free definitive guide to Google Shopping for Shopify (no optin required): https://www.digitaldarts.com.au/google-shopping
JoesIdeas
Shopify Expert
1670 143 428

@robedwithlove my team built an app to split test themes, so you you can test 2 different product pages like you want. And it switches the actual theme so there's no performance lag.

It's called Theme Scientist - apps.shopify.com/theme-scientist.

I'm the lead developer so if you need any help setting up just send me a message.

• Creator of Order Automator (automation app to save you and your team time on admin tasks)
• Shopify developer for +10 years
• Contributor on the Speed Boostr blog
• Guy who likes to help people