A/B Testing different themes

[Jessica_Lafleur]

Hi all,

I recently switched to a new theme, and think that sales have dropped as a result.
It's silly to be guessing about this, when I have Optimizely installed.

Is there an easy way to A/B test a completely different theme?

What I am investigating now is: Shopify has a "?preview_theme_id=1234567" query parameter... I am wondering if I can do a Optimizely redirect from http://www.starkskincare.com/   to  http://www.starkskincare.com/?preview_theme_id=1234567  as an A/B test.

Has anyone done this?

Thanks in advance for any ideas & help!
Adriano 

[BrennanM]

Hey Jessica/Adriano

From what I've heard you can't do A/B testing on shopify because everything is live :(. You can take a look at your new theme though and see if you've lost a bit of SEO goodness in the change. Sometimes the change can erase a few things. Just double check everything and if that's not the problem, maybe ask a few friends if they find it as appealing as the old theme. That's probably the best way to find out what the problem is. 

I really like the look of your store by the way, it looks really professional and the pictures you're using are great! 

Brennan

[Shopify_Craig]

Brennan,

That isn't actually the case.  You can use tools like Optimizely to do a/b tests on Shopify.  That said, I haven't heard of anyone A/B testing different themes, but I really do like the idea.  The only caveat is that you often need at least 100 conversions on both A and B before you can see any significance.  That means you'll need to wait for over 200 orders before you can figure out which theme works best.

Craig

[BrennanM]

Hello Craig

Ah, ok, good to know! Yeah testing different themes would be a huge benefit for stores, knowing the "click through" and "conversion rates" would be great advertising for different themes too, (our theme converts more customers etc.) hmmm.... 

Brennan

[robedwithlove]

Craig, just picking this thread up...

Is it now possible to a/b test a product page from a 2nd theme?

We want to determine how a redesigned product page performs with FB traffic vs. our current theme.

Thanks,

Lindsay

[Josh_Uebergang]

I wrote about a theme spit-test strategy many years ago for Shopify. For the most reliable theme split-test site-wide, the best strategy is to do it on two domains with redirects.

[JoesIdeas]

@robedwithlove my team built an app to split test themes, so you you can test 2 different product pages like you want. And it switches the actual theme so there's no performance lag.

It's called Theme Scientist - apps.shopify.com/theme-scientist.

I'm the lead developer so if you need any help setting up just send me a message.