A/B Testing different themes

Jessica_Lafleur
Tourist
5 0 3

Hi all,

I recently switched to a new theme, and think that sales have dropped as a result.
It's silly to be guessing about this, when I have Optimizely installed.

Is there an easy way to A/B test a completely different theme?

What I am investigating now is: Shopify has a "?preview_theme_id=1234567" query parameter... I am wondering if I can do a Optimizely redirect from http://www.starkskincare.com/   to  http://www.starkskincare.com/?preview_theme_id=1234567  as an A/B test.

Has anyone done this?

Thanks in advance for any ideas & help!
Adriano 

BrennanM
New Member
10 0 0

Hey Jessica/Adriano

 

From what I've heard you can't do A/B testing on shopify because everything is live :(. You can take a look at your new theme though and see if you've lost a bit of SEO goodness in the change. Sometimes the change can erase a few things. Just double check everything and if that's not the problem, maybe ask a few friends if they find it as appealing as the old theme. That's probably the best way to find out what the problem is. 

I really like the look of your store by the way, it looks really professional and the pictures you're using are great! 

 

Brennan

0 Likes
Shopify_Craig
Shopify Staff (Retired)
Shopify Staff (Retired)
333 0 26

Brennan,

That isn't actually the case.  You can use tools like Optimizely to do a/b tests on Shopify.  That said, I haven't heard of anyone A/B testing different themes, but I really do like the idea.  The only caveat is that you often need at least 100 conversions on both A and B before you can see any significance.  That means you'll need to wait for over 200 orders before you can figure out which theme works best.

 

Craig

 

0 Likes
BrennanM
New Member
10 0 0

Hello Craig

Ah, ok, good to know! Yeah testing different themes would be a huge benefit for stores, knowing the "click through" and "conversion rates" would be great advertising for different themes too, (our theme converts more customers etc.) hmmm.... 

 

Brennan

0 Likes
robedwithlove
New Member
1 0 0

Craig, just picking this thread up...

Is it now possible to a/b test a product page from a 2nd theme?

We want to determine how a redesigned product page performs with FB traffic vs. our current theme.

Thanks,

Lindsay

0 Likes

I wrote about a theme spit-test strategy many years ago for Shopify. For the most reliable theme split-test site-wide, the best strategy is to do it on two domains with redirects.

Run Google Shopping ads? Get the free definitive guide to Google Shopping for Shopify (no optin required): https://www.digitaldarts.com.au/google-shopping
0 Likes
JoesIdeas
Shopify Expert
1244 111 310

@robedwithlove my team built an app to split test themes, so you you can test 2 different product pages like you want. And it switches the actual theme so there's no performance lag.

It's called Theme Scientist - apps.shopify.com/theme-scientist.

I'm the lead developer so if you need any help setting up just send me a message.

Founder of Speed Boostr (Shopify optimization experts, theme customization, custom app development).
Creator of Shopify Analyzer (first performance analysis tool for Shopify sites, free for the Shopify community).
Creator of Order Automator (auto tag orders and customers + auto fulfillment + more automations).
More apps: Theme Scientist (A/B testing), Tip Jar (add a tip button), File Optimizer (optimize CSS, JS, Liquid).
0 Likes