My name is Lilith, I'm on the Social Care team at Shopify.
Sounds like you are refining some areas of your homepage. A/B testing is the fun part!
Below, I've gone ahead and listed some of my top picks from experience and from hearing what works for business owners as well.
I’ve always recommended Google Optimize as a way to help test different homepage setups. It's free and it integrates with Google's Tag Manager as well to help keep all of your Google Analytics data neatly organized.
You'll want to install their script into your Theme by going to Online Store > Themes > Actions (on the theme you wish to A/B Test) > Edit Code. Elevar's blog post can also walk you through how to connect Google Optimize to your store.
There is another site that can help with this as well. Optimizely isn't free like Google Optimize, but it is effective. They've also got an app that can help integrate their code into your theme for easier setup. Plus, it already has conversion events built-in as well.
I'd love to include a couple of resources to help you plan out and work on those split tests. These have been very useful for guiding business owners in the right direction.
Let me know if that helped! I'm happy to answer any questions you think of as you work on your business.
I will note, from experience, that Optimizely takes time to load the modified version.
So, especially with heavy images and multiple changes, there is an inherent bias, where the initial version loads quicker than the modified one.
That's great insight, @adiab.
I'm glad that you could provide some feedback on Optimizely based on your experience.
If you encounter something similar at any point in the future, they do have a troubleshooting section of their help center to tackle slow load or glitch load issues. Also, this situation could potentially happen in some circumstances when previewing since the snippet used during live testing or production is different than in preview mode.
Thank you again for bringing this important point up!
Few months late to the party but I have a solution - I led the development on the A/B testing app Theme Scientist for this exact scenario - you can load up different images in 2 themes, set a test, and they'll automatically swap and calculate metrics like views, adds to cart, orders, sales, conversion rate.
We set up a sliding scale so lower volume stores have a lower fee + included an entry level theme scheduler feature so you can schedule theme changes and track the data manually if you want to save some cash.
I love AB testing, if you want ideas for your site just give me a shout. 👍