I plan to test a few variations:
1. homepage (different images)
3. product alignment (e.g. first product shoppers see is X, then Y)
I assume there will be at least 5-8 variations I would need to test. Any suggestion on an app that allows to A/B test all these elements?
Would be a nice project to create a custom app for too.
You duplicate your live theme, then make the edits on the second theme, then the app swaps them at the interval you choose plus calculates the stats of each.
I recommend only testing 1 element at a time for more accurate results (rather than 3 different areas, because then you can't be sure which change affected the conversion rate).
My team built that app Theme Scientist, so we can help you get set up if you need it, just contact us through the app listing and ask for Joe: https://apps.shopify.com/theme-scientist