You duplicate your live theme, then make the edits on the second theme, then the app swaps them at the interval you choose plus calculates the stats of each.
For #3, you might have to integrate some custom JavaScript into your theme… I think the way I would approach is to set it up by sort. For example testing “best sellers” vs “new arrivals”. Then on page load, have some code that sets the sort order based on whether it’s theme A or B.
I recommend only testing 1 element at a time for more accurate results (rather than 3 different areas, because then you can’t be sure which change affected the conversion rate).
My team built that app Theme Scientist, so we can help you get set up if you need it, just contact us through the app listing and ask for Joe: https://apps.shopify.com/theme-scientist
I found an app for AB testing page titles and meta descriptions (what gets shown on Google). Slightly off topic to what you are trying to test, but might be worth checking out! The app is called SEO AB Tester, it’s helped my store.