This is a great question! My name is Charles with Shogun Page Builder.
Shogun has the ability to A/B test pages built in our application. If the app you are hoping to AB test 2 different apps you can integrate a third party app into Shogun using an HTML element and embed, and then test the results. :)
Let me know if you have any further questions.
Hey @ivy_help , here's how I would approach this:
1) Ask the app developers if you can load their app via snippet. Some apps load with a theme snippet, while some inject the scripts into the live site.
Even apps that don't have a theme snippet, I've had success converting to that load style just by asking them.
2) Now you have the snippet that controls whether the app loads, so duplicate your live theme, then remove the snippet from this duplicate theme.
3) Now rotate the 2 themes every day at the same time, so each theme / app gets the same exposure.
4) For best results make the change at exactly 00:01 in your local store time. This will give you the cleanest analytics.
If you want this automated, check out Theme Scientist A/B testing app. I built this as a way to run quality A/B tests on autopilot, then turned it into an app for everyone.
I know it's been a few months but if you still have ideas and need help setting up the app test let me know. I don't think I've A/B tested 2 different apps before, I like the concept though and would happy to help get you set up.