How to best implement A/B testing in a new app?

[snflwr]

Hello,

I am trying to build a Shopify app for A/B testing and experimentation. I am still new to the Shopify apps ecosystem, and I am struggling a bit with finding the right architecture/implementation, so all ideas and feedback are welcome.

A simple use case would be trying different product images, product descriptions, or how the product page is organized, then tracking the performance of the different variations (e.g. conversion rate).

Problem 1: How would you implement the decision on which variation to display?

Going through the documentation it is recommended to use an App Block for such things, then let the user add them to the page layout (i.e. an alternative "product description" block), but how would this work? Should I extend the database model to include new fields such as produc_decription_{1,2,3,...} and product_image_{1,2,3,...} then have the front end decide on which one to display?

Is there a more practical way that does not require the merchant to update the store layout? Would ScriptTag be recommended for such a use case?

Problem 2: How would you track the user and relate that to a variant?

For this, I thought of using JS local storage + App Pixel to send the analytics. But who would this logic figure out which variants are running and make them consistent for that user? I would need some piece of logic from Probme 1 above that stores the value of which variants are shown to this specific user session, would you use ScriptTag for this? Or is there a better way that does not need injecting code into the theme?

Thanks in advance!

- Mansour