Hi. We are currently using Google Optimize for our AB Testing app. It’s been working pretty great for us but I am having a really hard time figuring out how to run this new test on our Homepage. We currently have 2 videos on our site and I want to test to see if they are having a negative effect on our conversions/engagement.
I have tried display:none but that only hides the videos, it does not prevent the videos from loading. So the payload and number of requests doesnt change and defeats the purpose of my test.
Has anyone done testing where the variant without the videos actually does not load into the DOM?
One approach I’ve tried is to run a redirect test and host the variant experience on a separate page that renders the exact same Sections (except for the video content) as the control. Optimize will then split sessions between the original ‘website.com/’ path, and ‘website.com/pages/homepage-variant-name’, each of which will load different Liquid/HTML.
I’ll say that this approach feels really messy to me, and setting this up can be tedious if you have a lot of dynamically-set content on your page, but I’m not sure of a cleaner way to do this.
It would be great if Shopify offered their own A/B testing functionality –– the lack of control over Theme compilation can be a little frustrating sometimes.
If you are using a video tag, try adding the preload=“none” attribute else you could try lazy loading the video https://web.dev/lazy-loading-video/ which will give you more control over which video gets loaded.