Split theme testing - removing the preview bar in checkout

[danmaf]

After our developer followed a combination of these two articles (1 & 2) to help us split test our live Shopify theme with one in preview where he coded the edits, we've ran into the issue that the preview bar indicator appears in checkout. We were able to successfully hide it on the main site (homepage, pdp, etc.).

We've received various answers from support including:
- not possible
- can't edit that because of compliance restrictions
- use an individual's timed testing tool (which we're very against for various reasons - it switches which theme is published ~daily)
- upgrade to shopify plus
- to comment here on the forums

Now unfortunately the change we'd like to make is too complicated for VWO's A/B visual editor, which is why we'd like to split test/redirect test.

----------------

Changes we're making:

• Nested Nav (must be a persistent change on the whole site)
• Homepage CTA update from redirect to an anchor
• One section overhaul (this is the complicated change that VWO doesn't really suffice - going from a tall design to a more horizontal, mobile-friendly one)

----------------

Our developer has said he's tried the following and both don't work on checkout:

1. injecting display none to the iframe by javascript in online store > preferences > additional scripts

2.
#preview-bar-iframe {
display: none!important;
}

------------

I'm not technical, but these were some of my ideas:

1. Publish the variant version of the site and use that as the control. Change to A/B test (rather than redirect) and use the VWO editor to ~use it to undo those changes for the variant.… probably will run into the same issue with its limited capabilities.
2. Somehow hide the changes in Shopify and then in VWO editor, unhide them via HTML editor
3. Similar to #2 but maybe if a param is present then that triggers something to display version B (the variant) to the user instead of A (the control)

I also thought Shopify could disable it on their end for us like they disable captchas on forms but chat support was unable to find out.

-------------

Please let us know if you've split-tested themes before and how you've gone about it or are able to help with the issues we're running into.

Thank you in advance,

Dan

[marie_h]

Have you tried to add pb=0 url parameter to your theme preview urls?

See https://community.shopify.com/c/Technical-Q-A/how-to-remove-preview-bar/m-p/1023134/highlight/true#M... 

[RobertBotto]

Hello Dan,

Have you tried using an A/B Testing tool to test out themes? 

If not, I work at Neat A/B Testing and we offer Theme tests with the appropriate data that will help you evaluate what performs better at a significantly lower price than VWO. 

Our app also has price testing, product page testing, homepage tests, and split URL tests. 

If you or you developer need any help with setup, please don't hesitate to reach out as we are happy to help you get up and running!

[kconv]

@RobertBotto, do you have a way to hide the preview box on the checkout page too?

[danmaf]

You need to be on Shopify Plus to remove it in checkout which is why I found hiding preview with the param alone didn't suffice for the majority of our clients.

One old colleague of mine tossed around a dopey idea that I'd love to hear back from any of you if your dev team finds the time - she said potentially code the changes on the site but comment out those DIVs/edits. Then go into your Testing tool like Google Optimize and un-comment it out for the variant. Very curious if this works - please reply here if anyone tries this!

[kconv]

Thanks, @danmaf! For confirming this to me. So if someone is using Shopify Plus, does the hiding with-param work on the checkout, or no? Does it need CSS hiding?

For your colleague's idea, basically yes it is doable, but since the tool can uncomment, it can also add code and do countless things, so basically not really useful. You would also you would have to do it in a way that the uncommented blocks get re-executed, so not that trivial.

[danmaf]

It needs CSS. Across the site, the following CSS should be present for all pages including the checkout flow: .shopify-preview-bar { display: none }
I believe that is all, but feel free to read this documentation - specifically, at the way bottom of the 'How can I test alternate templates?' and 'How can I test a new design site-wide?' sections of the article. I found this to be the working solution, but again it's sadly only for Shopify Plus clients because at that plan you're able to edit checkout. Hopefully, Shopify adds just this one edit for non-Plus accounts because I'm really surprised split testing themes isn't built-in.