A wine online shop is planning major changes but faces challenges testing them through A/B testing, as rebuilding in their testing app is impractical and developing changes in a preview theme requires significant upfront investment. They’re also questioning when to refresh their brand icon and how to assess its value to customers.
Key Recommendation:
Avoid A/B testing visual design and logo changes, as users switching devices or clearing cookies can’t be reliably isolated into single groups, creating inconsistent experiences.
Suggested Approach:
Use user testing and prototype testing methods instead to gather qualitative feedback on different designs from real customers.
When implementing a rebrand or reskin, change only the visual styling in isolation—don’t bundle it with UX changes.
Test UX improvements separately from pure aesthetic updates.
The discussion remains open regarding specific methods for evaluating brand icon value and determining optimal refresh timing.
Summarized with AI on October 31.
AI used: claude-sonnet-4-5-20250929.
We are working on big changes for our wine online shop.
These changes are very difficult to test via A/B testing, as we cannot rebuild those changes in our testing app. On the other hand, if we want to test it within another preview theme we need to invest the workload to develop the changes already.
Do you have an advice how to bring big webshop changes live and what needs to be tested A/B?
Another question:
When do I know the time to refresh my brand icon?
We are wondering if our brand icon is important for our customers and whether it still belongs to our core brand values.
Do you have an advice how to estimate the values of a brand icon and what needs to be considered while refreshing the brand icon?
Thansk in advance
I wouldn’t advise trying to A/B test visual look and feel and brand/logo. It would be nice, but customers will come on different devices or delete cookies, so you can never completely isolate someone into one group, which means a jarring experience when they see different looking sites.
If you want to get feedback from real customers on different designs, it’s better to use user- and prototype-testing methods to get more qualitative data on the different designs.
One important thing though. If you’re reskinning and changing the brand, do that in isolation i.e. don’t use it as an excuse to throw in a lot of UX changes as well. Just change the pure styling by itself, and test those other UX changes separately