Accelerate your A/B testing with a targeted strategy based on Customer Experience data

To stay ahead of the competition, you must win more customers at every turn. To achieve this, your brand’s digital customer experience (CX) must be continuously optimized, and all improvements must be made faster than the competition. A/B testing is a powerful tool for achieving data-led optimizations, however these also experience a high failure rate and it’s hard to know which elements must be tested as a priority.

In this article we’re going to look at the WUA approach to optimization, and learn some important strategies for making your A/B tests more effective and more successful from the start.

When your A/B tests are led by the ideal approach outlined below, you can achieve better results, faster.

First: A/B testing – what’s it for?

A/B testing is one of the most important weapons in your arsenal. It’s a simple but rigorous method. A/B testing ensures every change you make to your digital experience will have predictable, positive effects – based on hard data. To get the most out of your A/B tests, it’s important to understand the purpose of A/B testing – and its limitations.

The principle behind A/B testing is logical and scientific: you change one variable at a time, creating two variations that are compared to a ‘control’. The control gives a baseline that allows you to generate meaningful comparisons between each variation. The numbers don’t lie.

By taking this methodical approach, incremental improvements can be made to your website’s branding, images, information, product/offer, navigation, layout, icons, style – or any other visible part. And with each change, you can measure the effect on the digital customer experience (CX) before enacting them permanently.

The limitations of A/B testing

In competition, speed is of the essence. And so is accuracy. Despite the built-in scientific methodology of A/B testing, it can be frustratingly slow and imprecise, due to its fundamental limitations. First, the failure rate of A/B tests is often high – especially at the start of optimization. In this scenario, many tests are conducted, but few indicate CX improvements and may detract from the digital experience.

A success rate of 50%+ might be desirable, but many only achieve 10-30% success from A/B testing. Eliminating wasted tests is a priority.

With deeper customer insights, you can accelerate the results from more intelligent testing. You can also gain a holistic vision of the process leading to conversion. Let’s see how you can achieve this.

Knowing what to test as a priority can be a huge boost to A/B testing, and can accelerate results for any business – B2B or B2C.

Common limitations of A/B testing:

  • One variable per test – You can only test one variable at a time with A/B testing while still producing meaningful results, this limits the number of tests per month that are possible.
  • No clear priority – Without knowing which elements to test first, it’s hard for web designers to use A/B testing effectively.
  • Minimum dataset size – This is especially tough for B2B companies as the pool of potential customers is shallower.

The ideal approach: Evidence based, thesis-driven A/B testing

Our approach is to guide faster optimizations by increasing the impact of each test, and to make each test more successful. This is the ideal, most powerful way to extract successful results from A/B tests.

At WUA, we’ve seen that companies get better results from evidence-based hypotheses that guide the process. When helping clients to formulate their testing strategy we find that a thesis-led A/B test massively accelerates the optimization process by increasing the impact and success of each test.

This is because a hypothesis is based on a ‘working model’ of your conversion process and an understanding of what makes it successful. This understanding is only possible using extensive, granular data about the customer experience during their journey.

Without this insight, researchers are faced with a real-life ‘trolley problem’: they must choose between multiple options – urgently – without being able to distinguish which one is most valuable.

How CX benchmarking helps build a 360-degree view of the customer

To create a solid thesis, you first need data. And not just any data – it must capture each part of the digital customer experience and help build a framework for how these work together. A 360-degree view is needed. So, let’s look more closely at the journey each customer takes.

First, an important fact: customers ultimately choose just one provider.

It might be you, or your competition – but, before they reach this decision, they must eliminate all other options along the way. In some cases, the driving factors behind their decision to eliminate a potential provider can be surprising. It may come down to something as simple as the color palette or tone of voice. Or it may come down to a single, key piece of information. With so many possible website elements involved in their entire experience, how do you know which will have the greatest impact?

The answer is simple: we ask them.

WUA uses a refined research methodology that means uniquely granular insights can be extracted. These insights help you form a better understanding of their digital experience, and which factors have the greatest impact for winning their eventual custom.

How WUA’s unique CX benchmark methodology provides rich experience data

The methodology behind WUA’s CX Benchmark data means that we can get a deeper view of the customer experience. We ask hundreds of potential customers each time a benchmark study is conducted on a sector, service, or product. It combines ‘hard’ statistical data with ‘soft’ data from qualitative responses.

Also read: What is Digital Customer Experience Benchmarking and why is it important?

WUA’s benchmark data is also not restricted by the imagination of the researchers and data scientists – customers freely identify their priorities by volunteering them instead of only choosing from a ‘drop down list’ of options. This means you can identify critical issues that could have gone unnoticed or unimagined.

Each answer is weighted by volume and the overall effect on conversion, ensuring that frequent ‘complaints’ are only considered relevant if they actually impact the final decision.

Example from the priorities module in the WUA Customer Experience Dashboard

So, for example, while 80% of visitors might hate your brand colors, this might not affect the end result as much as changing layout or product information.

The result of this methodology and the data it collects is a very special kind of knowledge of the entire experience. It can point out which factors are most critical for success. The synergy of using large datasets with both empirical and qualitative customer experience data means that companies can target their A/B tests on only the elements that will have the greatest positive effect on conversion.

With CX benchmarking you know exactly where to target your resources.

How to put CX benchmarking to work with A/B tests

WUA makes it easy for you to extract value from CX benchmarking studies with a user-friendly platform that gives you the depth of vision you need to improve your digital experience.

There’s a limit to how many A/B tests you can do, so our data helps optimize this by only testing the most relevant elements as a priority. This ensures you get the most value from each test.

When it comes to actually applying this data to A/B testing, our researchers can advise on strategy. This can accelerate results from the very start. It isn’t necessary for us to even see the results from the A/B tests themselves, as the strategy is already directly informed by the customer experience. This means we can offer strategic advice that is impartial and based purely on solid and reliable data.

With WUA’s strategic guidance, your A/B tests can focus on the most pressing priorities. The results can be seen from an improved benchmark performance, and an improved conversion.
By only testing where it counts, your optimization can happen faster and more accurately – helping you maintain your market position and your edge over the competition.

Start today with targeted optimization

Get in touch to discuss the possibilities.

Get in touch

Read our cases


How ING improves customer journeys in an agile and global organization, using an optimized digital CX.

Read more

How T-Mobile Home Doubled their Conversion with More Successful A/B Tests

Read more
VGZ Screenshot

How Dutch health insurer VGZ is aiming for friction-free online CX

Read more