Conversion rate optimisation (CRO) is the process of improving the percentage of visitors to a website who take a desired action, such as making a purchase, signing up for a newsletter, or downloading a white paper. CRO tests are a powerful tool used to identify the elements on a website that are most likely to influence conversions.
There are many different types of CRO tests, but some of the most common include:
- A/A test: This simple test compares two identical versions of a page. It is sometimes called a “sanity check” or “control experiment”. This type of test helps verify that your A/B testing software is working correctly. By running the same version of the page against itself, any inconsistencies or biases in the testing platform can be identified. The outcome of this test should be that the variants perform equally. If there is a difference between the two, there may be issues with the testing setup or tracking, and further investigation is required before moving on to A/B or other testing.
- A/B test: This is the most common type of split test. It compares two-page versions, often called Variant A and Variant B. Each variant represents a specific change or variation in the design, content, layout, or functionality. For example, you could test separate headlines, call-to-action buttons, or product images. The purpose is to identify which variant generates or performs at a higher conversion rate. This can be tracked through increased purchases, sign-ups, downloads, or any other predefined action.
- A/B/n test: This type of test allows you to test multiple alternative elements on a page at a time. Each variation receives its own share of your total audience. This is beneficial for high-traffic sites to try much broader setup changes—for example, three or more variations of widgets that display customer testimonials on the basket page.
- Multivariate testing: This type of test allows you to test multiple variations of designs and elements on the page to understand which combination is the best. Unlike A/B and A/B/n testing, where each variant tests a single element or variation, multivariate testing simultaneously tests multiple elements and their variations. This approach enables you to evaluate the interaction and cumulative effect of various factors on the overall performance of a webpage.
Here is a simple example:
- V1 – Control Variant
- V2 – Copy Change
- V3 – Video Item
- V4 – Copy Change and Video Item
Multi-armed bandit (MAB) testing:
This approach allows for the dynamic allocation of traffic to different variations of a webpage or user interface based on their performance. This type of test is similar to A/B testing but uses a more sophisticated algorithm to split traffic into different variations evenly. This testing refers to a metaphor derived from the concept of a slot machine with multiple arms or levers, each representing a different variation. The goal is to find the variation that provides the highest conversion rate while maximising the total reward. This can improve the efficiency of your tests and get you results faster.
Which type of CRO test should you use?
The best type of CRO test for you will depend on your specific goals and objectives and the stage of your optimisation process. If you’re starting with CRO, an A/A test is an excellent way to ensure your A/B testing software works correctly. Once you’ve got the hang of A/B testing, you can experiment with more complex tests, such as A/B/n testing or multivariate testing.
How to run a successful CRO test
Following a few key steps is crucial to run a successful CRO test. First, you must pick the right tool that aligns with your requirements and available resources. Factors such as budget, experience, developer knowledge, test complexity, and data analytics significantly determine the CRO “maturity” level you need.
Here are some recommended tools based on your CRO journey:
- Start: If you are starting with CRO, a tool such as Convert Experiences is a great place to start. Convert is an affordable option that allows you to set up A/B testing, split testing, multivariate testing, multipage testing, complete stack experiments, and personalisations. Its visual editor will enable marketers to set up tests, and Convert offers excellent support.
- Growth/Pro: VWO also has a user-friendly visual editor that allows users to make quick changes without the need for technical help; however, with advanced targeting and surveys, heatmaps, and session recordings built-in, it is an ideal tool for those wanting to ‘level up’ their CRO.
- Scale: Optimizely and Sitecore Personalize are amongst the tools for those who are mature and continuing to scale their CRO programme. Sitecore Personalize activates your data across all customer touchpoints to create seamless and consistent customer experiences. Sitecore Personalize allows you to conduct web, interactive, and triggered experiments in real time, allowing for more accurate data collection and analysis. This requires high technical expertise, which is crucial to remember when choosing your tool.
- Analyse user behaviour: The best tests often come through a thorough analysis of the current website using tools such as session recordings, heatmaps and analytics. A tool in place is essential to facilitate this, as it will also help illuminate and analyse how well your tests ran. Hotjar is an excellent example of a tool for this, which is also efficiently utilised by all stakeholders.
- Set clear goals: What do you want to achieve with your test? Do you want to increase checkout completions by 5%? Reduce the bounce rate by 10%. Once you know your goal, you can design your test accordingly.
- Choose the correct elements to test: Not all elements on your page will significantly impact conversions. Choose the elements that are most likely to make a difference.
- Run your test long enough: You need to give your test enough time to collect statistically significant results. You must run your test for a short period to get accurate results.
- Analyse your results carefully: Once your test is complete, you need to analyse the results carefully. Look at the conversion rates for each variation and metric to see which performed best.
- Make changes to your page: Once you know which variation performed the best, you can make changes to your page to reflect those improvements.
CRO tests are a powerful tool that can help you improve your website’s conversion rates. Understanding the different types of CRO tests and how to run them can increase your chances of success. Most importantly, CRO testing takes time. Wait to see results overnight; keep running tests and analysing your results, and you’ll see improvements in your conversion rates.
Consider each test type’s statistical significance, sample size, and practical feasibility, given your specific circumstances. It’s also beneficial to consult with CRO experts and use specialised CRO tools to help determine the most appropriate test type for your goals and constraints.
