E-Commerce Intelligence Series: Best Practices to Get A/B Testing Right

A/B testing is a great way for marketers to optimise their website, ads, and campaigns. This optimisation can result in higher ROI, conversions and engagement, and at times even enable brands to stay ahead of the competition. In this blog, we look at what is A/B testing and what best practices brands can follow to run effective testing scenarios.  

Can you recollect the number of times you’ve taken a few minutes to fill a survey or give user feedback? Not often, we’re guessing. As repeat customers of brands and products, we’ve often been asked to leave behind a review or share a photo of how we’re using their product on our social media profile. But, it’s not likely to be done unless we’ve been on a purchase high or have been incentivised. This user feedback mechanism is what HubSpot, the marketing and sales automation software company, attempted to fix through A/B testing.  

A/B testing is the process of optimising a website or app by building varied versions of content, design, layout or navigation experience and exposing these to randomised splits of an audience to measure which variation resonates better with the demographic of the chosen audience.  

In this A/B testing exercise, HubSpot’s main objective was to get more users to fill out the feedback form. How did they do it? 

They set up three testing scenarios via email and through in-app notification. In the first one, they sent a simple email to their champion users of the month, requesting them to leave a review. Once they’re done, they would be given a US $10 gift card as a token of appreciation. 

In the second testing scenario, they designed a certificate within the email, to add some personalisation to their campaign. 

In the third testing scenario, they pushed an in-app notification encouraging their champion users to leave a review.  

By running all three testing scenarios, what HubSpot noticed was that its customers were more receptive to emails than in-app notifications (which they either missed or skipped often). They recorded a 1.4 times higher response rate via email when compared to in-app notifications, and 24.9% of the users who opened the email left a review. The same number stood at only 10.3% via in-app notifications. 

This is just one of the many examples of how brands can use A/B testing to gain measurable insights into what needs to be optimised to increase engagement with new and existing customers. 

Why is A/B Testing Important? 

A Merit expert says, “A/B testing gives measurable insights into what components of your website or app can drive higher return on investment. When you create multiple test scenarios and measure which one is generating a higher conversion, you’ll be able to develop more targeted messaging, design, content, and layout that will resonate better with your customers.” 

In a way, it will also help you stay ahead of the game and keep you on your toes when it comes to developing a continually great user experience. 

Planning an A/B Test 

Typically, brands perform an A/B test on the website or app, or for ads and marketing campaigns. There are three fundamental criteria that the testing should fulfill; 

• It should be based on a specific problem or issue that needs to be resolved. For example, if a brand website is witnessing more bounce rates on its homepage, it can set up testing scenarios (tweak the content, layout and design) to identify which format is performing better and garnering more visitor attention. 
• It should be a problem that can be resolved with data-backed decisions. 
• The results should be measurable. For example, A/B testing can be done to measure the number of CTRs (Click-through-rates), conversions, bounce rates, or time spent on a landing page. 

When it comes to website or in-app testing, it typically includes; 

• Headline 
• Blurb 
• Featured Images 

• Call to Action 
• Pop-Up Notifications 
• Design (Including choice of fonts and colours) 
• Overall content on the page 

When it comes to offline testing, it includes; 

• Ads 
• Marketing Campaigns 
• Emails 
• Push Notifications 

And so on. 

How to Get Started: Here’s an A/B Testing Checklist 

While you can test practically everything that can be changed on your website, app or marketing material, identify critical pieces that can actually make a difference or be noticed when tested.  

Let’s look at a few quick best practices you can use when setting up your A/B testing process; 

1. Identify specific elements that you want to test. For example, one of the landing pages in your website may not be seeing as many conversions as a few other pages. So, your objective will be to set up split variations of the content, design and layout (including CTA) in that landing page to test which variation performs better.  

2. Have an insight into how the element has been performing currently, in order to measure the effectiveness of the change post A/B testing. For example, if you’re testing the impact of a marketing email, you need to know the current open or conversion rates it is recording, and then find out if the variations have performed better, worse, or neutrally. 

3. Make sure the audience groups you’re exposing different variations of your content to, are the same. Ensure that they are not shuffled or mixed across groups. This will ensure your A/B tests are consistent and effective. 

4. Ensure your tests are running simultaneously, that is, different variations are being exposed to different sets of customers at the same time.  

5. Launch the test and wait for a week or two to then measure how the tests have performed. 

6. Use insights from the test to make changes to your website, app or campaign accordingly. 

In conclusion, A/B testing is a great asset for marketers to optimise their website, app, ads, campaigns and emails. The only important thing to remember here is that while any changeable element can be A/B tested for improvement, marketers can’t spend all their time running testing scenarios on every small element. They need to identify parts of the content that can make the most impact on customers, and run tests accordingly. 

Merit’s Expertise in e-Commerce Data and Intelligence  

Our state-of-the-art eCommerce data harvesting engine collects raw data and provides actionable insights  

• Three to four times faster than standard scrapers  
• At lower cost  
• With Increased accuracy (up to 30% compared to standard scrapers)  

Our powerful, new scraper engine can gather massive data sets from multiple sites and geographies in real-time so you can stay informed on customer behaviours and market trends.  

Merit’s eCommerce data engine provides a high degree of confidence in insights generated from analytics – thanks to confidence in the data quality and access to enriched data.  

To know more, visit: https://www.meritdata-tech.com/service/data/retail-data/