At its simplest, this kind of testing, sometimes called split testing, involves comparing two versions of a web page to see which one performs better with customers.
According to Jonathan Epstein, Sentient’s senior vice president of international, traditional approaches and tools for A/B testing are time-consuming and resource-intensive, and often produce misleading or hard-to-interpret results. Only around one in six tests reap positive results, in any case. All these challenges make this tricky process a great candidate for that shot of AI, he claims.
In September 2016, Sentient launched a new ‘conversion rate optimization’ product, called Ascend. It’s based on much of the same AI technology seen in products that Sentient, a company founded by the team that originally invented Apple’s Siri, already sells to financial services companies to manage large, complex hedge funds. This technology, which it refers to as ‘evolutionary algorithms’, mimics natural selection to find the optimal answers to large-scale problems, Epstein explains - but more of that later.
Among the first companies to sign up for Ascend was Cosabella, a US-based, family-owned company that manufactures luxury lingerie, sells it B2B through international retailers including Nordstrom and Selfridges, and direct to customers through its e-commerce website.
Cosabella had been looking for a way to more efficiently test designs for its online storefront for some time, according to Cosabella CEO Guido Campello. It has been using Ascend since October last year and, in that time, Campello says:
We’ve discovered things about our brand, our site and our customers that we would never have believed without seeing for ourselves the results of this testing. These new findings have really influenced the way that we think about our brand.
That’s not to say that Cosabella had never done A/B testing before. In fact, the company had tried more traditional tools and third-party agencies on testing in the past, but found that not only did the company have too many ideas to test in an economically viable way, but also that testing didn’t lead to the uplift in conversion rates and email sign-ups that its management was hoping to see.
With Sentient Ascend, by contrast, Cosabella saw a 38% sales conversion uplift by simultaneously running tests across four different site elements (their header, button color, image size and call-to-action placement) and incorporating 15 different ideas for changes throughout those elements, amounting to some 160 potential designs in that first test, which was run over a space of seven weeks. Traditional testing of so many elements might have taken Cosabella between six months and one year, reckons IT manager Thomas Lyne.
Survival of the fittest
All this is hard to explain without returning to Sentient’s idea of ‘evolutionary algorithms’. Basically, it’s all about survival of the fittest. You reach the ‘fittest’ version of your website by testing large numbers of ‘mutations’, or changes to design – but with standard A/B testing, those tests need to run over an extended period of time before it’s possible for humans to identify the ideal combination of mutations.
With AI, however, it’s possible for the tool to very quickly identify those mutations that look most promising and then combine them, rapid fire, with other promising mutations for further testing.
In this way, Ascend uses machine learning to reach a faster understanding about the likely results of ‘cross-breeding’ different design elements to reach a superior end result. Sentient claims that Ascend enables marketers to carry out new website experiments between 10 and 100 times faster than existing A/B and multivariate testing tools.
Cosabella discovered, for example, that bigger images helped with conversion rates - no surprise there. But it also found that pink buy buttons worked better than black ones, even though black ones were originally used to give the brand an added feel of luxury. But no: customers preferred pink.
And, in the site header, intended to convey an important but snappy message about the brand, it found that communicating about the company’s heritage worked better than messages about free shipping, luxury fashion, different colours each season or Italian manufacture. That’s how it arrived at the header: ‘Family owned since 1983’. Says Campello:
Given all the attributes that convey who we are as a brand, we were able to determine which our customers actually care about and which emotionally appeal to them. Turns out it’s our ownership and age, that’s what matters.
Ascend has made a big difference for Cosabella IT manager Thomas Lyne. Like most A/B testing tools, he says, it’s simply a matter of him adding a piece of code to every page to prepare it for testing:
After that, our biggest challenge is getting together enough ideas to test at once, because there are so many ideas we can test simultaneously and you want to pack in as many as you can.
It’s made a big difference too to the way that Cosabella now thinks about future testing, Guido Campello adds:
I’d say our first goal is just to keep testing. Fashion’s a seasonal business, but we can test so many elements now, so quickly, that it’s a very different process to make things happen. In the past, we might have tested in Spring and then put the results into effect for the next Spring season, a year later. Now, we’ll test our ideas for Spring in January and put them into effect for February and beyond.
Our team went from having to be driven by instinct, ideas around aesthetics and some basic testing to now really being in control and able to determine the best actions quickly, to create constant new ideas and put them into effect where they’ve been shown by testing to work. We’ve all got involved in brainstorming and making suggestions. We’re empowered now: we have an idea, we set out to prove the theory and, where it’s proved, we put it into effect in a matter of weeks.