It is interesting how counter-intuitive A/B testing results can get. One of the best examples of that counter-intuitiveness is a recent A/B split test. This test was done by a Visual Website Optimizer customer Vendio, an Alibaba.com company that specializes in providing free e-Commerce stores to its merchants. They have special landing pages for the free store signup and the objective of this test was to increase signup conversion rate on one of those landing pages.
A/B test details
Their original landing page (control) combined marketing content and registration fields in an attempt to reduce the number of clicks for a successful registration. This is how it looked:
Original Landing Page (with embedded signup form)
Note that it uses the so-called “best-practice” of embedding the signup form in the landing page itself. They had long been using the layout of the original page because as a best practice they presumed that reducing the number of clicks for a registration increased conversion rate. Although the page was performing relatively well, they wanted to make sure the included registration fields weren’t too aggressive or limiting us in any manner.
So, the variation that they tested had somewhat similar imagery and content but the page didn’t include any registration fields and had slightly different styling. Clicking on ‘Signup Now’ button simply took the visitor to a page with signup form. In other words, Vendio added an extra step in their conversion funnel. Not a smart move, huh? This is how variation looked like:
Variation (no signup form) – 60% increase in signups!
A/B test results
Guess what? The page without the registration fields performed better – much better – to the tune of a 60% increase in conversions! Here is what Vendio had to say about the results:
Best practices are NOT always true! It’s still hard to believe, but the numbers don’t lie.
So, that’s the biggest lesson here and it is worth repeating: “Best practices are NOT always true”. If you do changes on your website or landing pages without A/B testing them, you are actually flying in the dark. Another lesson here is that it is it is worth testing radically different ideas – which, on the first glance, may appear not-so-smart (like removing signup form from the landing page!).
Multi-variate Testing or A/B testing is a very important tool for any performance campaign. As the article shows, best practices may not always be true for your campaign. We successfully used this approach in a performance campaign for a leading bank last year and found the ‘winning’ webpage had a CTR of 20% more than the average.
87% Muscle, 100% Awesome 