Video Screencast Help
Symantec to Separate Into Two Focused, Industry-Leading Technology Companies. Learn more.

A/B Test Shows Being Number One and Message Matters

Created: 07 Sep 2010 • Updated: 03 Jun 2014
Reshma Kumar's picture
0 0 Votes
Login to vote

In an effort to optimize the effectiveness of the homepage banner, we recently did some testing to maximize response rates. The homepage banner is a multi-message banner with four 'buttons' each representing four individual messages.

The Goal
The goal was to test which sequence of buttons was the most compelling and therefore, garnered the most clicks. We were not testing the copy, layout, imagery, or color palette as part of this test; this was previously done in usability testing.

The Test
Along with the control, we deployed different versions of the banner with the order of the first button being displayed varying. What we found was that the button positioned first (from left to right) always got the most number of clicks - likely by virtue of being the first and easiest and quickest to access, and the other three buttons got varying numbers of clicks.

The Results
The real-time results were interesting as they usually are. Of all the permutations, when the "Protect Yourself Online" button was displayed as the first button in the sequence, it not only got the most number of click-throughs but this is the offer that got the second most number of clicks in the test cells - in essence, producing the most improvement. As a result, we have launched this version of the banner which you can see live today on the homepage.

What this demonstrates is that sequence does matter! And in this case, the number one position is what matters. To get something noticed and clicked on, putting it as number one where users are reading from left to right, was the magic order - no real surprise here. The other conclusion, of course, is that users like some messaging/offers better than others. While users consistently clicked more on the first button displayed, they clicked more on the winning message - Protect Yourself Online!

Homepage banner
(Note: In case you click and don't see it, we are running a different test.)