Growth Giant A/B testing tool
Growth Giant is an A/B testing tool that dynamically directs traffic to better performing pages to ensure that you get the maximum conversion rate.
The challenge
A/B split testing tools are fast becoming a popular way for companies to improve sign-ups and sales on their websites. The way it works is pretty simple. Let’s say that your current web page (version A) gets an average of 100 sign-ups per month. You then design a new web page (version B) that you believe could increase sign-ups. A/B split testing tools basically split your visitors in half, with half going to version A and the other half to version B. The number of sign-ups is recorded against each version and once there is enough data, a winner is determined (if it exists).
The downside to this is that half of your visitors are being sent to a poorly performing page for the duration of the test, which could be months. What if you didn’t need to lose those sign-ups? What if traffic allocation could be dynamically changed during the test, with more visitors being gradually sent to the winning version? The below diagrams compare regular A/B splits against dynamically changing splits.
A few friends and I stumbled across an algorithm that could be used to dynamically allocate visitors in an A/B test. It uses multi-armed bandit theory to gradually divert more and more visitors to the winning version, rather than being stuck with a fifty-percent split for the duration of the test. We saw an opportunity to create our own A/B testing tool and thought that it could be a fun challenge.
Research and discovery
We had been using A/B testing tools for quite some time, so we knew what worked and what didn’t. Actually being a user of the product you’re creating is great, as you don’t have to spend as much time on user testing and gathering requirements. This tool would be targeted towards people like us who had the resources to create and edit their own website, rather than those with limited technical knowledge.
We wanted users to be able to create an A/B test with minimal friction, so we created the shortest user journey we could (as shown in the diagram above). We assumed that the users would already have their new web page on a live URL, so they would basically just need to input the URL of their current web page, the new web page they wanted to test and the URL of the conversion goal, e.g. the sign-up page. Users would then simply need to copy and paste a small snippet of JavaScript into their website header to allow us to collect data and divert traffic accordingly.
Sketching wireframes
After checking out the competitors and figuring out the information architecture, it was now time to sketch out some wireframe ideas (above) for the three main parts of the tool: the dashboard, the create test funnel and the test details page. The function of the dashboard was to simply list A/B tests and allow users to create a new test or edit an existing one. The test creation funnel needed to be quick and easy so we tried to keep the process as short as possible. The biggest challenge was the test details page, as we wanted to design something very different to most A/B testing tools.
As I worked on the wireframes, our developer started planning the build. Our initial goal was to get a basic version of the app working so that we could test it out on one of our own websites and then on some potential users. We used the initial wireframes to help write out user stories and prioritised them to ensure we stayed on track.
Pushing pixels
Once we were happy with the wireframes, it was time to start on the visual design. This is one of my favourite parts of the design process as you can finally see things come together. I already had a pretty good idea of the aesthetic I wanted to achieve from working on the branding earlier on. It needed to be bright and fun with a strong focus on content. I liked the idea of laying white boxes on a grey background to help group elements on the page.
I put together the above UI style guide containing the main elements I needed for the design. The colour palette consisted mainly of shades of teal, but I used a complementary bright orange colour to emphasise important elements.
For the dashboard design, I used the wireframes as a guide and applied the styles from the style guide. I didn’t want users squinting to see the interface, so I exaggerated the size of typography and click targets. Bright orange buttons and teal-coloured links worked well together to create visual hierarchy.
Forms are scary
Nobody likes filling out forms, so my main focus when designing the create test funnel was to make sure the form wasn’t scary to fill out. I kept form inputs to a minimum and stacked them to make the form easier to scan. I also centre-aligned the form and removed any unnecessary distractions.
Once an A/B test was running, users would be monitoring their results on the test details page below. Unlike most A/B testing tools, which simply compare two pages, we wanted Growth Giant to be more focused on optimisation of your web page as a whole over time. So the average conversion rate of the whole test is graphed at the top of the page. The performance of individual page ideas can also be seen under the graph, with better performing pages gradually receiving more visitors. New web page ideas can be added to the test at any time and better performing ideas will bubble to the top automatically.
Some things just aren’t meant to be
Although we got pretty close to building an initial beta version of the application, work, life, time constraints and priorities got in the way and we had to abandon the project. It was great fun working on it though and we learned a lot in the process.
