How to create better A/B tests based on User Research

Karl Gills, founder of AG Consult and an expert in optimising websites through A/B testing and quantitative user research, gave this talk at UX Poland in 2015. A/B testing is great, he says, but if you test every idea that pops into your head, you can end up doing a lot of testing and facing bitter disappointment.

Gills shows A/B tests that made a real difference because they were based on user research: analytics, targeted surveys, user session recordings and user testing. His cases come from car brands, web shops, luxury products and health products. Among the examples: why big pictures in the header area aren’t always a good idea, how explaining how personal data will be used encourages people to fill in forms, why repeating calls to action at the end of a page makes sense, and why it is sometimes better to promote product categories than individual products on e-commerce sites.

With tools like VWO and Optimizely, A/B tests are easy to run, but Gills stresses planning tests in advance. Every site is different, so copying someone else’s tests is pointless. Online surveys can help you understand user expectations: instead of asking for opinions, simply ask, “What is the purpose of your visit today?”