A/B Testing talks

A/B testing, also called split testing, compares two or more versions of a page, screen or flow by showing each to a share of real users and measuring which one performs better against a chosen goal, such as sign-ups, purchases or task completion. Instead of arguing about which design is better, a team lets user behavior settle the question.

For UI, UX and product design work, A/B testing turns design decisions into experiments. It helps teams check whether a change actually improves an experience, catch changes that quietly hurt it, and build up knowledge about what their users respond to. Good tests depend on good hypotheses, enough traffic and careful measurement; a poorly set-up test can produce confident but wrong answers.

The two talks here, both from 2015, look at the quality of testing rather than the mechanics. Karl Gills explains how to create better A/B tests based on user research at UX Poland, linking qualitative insight to what gets tested. Craig Sullivan’s talk at conversions@google, The Myths, Lies and Illusions of Cross Device Testing, examines the pitfalls of testing when people move between devices. Persuasive patterns in the UI-Patterns.com library, such as framing and social proof, are common subjects of this kind of experiment.