I recently attended a half-day seminar on best practices for handling redesigns of websites. The seminar itself wasn’t of much value – the only thing I learned is that we rock in our approach to building websites and their relaunches at my work, Benjamin Interactive.
The attendees at the seminar all reminded me of university professors engulfed by their scientific background. There are many ways of conducting user studies and tests. The “correct” way of conducting such investigations according to the academia is to ensure scientifically proofed data. This includes collecting enough data for the findings to be empirically true.
That means not conducting just one or two studies, but 10 or 20 – or even more. That means renting or building expensive user-test labs. That means hiring expensive experts to ensure unbiased data and maybe much more important: adding a difference between who is building a product and who is testing it.
This approach seemed to be the “correct” way of conducting user research when talking to the attendees at the seminar. I must have provoked more than a dozen when I told them all that I couldn’t care less for that kind of approach.
In my view, user studies should be conducted by the people building the product. A developer is not only a developer and neither is a designer. Developers and designers – the ones building the product – should know who they’re building it for – in person. Having somebody else do it for them allows for double interpretation of who the users are: one by the user experience experts writing it down on paper, and another by the developers and designers reading that paper.
Having studied HCI at the university, I’ve both tried the “correct” way of doing user research and a more loose guerrilla HCI approach.
In my view, the most valuable thing to gain from user research is feedback. The more rapid the feedback the better. From my experience, I haven’t gained much more knowledge from conducting user research the “correct” way than from grabbing a victim in the cafeteria for a 30 minute session.
My goal with user testing is to be able to rapidly test what I have just been building, correct what I’ve build with base in what the user test showed, build something more, test that, correct it, build some more, etc.
Have any of you out there experience situations where the “correct” way gave better results than the budget way of doing user tests? I can’t find any other reason than bureaucratic and organizational ones.
2 comments
Warren on Apr 21, 2009
Maybe you’re designing too much stuff for people like you. I pretty much always find it useful to talk to real users and particularly the case of tightly targetted sites/apps. I don’t see that it necessarily has to be expensive, it just means that it takes a little planning and effort – a rare thing in this modern age.
Anders Toxboe on Apr 22, 2009
@Waren:
I disagree. It is my experience that up until a certain point, the main goal of user testing is just to make everything flow much better. It’s about getting basic functionality correct and fix obvious bugs that you’re too involved in the project to notice yourself.
When these bugs have been wiped away, it is time to bring in the heavy artillery and go for user test sessions in special laboratories with totally unbiased users from the absolutely correct target group.
The problem is just, that not many companies actually get this far. They use their expensive usability sessions to wipe out stupid bugs – or to find out that their product suck. If they would just get past that point of correcting “obvious” things first, they would get so much more value out of their expensive sessions.
So my point is just that with the first X user test sessions, you’re most likely better off conducting a cheap and fast user test than going for the expensive one.
Comments have been closed