Conversion On A Limited Budget: Users Vs Experts

Yesterday, I was talking to a potential client. Like most, he wanted to improve his conversion rate, and like even the biggest companies, he had to live within a budget. We discussed the pros and cons of doing user testing with 5-8 users vs. my doing an “expert” conversion analysis. The costs weren’t different enough to be a consideration.

“Don’t take this the wrong way,” he said, “But I highly doubt that I am going to learn anything with 5-8 users.”

Now, at some level he was absolutely right. Five to eight users would not be statistically significant. Nor, for that matter, would fifty or eighty.

So would one conversion analyst be more statistically significant? No. I can say, “Studies show that making the button red tends to work better,” but it might not work better for your site. Plus, it is incredibly hard for a best practices person to discard the “professional blindness” that knowing websites brings with it. (This is the reason that I don’t allow web designers to sign up for our user tests – same problem.) Web analytics are statistically significant, but we only know that everyone bails out on a page, we don’t know why. Multivariate testing is the best, but you still need to know what to test.

This is why I love user testing. It’s not that expensive, especially if you do it yourself. Sure, you won’t learn everything you always wanted to know, but when you hear four people out of five tell you that the deal isn’t good enough for them to be interested, you sure do know what to test.

Speaking of which, I think too many people think user testing is just usability. Usability does go hand in hand with conversion, but you learn about pricing, about trust, about what people notice on the page. And a great idea (I owe this to a LunaMetrics customer) is to do user testing on your competitors’ sites. That’s when you learn where they have elements that are worth testing on your own site.

Robbin