Five Second Tests: An Open Letter To Jared Spool

Dear Jared:

Before I say how much I disagree with your recent comments on five second tests and homepages, I should point out that I love your new series of podcasts. Not the stuff you do with Josh or the seven other usability experts who pontificate about what Brown University should do. That other stuff was so boring that I deleted you from my iPod. (Do I win the award for the Queen of Criticism or what?) But as luck would have it, I bought a new iPod and “made” my daughter set it up for me (I am just too busy) and she set it up to get ALL my podcasts. So I was surprised and delighted to hear that you and Christine Perfetti are doing a “usable” series on usability — stuff that I can really use. I listened to home pages and store pages and gallery pages. And then I heard you talk about five second tests.

(Now, for the other thousand people who may eavesdropping: a five second test is when you show a user a page for five seconds, take it away and ask them, “What does this page do?” Reason being: that’s about what users give a page, five seconds, before they decide that they are in the right place or not. It’s fast and cheap and helpful.)

But Jared – I think you are wrong when you say that five second tests aren’t helpful with home pages. I use five second tests with users to test home pages all the time.

In your podcasts, you said something like, “Users already know what the site is about. People don’t need to know what the homepage of UPS is all about, they know what UPS does, they go there to track a package, for example.” Well sure, that’s true if the site is UPS, or JC Penney, or Fidelity Investments, or your favorite site, the Bureau of Labor Statistics. But do you really believe that all municipal sewer engineers are familiar with RedZone Robotics? Do you think that every realtor knows HomeFeedBack? These non-Fortune-500 sites may be found in the organic listing (by their home page) and the user will give the site — what, maybe five seconds? — before deciding to bail. Bloggers and linkers like me almost always link to a homepage, and if someone follows a link, they may end up on a homepage of a site they don’t know, giving it what — five seconds? — before deciding to leave. Or not.

I have yet to start a user test with a five second homepage test and hear the user say, “Oh them. I know them.”

You also pointed out that when someone goes to a site with a specific purpose — like finding instructions on their HP printer — they only care if they can find the link to their printer. They don’t care if the page “looks” like a company that sells printers and computers.

But the example is somewhat lacking. It’s really not any different from the UPS example – the user has a task to complete and knows what site he should complete it on. What about when you are searching, when you are following a link, when you are typing a link in that you heard on the radio?

Recently, we did some user testing on a site that sells do it yourself pieces to construct parts of your home. As part of that test, we did a five second home page test. “I would use this site if I wanted to have the company do the construction for me,” users said, almost uniformly. They were all really surprised that the site was a place to buy the parts so that they could do the construction themselves. In retrospect, it’s clear as the nose on my face, but my nose was way too close to that site. Now, we are going to test the pieces of the home page that gave the users the wrong impression.

All because we did five second home page tests.

Robbin