Greg Niland On Testing: Not So Excellent

Tags:

June 2, 2007

Since it’s the weekend, I thought I could rant a little about Greg Niland’s recent podcast on testing your website. But first, a few nice things about Greg:

  • He has the world’s most adorable laugh. And it is even more fun to hear it over the phone after you have heard him do it on fifty podcasts
  • Unlike his SEO peers, he does a podcast for newbies. Most advanced web marketing professionals, be they in SEO or web analytics, want to show off their advanced techniques.
  • He devotes a lot of his time to charitable causes.

But on to the main attraction. I only this week had a chance to hear his podcast with Shimon Sandler. The exact topic was “Making money the Unfun way.” Hmm, I thought as I started to listen. What will be so unfun here? Rewriting URIs? Playing with your robots file? Canonical issues?

Oh, I was wrong. The major unfun issue they wanted to talk about was testing.

Testing? Unfun? Well, I guess what is fun to one person might be boring to another.

But then guest Shimon Sandler, who is an SEO, put testing into two categories, A/B and Mutivariate, “also known as Taguchi.” Really? Greg asked, I’ve never heard of multivariate testing called Taguchi. Maybe multivariate testing was designed by someone named Taguchi, Shimon answered.

(For the record, very simplified: the Taguchi method is one way of decreasing the number of page views and conversions your test needs, as opposed to a full-factorial analysis, which uses every variation. Offermatica uses Taguchi; Google Website Optimizer uses a full-factorial analysis. The Taguchi Method was designed by someone named Taguchi, but as a way of applying statistics to manufacturing.)

Sandler rattled off a few names in the MVT field, but forgot (or didn’t know) most of the important ones. He knew Offermatica, Vertster and at least one other, but missed Optimost, Sitespect and Website Optimizer. And then, the two of them talked about what you could test. But being SEOs instead of conversion “scientists” or MVT jocks, they could only come up with SEO-type things to test. Your PPC ads. Your urls.

Well, everyone who works in testing knows that to make a difference, you should start by testing BIG things. Your headlines. Your buttons. Your shopping cart. Your call to action.

But what was I expecting? Greg needed to have someone from the testing world on that show. How about Sean Purtell from ROI Revolution? How about Ophir Prusack? How about Bryan Eisenberg?

Well, in closing my rant, I do want to give credit to Shimon Sandler for addressing the web analytics issue. When I called Greg and pitched him on doing a show on web analytics (back when our company wasn’t a Google Analytics Authorized Consultant, life was sane and I didn’t spend every weekend working), he told me that his listeners “just aren’t interested in logfiles.” (I tried to explain that not all WA is server side, but I don’t think he cared.)

Robbin