Observation Techniques

[Bill Sullivan <bsullivan -at- SMTPLINK -dot- DELTECPOWER -dot- COM>]

Lisa asked:

>I would like to see a discussion concerning interviewing and =
observation techniques.  I have been to several workshops on how to =
interview for requirements gathering and on usability labs.  I am =
interested in hearing about your experiences and training in these
areas = because I do think listening is an oft overlooked skill.

I have run a few usability tests; not enough to claim to be an
expert, but enough to have gained a feel for the experience.  They
are easier talked about than carried off smoothly.  I suppose the
biggest lesson I have had to learn is to shut up and let the test
subject carry on with the task at hand.  It is easy to forget that
the thing being tested in a usability test is your product package.
If your test subject can't figure it out, it's the product that
fails, not the person doing the testing.  As the person running the
test, you should bite your tongue, shut up, and take notes on what
happens.  I personally have had to learn this, and it may be correct
to say that I am still learning this.

A test is a lot of work.  You have to plan the thing, and figure out
what tasks you want performed.  You have to select test subjects.  I
used a combination of new hands and older hands at the company where
I work.  You have to reserve a room and set it up.  You have to get a
video camera and tape, and learn how to use it.  Then you have to dry
run the thing to be sure your equipment and software work, and in a
place like ours with a dozen different versions running around this
isn't that simple.  In my case, I had to order pizza delivered.  We
did this over the lunch hour.  Then, after the test, you have to
strike the set, get the video camera tape converted to VCR tape, and
then you have to sit down and evaluate what happened, creating a test
report.  I held a screening for all of my fellow writers and the
marketing writers.  Also, I found I had to put on a sort of what
football players call a game face because the office scuttlebutt
picked up that I was doing these things, and the word got out that I
had videotape of people looking not always too cool, and something
told me it better if people treated these things as serious
professional exercises and not jokes.  And then, of course, the real
hard part: finding out something meaningful about the product and
carrying it to engineering and getting them to modify the product.
Oh yeah, sure.  Other than that, they're a pretty simple thing to do.

One other thing I have learned is that I as test coordinator learned
a lot.  To anyone wanting to do these things, I would strongly
recommend doing a few as practice.

Bill Sullivan
bsullivan -at- deltecpower -dot- com
San  Diego, California