What to ask guinea pigs? (Usability testing user guides)

["Hart, Geoff" <Geoff-H -at- MTL -dot- FERIC -dot- CA>]

Neha Sharad <<Just found out that the powers-that-be WANT to test out the
user guide  (which is still in progress) during an informal training session
tomorrow.>>

Marvelous! Here's hoping you can make this standard operating practice. That
would be a great precedent.

<<1. Make the user guide the agenda for the training session. These are the
topics we're going to cover...  The instructor should ask himself whether
the organization flows smoothly, are important topics covered, are we at the
right level of detail, is content accurate, are participants' questions and
concerns are covered in the book?, not covered in the book?>>

This depends strongly on what type of guide you've created. If it's a
tutorial, then it will probably work well with an instructor-led
presentation because both the manual and the class try to accomplish the
same goal: teaching people to use the software. If it's a reference manual,
it won't work nearly so well because reference material isn't designed to
support learning; it's designed to support performance of tasks. In that
case, your instructor must create an agenda that both accomplishes the goals
of the training session (i.e., training people to use the product) and lets
you walk the students through using specific parts of the reference material
(i.e., asking them, as part of learning certain procedures, to evaluate the
description of the procedures). It may not be easy to reconcile these two
needs, particularly on short notice.

<<2. Before beginning, ask participants to write down as many questions as
they can about the product, how to use it, troubleshooting, etc. Be as
specific as they can. Then I can compile a list of common questions and make
sure they are included in the guide (and maybe include it as an FAQ?).>>

A great idea, but don't stop at written comments; turn it into an open
brainstorming session. Although this may intimidate some of your shy users,
they'll still have the chance to make their opinions known by writing them
down, but the great advantage of brainstorming is that one person's question
may inspire three other people to ask different questions (or to confirm
that this question concerns more than just the person who asks it). If you
expect a large or enthusiastic class of learners, consider using a tape
recorder or a couple fast typists to be sure that you record all the
comments. Don't just create an FAQ, since these documents are inevitably
difficult to use: they're never indexed, introduce questions and answers in
seemingly random order, and force readers to skim the entire document just
to find what they're looking for. Instead, use any patterns in the questions
and how they're asked to give you an understanding of how to structure
introductory material (including a tutorial) and gain insight into the types
of things your audience doesn't understand before they've been trained. Pay
attention to their word use, their misconceptions, their fears, their hopes,
their whimpers of fear, their laughter, their silences, etc. These can all
be clues that something important just happened: for example, if all the
joking stops and everyone pays close attention when someone asks about
viruses and your software, you know this is important to them and you'll
have to say something about it.

<<3. After the session, ask participants what questions/concerns were not
covered in the session and/or in the guide. Make sure these are added in the
guide.>>

Since they're coming to be trained, albeit informally, you'll probably hear
lots of concerns about the quality of the course. So make sure that you
clearly indicate before they begin that part of the goal of the training
session is to critique the docs, not just to learn about the product. (In
fact, this is a great time to spread the seditious message that the docs
_are_ part of the product. <g>) If the students are primed to expect this,
then they'll be less annoyed that the training was "interrupted" by all this
documentation stuff. They'll also greatly appreciate the efforts you're
making to consider their documentation needs, even if they don't jump up and
start cheering--it's one of those subliminal "they actually care what we
think" messages that you're sending.

<<4. At the beginning of the session, ask participants to use the guide to
find specific information about the product, and assess how easily they can
get at the answers.>>

That's an important thing to test, and particularly so if you can somehow
arrange to tie it in with the FAQ you proposed. That is, rather than
arbitrarily deciding what _you_ want them to try to find, use their
questions as the basis of the test. Can they find information on the things
that concern _them_? That's the true test.

--Geoff Hart, FERIC, Pointe-Claire, Quebec
geoff-h -at- mtl -dot- feric -dot- ca
"User's advocate" online monthly at
www.raycomm.com/techwhirl/usersadvocate.html

"The most likely way for the world to be destroyed, most experts agree, is
by accident. That's where we come in; we're computer professionals. We cause
accidents."-- Nathaniel Borenstein

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

*** Deva(tm) Tools for Dreamweaver and Deva(tm) Search ***
Build Contents, Indexes, and Search for Web Sites and Help Systems
Available now at http://www.devahelp.com or info -at- devahelp -dot- com

Sponsored by Cub Lea, specialist in low-cost outsourced development
and documentation. Overload and time-sensitive jobs at exceptional
rates. Unique free gifts for all visitors to http://www.cublea.com

---
You are currently subscribed to techwr-l as: archive -at- raycomm -dot- com
To unsubscribe send a blank email to leave-techwr-l-obscured -at- lists -dot- raycomm -dot- com
Send administrative questions to ejray -at- raycomm -dot- com -dot-  Visit 
http://www.raycomm.com/techwhirl/ for more resources and info.