RE: Usability Testing--Interim (I hope) summary

Subject: RE: Usability Testing--Interim (I hope) summary
From: Tom Johnson <johnsont -at- starcutter -dot- com>
To: "'Tom Murrell'" <trmurrell -at- yahoo -dot- com>, TECHWR-L <techwr-l -at- lists -dot- raycomm -dot- com>
Date: Wed, 5 Jul 2000 08:33:39 -0400

Tom,

My point was that hundreds of people are using the procedure and getting
the job done without problems. That should tell us the documentation is
working and is as valid as any formal testing.

One customer couldn't use the procedure because they were trained using a
different method. Let me explain it this way. Suppose I give you driving
instructions to get to my house. This is the same set of instructions that
I've given to dozens of people and they all find my house without
difficulty. Now you have a friend who offers to ride with you and he guides
you to my house by way of back roads. Some time later you decide to return
to my house on your own and your memory isn't perfect. You start off trying
to follow the directions and you realize you're not making the same turn
you made before. Instead of following the instructions, you try to go the
route you drove before. Then you find yourself hopelessly lost and the
directions don't make any sense because your not in the correct starting
point. The fault that you're lost has no bearing on the quality of the
instructions you were given. It is because you decided the instructions
were wrong and chose not to follow them in the first place.

As far as our manuals being in maintenance mode and not conducting testing
for minor changes. It would be about the same as the name of a road being
changed (in my example above) and I updated the manuals to reflect that
change. So if I told you to turn left on Hammond Road and the road's name
is changed to Johnson Road, I would just change it to "Turn left on Johnson
Road." There's hardly a need for more testing.

No, I don't think we are doing it as a bureaucratic exercise. In the case
that I pointed out, our manuals are being tested on a daily basis by real
users. We had one complaint, we looked into it and found out where the
problem was.

When we produce NEW documentation, we train real users in our plant and we
watch them use the manuals. We quickly spot problems and change them almost
on the spot. It only takes a couple of groups going through training before
we have most of the bugs worked out and we go into maintenance mode for the
next couple of years. Any time we have any kind of a complaint about the
manuals, we look into it. Those complaints are rare and our clientelle is
not shy about letting us know when they need any kind of help or have
questions.

I guess my point is that we keep close tabs on how well our customers are
doing with our products. We listen to their complaints and documentation is
the result of very few complaints so we should be able to assume our
manuals are usable. We conduct training sessions on a regular basis and
these newbies are generally able to complete tasks without difficulty. We
try to look at complaints as an opportunity to make improvements. I think
our testing is a success story. We don't routinely sweep things under the
carpet to make ourselves look better and most of our business comes from
repeat customers.


On Tuesday, July 04, 2000 10:37 AM, Tom Murrell [SMTP:trmurrell -at- yahoo -dot- com]
wrote:
snip>
> Tom Johnson says that his documentation is all in maintenance mode, so
they
> don't do testing for minor changes. Tom does report a documentation
problem,
> but it turns out to have been a service rep. who arrogantly insisted that
his
> way was better than the documented way. So we conclude that the documents
were
> okay after all.
>
snip>
> Forgive me for seeming so cynical, but if we are writing to meet the
needs of
> an audience, should we not be able to (a) define those needs, and (b)
test our
> documentation to determine how well we are meeting those needs with an
eye
> toward doing even better as we move forward?
>
> Apparently, where usability testing IS occurring, it is a mere
bureaucratic
> exercise. You all have made me feel at least honest in not pushing it. It
> sounds like a waste of time.
>
> Does anyone have any success stories for usability testing?
>
> =====
> Tom Murrell
> Senior Technical Writer
> Alliance Data Systems
> Columbus, Ohio
> mailto:trmurrell -at- yahoo -dot- com
> http://home.columbus.rr.com/murrell/


Tom Johnson
Technical Writer
Elk Rapids Engineering Div., Star Cutter Company

johnsont -at- starcutter -dot- com - work
thomasj -at- freeway -dot- net - personal





Previous by Author: Usability testing
Next by Author: RE: Making Word documents dead-only?
Previous by Thread: Re: Usability Testing--Interim (I hope) summary
Next by Thread: Front Page ...


What this post helpful? Share it with friends and colleagues:


Sponsored Ads