RE: Ideas in Motion

["Mike Starr" <writstar -at- wi -dot- net>]

Well, having specific metrical targets is a nicety that I've never had the
luxury of and from what I can fathom of it, they don't help me do my job,
either. In the world I've lived in, the metrical target has been the ship
date. The development team keeps coding and I keep on trying to put
together a document that provides the best possible documentation for the
user before they rip it from my hands and ship it. I've never been
completely satisfied with what I've had to ship because there are always
things that I've wanted to include but was prevented from doing so by the
ship date.

I've never concerned myself with having the boss say "Hey, I like it!"
because I can do that with one hand tied behind my back. Most of the time,
the boss has started to say that well before I'm even remotely satisfied
with it. 

The metric I tend to employ is this: is there a question the user can ask
about this product that I haven't provided an answer for? If the answer is
yes, then I'm not satisfied.

I've been involved in usability testing and I was completely dissatisfied
with it. I wasn't given any input into the design of the usability testing
at all. The usability folks completely disregarded concerns that I had. No,
thanks. You're right, though... usability testing isn't arcane or hellishly
expensive but then again, I don't consider the results of eight to ten
subjects anywhere near statistically significant, either. Now if that
number of subjects all exhibit the same difficulty then of course there's
cause for investigation. However, I object to a sample of eight to ten
subjects being used to determine things that are much better measured by
true statistical analysis. I've seen usability testing used to make
decisions about a product with sales of 30,000 units per year based on
testing five subjects, three of whom weren't even close to being typical
users. This by a major corporation in its industry.

With respect to heuristics, it's not a term I'm familiar with so I'll not
address it. What I will tell you is that I rely on the product knowledge I
develop during the documentation process as well as my own
seat-of-the-pants guesstimates of what's the right thing to do. I'm happy
with that and the end users I've rubbed up against have been happy with my
product.

Mike

Mike Starr - WriteStarr Information Services
Technical Writer - Online Help Developer -  Technical Illustrator
Graphic Designer -   Desktop Publisher   -       MS Office Expert
Telephone (262) 694-0932 - Pager (414) 318-9509 - mailto:writstar -at- wi -dot- net

-----------------------Original Message-----------------------
From:  Tim Altom
To:  Mike Starr;TechDoc List
Date:  3/12/2000  9:05 AM
Subject:  Re: Ideas in Motion

All specifications are subject to change, but that's in the details. You
add, subtract, rethink. That's to be expected.

At the same time, every project should have specific metrical targets to
hit, because otherwise projects piddle off into the sands rather than having
a stopping place. Even ongoing projects, such as updates and improvements,
need milestones that are measurable.

I say again, with feeling...if a manual is produced without metrics for its
outcome, you do not know if you have done your job. The boss's subjective
impressions don't count. They may be important within the context of the
company you work for, but that says more about the company than about the
manual. If the standard for performance is the boss's "Hey, I like it!",
then you are not even being given the same potential for meeting standards
that's awarded to assembly workers or janitors. You're expected to produce
art, not measurable communication.

There's no reason why a manual can't have measurable performance metrics. A
simple test might be "During usability testing, users must be able to locate
applicable assistance for unfamiliar tasks within X seconds". Write such
metrics to factor out the difficulty of the product. Observe only how often
a user flips from page to page, tinkers with menus, gets frustrated and
clicks out of the help file before completing the task. Set actual targets
of seconds elapsed or pages flipped.

Usability testing isn't arcane and it isn't hellishly expensive. One week,
eight to 10 test subjects.

In the absence of testing, you have to rely on heuristics. If that's the
case, then in my view you're obligated to have the very best heuristics,
which come to us courtesy of research and reading. This isn't ruining the
profession, as some have said on this forum; it's enhancing it.

If a writer has flimsy heuristics and no testing, then how do she know she's
doing anything right? Because users don't go on rampages, thirsting for her
blood? Users are all too familiar with documentation that's more suitable
for lighting fires than for reading. They're tragically used to it, so much
so that really good doc excites them. We'll know we've become a great
profession when fabulous doc is so common that substandard doc cause a howl
of outrage.

Tim Altom
Simply Written, Inc.
Featuring FrameMaker and the Clustar Method(TM)
"Better communication is a service to mankind."
317.562.9298
Check our Web site for the upcoming Clustar class info
http://www.simplywritten.com


> I have never produced a manual that conformed to what I was asked to
> produce at the start of the project. Now, some may consider that to be
> just cause for banishment but in every case, the person who asked me to
> produce the manual didn't really know what was needed. It takes a lot of
> blood, sweat and tears as well as the occasional discreet application of
> the Official Technical Writer's 2x4(TM) to convince the powers that be
> what a real manual should and shouldn't contain.