Evaluating help files?

[Geoff Hart <ghart -at- videotron -dot- ca>]

Carrie Baker wondered: <<The company are trying to implement a  
process of measurements. For our department our boss likes us to  
deliver surveys, get the answers, and send the same survey out next  
year and get better results!>>

You have my sympathies. <g> On the other hand, the underlying concept  
(trying to improve what we do) is good.

<<However, it seems that my non technical writer boss has been asked  
for us to send out a survey on our help files... I think that I will  
have to write and distribute a survey about our help files. (i.e. are  
the help files helpful:-) ) Do people have any suggestions of useful  
questions to ask, or can tell me whether such a survey already exists?>>

How about thinking about the problem from the other side? Instead of  
asking "is our file good?", why not ask "how do you use the Help, how  
easily can you use it to solve your problems... oh, and by the way,  
what are those problems?" This turns an exercise designed to earn you  
a pat on the back ("we earned a B+ for our manual") into a form of  
usability testing: you identify the problems that need to be solved,  
then pick a few people to test the results to see whether you  
succeeded. The goal then becomes continuous improvement of usability,  
not getting a better grade. (After all, once you get an A+, your next  
performance appraisal will say "no raise... you didn't improve your  
grade!" <g>)

For example, you can time a group trying to find a specific topic  
using the old files, and time a second group (matched for computer  
experience and work background) using the new version to see whether  
the second group succeeds more often or faster. You can then time  
both groups trying to actually accomplish the task once you've shown  
them where that part of the Help system lies. What tasks should you  
test? The ones that account for the majority of the calls to  
technical support! This has a huge additional advantage: if you track  
the number of calls related to a specific task before you made the  
change, and track the number afterwards, and can clearly show a  
decrease, you've just proven your value to the company. Not a trivial  
accomplishment!

This kind of testing is a bit more complex than that -- usability  
studies require some knowledge of experimental design, and there are  
various "best practices" I'm not up to date about. But it's a far  
more productive approach because you're getting at the real goal  
(solving problems) rather than measuring satisfaction with the  
documentation. If you want to learn more, STC's Usability SIG (http:// 
www.stcsig.org/usability/) would be a good place to start,  
particularly if you're already an STC member.

Ideally, you want to work with a diverse range of your real  
customers. Realistically, unless you already have a well-entrenched  
culture of usability, you may need to use proxies such as office  
staff who aren't involved in development of the product, but who  
might realistically be expected to use it. If you can demonstrate  
good results with this group, and can demonstrate savings in  
technical support calls, you can make a business case for reaching  
out to test actual customers in the future.


----------------------------------------------------
-- Geoff Hart
ghart -at- videotron -dot- ca / geoffhart -at- mac -dot- com
www.geoff-hart.com
--------------------------------------------------
***Now available***  _Effective onscreen editing_
(http://www.geoff-hart.com/home/onscreen-book.htm)


^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Create HTML or Microsoft Word content and convert to Help file formats or 
printed documentation. Features include support for Windows Vista & 2007 
Microsoft Office, team authoring, plus more.
http://www.DocToHelp.com/TechwrlList

True single source, conditional content, PDF export, modular help.
Help & Manual is the most powerful authoring tool for technical
documentation. Boost your productivity! http://www.helpandmanual.com

---
You are currently subscribed to TECHWR-L as archive -at- web -dot- techwr-l -dot- com -dot- 

To unsubscribe send a blank email to 
techwr-l-unsubscribe -at- lists -dot- techwr-l -dot- com
or visit http://lists.techwr-l.com/mailman/options/techwr-l/archive%40web.techwr-l.com


To subscribe, send a blank email to techwr-l-join -at- lists -dot- techwr-l -dot- com

Send administrative questions to admin -at- techwr-l -dot- com -dot-  Visit
http://www.techwr-l.com/ for more resources and info.