Re: Productivity Metrics

[Stephen -dot- MacDonald -at- Aspect -dot- com]

Curtis Ward wrote:

>My department (Tech Pubs/Training Development) has been charged with
>developing a set of productivity metrics to be used for both individual and
>department evaluation and planning. I'm pretty underwhelmed with most of
the
>material I have found researching this issue.
>
>Does anyone know of a good model or have any experience with this area?
I'm
>particularly concerned about validity of any numbers we are able to
develop.

In a previous life I spent several years as an internal consultant in a
quality group chartered to do process improvement, metrics development, etc.
for Digital Equipment.

Assuming this is a "new" thing in your company, I would discourage
developing   individual productivity metrics.  First, individual
productivity usually has more to do with the process than the efforts of the
individual.  Secondly, organizations new to gathering and using metrics
often fall into the trap of using metrics to beat people over the head
rather than to identify ineffecient processes and ways to improve them.

First, I would focus on the organization.  There may not be lots of
information about how to use metrics in a documentation organization, but
there is a wealth of stuff about how to use metrics in software development.
The Software Engineering Institute (SEI - used to be housed and hosted by
Carnegie Mellon in Pittsburgh) developed the CCM, Capability Maturity Model,
which was a way to evaluate the effectiveness of a software development
organization.   This model and approach could be applied to documentation
without a lot of effort.  The major difference would be the metrics
themselves; the SEI would have examples of software metrics.  This would
give you a firm grounding in the principles of developing and applying
metrics.  Your concern about metrics validity is sensible.
  
Second, once I had a measurement philosophy and approach in place, I would
develop my own metrics.  What you measure is how the work is done in your
organization.  That is why you need locally developed metrics.

I would not implement an individual metrics effort until I had a mature,
well-managed program at the departmental/organizational level.  The
departmental program would identify training, skills, experience, etc.
needed to achieve the required level of productivity.  The individual
metrics would identify the achieved level of training, skills, experience
etc. so you could develop plans for individuals who needed further
development.

A metrics program can be a huge benefit if it's well thought out and does
what you can reasonably expect metrics to do (metrics per se are not problem
solvers).  It can also be a nightmare causing much more trouble than you
have already if you are not careful.

Steve MacDonald
Aspect Communications, Inc.