[no subject]

     --clarity (0 to 4)
     --conciseness (0 to 4)
     --correctness (0 to 4)
     --content/completeness (0 to 4)
     --courtesy (0 to 4)

The applicant is placed in a room with some source documents and
computer equipment.  He or she receives five to six hours to produce up
to five pages of a user's guide.  For the sake of time, I'd provide an
existing document requiring moderate to heavy revisions plus three to
five new, small sections of about one or two paragraphs each.  Markups
should be in the form of editorial comments/suggestions in the margin.
Leave all mechanical and usage errors unmarked so that the writer can
correct them on his or her own during the test.

Scores under each category are defined on a grading sheet that the
applicant doesn't see, but is distributed to scorers from within your
department.  For example, "correctness" and "conciseness" might read:

     CORRECTNESS

     4 pts.      Three or fewer errors in the finished document

     3 pts.      Four to six errors...

     2 pts.      Seven to nine errors...

     1 pt.       Ten to twelve errors...

     0           Thirteen or greater errors...

     CONCISENESS

     4 pts.      Three or fewer instances of passive voice,
                 expletives, etc. (add your own bugaboos)

     3 pts       Four to six instances...

     2 pts       Seven to nine instances...

     1 pt        Ten to twelve instances...

     0           Thirteen or greater instances...

For "completeness," list the pieces of information that need to be
included in the finished product, count them up, and construct your
scoring scale from there.

You and another person should grade each test (which has been assigned a
code number by your department support staffer or HR).  All errors,
including missing information, need to be carefully documented.

If you were to use the criteria stated at the beginning of this post,
the test would be worth 20 possible points.  An average score of 14
(70%), 16 (80%), or 18 (90%) could be your minimum standard for whatever
the next step for the applicant might be--say, a second interview or
hire/no hire.

The scores assigned should be within two points of each other.  If they
aren't, get a third party to grade.

My information is based on procedures that many schools, colleges, and
universities--and organizations such as the Educational Testing
Service--use for standardized essay tests.  Hope this is of some help!

Take care,
Kelli Bond
Principal Consultant
KBA/DesignWrite (Orange County, California)