Usage: Keeping the number of digits consistent?

[Geoff Hart]

Thought this might be of interest to folks who work with numbers...

One standard stylistic approach in the sciences is to ensure that  
numbers have the same number of decimal places so that comparisons  
between numbers are facilitated. For example, in the series 0.02,  
0.031, and 0.025, the first number should be changed to 0.020.

The rationale evinced to explain this is interesting (at least if  
you're a cognitive psychology geek like I am): adding the trailing  
zero makes the comparison of the last digits of the numbers become 20  
(not 2) versus 31 versus 25. (The same effect, but in the reverse  
order, probably explains why something selling for $49.99 seems less  
expensive than something selling for $50.) Is this just theory with  
no significant consequences in reality?

As it happens, it does indeed have consequences. I'm currently  
editing a manuscript that has already been through an edit by another  
editor, and both the editor and the author failed to change the 0.02  
to 0.020, and in so doing, failed to notice that the detection limit  
of the equipment used to perform the analysis (reported as 0.020) was  
larger than many of the numbers (e.g., 0.003, 0.018) presented in the  
data. Since the other editor is ordinarily quite careful, I take this  
as one data point to support the theory: if both a good editor and an  
intelligent author missed this, something more than chance is  
probably going on.

Also note that I'm not reporting this to criticize the other editor,  
but rather to make the point that sometimes, seemingly trivial style  
choices have important underlying reasons.


----------------------------------------------------
-- Geoff Hart
ghart at videotron.ca / geoffhart at mac.com
www.geoff-hart.com
--------------------------------------------------
***Now available***  _Effective onscreen editing_
(http://www.geoff-hart.com/home/onscreen-book.htm)