Path: utzoo!utgpu!news-server.csri.toronto.edu!rutgers!ucsd!swrinde!zaphod.mps.ohio-state.edu!rpi!bu.edu!mirror!necntc!necssd!harrison
From: harrison@necssd.NEC.COM (Mark Harrison)
Newsgroups: comp.software-eng
Subject: Significant digits (was Re: recap so far)
Message-ID: <399@necssd.NEC.COM>
Date: 28 Jul 90 17:26:45 GMT
References: <1990Jul19.154906.20518@newcastle.ac.uk> <141@srchtec.UUCP> <152@srchtec.UUCP>
Organization: NEC America Inc. SSD, Irving, TX
Lines: 22

In article <152@srchtec.UUCP>, johnb@srchtec.UUCP (John Baldwin) writes:
> In article <1990Jul21.191009.14229@newcastle.ac.uk>
> Chris.Holt@newcastle.ac.uk (Chris Holt) [in response to me] writes:
> > ...  many people still think if you have digits of precision, that
> >those digits are necessarily meaningful.  A garbage number is worse than
> >a very imprecise one *because* it is misleading.
> 
> Very well-said.             --john b.---

An interesting example of this appeared in Byte several years ago.  The
reviewer was reviewing a statistics package for the Mac, which had a
feature that as significant digits were lost, they were "grayed out."
The reviewer was quite surprised when some statistical models he had
been using for some time returned results that were entirely "grayed out."

A similar story concerns Boeing, when they moved from an IBM to a VAX.
Their programs started aborting in the VAX trig functions with a "Total
loss of precision" error, which the IBM system had not reported.
-- 
Mark Harrison             harrison@necssd.NEC.COM
(214)518-5050             {necntc, cs.utexas.edu}!necssd!harrison
standard disclaimers apply...