Path: utzoo!mnetor!uunet!husc6!rutgers!columbia!garfield.columbia.edu!andy
From: andy@garfield.columbia.edu (Andy Lowry)
Newsgroups: comp.databases
Subject: Re: Problems with data-type FLOATING in VAXDBMS
Message-ID: <5393@columbia.edu>
Date: 7 Mar 88 22:49:15 GMT
References: <482@iraun1.ira.uka.de>
Sender: nobody@columbia.edu
Reply-To: andy@garfield.columbia.edu.UUCP (Andy Lowry)
Organization: Columbia University CS Department
Lines: 22
Summary: FP numbers "accurate to n digits"

In article <482@iraun1.ira.uka.de> rader@ira.uka.de (Klaus Radermacher) writes:
>However, DEC claims in the manuals, that FLOATING has a precision of about 
>7 digits, and the stored value was exactly 8.05!

It's important, when relying on those advertised precisions, to
remember that the mantissa in the floating point representation is
stored, in most cases, as a *binary* fraction.  Thus any statement as
to number of correct decimal digits can be a bit misleading.  As an
example, in the present case, the fractional part 0.05 cannot be
stored exactly, since the binary representation of 0.05 is
non-terminating.  It starts out as 0.0000110011000100....  In fact,
there are very few fractions that can be expressed exactly in seven
decimal digits and can be expressed exactly in binary at all (like
0.5).  So when you type a number like 8.05 to a program that must use
a binary floating point representation, internally you will
necessarily get only an approximation, no matter how big the mantissa
is.

Of course, these comments don't apply to machines that can directly
manipulate numbers stored in a decimal representation.

-Andy