Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!csd4.milw.wisc.edu!cs.utexas.edu!uunet!zephyr.ens.tek.com!tekcrl!tekgvs!larryh
From: larryh@tekgvs.LABS.TEK.COM (Larry Hutchinson)
Newsgroups: comp.sys.mac
Subject: Re: Bug in EXCEL v. 2.2, and a flame
Keywords: Excel
Message-ID: <5680@tekgvs.LABS.TEK.COM>
Date: 31 Jul 89 17:38:26 GMT
References: <4134@utastro.UUCP>
Reply-To: larryh@tekgvs.LABS.TEK.COM (Larry Hutchinson)
Organization: Tektronix, Inc., Beaverton,  OR.
Lines: 45

In article <4134@utastro.UUCP> bill@utastro.UUCP (William H. Jefferys) writes:
>Nearly two years ago, I reported in this forum on
>a bug in Microsoft EXCEL. I also reported the bug
.
.
>Here is the bug. It shows up on a Mac+ with no coprocessor.
>It does NOT show up on a Mac II. I don't know about the SE,
>but I expect it will show up there too (anyone want to
>try it?)
>
>Compute the square root of 0.999975. On a Plus, EXCEL
>gives the incorrect value 0.9999875. The correct value
>is 0.9999749992187, a difference of 1.6e-10. This may


I am not convinced that this is a bug at all.

Directly using either the 68882 OR using SANE (which probably calls the 882)
I get:

sqrt(0.999975)=
    0.999987499921874

as a check on the answer:
0.999987499921874 * 0.999987499921874=
    0.9999749999999998
which is not too shabby.

Using the value you state, I get
0.9999749992187 * 0.9999749992187=
    0.9999499990624391
which is obviously far less accurate.


This is on an SE/30.  But if this (and your Mac+) is the correct answer
then that implys your Mac II gave the wrong answer!  Pretty wierd stuff.
(My HP15C gives the same results to 10 digits.)

Where did you get your value?  How did you check it?


Larry Hutchinson, Tektronix, Inc. PO Box 500, MS 50-383, Beaverton, OR 97077
UUCP:   [uunet|ucbvax|decvax|hplabs]!tektronix!tekgvs!larryh
ARPA:   larryh%tekgvs.LABS.TEK.COM@RELAY.CS.NET
CSNet:  larryh@tekgvs.LABS.TEK.COM