Path: utzoo!attcan!uunet!aplcen!uakari.primate.wisc.edu!zaphod.mps.ohio-state.edu!usc!apple!agate!violet.berkeley.edu!lindahl
From: lindahl@violet.berkeley.edu (Ken Lindahl   642-0866)
Newsgroups: comp.sys.mac.programmer
Subject: Re: Why can't the Mac add?
Message-ID: <1990Sep25.174006.11979@agate.berkeley.edu>
Date: 25 Sep 90 17:40:06 GMT
References: <45060@apple.Apple.COM> <20360@dime.cs.umass.edu> <45108@apple.Apple.COM>
Sender: usenet@agate.berkeley.edu (USENET Administrator)
Organization: University of California, Berkeley
Lines: 29

In article <45108@apple.Apple.COM> das@Apple.COM (David Shayer) writes:
>huh-HUM (throat clearing noises.)
>The calculator DA can do this math correctly.  If you add 0.2 fifty
>times, you get 10.0.  Exactly.  Not 9.999999 or 10.0000001.  I tried
>changing my float variable to an extended, as someone suggested.  No dice.
>Does the calculator DA have its own special math package?  (If so, the
>dCad calculator does too.)  No one seems to think SANE has calls which
>take care of this.  So why does it work in the calculator?
>
>David

huh-HUM (throat clearing noises.)

I don't think this description of the calculator DA is correct. If I
add 0.2 five times, I get "1" not "1.0". I'm not going to do it 50
times, but I'll bet if you do you'll get "10" not "10.0"! This should
be your first hint: calculator DA is rounding the result that it displays.
Even if the result was 9.9999999999 or 10.000000000001, you'd never see
it because the DA would round to 10. Try this in calculator DA:
1.00000000001 * 1.  The answer is "1". Actually, if you watch closely,
you'll see that the rounding occurs when you hit the "*" key! Note:
the number of zeroes to the right of the decimal point must be 10 for
you to see the behavior I'm talking about. This is determined by the
width of the calculator display.

Ken Lindahl				lindahl@violet.berkeley.edu
Advanced Technology Planning,
Information Systems and Technology
University of California at Berkeley