Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!mips!pacbell.com!ucsd!ucbvax!DLRVMKP.BITNET!AT1F
From: AT1F@DLRVMKP.BITNET
Newsgroups: comp.sys.transputer
Subject: Re: Improper results on T800
Message-ID: <9104261440.AA16404@theory.TN.CORNELL.EDU>
Date: 26 Apr 91 15:07:03 GMT
Sender: daemon@ucbvax.BERKELEY.EDU
Organization: The Internet
Lines: 21
X-Unparsable-Date: Fri, 26 Apr 91 15:50:06 MEZ

hi all,

your floating point number problem stems from limited representation
capabilities of numbers on your computer.

the resulting error comes from calculating
(we * yin&nsitea + xdin&nsitea)

we * yin&nsitea = 1168.106778
xdin&nsitea     = 1168.106812

the two numbers differ in the 8th significant digit!
IEEE says single precision handles 7 significant digits. the results
obtained are pure random.

to overcome the problem change to 'double'.

have a nice weekend.
Stefan Pokorny, DLR (german aerospace research establishment), Cologne,
                Germany
E-Mail: at1f@dlrvmkp.bitnet