Path: utzoo!attcan!uunet!husc6!bloom-beacon!mit-eddie!uw-beaver!apollo!oj
From: oj@apollo.uucp (Ellis Oliver Jones)
Newsgroups: comp.sys.apollo
Subject: Re: Problem with FPX on 580-T
Summary: Lost precision
Keywords: epsilon, IEEE
Message-ID: <3cacb21e.d5b2@apollo.uucp>
Date: 15 Jun 88 12:55:00 GMT
References: <471@nikhefh.hep.nl>
Reply-To: oj@canyon.UUCP (Ellis Oliver Jones)
Organization: Apollo Computer, Chelmsford, MA
Lines: 50

In article <471@nikhefh.hep.nl> i91@nikhefh.hep.nl (Fons Rademakers) writes:
>I have the following problem...
>      PA(1) = 100.00001
>      PA(2) = 100.00001
>      PA(3) = 100.00001
>      SAVXYZ = PA(1)*PA(2)*PA(3)
No doubt you realize what you're asking for arithmetically...
Let's analyze it as follows:  

You want to compute (a+b) ^ 3 , where a = 1e2 and b = 1e-5

The result is  a^3 + 3*(a^2*b + a*b^2) + b^2    
or 1e6 + 0.3 + 3e-8 + 1e-15
or 1000000.300000030000001

Single-precision IEEE floating point has a 24-bit mantissa (counting
the "hidden" bit) so at most accuracy is 1 part in 2^24, or about 5.96e-8 .

So, the precision of your PA(1)*PA(2)*PA(3) calculation is always getting sawed
off and more so when you move it from a register to memory.

>What I would like to know is 
>  1) does this happen on all DN5(8|9)0-T with a FPX 
It happened on the one I use.

It has to do with the optimization strategy used by the compiler.
The fpx's internal registers are good to double precision, whereas
once the number's been stored in memory precision is lost.  Code
generation for the FPX is different than for -cpu any 
or -cpu 580  (which gets you 68881 code).  Register-lifetime
differences could account for the differences you saw.

Try using the "-exp" option in your compilation commmand, and you
can see the compiled code for yourself.

>  2) how come that it also goes wrong in the case when compiled without
>     -cpu fpx, is the FPX always used?
Yes.  When you don't specify -cpu fpx, you get generic code which
calls syslib to do floating point operations.  The version of syslib 
which runs on an fpx-node uses the fpx.

> 3) Microcode fix?   

I'm (for what my opinion's worth) not convinced the FPX is broken,
considering the requirements of your computation compared to the
precision provided by single-precision floating point.
We can check it out, though.  Would you mind submitting your net posting
in APR form?

/Oliver Jones     (speaking for myself, not necessarily for Apollo Computer, Inc).