Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!usc!zaphod.mps.ohio-state.edu!uakari.primate.wisc.edu!ames!haven.umd.edu!udel!nigel.ee.udel.edu!mccalpin From: mccalpin@perelandra.cms.udel.edu (John D. McCalpin) Newsgroups: comp.arch Subject: Re: IEEE floating point Message-ID: Date: 29 May 91 20:46:41 GMT References: <9105250030.AA08036@ucbvax.Berkeley.EDU> <1991May25.222551.16365@zoo.toronto.edu> Sender: usenet@ee.udel.edu Organization: College of Marine Studies, U. Del. Lines: 26 Nntp-Posting-Host: perelandra.cms.udel.edu In-reply-to: mccalpin@perelandra.cms.udel.edu's message of 27 May 91 13:29:50 GMT > On 27 May 91 13:29:50 GMT, mccalpin@perelandra.cms.udel.edu (John D. McCalpin) said: > On 26 May 91 14:02:57 GMT, mccalpin@perelandra.cms.udel.edu, I said: Me> Errors in solution of 1000x1000 system of equations Me> from the LINPACK benchmark suite Me> machine precision RMS error Me> --------------------------------------------------------- Me> IBM RS/6000 64-bit 1.3e-10 ^^^^^^^ WRONG! Me> IBM RS/6000 (-qnomaf) 64-bit 2.2e-12 Me> IEEE (Sun-3) 64-bit 2.3e-13 Me> --------------------------------------------------------- Please note that the number indicated is WRONG. At least, I am not able to reproduce it. The reproducible number is 1.2e-12 for the RMS error. This is very close the the -qnomaf result, but still inexplicably about 5 times larger than the IEEE number. I apologize for the misunderstanding, and any resulting heartburn at IBM :-)..... -- John D. McCalpin mccalpin@perelandra.cms.udel.edu Assistant Professor mccalpin@brahms.udel.edu College of Marine Studies, U. Del. J.MCCALPIN/OMNET