Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!dali.cs.montana.edu!uakari.primate.wisc.edu!xanth!mcnc!uvaarpa!polaris.cv.nrao.edu!polaris.cv.nrao.edu!dwells From: dwells@fits.cx.nrao.edu (Don Wells) Newsgroups: comp.arch Subject: Re: IEEE floating point Message-ID: Date: 26 May 91 02:45:41 GMT References: <9105250030.AA08036@ucbvax.Berkeley.EDU> <1991May25.222551.16365@zoo.toronto.edu> <12805@mentor.cc.purdue.edu> Sender: news@polaris.cv.nrao.edu (News Manager) Organization: National Radio Astronomy Observatory, Charlottesville, VA Lines: 22 In-Reply-To: hrubin@pop.stat.purdue.edu's message of 26 May 91 00: 25:17 GMT In article <12805@mentor.cc.purdue.edu> hrubin@pop.stat.purdue.edu (Herman Rubin) writes: ... The IBM hex FP is not that much worse than any other FP with 32-bit words. The average precision of SP hex normalization, something like 22.5 bits (compared to the 24 bits of VAX or IEEE), does not determine the overall precision of complicated numerical operations like FFTs; instead the 21_bit worst case causes random truncation errors in intermediate sums, and thereby eventually corrupts most of the answers. The loss of precision is easily detectable when SP results from 370-architecture-CPUs are compared with SP results computed on VAXen and IEEE-FP machines. For signal processing applications where the signals and results have high dynamic range (e.g., more than 10,000:1), hex-normalized SP FP must be viewed with suspicion. -- Donald C. Wells Associate Scientist dwells@nrao.edu National Radio Astronomy Observatory +1-804-296-0277 Edgemont Road Fax= +1-804-296-0278 Charlottesville, Virginia 22903-2475 USA 78:31.1W, 38:02.2N