Path: utzoo!utgpu!water!watmath!clyde!rutgers!sunybcs!boulder!hao!gatech!mcnc!ecsvax!hes
From: hes@ecsvax.UUCP (Henry Schaffer)
Newsgroups: comp.arch
Subject: Re: Round-off
Summary: HEX roundoff in floating point - vxs. - exponent size
Keywords: IBM HEX ???
Message-ID: <4404@ecsvax.UUCP>
Date: 10 Jan 88 05:06:03 GMT
References: <189@mithras> <614@PT.CS.CMU.EDU>
Organization: NC State Univ.
Lines: 36

In article <614@PT.CS.CMU.EDU>, lindsay@K.GP.CS.CMU.EDU (Donald Lindsay) writes:
> In article <189@mithras> sims@stsci.EDU (Jim Sims) writes:
> >I have been told and am seeking to confirm that the 309x (and others?) series
> >from IBM does HEX roundoff. Is this for real?
> 
> Yes. The floating point format normalizes to the hex digit, not to the 
> bit level. This means that a normalized number can still have one (or two
> or three) leading zero bits.
> 
> Gene Amdahl has claimed that he did this in the hope of saving hardware.
                                                          ^^^^^^^^^^^^^^^
This tradeoff goes back to the original 360 - and seems to have originated
with the desire to have a floating point word fit in 4 bytes, and so the
exponent was in 1 byte.  In order to have an acceptable range of
magnitude, the exponent had to shift more than binary, and HEX was chosen.
Of course with hex normalization, any hex digit could be most significant,
and so necessarily there could be 1-3 leading zero *bits*, but that is just
part of the leading non-zero hex digit.

Many of us were less than happy with the 4 byte floating point word -
feeling that it had less precision than desirable in the mantissa and
less range than desirable in the exponent.  (The 48 bit floating point
in the CDC 1604 had room for a range of 10^+-300 and about 13 decimal 
digits of precision.)  The answer of course was the doubleword REAL*8, but
it still had the limited dynamic range.  I believe the reason the short
precision was chosen as the default was to make the most of the small
memories of those days.  256K or 512K was considered *big*.

> Apparently the saving wasn't worth much. The downside is mostly apparent
> to numeric analysts, who can't characterize accumulated roundoff as well
> as they'd like.
Why not?  Why can't they just treat it as hex roundoff - without worrying
that the first bit in hex 1-7 is a 0?
> -- 
> 	Don		lindsay@k.gp.cs.cmu.edu    CMU Computer Science
--henry schaffer  n c state univ