Xref: utzoo rec.humor:19786 comp.misc:5464
Path: utzoo!utgpu!utstat!jarvis.csri.toronto.edu!mailrus!purdue!decwrl!ucbvax!agate!saturn!ucscc.UCSC.EDU!haynes
From: haynes@ucscc.UCSC.EDU (Jim Haynes)
Newsgroups: rec.humor,comp.misc
Subject: Re: Looking for Computer Folklore
Summary: Yet another GE625/635 story
Message-ID: <6665@saturn.ucsc.edu>
Date: 13 Mar 89 05:50:18 GMT
Sender: usenet@saturn.ucsc.edu
Reply-To: haynes@ucscc.UCSC.EDU (Jim Haynes)
Organization: California State Home for the Weird
Lines: 40

Can't remember if I posted this one already.  The GE 635 and its
Honeywell successors use 2-s complement notation even for floating
point numbers.  During computation the fraction part is kept in
36-bit registers, and the exponent is kept in a separate register.
When a floating point result is to be stored in memory the floating
store instruction shifts the fraction part to the right to make
room for the exponent, so the whole thing fits into one 36-bit word.
Now when you shift a 2-s complement number to the right, letting
bits fall off the end, the result is that the error is biased.
Positive numbers become smaller, less positive, and negative numbers
become bigger, more negative.  So the error in a large sequence of
calculations doesn't tend to average toward zero.

The folklore part:  I was told that this was recognized as a serious
problem when the computer was used to compute instructions for a
numerically-controlled cutting torch.  The torch was cutting a
really big, something like 16-foot diameter circular plate of steel.
Well, when the torch came around back near the starting point the
accumulated error was such that the track failed to close on itself;
there was a jog of a couple of inches between the starting and ending
points.

The obvious way to fix this is to change the floating store operation
so that it rounds the number to be stored, rather than just lopping off
bits.  However that couldn't be done, because some software writer had
discovered that the floating store operation was a fine way to pack
a byte, placed in the exponent register, on to the end of a string
of bytes stored in a word.  If the bytes happened to be characters
then rounding on a character string might make a mess of things.

Hence it was necessary to introduce a new instruction, floating store
rounded.  The compilers were all modified to use this instruction
to store floating point values and to never use the original floating
store instruction.
haynes@ucscc.ucsc.edu
haynes@ucscc.bitnet
..ucbvax!ucscc!haynes

"Any clod can have the facts, but having opinions is an Art."
        Charles McCabe, San Francisco Chronicle