Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!mailrus!cs.utexas.edu!uunet!motcid!murphyn
From: murphyn@cell.mot.COM (Neal P. Murphy)
Newsgroups: comp.software-eng
Subject: Re: Information on current state of software safety desired
Summary: No potentially lethal system should rely solely on software for control.
Message-ID: <209@cherry5.UUCP>
Date: 9 Oct 89 13:53:50 GMT
References: <1321@cs.rit.edu> <195@cherry5.UUCP> <1989Oct4.055359.15145@paris.ics.uci.edu>
Organization: Motorola Inc. - Cellular Infrastructure Div., Arlington Heights, IL 60004
Lines: 54

From Article 96 of comp.software-eng:

>unable (at this time) to provide details.  However, the failure resulted from
>software bugs, not from system design flaws.
>     .
>     .
>     .
>This is not the responsibility of the software developers, but of the system,
>nuclear, and safety engineers.  

So, in one breath, the deaths and injuries resulted from software
failures.  In the next breath, you state that desirable safety interlocks
are the responsibility of the system designers.  So whose failure was it?
Did the software engineers fail?  Or did the system designers fail?  Or was
it (as I think) a breakdown of the development process?  I accept that
there were errors in the software.  But I must insist that that system
should have been PHYSICALLY INCAPABLE of producing those lethal dosages,
REGARDLESS of what the software controller was telling it to do!

While I have not been an expert witness at any trial, I do know something
about this topic, as I spent a number of years in the radiation testing
field.  All that would have been needed to prevent these tragedies is a
circuit to compute the area beneath the pulse - a diode, analog integrater,
comparator and a relay switch (for an analog solution) or a diode,
digitizer, dedicated processor (for a digital solution) whose sole function
is computing the area beneath the pulse and to turn on a relay to disengage
the LINAC should the total dose exceed safety margins.  This is a version
of the `crowbar circuit', which function it is to shunt, to ground, power
surges in a power supply's mains, much like putting a crowbar across the
hot and common terminals.  How much did the system cost?  One million
dollars?  Two million dollars?  Even as little as $500,000?  Would a
$30,000 fail-safe system have made the system uneconomical?  I think not.

A terrible tragedy it is.  But I will not persecute the designers of the
system.  I imagine they feel pretty rotten anyway.  Did they REALLY do all
they could to ensure the safe use of the machine?  (Look at how many people
are killed on the highways and by-ways of the world.  I don't hear much
call for educating the drivers to make them better, or confiscating the
vehicles of drivers who repeatedly kill and maim people while behind the
wheel.  But I digress.)  I stand by what I said.  What is done, is done.
What can we learn from our mistakes?  We are only human.  We WILL make
mistakes.  I will never claim that any software I create will be so perfect
that it won't need to be independently checked.  Try as I might to avoid
them, I DO make mistakes.  I HAVE hurt people in my lifetime.  And I have
felt that hurt.

All we can do is our best.  And when that best is not good enough, we hurt,
because we realize that we could have done better.  Those of us who care
stand up and accept responsibility for our actions.  To mis-quote the Good
Book, "Let him, who has made no mistakes, cast the first stone."  If there
were more of us working together instead of attacking each other, mayhap we
would make some progress.  No?

NPN