Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!cs.utexas.edu!ico!vail!rcd
From: rcd@ico.ISC.COM (Dick Dunn)
Newsgroups: comp.software-eng
Subject: Re: The Errors of TEX
Summary: bizarre testing technique...better to ask why it worked
Keywords: testing debugging structured-programming OOD
Message-ID: <16207@vail.ICO.ISC.COM>
Date: 13 Oct 89 20:36:44 GMT
References: <2402@munnari.oz.au>
Organization: Interactive Systems Corp, Boulder, CO
Lines: 45

sharam@munnari.oz.au (Sharam Hekmatpour) writes about D. Knuth's "The
Errors of TEX":
> ...I find some of his suggestions very useful, but there also ones
> which I find inappropriate if not shocking. This one more than any other:
>...I could wait until the whole program was written, before trying to debug any
>    of it. This saved a lot of time, because I did not have to prepare 'dummy'
>    versions of non-existent modules... I could test everything in its
>    final environment..."

>...This is the worst testing technique I have ever heard of.

I tend to agree that it sounds pretty awful...and I've experienced it.  I
was once called in to fire-fight on a project where a large program had
been going for months and months of debugging without getting to even the
rudimentary stages of doing bits of useful work.  The debugging approach
was "bang on it 'til it breaks, then find it and fix it."  What was hap-
penning, quite predictably, was that there were bugs at all levels of the
code down to the most basic utility routines (like I/O and storage manage-
ment).  When something caved in, there was no easy way to isolate it.
There was no way to make a decent probability-guess on where to look,
because ALL of the code was suspect.  Multiple errors at different levels
masking one another was the rule, not the exception.

Still, Knuth is not a fool, and TeX is a serious program.  I think it would
be better to look at WHY this technique seemed to work for him, instead of
just recoiling in horror.  (It's OK to recoil in horror first...but then
look beyond it.:-)

> But let's not forget that he decided to scrap the original version and
> rewrite it from scratch. I wonder if Big Bang testing had something to
> do with this.

I tend to doubt it.  Usually if you decided to rewrite from scratch, it's
to do a major reworking of the program structure.  Program structure isn't
(or shouldn't be) much affected by testing technique; the structure is all
set by the time testing comes around.

Most programs of any size need to be scrapped and rewritten from scratch.
The first version teaches you the things you needed to know but couldn't
find out just by thinking about the problem.  With reasonable luck, the
first version also keeps you going with the breathing room to write the
second version.
-- 
Dick Dunn     rcd@ico.isc.com    uucp: {ncar,nbires}!ico!rcd     (303)449-2870
   ...No DOS.  UNIX.