Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!uunet!pilchuck!dataio!bright
From: bright@Data-IO.COM (Walter Bright)
Newsgroups: comp.windows.ms
Subject: Re: Microsoft Support (or lack of)
Message-ID: <2702@dataio.Data-IO.COM>
Date: 10 Sep 90 18:56:49 GMT
References: <1990Aug28.195832.1853@chinet.chi.il.us> <2684@dataio.Data-IO.COM> <4053@rtifs1.UUCP>
Distribution: na
Organization: Data I/O Corporation; Redmond, WA
Lines: 37

In article <4053@rtifs1.UUCP> bcw@rti.rti.org (Bruce Wright) writes:
<In article <2684@dataio.Data-IO.COM>, bright@Data-IO.COM (Walter Bright) writes:
<< You need a test suite for your product.
<To some extent I agree with this sentiment (that you need a test
<suite for a commercial product), but I think you are putting too
<much confidence in what a test suite can find.  It isn't reasonable 
<to think that a test suite can find all possible bugs - by definition
<a test suite is finite, and many sorts of software can be subjected
<to a countably infinite number of possible inputs.

Quite true. But it also isn't reasonable to expect that the compiler
or other tools you are using have zero bugs either.

I've been involved in several major projects. At my insistence, the
ones without test suites got them developed. The idea was to create
a test suite that could be run automatically. Once the system was
in place, whenever a bug got fixed or a new feature added, a test for
it was added to the suite.

One project used a coverage analyzer to verify that the test suite
executed *every* line of code in the program. (We discovered a lot
of dead code this way, and removed it!) Not coincidentally, this
turned out to be the most bug-free major project I've ever been
familiar with (6 people, 60,000 lines of code). The product has been
maintained and sold for 6 years now, and that old test suite proves
its value with every new release.

I've heard all the arguments about why a test suite could not be done
for a particular project. After doing it anyway, nearly all those involved
conceded that the total time spent on a project was *reduced* by having
the test suite, and the reliability of the product was greatly increased.

Test suites are as imperfect as anything else. But they do greatly improve
software quality and significantly reduce software maintenance costs.

P.S. I regard the test suite for the Zortech compiler as one of the
most valuable assets we have...