Path: utzoo!utgpu!jarvis.csri.toronto.edu!rutgers!ukma!xanth!lll-winken!maddog.llnl.gov!brooks
From: brooks@maddog.llnl.gov
Newsgroups: gnu.gcc
Subject: Reliability of GCC
Message-ID: <27145@lll-winken.LLNL.GOV>
Date: 19 Jun 89 16:40:22 GMT
Sender: usenet@lll-winken.LLNL.GOV
Reply-To: brooks@maddog.llnl.gov ()
Distribution: gnu
Organization: Lawrence Livermore National Laboratory
Lines: 32

Several posters have indicated, and later strongly supported,
statements about reliability problems with GCC.  As you might
guess at this point, the reliability problems are real.

The reason why new releases are often flakey is probably
the lack of sufficient regression testing.  A regression
test package should be built and distributed as part of
GCC. Before each publicly announced release FSF should
ship the new release to several friendly sites which will
run the regression tests on hardware FSF does not
have direct access to.  The major problem with regression
test packages is the sheer volume of code which must be
written, with the writer thinking that he is not doing something
useful (like working directly on the compiler).  The solution
to this problem is to request that bug reports include if
possible a test program which is in the standard regression
test format.

Suitable requirements:

A single program which runs and produces the message PASSED or FAILED
on standard output.   The regression test driver can trigger on these
keywords and inform the tester.  Liberal use of tests for failure
with suitable printing of line and file numbers of where the compilation
failure occured.  A pointer to the source of the test program in case help
is required in bug shooting the compiler.

With all the bug reports which are coming in, it should be no time
at all that we have a large and useful regression test library for
GCC.  At least we wont have repeat visits on previous bugs.

brooks@maddog.llnl.gov, brooks@maddog.uucp