Path: utzoo!utgpu!news-server.csri.toronto.edu!rutgers!tut.cis.ohio-state.edu!pt.cs.cmu.edu!sei!bwb
From: bwb@sei.cmu.edu (Bruce Benson)
Newsgroups: comp.software-eng
Subject: Re: Specification Tools and Code Testing
Message-ID: <8316@fy.sei.cmu.edu>
Date: 23 Aug 90 13:46:57 GMT
References: <5456@stpstn.UUCP> <1990Aug12.134735.22528@cbnewsm.att.com> <1990Aug13.140347.9441@nixtdc.uucp> <19578@well.sf.ca.us>
Reply-To: bwb@sei.cmu.edu (Bruce Benson)
Distribution: usa
Organization: Software Engineering Institute, Pittsburgh, PA
Lines: 56

In article <19578@well.sf.ca.us> jjacobs@well.sf.ca.us (Jeffrey Jacobs) writes:
>
>> testing a big system is *hard*
>
>It shouldn't be "hard", but it is a major cost factor.
>
>Some general comments on testing large systems:
<agreeable items removed>

>3.  Most phases of testing should be conducted by a separate test
>organization.
>This test organization should include some senior people; one of the most
>common mistakes is assuming that testing (and Q/A) is something that only
>requires a BS/CS and a couple of years of experience.

The article makes a good argument for a methodical planned test approach. But
then "cops out" by the age old refrain of needs "experienced senior people". 
This is also sometimes worded as "put your best people in activity X". 

What activity does not need experienced people?  Give me some compelling
reasons why one activity (such as test) should have more or less experienced
people then another activity (QA, design, coding, etc).

The entire software engineering process must work well.  Throwing
"experience" at part of the process is usually a symptom of not yet
having that part of the process under control.  We put our independent
testers out of business of catching errors by introducing a methodical test
method in DT&E (prior to that the programmers had the mind set
that someone else was responsible for showing the code worked).

Quality of the product (code working well) should remain the responsibility of
those who produce the code.  Inspecting and testing is clearly part of the
activity of the programming staff.  Independent checking activities should be 
used in a manner to verify the quality of the product (and process) and not as
a substitute for the checking (and feedback) activities of the programming 
staff. 

>6.  Despite all the emphasison testing, the single most effective
>technique for eliminating and preventing defects is "inspection".

Well, getting it right the first time works pretty well too.  Most code 
written is correct.  The trick is to increase this percentage.  Programmers
doing methodical reviews and testing of their code *learn* faster about
what their individual strengths and weaknesses are.  My reference above to
putting testers out of business refers to programmers being required to
"structurally" test their code (ie, exercise every line of code).  This
required the programmer to relook at just about every line of code.  This
simple combined inspection and test eliminated (yup - zero) code errors.
Independent test did the functional test, and we argued a lot over if the
function was implemented correctly, but the system never died, blew up, or
corrupted anything.  Your mileage may vary.

* Bruce Benson                   + Internet  - bwb@sei.cmu.edu +       +
* Software Engineering Institute + Compuserv - 76226,3407      +    >--|>
* Carnegie Mellon University     + Voice     - 412 268 8469    +       +
* Pittsburgh PA 15213-3890       +                             +  US Air Force