Path: utzoo!attcan!uunet!cs.utexas.edu!samsung!sdd.hp.com!zaphod.mps.ohio-state.edu!ceres.physics.uiowa.edu!news.iastate.edu!ux1.cso.uiuc.edu!cs.uiuc.edu!marick
From: marick@cs.uiuc.edu (Brian Marick)
Newsgroups: comp.software-eng
Subject: Re: Code inspections
Message-ID: <1991Feb1.151945.22863@cs.uiuc.edu>
Date: 1 Feb 91 15:19:45 GMT
References: <14964@megatest.UUCP> <156342@felix.UUCP> <1991Jan31.151559.16415@bigsur.uucp>
Organization: University of Illinois, Dept. of Comp. Sci., Urbana, IL
Lines: 42

ppblais@bcars305.bnr.ca (Pierre P. Blais) writes:

>From my experience, I think that code inspections should be started
>after a certain amount of (sanity) testing is done. This prevents
>wasting the inspector's time in uncovering (obvious) defects. This
>is the same argument as using the compiler to find syntax errors
>instead of having a human spend time reading the code looking for
>them.

I have a similar approach.  I find that inspections and testing are
good at discovering different kinds of faults.  For example, dynamic
testing is poor at discovering what Dewayne Perry calls "obligation
faults", cases where, for example, heap-allocated memory is not freed
or open files are not closed.  But dynamic testing (when backed up by
tools to measure test suite coverage) is effective at discovering
off-by-one errors or wrong-variable-used errors; consequently, it's a
waste to check for these during code reads.  (It's a waste because, I
believe, the long-term cost of detecting a fault with a code read is
higher than with dynamic testing.  The reason is maintenance: that
dynamic test, if written sensibly, can be rerun quite cheaply.
"Rerunning" a code read when you change a module is roughly the same
as the cost of the original code read.)

What I do nowadays is keep a catalog of explicit questions to ask when
reading the code.  I test a module by writing black box tests (pretty
much the standard technique, except that I maintain a catalog of
special test cases for data structures, operations, and combining
rules that recur in specifications), then I look inside the code to
write more black-box-style tests based on the cliches I find there.
(The idea being that people implementing cliched code tend to make
cliched mistakes.)  At this point, while I'm reading the code anyway,
I apply the inspection checklist to look for those kinds of faults I
expect the other tests won't catch.  Then I run the tests and add new
ones until I've satisfied branch, loop, multi-condition, and weak
mutation coverage.  

This seems to work pretty well and hasn't been as time-consuming as
I'd earlier expected.

Brian Marick
Motorola @ University of Illinois
marick@cs.uiuc.edu, uiucdcs!marick