Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!swrinde!cs.utexas.edu!uunet!dg-rtp!farmhand!cole From: cole@farmhand.rtp.dg.com (Bill Cole) Newsgroups: comp.software-eng Subject: Re: Personal growth and software engineering! Message-ID: <1991Apr12.183600.15528@dg-rtp.dg.com> Date: 12 Apr 91 18:36:00 GMT References: <9233@castle.ed.ac.uk> <1991Mar25.164133.29674@unislc.uucp> <549@tivoli.UUCP> <1991Apr12.134908.20245@m.cs.uiuc.edu> Sender: cole@farmhand (Bill Cole) Distribution: comp.software-eng Organization: Data General Corporation, Research Triangle Park, NC Lines: 42 In article <1991Apr12.134908.20245@m.cs.uiuc.edu>, marick@m.cs.uiuc.edu (Brian Marick) writes: |> Jorge Gautier) writes: |> |> >... By effectively I mean that the metric should give you a |> >nonambiguous message of what needs to be done given your intentions. |> >For example, a coverage measuring tool measures the amount of code |> >exercised by tests. If you want 90% coverage and you are getting 30%, |> >the message is clear: write more and/or better tests. Testing is a |> >well formalized process and can be meaningfully measured in this way. |> |> (Much stuff deleted) |> |> And what's the difference between 90% coverage, 80% coverage, and 100% |> coverage? Does it depend on the program? |> |> I don't mean to say that measuring test coverage is useless, because I |> happen to find it very useful -- when interpreted in context. The |> best numbers focus your attention, say, "Hey! Look over here; there's |> something odd going on." Without them, problem solving is next to |> impossible; with them, it's merely difficult. (Shewhart control |> charts, used in manufacturing quality control, are a perfect example |> of what I mean.) |> Tell me what you mean by 'coverage'. I submit that if you can define 100% coverage that testing the software might not be meaningful (or challenging since we're talking about personal growth). My worry is always that we haven't covered some specific, but minor, situation that will eventually bite a user in the ear -- or lower. We try to do at least 100% coverage on any new feature with both positive and negative tests. The interesting thing is that we can't build all the scenarios that our customers will build in their own shops. We do, however, learn from reported bugs. Part of the bug closure process in my shop requires that a test be added to the test suite so that we keep the bug out of future released software. The down side to all of this is that we know that we're not going to find all the bugs in our software, but the positive side is that we're going to keep getting better. The views must be my own because nobody wants to share them, /Bill