Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!usc!wuarchive!m.cs.uiuc.edu!marick From: marick@m.cs.uiuc.edu (Brian Marick) Newsgroups: comp.software-eng Subject: Re: Personal growth and software engineering! Message-ID: <1991Apr15.141457.11271@m.cs.uiuc.edu> Date: 15 Apr 91 14:14:57 GMT References: <9233@castle.ed.ac.uk> <1991Mar25.164133.29674@unislc.uucp> <549@tivoli.UUCP> <1991Apr12.134908.20245@m.cs.uiuc.edu> <1991Apr12.183600.15528@dg-rtp.dg.com> Distribution: comp.software-eng Organization: University of Illinois, Dept. of Comp. Sci., Urbana, IL Lines: 47 cole@farmhand.rtp.dg.com (Bill Cole) writes: >Tell me what you mean by 'coverage'. I was referring to code coverage. For example, 100% branch coverage means that running the program against your test suite forces every branch in the program to be taken in both the TRUE and FALSE directions. 100% statement coverage means that every statement has been executed. 100% loop coverage is usually taken to mean that every loop has executed 0, 1, or many times. Code coverage is a measure of how thoroughly your test suite exercises the actual text of the program. The problem with coverage is that it does NOT measure how thoroughly you've explored the program's specification. That is, it doesn't directly address your worry: >... that we haven't covered some specific, but minor, situation >that will eventually bite a user in the ear -- or lower. In general, there's no way to measure specification coverage, so people resort to using code coverage as an approximation. My point was that the approximation gets better when you use it as a springboard for thought. For example, this has happened to me: I have an unexercised branch. I could simply derive a test case to exercise it and be done with it. Instead, I treat this as a signal that there's some entire area of the specification that I've failed to explore. I develop several tests, only one of which exercises that branch. One of the *other* tests -- seemingly useless as far as coverage is concerned -- finds a bug. The next step is then to think a bit about why you missed that specification area in the first place and how you can avoid missing ones like it next time, when you might not have a coverage clue. See also Scott McGregor's recent note in this newsgroup. Of course, even if you could measure specification coverage -- whether you tested everything the program's supposed to do -- now you get to worry about requirements coverage -- does the program do what the user wants? This is where it gets really interesting, where a lot of payback lies. Testing techniques are applicable. Hope I've answered your question. Brian Marick Motorola @ University of Illinois marick@cs.uiuc.edu, uiucdcs!marick