Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!sun-barr!lll-winken!uunet!stanford.edu!unix!CRVAX.Sri.Com!hlavaty From: hlavaty@CRVAX.Sri.Com Newsgroups: comp.software-eng Subject: Re: metrics and the SAT example Message-ID: <24822@unix.SRI.COM> Date: 28 May 91 16:58:48 GMT Sender: news@unix.SRI.COM Organization: SRI International Lines: 57 In article <1991May24.192101.22317@grep.co.uk>, frank@grep.co.uk (Frank Wales) writes... >In article <1991May23.014904.5896@netcom.COM> jls@netcom.COM (Jim Showalter) writes: > >It must be possible to establish credible causality too, otherwise you can't >be sure what you're measuring. Say you notice that levels of ice-cream >consumption correlate strongly with deaths at the beach. Does this mean >ice-cream is a killer? Not if you realise that both variables have >a common influence, such as sunny weather. While causality is important and helpfull, I don't agree that it prevents the use of the metric. In your example, if you wanted to prevent deaths at the beach, you could use the ice cream metric to alert you when more deaths at the beach were immininent. Then you could go to the beach and figure out what was going on. While a simple (and silly) example, the same principle applies to something more arcane like the health and status of an integration effort. If you have noticed that more overtime usually indicates that your effort is in trouble, you certainly don't prevent people from working overtime. You spend some time going over your integration effort and try and figure out what the problems are. The major problem facing any software development is that so many potential things can go wrong that are *not* obvious, or that "seemed like a good idea at the time" but turned out later to be short term thinking. The quest for metrics is a search to find something (ANY something) that you can demonstrate is a good indication of success or failure, OR allows you more insight into the inner workings of the project. An example of the former would be looking at overtime or number of compiles/day. A good example of the latter would be tracking test cases completed over time and comparing it to your original plan. >Indeed. Just like looking at local death rates makes hospitals hazardous. Well, yes. Being in a hospital is a good metric to indicate that your chances of dying in the near future MAY have gone up. So you look closer. Why am I here? If it's because a broke my foot, I relax and conclude that there is no cause for concern. If it's because I am having an operation that requires me to be knocked out, I get a little more concerned and may look at this hospital's track record for 1) knocking people out and not killing them and 2) their overall success at this type of operation. Applying the analogy directly to software development, let's say I have been tracking the success rate of all my programmers (How I am doing this or what my definition of success is I will leave to future discussions). Now, when I realize that a particular module that I am concerned with is being worked on by a programmer with a demonstrated poor performance, I get concerned and proceed to look closer at the situation. Has that programmer ever done something similar to this module? How did he do on that one? What did he learn (if anything) from his mistakes? After going through this process I may decide that no action is warranted, or that some help is in order, or that the module should be given to someone else entirely. In fact, here I am not concerned too much with causality. Last time the programmer tried this it got all bungled up. I am now concerned, whether or not I know why he bungled it up. If I know why it got bungled, I can possibly make a more eductaed decision. If I don't know why it got bungled, I can decide to 1) watch the development vert closely this time and try and figure out why the programmer has problems or 2) decide the module is too important to risk and give it to someone else. The metric is more valuable to me with causality, but still usefull even without it. Jim Hlavaty