Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!sample.eng.ohio-state.edu!purdue!haven.umd.edu!mimsy!cml
From: cml@cs.umd.edu (Christopher Lott)
Newsgroups: comp.software-eng
Subject: Re: use of metrics
Message-ID: <35626@mimsy.umd.edu>
Date: 14 Jun 91 14:05:26 GMT
References: <1991Jun12.003809.24084@netcom.COM>
Sender: news@mimsy.umd.edu
Organization: University of Maryland Dept of Computer Science
Lines: 44

In article <1991Jun13.235937.24165@netcom.COM> jls@netcom.COM (Jim Showalter) writes:
>I came up with automatic ways to measure the
>number of changes to a particular unit and to tie those changes to a particular
>bug fix, ways to track the error-proneness of particular units, ways to
>compute both static and dynamic metrics of various kinds, and so forth,
>and all of it possible with off-the-shelf technology I've been working
>with for the past four years

I for one am extremely interested in hearing more about your successes,
especially if this is as easy as you make it seem to be.

Let me pose some questions, hopefully more meat than gristle ;-)

1.  how you define a change?  does fixing two bugs in a single module
    constitute 1 or 2 changes?  Is it every word?  Or an editing session?
    if this is collected automatically, I suspect you're using editing
    sessions, because keystrokes would be very difficult to interpret.
    so what if I run 17 editing sessions of the same document, and wind
    up changing only comments 16 of 17 times?  how do you measure granularity
    of the changes?

2.  how do you count errors?  are errors == changes?  are you familiar with
    IEEE standard notation for bugs?  It is:
	error == human misconception
	fault == error as manifested in a document (possibly code)
	failure == visible manifestation of a fault (runtime, etc)
	(note that faults don't always cause failures)

    so is an error really a fault?  Do change forms (do you collect forms?)
    distinguish faults from enhancements?  Do you count failures?

3.  what sort of static and dynamic metrics do you compute?  what is a dynamic
    metric, runtime performance?  profile data?  did you write tools to compute
    these metrics?  

4.  do you see any interesting correlations between data from static analysis
    and error(fault)(?)-pronenes of modules?

Whew, got a little carried away.  Well, I look forward to the discussion.

chris...
-- 
Christopher Lott \/ Dept of Comp Sci, Univ of Maryland, College Park, MD 20742
  cml@cs.umd.edu /\ 4122 AV Williams Bldg  301 405-2721 <standard disclaimers>