Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!swrinde!elroy.jpl.nasa.gov!ames!dftsrv!mimsy!cml From: cml@cs.umd.edu (Christopher Lott) Newsgroups: comp.software-eng Subject: Re: use of metrics Message-ID: <35353@mimsy.umd.edu> Date: 7 Jun 91 03:14:42 GMT References: <795@tivoli.UUCP> <35121@mimsy.umd.edu> <801@tivoli.UUCP> Sender: news@mimsy.umd.edu Distribution: na Organization: University of Maryland Dept of Computer Science Lines: 44 In some article I write: >>Clearly the metrics person must justify that there >>will be some return on the metrics investment, o/w there's no point. In article <801@tivoli.UUCP> alan@tivoli.UUCP (Alan R. Weiss) writes: >My point, which you have NOT addressed, >is that ... If you can't produce >short-term improvements, you're probably in trouble It occurs to me that Mr Weisss is looking for the silver bullet. Count this, do that if the number is in this range, Boom, done. Swell. But I also think I'm selling him short here. It's a good point, but not a great one. Metrics programs are instituted with the goal (as I see it) of improvement in the way a shop does business. I believe that it is vital to understand what you're doing before throwing it away and doing something different. I hesitate to say that a metrics program is inappropriate to the short-term payback, because I lack experience. But I do believe so. A metrics program must begin with a goal. [ I refer folks to the Goal/Question/Metric paradigm developed by my advisor, previously mentioned here, published in IEEE TSE some years ago, don't have the ref at home. ] Your goal could be short term, fine. Your questions and metrics could be very limited and inexpensive to answer and collect. But evaluation (which is what I think you want) is impossible without some corporate memory, some record of prior efforts. Without reference data, you can't evaluate the numbers you get very well. Without an evaluation, how can you improve? Still, the numbers give you insights that you lacked before you measured. Maybe you can justify these early insights as a short-term payback, and a chance to improve. Once you understand your environment thoroughly (probably as a result of a long-standing metrics program), the possibilities for improvement will seem endless. But characterization does not lend itself well to the short term. It takes time and a lot of data to allow you to construct an environment profile, and without a basis for evaluation, your numbers are too likely to be just numbers, subject to any interpretation you come up with. chris... -- Christopher Lott \/ Dept of Comp Sci, Univ of Maryland, College Park, MD 20742 cml@cs.umd.edu /\ 4122 AV Williams Bldg 301 405-2721