Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!sdd.hp.com!elroy.jpl.nasa.gov!decwrl!netcomsv!jls
From: jls@netcom.COM (Jim Showalter)
Newsgroups: comp.software-eng
Subject: Re: metrics and the SAT example
Message-ID: <1991May23.014904.5896@netcom.COM>
Date: 23 May 91 01:49:04 GMT
References: <24563@unix.SRI.COM> <1991May21.223401.27023@netcom.COM> <1991May22.222646.10571@ico.isc.com>
Organization: Netcom - Online Communication Services  UNIX System {408 241-9760 guest}
Lines: 38

>Now, note that this doesn't make the SAT inaccurate--it DOES predict what
>it's supposed to predict (for whatever reason), just as Jim said.  But we
>have to be careful that the metrics, particularly if two levels deep,
>predict something useful in the end result.

Agreed. It may seem silly, but my basic point is that if it turned out
that there was a very strong correlation between a metric that measured
the monthly rutabaga consumption of a development team and that team's
success on a project, then it is quite arguable that rutabaga consumption
is a VALID metric. I realize this is a reductio ad absurdum, but consider
this: the SAT's only really measure one's ability to take multiple-choice
tests...and yet there is apparently a very strong correlation between
that ability and one's success in college (this may well say something very
nasty about the state of education in this country, but that's for another
newsgroup!). Hopefully we can even do BETTER than rutabaga consumption,
but if it works, what the hell...

This is all a rather murky area, really. My father pointed out one time
that the statement "it only provides symptomatic relief" was stupid: if
the symptoms of a broken arm are pain, bone jutting from muscle, and an
inability to lift objects with the arm, then relieving those symptoms
is the same as curing the problem--so what's the objection? Similarly,
if SAT's only measure the ability to take multiple-choice tests, but this
predicts success in college, then that's functionally equivalent to
directly measuring college-success-ish-ness. You've provided symptomatic
relief.

The real danger is in arguing backwards from a metric. For example, I
remember reading somewhere that 90% of all violent felons in prison
were determined to have eaten potatoes in one form or another within
the 48 hours preceding their commission of the crime for which they
were convicted. Obviously, then, potato consumption is a valid metric
for violent criminal behavior... ;-)
-- 
**************** JIM SHOWALTER, jls@netcom.com, (408) 243-0630 ****************
*Proven solutions to software problems. Consulting and training on all aspects*
*of software development. Management/process/methodology. Architecture/design/*
*reuse. Quality/productivity. Risk reduction. EFFECTIVE OO usage. Ada/C++.    *