Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!wuarchive!sdd.hp.com!elroy.jpl.nasa.gov!decwrl!ads.com!saturn!jgautier From: jgautier@vangogh.ads.com (Jorge Gautier) Newsgroups: comp.software-eng Subject: Re: bridge building and discipline Message-ID: Date: 22 May 91 19:31:30 GMT References: <24563@unix.SRI.COM> Sender: usenet@ads.com (USENET News) Organization: Advanced Decision Systems, Mountain View, CA 94043, +1 (415) 960-7300 Lines: 75 In-Reply-To: hlavaty@CRVAX.Sri.Com's message of 21 May 91 16:43:36 GMT In article <24563@unix.SRI.COM> hlavaty@CRVAX.Sri.Com writes: > more than one person enters the discussion. I do not have your values, > experiences, or knowledge (at least, not necessarily) so I can not possibly > accept qualitative information/decisions from you with the same impact that > the information has on you. How do *I* know your right? Why do I care? What > if it's my job to care? Am I supposed to accept your "feelings" on a multi- > million dollar development? If you had a fine track record on a similar > project in the past, I just might do that. But if I am not familiar with > your past history, or this project is new for you, I cannot in good conscience > accept your unsubtatianted opinion. If you don't trust me, why the heck did you hire me to do the job? Why don't you just do it yourself if you're the only one who can interpret information and make decisions? Don't you think that I can substantiate my "feelings"? (Or does everything have to look like a number? :) Who's got a problem here, the person who is reporting something or the person who refuses to accept the report? If the report is irrelevant to the project, just say so. If you don't want to accept it because it doesn't come in "metric normal form," that is your problem. > Where qualitative judgements really break down is when two people with > different opinions view the same situation differently. Who's right? Many > times you just wind up in a pissing contest (to think how many meetings I've > sat in and watched that happen!). Qualitative information is by definition > only reliable to yourself. If that's all that matters in your organization > (if so, I would assume it to be small), you can get away with it. I would > still say your missing an oportunity, however. I would say you're the one who's missing an opportunity if you're ignoring qualitative information. Many things that happen at the reality level have not been quantified, and yet they can have a dramatic impact on the project. For example, how do you measure a bad design? This can be very problem-dependent; are you including a model of the problem in your metrics? Or would you wait until it is implemented so you can measure the defect density, because this is *something* (in your words) that can be measured? What is your early warning for this situation? > The advantage of metrics is that they facilitate a common ground for discussion > between people and organizations. You may disagree with me what the numbers > mean, but we now have a common reference point. The real trick to metrics is > really just to start measuring *something*. After trial and error you will > arrive at "things" to track that will work for you and your organization (all > of which are peculiar, IMHO). What you are after are "early warnings" of > impending problems that allow you time to fix them up front - before they > are problems. I would argue that someone with a lot of experience that isn't > using metrics consciously is actually using them unconsciously (or intuitively). "When searching for the least common denominator, beware of the occassional division by zero." A disadvantage of metrics is that it is easier to LIE with them. I recall a meeting with a previous manager during which he "re-weighted" the severity of the outstanding defects in a system because, well, they weren't really that bad, were they. These were cases were the system clearly did not satisfy the requirements, and worse. Our common ground for discussion simply facilitated the lie. Nobody had the guts to say "we don't care if the system doesn't work sometimes," they simply changed the numbers to make everything look good. Similar problem with other metrics: An unreported defect is not a measured defect. The number of reported defects does not tell you how many defects the system has. The rate of defect discovery depends on how hard you look for defects, and this is not constant over time. The amount of time spent looking for defects and/or the reporting and classification of defects can be fudged to make the rate look better or worse. Lines of code or other size metrics can be twisted to make defect density look better or worse. Test coverage metrics don't tell you how good your test suites are. Etcetera. If you think that the numbers correspond to the reality of the project and that a change in one implies a change in the other, think again. -- Jorge A. Gautier| "The enemy is at the gate. And the enemy is the human mind jgautier@ads.com| itself--or lack of it--on this planet." -General Boy DISCLAIMER: All statements in this message are false.