Xref: utzoo comp.society.futures:1602 comp.software-eng:2957
Path: utzoo!utgpu!jarvis.csri.toronto.edu!rutgers!bellcore!dduck!duncan
From: duncan@dduck.ctt.bellcore.com (Scott Duncan)
Newsgroups: comp.society.futures,comp.software-eng
Subject: Re: Retrospective Forecasting
Message-ID: <20406@bellcore.bellcore.com>
Date: 28 Feb 90 12:26:54 GMT
Sender: news@bellcore.bellcore.com
Reply-To: duncan@ctt.bellcore.com (Scott Duncan)
Followup-To: comp.software-eng
Organization: Bellcore, Piscataway, NJ
Lines: 78

In article <5473@bgsu-stu.UUCP> klopfens@bgsu-stu.UUCP (Bruce Klopfenstein) writes:
>I am very interested in the discussion going on about forecasting.
	[In comp.society.futures]
>I have studied technological forecasting, and what surprises me is
>the seeming lack of research into past forecasts.

>                                                                What
>better way to learn about forecasting than to re-analyze past forecasts 
>and look, for example, for systematic errors in the process.

>                                                         Little is
>learned from past mistakes.  I'm curious what you forecasting theorists
>and practitioners think about that.


>Dr. Bruce C. Klopfenstein      |  klopfens@andy.bgsu.edu
>Radio-TV-Film Department       |  klopfenstein@bgsuopie.bitnet
>Bowling Green $tate University |  klopfens@bgsuvax.UUCP
>Bowling Green, OH  43403       |  (419) 372-2138; 352-4818
>                               |  fax (419) 372-2300

It struck me that this applies equally well to software projects.  In discus-
sing software quality and productivity efforts with a number of people over the
past couple of years, this has been one of the major failings noted.  There is
very little effort put into retrospective analysis of what happened in a pro-
ject.

My sense is that some large projects (especially those for the military) may
have this done to them and some very small projects, where the individual de-
termination of a few people prevails, do this.  However, the 100K - 5M range
(lines of code) does not seem to attract much of this.

I have heard many reasons given for this, but the ones repeated most often are:

	o by the time a project is done, folks are already committed to
	 the next project (or, at least, the next release of the one they
	 just completed), i.e., there is no time alloted in the original
	 (or following) schedule for such activity;

	o people keep very poor records of what went on in the project,
	 e.g., history, documentation of decision-making, etc., so there
	 is little data to operate on other than people's memory of what
	 happened, i.e., people won't trust or find worth in the effort
	 with such "unreliable data";

	o the impression is that this effort is to find out what went wrong
	 (rather than what worked well and then deciding how to repeat those
	 successes) and folks involved don't want to spend time on an activity
	 that can only make them look bad;

	o it takes a lot of time and human effoprt, i.e., it costs a lot,
	 and nobody's risked doing it enough (and written about it) to show
	 real benefit in subsequent projects from having done retrospectives;

	o the original teams usually move on to other projects so, even if
	 you uncovered things of interest, the next project will have a dif-
	 ferent staff and is likely, because of individual differences, to
	 display a different profile of behavior.

While there is truth in all of them, I think the negative aspects of that
"truth" can be overcome.  In particular, if such retrospective activity and
project history is a pre-planned part of the project, I think much of the prob-
lem goes away.  What is a problem is justifying, in the short term, the cost of
such effort since the long-term benefit is generally where you'd expect to see
the cost recouped.

On the other hand, my personal belief is that the very next project would prob-
ably benefit from such an effort.  I look at it as a parallel to code inspec-
tions (also labor intensive but effective):  there's an up front cost, but peo-
ple begin to see the back-end benefit rather quickly once they actually get in-
to doing it.

I'd be interrested in people's comments about these points.

Speaking only for myself, of course, I am...
Scott P. Duncan (duncan@ctt.bellcore.com OR ...!bellcore!ctt!duncan)
                (Bellcore, 444 Hoes Lane  RRC 1H-210, Piscataway, NJ  08854)
                (201-699-3910 (w)   609-737-2945 (h))