Path: utzoo!attcan!uunet!ogicse!cvedc!nosun!qiclab!m2xenix!quagga!ucthpx!undeed!fakefeed!news
From: m@Daisy.EE.UND.AC.ZA
Newsgroups: comp.software-eng
Subject: Re: recap so far
Message-ID: <900627180723.softweng.26.msg@f4.n494.z5.fidonet.org>
Date: 27 Jun 90 12:41:37 GMT
Sender: guest@ucthpx ()
Lines: 310
Approved: news@Daisy.EE.UND.AC.ZA (rlist 1.10)
X-Gatewayed: by Daisy.EE.UND.AC.ZA (rlist 1.10); Thu, 28 Jun 90 12:30:55 GMT

cgregor@hemlock.Atherton.COM (Scott McGregor)
Date: 26 Jun 90 02:44:21 GMT
Organization: Atherton Technology -- Sunnyvale, CA
Message-ID: <25793@athertn.Atherton.COM>
Newsgroups: comp.software-eng
  
In article <31097@cup.portal.com>, cliffhanger@cup.portal.com (Cliff C
Heyer) writes: 
  
> The whole problem is that SOFTWARE HOUSES REFUSE TO TREAT
> SOFTWARE LIKE ENGINEERING!  They try to treat it like production
> and manufacturing, and then they cry when deadlines are missed.
  
While I agree that software houses refuse to treat software
like engineering, I do not believe that this is why they cry
when deadlines are missed.  Even other engineering businesses have
deadlines and schedules.  Accuracy of prediction varies across
disciplines and even companies.  But people still have them.
Companies "cry" when people miss deadlines because the economic
consequences of missing them can be threatening to either the
individuals or to the organization as a whole.  This is as true
in engineering as in manufacturing and is a psychological and economic
factor, not a scientific or engineering factor.
  
> The point is that "deadlines" are inappropriate in engineering, because
> the quality of the product suffers (defective cars, TVs, etc.) You don't run
> an engineering firm with that mentality. You don't run a software house
> with that mentality, but many do. An engineering firm with a profit only
> motive will go out of business after all the structures it designs collapse.
> Actually, this is what is happening with a lot of software houses
these days.
Actually, since much of engineering IS about tradeoffs it is entirely
appropriate to consider deadlines in development.  Building an elegant
product  to 80% completion and then abandoning it due to lack of money,
or because another company has already nailed down the market is not
good just because the quality of the implementation that was completed
was high.  Success comes only if you survive long enough to deliver
your products.  Money, manpower and time not being unlimited, you have
to meet some constraints to ensure success.  Within those constraints
it is entirely appropriate to concern oneself about product quality.
I also don't doubt that some people paint the constraints as being
tighter than they really are, thereby making inappropriate decisions
about product quality.   But that doesn't mean that one should
abandon managing to constraints--it just means you should get more
realistic managers.  I am loath to make any blanket claims about
any class, but in general, many engineering firms go out of business
not from too much attention to cash flow and profits, but from too little
attention to short term finances to ensure their day to day survival
long enough to reach the long term.
  
> I predict "software building blocks" to be a dismal failure, unless one is
> happy with resource hog programs. There is too much tuning that a craftsman
> can do to double and triple performance in certain situations. Adding
"black boxes"
> to eliminate this process will guarantee mediocre programs. That is,
until we have
> 10000 MIPS on every desktop, then maybe "how well" software works will no
> longer be an issue.
  
Note that this is already happening.  Some machines are getting fast
enough that people are accepting resource hog programs that are easy
to build and maintain even though more efficient systems are possible.
The most obvious cases are 4GLs and spreadsheets that are coded by
non-programmers.  Higher efficiency programs (in terms of transactions
per second, etc.) are possible by trained programmers in languages
like assembler and C.  But these programs are already "fast enough" for
their users, and programmers are not consulted.
  
Additionally, more and more programs are used "off the shelf".  Once
upon a time almost every company's payroll, Accounts Payable and
Accounts Receivables were done with specialized programs suited
especially for that company.  While this is still true for many
large or older companies, most small or new companies now either
purchase canned software packages of this sort or subscribe to banking
services that use one set of these programs for all their clients.
  
> /.. line of code level... [same as] describing the depth of
> /every groove at each point (external interface) and the precise molecule
> /by molecule composition of the material.
> Every item is accounted for in the building estimate, while every item
> IS NOT accounted for in a software estimate. No wonder software
> estimates are wrong so much of the time. But then as Scott points out,
> you can't complete the whole program just to get the software
> estimate. The software business is a tough one to be in.
  
My point is that a succinct description is sufficient to describe
an item in a building estimate, but that without standardized components
this is not possible in software.  If the precise threading depths and
widths and nut and bolt lenghts were not standardized, you would have
to write a paragraph or two about each one used in the building
describing not only these depths and widths and lengths but the
tensile strengths of the materials, their thermal properties, etc.
If this was done for building, instead of relying on standardized components
for which these paragraphs are already written and well-known then
detailed design costs would dominate construction costs too, and
probably fewer buildings would be done with so many plans (in fact,
back when timbers were custom cut and nails were hand made there was
less design plans done.  Even huge cathedrals were often designed "on
the fly" over years and years with different parts of the buildings
done differently by different craftsmen.
  
Now some building projects have really immense planning phases that
dwarf the complexity of some software projects.  However,  for those
complex building projects we find that the rest of the costs of the
project are giant too.  I'll bet that if you compare like size building
projects to like size software projects (say like being in total man
hours to completion of the entire project) you'll find that software
design phase consumes a larger amount of the relative budget than
the building project does.  Increasing this to even larger amount
spent on the project to ensure even more precision is often uneconomic
for software.
  
> /Moreover, because the engineering costs are small
> /compared to the overall construction costs, people are willing to pay for
> /a "paper study"
> This is true. In a big project, the engineering cost is small compared to
> MATERIALS. But the accuracy is good because all the components
> are known quantities to estimate.
  
Yes.  Now if you are a little less specific, you'll save more on the
up front engineering cost, but at increased variability of the production
phase.  At some point you reach an individual's or organization's balance
for predictability vs. cost (the flip side of risk vs. reward).  That's
what determines what the actual tradeoff is.  Unfortunately, largely
because of the lack of software components, the specification process
is very expensive.  So many people trade that off considerable design
specificity to achieve lower costs.  Granted that comes at some cost
in predictability as well.  People would like both increased predictability
and lower cost.  But sometimes the cost factor is the gating factor
since if it costs more you might go out of business or at least
cancel that investment.   In general in the software business today
these tradeoffs tend to be toward lower up front costs at the cost
of poor predictability.  Apparently this common tradeoff is a reasonable
one, since companies that do more complete specs are not by and large
dominating the industries profits yet.
  
> /However, in Cliff's suggested version.....management has to totally
> /commit to a project even before they know what it will cost.
> /...[this] shuffles the cost prediction problem to someone else it
doesn't solve
> /it.
> My point is that software is not the business to be in if you want the
> accurate cost prediction that occurs in the construction industry.
  
I agree.  And I don't think that people are in it for that reason.  I
think that they are in it because there is some money to be made there
and because they think they have the special skills to be successful
there.  I believe this is true at both the macro and micro levels.
Even so, I know that people have different tolerance levels for accuracy
and predictability.  So within any given organization there is some
tension about the precise level of tradeoffs.  It is by no means always
this way, but I have frequently found that many of the programmers have
a lower tolerance for ambiguity than their managers.  So while a manager
WANTS the same low level of unpredicitability as their engineers, and they
may REWARD the more predictable or favorable outcomes, they will be less
tense personally about possible variance in the predictions than the
engineers.   Now this reward structure may tend to aggravate the self-induced
tension that engineers have about lack of precision in their estimates, and
I suspect that this psychological fact may be the real root of this discussion.
  
> I would rather see software classified as engineering, with no construction
> component, and have all software ventures classified as "engineering
> studies". Such studies are not fixed-priced, and the accuracy is equal
> to the amount of cash you pump in. If you want an accurate estimate, you
> PAY FOR IT. Don't do it breaking the backs of programmers who make the
> educated guesses, which is all that can be made unless to program is
> coded and finished. Programmers are really getting a raw deal in the
> current scenario. That's why so many of them change jobs and are
> generally disillusioned with their career.
  
I truly believe that no semantic change in whether we classify the work
as engineering or construction will make a difference in this matter.
As I noted above.  The desire for price control and predictability is
a basic economic need for survival of the individuals (managers) and
organizations in charge.   It is also a basic psychological need of
the individuals for some sort of stability, purpose, and feeling of
control over their future.  I do not believe that you can make these
psychological needs go away by a change of classification or terminology.
  
I do believe that Cliff is correct that if you want an accurate estimate you
have to pay for it.  I believe that any competent manager who has done this
for a while will recognize that this is necessarily true.  But some (and
I include myself) will tolerate some amount of unpredictability ("I just
want a 'ballpark' figure") in order to save some cost.  We don't require
absolute accuracy--close is fine.  Closer is better, as long as it doesn't
cost much.  If it costs too much I settle for less accuracy. I try to
be humane about this, and let my people know that I understand the level
of variance implied (and I take care to consider that when I have to
make committments myself).  However, I have frequently found that
my engineers are personally less willing accept the same level of
ambiguity as I am.    As I say, I try to be humane about this, but
sometimes things do come down to personal differences,  I hope that
my engineers in the past haven't felt ill-served by this--perahps they
will teply and tell me different.  I do think that there is one other
exacerbating problem, and that is that the ability to predict seems
to be in part determined by years of experience.  Some people have
had many years but don't seem to have been able to convert any of
that experience to good use, but there are a many experienced people
who give MUCH BETTER estimates and predictions than others.  I rarely
find inexperienced people who seem to have a talent for good estimates
or predictions.  This is unfortunate for our profession at present,
because so many of the line manager slots are filled with people with
only a few years of management experience and often less than ten
years programming experience.   And so lots of engineers pay for this
poor experience of their management, leading to disillusionment, etc.
  
> /The truth of the matter as to how building contractor's can predict
> /schedules better than software engineers is that they have a book
  
  
> /of expected times to do standard tasks.  The tasks and materials
> /are pretty  standardized and have been done by large numbers of
> /people so that reasonable maximum times have been observed.
> THIS IS THE CORE OF THE ISSUE!
> I submit that this will NEVER occur in software because the components
> are necessarily different for each program's unique needs. All
> buildings are made of the same pieces. Why should we think that
> programs in THOUSANDS of industries will ever share many of
> the same components? I think the idea is poppycock.
  
Personally, I think that the jury is out on this one.  There are still
some "custom craft" tasks that don't show up in contractor's books
and which are subject to more variation and risk.  I think we will
always have some custom craft work as Cliff points out.  But it is not
clear to me that we won't have a growth in components.  In some ways
I think that we already have, but that you see the benefits most
strongly in the end-user systems and applications program areas.
I've already mentioned the re-use of accounting software, of spreadsheet
macros, of 4GL libraries by end users.  We are also seeing standard
libraries like Xtk, Motif, even the unix programming libraries that
people are adopting and reusing where before they would have written
their own window systems, etc. before.  Now I (re)used the Xtk widgets
a lot, and to the extent I did, I didn't have to worry about variance
estimating how long those functions would take to build.  Now, there
was a case where I needed a widget that wasn't in Xtk, so I had to
build it from scratch.  It was harder to estimate. That's life.
  
>
> /It is different in the software world.  ..people ... make
> /an estimate without any "book" [to] consult.
> / ... tasks...can be more variable.
> Exactly what I've said.
>
> /[distinguish] a schedule (i.e. a contract) and an estimate.
> [discussion of inclusion of variances in estimates, scheduling, etc.]
> Omission of these items surely contributes to erroneous estimates.
>
> /For many software projects time to market is
> /extremely important. Early entrants often make significantly more
> /money than late entrants.
> Try a small core of programmers who own enough stock in the company
> that they will make a nice nest egg if the product makes it big. Treat
> them as an R&D group rather than a production group with
> conventional management. I think this will produce more software
> of better quality in less time than conventional approaches. Financial
> backers of the company won't like this approach though because
> if one person leaves, the value of their investment will be diluted.
  
I think that in many cases if you ran this experiment you would see
the benefits you claim.  But I would attribute it to the fact that you
had selected a SMALL group and made the individuals more directly responsible.
Because of their economic incentive I would guess that some would take
a more personal interest in what the customer wanted too.  And those things
can lead to more successful products.  But if no one paid attention to
the costs I would not be surprised if several of these groups ran out
of money before completing their  projects.  And if the host organizations
ongoing success was dependent on these projects I would expect some of
the host organizations to collapse too.
  
> / So there is the natural pressure (mentioned
> /above) to look for optimistic estimates.
> YES! I find I always have to match the estimate my manager
> suggests, because if I don't I'm in trouble. Actually this
> makes estimating easy, except that I am still responsible
> when the estimate is wrong.
  
Well, I can understand that. But if you also say "no guts, no glory",
what does this imply about the need for "backbone" in setting estimates
and schedules that are reasonable?  Doesn't "guts" mean taking an
unpopular stand sometimes in the hopes that long term rewards will
pay off?
  
> Re: Discussion on OOP: 
> I have yet to see how OOP solves any of the problems regarding
> estimating time for writing a program. Someone at some point has
> to develop a sequence of steps that make each "object" work. The
> subjects I'm discussing aren't even relevant to CAD because using
> CAD isn't writing a program - CAD is a higher-level abstraction using
> an existing set of modules already written.
>
The way OOP helps is that it makes the components more standardized
and higher level.  If you can live with the performance and flexibility
trade-offs, OOP means that you will have fewer primitives that you
will have to specify.  Fewer, more standardized primitives means less
detailed and less costly designs relative to the final products. It
changes the cost of the paper study relative to the variability in
the final product, by reducing the cost of the paper study on a
per unit basis.  This doesn't mean that specification isn't done,
it means that instead of having to specify a page of code you say
"displays file name in an XwstaticWidgetClass widget" and you are
just as unambiguous.  That's less to write, less to read, less
possibilities for error in specification, which add up to a less
costly specification.
  
Scott McGregor
mcgregor@atherton.com
  
  
--- QM v1.00
 * Origin: Bink of an Aye - Portland, OR US - PEP/V32 (1:105/42.0)