Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!usc!jarthur!nntp-server.caltech.edu!mustang!data.nas.nasa.gov!amelia!eugene
From: eugene@nas.nasa.gov (Eugene N. Miya)
Newsgroups: comp.benchmarks
Subject: Re: performance vector approaches
Message-ID: <1991May7.203052.18096@nas.nasa.gov>
Date: 7 May 91 20:30:52 GMT
References: <1991Apr25.174542.100@skyler.mavd.honeywell.com> <APM.91Apr28021454@vipunen.hut.fi> <1991Apr29.125222.103@skyler.mavd.honeywell.com>
Sender: news@nas.nasa.gov
Reply-To: eugene@amelia.nas.nasa.gov (Eugene N. Miya)
Organization: NASA-Ames Research Center
Lines: 29

Performance vectors (done before) [do you need more references?].
This is a nice idea in theory, but very difficult to do in practice.
It is subject to many non-linear, hard to control effects:
operating system paging and swaping, compiler optimizations, cache
effects, etc.  I think we will inevitability be forced to do this,
but it would require some stabilization in the industry.  One might be able
to make allowances for these effects.  But we are talking major work.
It requires a "bottom-up" approach, and it won't be as portable
as the "bc/useless-benchmark."  So until companies are willing to make
public their compiler technology, and users take the time to analyze
their codes, and hardware stablizes.  I doubt it would happen.

Oh, I will say that I think it will happen on supercomputers first,
some consensus on theory is required (we think far too linearly:
we believe in the "mythical-MFLOPS/MIPS:" 1000 CPUs == a CPU which is
1000* as fast.  Not so.).  The theory will include: "semi-groups,"
vector-valued measures, and we need more metrics to characterize
software (I've thought about Halstead measures and I know the people
at the SRC are thinking about McCabe measures).  We must fight
useless measures.  I think it will happen, just not quickly because of
market and political considerations.  It will happen because some
applications need the performance.  I don't think workstations will
drive this work.

I'm just trying to catch up on this news group.

--eugene miya, NASA Ames Research Center, eugene@orville.nas.nasa.gov
  Resident Cynic, Rock of Ages Home for Retired Hackers
  {uunet,mailrus,other gateways}!ames!eugene