Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP Posting-Version: version B 2.10.2 9/18/84; site peora.UUCP Path: utzoo!watmath!clyde!burl!ulysses!mhuxr!mhuxt!houxm!vax135!petsd!peora!jer From: jer@peora.UUCP (J. Eric Roskos) Newsgroups: net.arch Subject: Re: Scientific Computing and mips Message-ID: <1573@peora.UUCP> Date: Wed, 4-Sep-85 08:12:02 EDT Article-I.D.: peora.1573 Posted: Wed Sep 4 08:12:02 1985 Date-Received: Thu, 5-Sep-85 08:21:48 EDT References: <419@kontron.UUCP> <2300001@uicsl> <1093@ames.UUCP> <1119@ames.UUCP> Organization: Perkin-Elmer SDC, Orlando, Fl. Lines: 78 Eugene Miya writes: > One problem with computer science is that we have poor empirical and > laboratory skills and tools. I wrote, about an earlier topic: >>This statement isn't true. For example, back when I was a graduate-student >>researcher in computer architectures, we used the Whetstones to test our >>vertical-migration software. Eugene Miya replies: > I would Still place you in that category. Another problem with computer science, at least as it is often practiced in the technical newsgroups, is a failing of the scientific method in other areas; we resort to ad hominems to justify our positions, and make contradictory statements without explaining why. If the only people who use Whetstones are the manufacturers of the computing machines, and if I am "still" a graduate student researcher because I use them, isn't that contradictory? Actually I cancelled the message on benchmarks because I decided it didn't say much. Unfortunately, the "cancel" command apparently often doesn't work. And also, actually I mostly agree with you (except the ad hominem parts, of course). But to answer the questions: > How do you convince of compiler independent? That's very hard. If you are doing the evaluations yourself, you write the algorithm in a high-level-language, then hand-compile it the way you think a good compiler would compile it. Thereby you at least convince yourself. This is better than being unconvinced by "black box"-type benchmarks, though I agree that it is far from ideal, and far from what would be acceptable in many scientific communities for final results. > What about compiler dependent? What tests which determine compiler > characteristics? Limitations? It's been my experience that simply looking at the code generated by a compiler often tells a lot about the quality of the compiler. Of course, this breaks down when you get to the point of comparing several good optimizing compilers with one another; but most of the compilers out there today, especially for the microcomputers, are not that good, yet; their characteristics and limitations are readily visible in the code they generate. > OS independent, too. Just don't make any OS calls. I don't think many benchmarks do. If you are benchmarking I/O, well, unless you are going to do the I/O yourself, then probably the OS performance IS worth benchmarking. > A problem with uniprocessor architectures, computers, and operating > systems is that all are constructed in such as way as to make > measurement difficult. Taking a measurement affects the > thing being observed. R. I. Winner, my committee chairman, used to call this phenomenon (but as it related to debugging) "Heisenbugs". (I'm not sure if he coined the term, or not.) I don't think they are constructed *to* make measurement difficult. Rather (a) the economics of putting the instrumentation on to monitor the performance from the outside tends to discourage that in the marketplace, and (b) there are always things you just can't measure from inside the system. It would be nice to have performance measuring support that worked from outside (e.g., small processors dedicated to watching the cache, peripherals, etc. without disturbing them), but when it came to making the machine cost-effective to market, those would be one of the first things to be eliminated, I'm afraid. -- Shyy-Anzr: J. Eric Roskos UUCP: ..!{decvax,ucbvax,ihnp4}!vax135!petsd!peora!jer US Mail: MS 795; Perkin-Elmer SDC; 2486 Sand Lake Road, Orlando, FL 32809-7642 Uryc! Gurl'er cnvagvat zl bssvpr oyhr! Jung n qrcerffvat pbybe...