Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!sun-barr!ames!pasteur!agate!shelby!eos!eugene From: eugene@eos.arc.nasa.gov (Eugene Miya) Newsgroups: comp.benchmarks Subject: Re: benchmark evaluations Message-ID: <7692@eos.arc.nasa.gov> Date: 14 Dec 90 07:58:16 GMT References: <12220@hubcap.clemson.edu> <1990Dec12.070209.3272@murdoch.acc.Virginia.EDU> <1990Dec12.140615.27870@cs.utk.edu> Reply-To: eugene@eos.UUCP (Eugene Miya) Organization: NASA Ames Research Center, Calif. Lines: 33 In article <1990Dec12.140615.27870@cs.utk.edu> Dave Sill writes: >In article <1990Dec12.070209.3272@murdoch.acc.Virginia.EDU>, gl8f@astsun.astro.Virginia.EDU (Greg Lindahl) writes: >The fact that you'd moderate this thread out is evidence that leaving >it unmoderated was the right decision. Just because you don't like >it, or think it's good enough, doesn't mean others should be denied it. Moderation != censorship. Moderation should differ little from editorship. There are numerous fine moderated groups. A moderator can cull many "me too postings." A good moderator can contribute knowledgeable information, simple queries can be answered before posting, etc. >Could be. Why doesn't someone keep a table of SPEC results? Actually, we keep tables of results, and we have one of the original SPEC programs. 1) it is not sufficient to simply have the results. 2) when we give the code to a vendor we insist (now) on getting the code back. We want to see modifications. We want to see portability changes. We want to see optimization enhancements. We want to see if iterative algorithms converge (low error bounds). We do not like it when some people claim to have our codes running when in fact, they execute incorrectly. Our code was developed for a 64-bit single precision machine. It takes considerable run time on a non vector architecture. But the biggest interesting problem is that certain manufacturers have a policy of not releasing benchmark results. We have to sign special non disclosure of benchmark result forms. --e.n. miya, NASA Ames Research Center, eugene@eos.arc.nasa.gov {uunet,mailrus,most gateways}!ames!eugene AMERICA: CHANGE IT OR LOSE IT.