Path: utzoo!utgpu!news-server.csri.toronto.edu!rutgers!usc!snorkelwacker.mit.edu!paperboy!meissner From: meissner@osf.org (Michael Meissner) Newsgroups: comp.benchmarks Subject: Re: bc is bs Message-ID: Date: 8 Dec 90 01:24:16 GMT References: <1990Dec6.133344.2544@cs.utk.edu> <109969@convex.convex.com> Sender: news@OSF.ORG Organization: Open Software Foundation Lines: 26 In-reply-to: tighe@convex.com's message of 6 Dec 90 18:55:10 GMT In article <109969@convex.convex.com> tighe@convex.com (Mike Tighe) writes: | Not only is the benchmark itself worthless, but so are the times reported | by the benchmarkers. Why? Because the benchmark requires elapsed (real) | times to be reported, and this number can only be reliable when the | benchmark is run in dedicated mode. ... That is certainly the conventional wisdom, but let me offer something. You should benchmark a machine in the state it's going to be used. For example, it is unrealistic to benchmark a large machine with nothing else running, if you can't afford to buy the machine for that single task, and that task only (ie, make the Cray your workstation). Thus, if fifty people are going to be sharing a machine in a timesharing fashion, you should test it with multiple jobs running. Similarly, when benchmarking single user, multitask workstations (ie, a typical UNIX box), you should have the network daemons running, X windows, a couple of xterms running, and possibly GNU emacs doing something useful. -- Michael Meissner email: meissner@osf.org phone: 617-621-8861 Open Software Foundation, 11 Cambridge Center, Cambridge, MA, 02142 Considering the flames and intolerance, shouldn't USENET be spelled ABUSENET?