Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!sdd.hp.com!wuarchive!emory!hubcap!carter
From: carter@iastate.edu (Carter Michael Brannon)
Newsgroups: comp.parallel
Subject: SLALOM Benchmark (Was FLOPS-data)
Message-ID: <12367@hubcap.clemson.edu>
Date: 20 Dec 90 15:27:27 GMT
References: <12330@hubcap.clemson.edu>
Sender: fpst@hubcap.clemson.edu
Reply-To: carter@iastate.edu (Carter Michael Brannon)
Organization: Iowa State University
Lines: 62
Approved: parallel@hubcap.clemson.edu


In article <12330@hubcap.clemson.edu Bernard Bauer writes:
> I want to build up a short clear table with informations about the fastest
> processors/machines (in comparison with parallel systems). I haven't found
> any comparable overview until now.

     I have just what you're looking for!  Here at the Ames Laboratory at
Iowa State University, we have developed a new benchmark: SLALOM (Scalable,
Language-independent, Ames Laboratory, One-minute Measurement)  This bench-
mark runs for a fixed amount of time (one minute), not for a fixed-size
problem.  Consequently, it can fairly benchmark any computer from a lowly
laptop portable to the most powerful supercomputer (and well beyond).

     The first public announcement of the benchmark appeared in the November
1990 issue of Supercomputing Review.  Since then, we have received floods of
mail from individuals wishing to run the benchmark on their machines.  A
comprehensive, and ever-changing, list of performances is available from our
ftp site, as well as source code for many versions of the code.  Our ftp
site's name is "tantalus.al.iastate.edu" and has an IP address of
129.186.200.15.

     The result of a SLALOM run is a single number expressing the "size" of
problem the machine was able to solve in one minute.  Machines are classed
according to this problem size rather than FLOPS for a number of reasons.
First and foremost, floating-point operations are hardly the only work that
a computer does in order to solve a problem.  One must fairly evaluate the
work a machine must do in order to solve a given problem.  One way of doing
this is to pose a standard problem to be solved, and allow people to solve
it in whatever way is most appropriate to the computer in question.  To
quote from the article:

     Most "benchmarks" are simply excerpts from programs that
     have been run on multiple machines, or "synthetic" combin-
     ations of sample operations that correlate poorly with
     entire problems.  Building a good benchmark is a challenge,
     since it has to be unprejudiced toward any machine or
     programming environment, free from undiscovered "shortcuts,"
     and capable of self verification.  Most challenging of all
     is to create a single program that captures salient features
     of scientific computing generally.

     SLALOM can be run on scalar, vector and parallel machines of
     all kinds.

     SLALOM solves a radiosity problem, which amounts to solving a large
system of linear equations, just like LINPACK.  The similarity stops
here, though.  SLALOM also times the I/O required to read in the
problem parameters and write the answer, and the matrix setup.  Further-
more, a sufficiently large problem is solved to allow the machine to
run for one minute.  This allows all machines to show off what they're
made of.

     We currently have a list of more than 50 machines which have run
SLALOM spanning over six orders of magnitude in performance and prices
ranging from $1,000 to $25M.  This list is also available via anonymous
FTP.

     Specific comments and questions about SLALOM can be sent to
slalom@tantalus.al.iastate.edu.

Michael B. Carter
carter@iastate.edu