Path: utzoo!utgpu!jarvis.csri.toronto.edu!mailrus!cs.utexas.edu!uunet!mcsun!ukc!strath-cs!jim
From: jim@cs.strath.ac.uk (Jim Reid)
Newsgroups: comp.sys.sequent
Subject: Re: timing parallel programs
Keywords: timing, sequent, multi-processor
Message-ID: <1803@baird.cs.strath.ac.uk>
Date: 12 Feb 90 15:28:54 GMT
References: <1324@csisles.Bristol.AC.UK>
Sender: news@cs.strath.ac.uk
Reply-To: jim@cs.strath.ac.uk
Organization: Comp. Sci. Dept., Strathclyde Univ., Scotland.
Lines: 38

In article <1324@csisles.Bristol.AC.UK> beaumont@CompSci.Bristol.AC.UK (Tony Beaumont) writes:
>We want to get accurate timings of the run-times of Prolog programs when we
>use 1, 2, ... 11 processors in order to accurately measure the speed-ups.

> Our current solution is to ask other users to log off when we make
>timing runs, ensuring the load on the machine is as low as possible.
>However, operating system processes are still running and although we
>always leave at least 1 processor idle, how can we be sure that the
>operating system processes do not interfere with (slow down) the processes
>we are timing?  Also this approach is rather unfortunate in that we
>require all other users to log off which in effect means that we have to
>make our timings during the night.

If you are hell-bent on timing *only* your application, you have little
choice but to dedicate your machine to that task. That means kicking off
all the users and booting the machine with a minimal number processes
active - essentially in single user mode. You wouldn't want the results
to be affected by a tty interrupt or a packet arriving from the ethernet
or whatever. This will give you accurate results but is somewhat
extreme.

Note that there will always be some kernel latency. [This is because of
the way the UNIX kernel works.] Your application process may have to
switch into kernel mode to do some interrupt handling, so unless you can
disable the interrupts, your results will be affected by the demands of
interrupt servicing.

Do you *really* need that accuracy? Most people treat benchmarks with
deserved scepticism so your super-accurate results are only likely to be
interpreted as a rough rule of thumb. If that's all your looking for,
you might as well run the timings while the machine is in use.
Personally speaking, I'd be happier if I knew that timings included the
noise of other system activity. I don't run applications on an empty
machine, so why should I run benchmarks in that way. To me, the absolute
numbers are not important - it's more useful to know what to expect when
the system's doing real work...

		Jim