Path: utzoo!utgpu!news-server.csri.toronto.edu!bonnie.concordia.ca!uunet!mcsun!ukc!pyrltd!pyra.co.uk!tonys
From: tonys@pyra.co.uk (Tony Shaughnessy)
Newsgroups: comp.benchmarks
Subject: How do maxed out users compare to users with think times
Message-ID: <676640053.AA19240@flaccid>
Date: 11 Jun 91 11:34:13 GMT
Sender: news@pyra.co.uk (News system)
Reply-To: tonys@pyra.co.uk (Tony Shaughnessy)
Organization: Pyramid Technology Ltd, Farnborough, ENGLAND
Lines: 62


I am running a benchmark that has been supplied by a customer. The benchmark
simulates n users (up to 100) entering transactions into the application
(a business accounting package) as quickly as possible, without any think
times or any simulated typing rate. Obviously this will be a far heavier
load than 100 real users with think times. The question is, how much heavier?

How do I calculate how many real life users these 100 maxed out users simulate?
I have looked at several approaches. Firstly you could look at the number
of transactions per second put through the database. If I was getting 100
transactions per second from my maxed out users, and the 100 real life users
would only put through 10 transactions per second, then each of my users
is looking like 10 real users. The problem is that when the response times
approach the real life think times, or are much greater than real life response
times, this breaks down because the number of transactions per second then
depends on the response time. Obviously as I add more users,response times
are going to get worse, and TPS is going to get lower. However, there is still
a constant load on the system no matter how many users I put on, because the
system is CPU bound all the time, and even a few of my users can use 100%
of the CPU when they are maxed out. Maybe you could look at the response times
that one of my users produces, and the TPS figure for one user, and compare
that instead, but then that doesn't take any multi user effects into account.

In real life, the load on the system will be 
some function of think time (smaller think times - larger load) and 
the response time (larger response time could be either the cause or 
the effect of larger load, or both). However, with my maxed out users, the
load is independent of the response time as a cause, although larger response
times will be an effect of a larger load. This assumes that load is defined
as percentage of CPU utilisation. Maybe I should be looking at other
definitions of load? See below.

If I try and say that in real life, I will do n TPS, where n is

	number of users / (response time + think time)

then how do I calculate how many real users each of my maxed out users
represents, because think time is zero, and response time varies with the
number of users.

I could use another definition of load - the size of the run queue (this
is a Unix system). I could look at the run queue in a real system and
on my system and compare. Unfortunately I do not have a real system to look
at, I only have my system. The only metrics I know are the predicted think
times, target response times, and actual response times. I can work out
predicted TPS, and I know actual TPS.

A further question is - how does my maxed out user load differ from the
equated number of users with think times? If each of my users is like 10 users
with think times, how do 30 of my users compare to 300 real users? If the
transaction rate is the same, then will the load be the same? I don't think so.
For one thing, contention may be less with 30 users, that is contention for 
locks, etc. . Also, overhead from context switches will be less. What other
effects will there be?

I am going to go away and think hard about this one. My mind is in turmoil.

--
Tony Shaughnessy
tonys@pyra.co.uk

	"Pedal away those tag-nut blues"