Path: utzoo!attcan!uunet!lll-winken!xanth!ukma!tut.cis.ohio-state.edu!bloom-beacon!EXPO.LCS.MIT.EDU!jim
From: jim@EXPO.LCS.MIT.EDU
Newsgroups: comp.windows.x
Subject: Re: X benchmarking
Message-ID: <8905051531.AA08249@expo.lcs.mit.edu>
Date: 5 May 89 15:31:05 GMT
References: <8905050502.AA03307@xuucp.ch.apollo.com>
Sender: daemon@bloom-beacon.MIT.EDU
Organization: X Consortium, MIT Laboratory for Computer Science
Lines: 50


> How can a benchmark client tell when the graphics requests
> have really finished?

Only by doing a request that uses the pixels in question to generate a reply,
such as GetImage.

Of course, doing even a single pixel GetImage will have a major impact on the
timing statistics, so the benchmark needs to be very careful to identify
exactly what it is trying to test (server response time, actual graphics
rendering time, etc.). 


> I saw one benchmark client that used
> a XQueryPointer() call for this purpose. According to my
> understanding, XQueryPointer() will flush Xlib's buffer
> and cause the server to process the graphics requests, but
> if, say, the graphics hardware is itself buffered, the
> graphics requests may not have actually finished at the
> time the server is responding to the XQueryPointer request.

Yup.  You'll see this in any multithreaded server (graphics coprocessors are
a "simple" form of multithreading).

This is one of many reasons why benchmarking X servers is really hard to do in
a general way.  A real benchmark will provide an analysis of every single
request series, describing:

	o  What situations is the test is trying to model?

	o  Why is it relevant to various types of applications (in other words,
		when might real applications do this same operation)?

	o  What "environmental" factors (e.g. transport type and loading, 
		client loading, application load on the server,
		length of time since server was started, length of time since
		client and server were booted, etc.) might affect the results?

	o  What efforts does the test make to explain different results on 
		different platforms and to avoid special cases (e.g. hardware 
		vs. software cursors being in the way, window size, depth, and
		location, alignment of graphics operations, particular bit 
		patterns in pixel values, special clipping, etc.)?


Even testing a single operation is an amazing amount of work.  


						Jim Fulton
						MIT X Consortium