Path: utzoo!attcan!uunet!snorkelwacker!bloom-beacon!EXPO.LCS.MIT.EDU!rws
From: rws@EXPO.LCS.MIT.EDU (Bob Scheifler)
Newsgroups: comp.windows.x
Subject: Re: Has anyone used the X Test Suite?
Message-ID: <8911171611.AA22044@expire.lcs.mit.edu>
Date: 17 Nov 89 16:11:50 GMT
References: <215@uucs1.UUCP>
Sender: daemon@bloom-beacon.MIT.EDU
Organization: The Internet
Lines: 37


    Has anyone actually used this test suite?

Yes, many people have.  Certainly we have at MIT.

    Have bugs been reported against it

Yes, a few.  Others are known but haven't been written down, due to lack of time.

    and, if so, are the fixes publicly available?

Not at present.

    I sure hope it isn't an orphan;

Me too.  But, you must realize that verifying such a suite is a rather
substantial task.

    Would anyone connected with the test suite care to comment on the state
    of the software

The most important aspect of the suite is that it is under-reviewed.  Until
a reasonable evaluation of the suite takes place, there's no good way to know
how good or bad it is.  Some evaluation has taken place, but not by enough
people or in enough detail to be satisfactory.  If you have any comments on
the suite, I'd be glad to collect them.  My own guess is that it's rather good
compared to many test suites, but considerably lacking in terms of the coverage
needed to really provide good confidence in an implementation.  For example, we
ran the suite against a pre-release of ours and the server didn't dump core, but
the very first operation with a particular window manager on the same server
did cause a core dump.

    and what we can expect in this area in the future (R4)?

You won't see anything in R4.  The testsuite is considered a separate
distribution.  Work on the suite is on my agenda, but I can't really give
you any more information at present.