Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!usc!apple!snorkelwacker!ira.uka.de!fauern!tumuc!guug!ecrc!micha
From: micha@.de (Micha Meier)
Newsgroups: comp.lang.prolog
Subject: Re: Prolog Efficiency and Benchmarking
Summary: Further issues
Message-ID: <744@ecrc.de>
Date: 8 Aug 90 13:26:19 GMT
References: <708@ecrc.de>
Sender: news@ecrc.de
Lines: 47

Some issues concerning the benchmark suite:

	- Several people mailed me that they would like to obtain
	  the benchmark suite. I'm going to make it available as soon
	  as I have enough interesting programs in it, together
	  with test data and output so that the testing
	  can be completely automatic. The first problem that comes
	  to my mind here is that the same output produced by
	  different Prolog systems might not necessarily be identical,
	  some Prologs put spaces after commas and around operators,
	  other don't, probably there are other differences as well.
	  I guess the output will have to be preprocessed for some systems
	  to be able to compare its correctness.

	  If you are interested in obtaining the test suite, let me know.
	  I'm not yet sure how I will distribute it, probably
	  on a tape for the tape+mailing costs, maybe also by ftp.

	- Whoever else has interesting programs, you can also mail
	  me where to ftp them, it is not necessary to send the
	  program directly.

	- Although this might not be of general interest, I would also
	  like to collect programs using coroutining, to be able to
	  compare performance of coroutining systems.
	  I have some rather small programs, mostly puzzles, but I'm
	  missing serious application programs using coroutining,
	  are there any such programs around? Small programs would
	  do as well (but no n-queens, primes etc.).

	- It is clear to me that every commercial company would like
	  to push its own programs into the benchmark suite, and
	  to avoid programs from other companies. I would like
	  to make clear that ECRC, my employer, is a non-profit
	  organization, and I'm not dependent on the performance of our
	  Sepia system (or others) on the benchmark. I am trying
	  to select the programs using some objective criteria,
	  based on profiling them. However, I do accept test programs
	  even when they are supplied by commercial companies,
	  if they fit into the suite frame.

--Micha Meier
--
USA     micha%ecrc.de@pyramid.com	MAIL	Micha Meier
						ECRC, Arabellastr. 17
EUROPE  micha@ecrc.de				8000 Munich 81
						West Germany