Path: utzoo!utgpu!news-server.csri.toronto.edu!helios.physics.utoronto.ca!ists!yunexus!oz From: oz@yunexus.yorku.ca (Ozan Yigit) Newsgroups: comp.lang.scheme Subject: Scheme Benchmarking [topics from hell, part 3] Message-ID: <22632@yunexus.YorkU.CA> Date: 3 May 91 03:18:44 GMT References: <1991May2.155838.20830@bronze.ucs.indiana.edu> Sender: news@yunexus.YorkU.CA Distribution: comp.lang.scheme Organization: York U. Communications Research & Development Lines: 29 jinx@zurich.ai.mit.edu writes: >The numbers don't tell the whole story. When analyzing benchmarks you >have to be very careful. [detailed analysis of a toy benchmark omitted] Bill's point is well taken, but it had been made at a grander scale by Richard Gabriel quite sometime ago, which brings us to the point: We seem to have very little by the way of benchmark code for Scheme implementations, some (most?) of Gabriel stuff is not quite up-to-date for latest n, R^nRS. The Repository does contain some code that can be used as benchmarks (see mixfib, conform[1], peval etc.) but these probably require some agreement, and some careful analysis a la' Gabriel (and a la' PI of Jinx). Now that we have an adequate standard for the language, maybe it is about time to come up with a standard-compliant benchmark suite. We have all learned that benchmarks results should be consumed with some care to avoid indigestion, but not having a consistent suite makes matters inedible. oz --- [Scheme versions of Gabriel benchmarks circa 1987/88 are placed in the repository. Also note prop.scm and proplist.scm] --- In seeking the unattainable, simplicity | Internet: oz@nexus.yorku.ca only gets in the way. -- Alan J. Perlis | Uucp: utai/utzoo!yunexus!oz