Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!uunet!mcsun!unido!uklirb!shell
From: srt@aero.org (Scott TCB Turner)
Newsgroups: comp.ai.shells
Subject: Re: KES Timing Result
Message-ID: <7667@uklirb.informatik.uni-kl.de>
Date: 22 Mar 91 17:07:27 GMT
References: <7642@uklirb.informatik.uni-kl.de> <7658@uklirb.informatik.uni-kl.de>
Sender: shell@uklirb.informatik.uni-kl.de
Organization: The Aerospace Corporation, El Segundo, CA
Lines: 27
Approved: shell@dfki.uni-kl.de
Posted-Date: Mon Mar 25 08:32:08 GMT 1991

(Klaus ten Hagen) writes:
>Unfortunately the problem is that such an ``benchmark'' not even gives
>``a general feeling'', since the speed determining parts of an
>rulebased system are not tested by such a crude trial.

Nonsense.  Repeated firing of a single rule tests simple conditions,
rule activation, and the internal representation of rules and data (to
the extent that compiled representations will be faster than
interpreted ones).  Simple tests are simple; that doesn't mean they're
worthless.

>...It is well-known that in more realistic examples around ~97% of
>the runtime is spent in the matching phase.

The most famous expert system ever, MYCIN, did little or no pattern
matching.  The telemetry-based diagnosis system I work on does no
matching, and I suspect that diagnosis systems in general do little
matching.  What they do is test symptom values.  Conversely, the
ART-IM people claim that large expert systems are dominated by the
time to select rules for activation; hence their concentration on an
efficient RETE algorithm.  (Although to some extent that subsumes the
matching problem.)  In the case of an embedded expert system that is
called repeatedly from scratch, execution time may well be dominated
by initialization costs.  To me it isn't at all clear that the right
metric for testing the speed of expert systems is the speed of the
matcher.  Perhaps you can share in more detail the results to which
your refer?