Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!uunet!seismo!sundc!pitstop!sun!decwrl!labrea!rutgers!rochester!PT.CS.CMU.EDU!SPEECH2.CS.CMU.EDU!kfl
From: kfl@SPEECH2.CS.CMU.EDU (Kai-Fu Lee)
Newsgroups: comp.ai
Subject: Re: Practical effects of AI (speech)
Message-ID: <326@PT.CS.CMU.EDU>
Date: Sun, 8-Nov-87 12:14:19 EST
Article-I.D.: PT.326
Posted: Sun Nov  8 12:14:19 1987
Date-Received: Tue, 10-Nov-87 05:38:10 EST
References: <267@PT.CS.CMU.EDU> <930001@hpfcmp.HP.COM>
Sender: netnews@PT.CS.CMU.EDU
Organization: Carnegie-Mellon University, CS/RI
Lines: 39

In article <930001@hpfcmp.HP.COM>, gt@hpfcmp.HP.COM (George Tatge) writes:
> >
> >(1) Speaker-independent continuous speech is much farther from reality
> >    than some companies would have you think.  Currently, the best
> >    speech recognizer is IBM's Tangora, which makes about 6% errors
> >    on a 20,000 word vocabulary.  But the Tangora is for speaker-
> >    dependent, isolate-words, grammar-guided recognition in a benign
> >    environment. . . .
> >
> >Kai-Fu Lee
> 
> Just curious what the definition of "best" is.  For example, I have seen
> 6% error rates and better on grammar specific, speaker dependent, continuous
> speech recognition.  I would guess that for some applications this is
> better than the "best" described above.
> 

"Best" is not measured in terms of error rate alone.  More effort and
new technologies have gone into the IBM's system than any other system, 
and I believe that it will do better than any other system on a comparable
task.  I guess this definition is subjective, but I think if you asked other 
speech researchers, you will find that most people believe the same.

I know many commercial (and research) systems have lower error rates
than 6%.  But you have to remember that the IBM system works on a 20,000
word vocabulary, and their grammar is a very loose one, accepting
arbitrary sentences in office correspondences.  Their grammar has a
perplexity (number of choices at each decision point, roughly speaking)
of several hundred.  Nobody else has such a large vocabulary or such
a difficult grammar.  

IBM has experimented with tasks like the one you mentioned.  In 1978,
they tried a 1000-word task with a very tight grammar (perplexity = 5 ?),
the same task CMU used on Hearsay and Harpy.  They achieved 0.1% error
rate.

> George (floundering in superlative ambiguity) Tatge

Kai-Fu Lee