Path: utzoo!utgpu!news-server.csri.toronto.edu!rutgers!rochester!pt.cs.cmu.edu!o.gp.cs.cmu.edu!andrew.cmu.edu!rr2p+
From: rr2p+@andrew.cmu.edu (Richard Dale Romero)
Newsgroups: comp.ai.neural-nets
Subject: Re: Connectionist Finite State Machines -- description of an architecture
Message-ID: <kbJrbE600Vok4vq0Vv@andrew.cmu.edu>
Date: 1 Dec 90 09:37:20 GMT
References: <7982@uwm.edu>,
	<14611@sdcc6.ucsd.edu>
Organization: Carnegie Mellon, Pittsburgh, PA
Lines: 14
In-Reply-To: <14611@sdcc6.ucsd.edu>

I'm also using this structure for a class paper to see how well it can do
predicting the next letter in a word.  More importantly, it should predict
what letters can not come next in a word.  One thing to watch for in this
sort of task is your training set.  If you don't produce a random distribution
of letter orderings, the network will tend to learn the probability of a
certain
letter appearing next in a word.  In other words, if you train it on twice as
many words that contain "st"'s as "sh"'s, it will learn that the probability
of a 't' following an 's' is twice as high as an 'h' following the same 's'.
It has learned quite successfully learned that certain letters signify the end
of a word, ie 'ly', 'y', 'ing', and that vowels can follow consonants, but I
am still working on getting my training dictionary up to snuff.

-rick