Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!sdd.hp.com!elroy.jpl.nasa.gov!ncar!noao!asuvax!mcdphx!hrc!gtx!al
From: al@gtx.com (Alan Filipski)
Newsgroups: comp.ai.neural-nets
Subject: Re: Observations on the State of NN theory
Message-ID: <1327@gtx.com>
Date: 24 Aug 90 20:01:22 GMT
References: <schraudo.651012988@beowulf> <3430008@hpwrce.HP.COM>
Reply-To: al@gtx.UUCP (Alan Filipski)
Organization: GTX Corporation, Phoenix
Lines: 36

In article <3430008@hpwrce.HP.COM> kingsley@hpwrce.HP.COM (Kingsley Morse) writes:
>I agree with Nicol. I encoded a neural net in a GA and fine tuned it
>with gradient descent, but as the net evolved to be larger and larger, it
>wouldn't learn more and more. The aggregate of brain material became
>computationally "stiff", in that the elements were too tightly
>coupled.
>
>We know that GAs can evolve true intelligence, because we've evolved to
>our present human intellect. But just knowing that GAs CAN work isn't
>enough. The question now is:
>
>    What genetic encoding will allow a large network to stay flexible and
>be trained with many patterns?

Complex biological (not just nervous) systems use strategies of
hierarchical and sequential decomposition.

Our brains, for example, are not randomly connected nets but have
thousands of recognizable discrete structures, from tiny nuclei to huge
cortical sheets.  Many of these are repeated and some are found within
others.

Hierarchy and repetition seem to correspond to context-free and regular
languages, respectively.  I don't know much about GA's, but is seems to
me that a grammatical encoding might be appropriate to get around that
scale barrier.  A "mutation" might involve not only tweaking some
weights or connections, but repeating whole structures or putting them
together in different  ways.

Just a half-baked idea.


  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 ( Alan Filipski, GTX Corp, 8836 N. 23rd Avenue, Phoenix, Arizona 85021, USA )
 ( {decvax,hplabs,uunet!amdahl,nsc}!sun!sunburn!gtx!al         (602)870-1696 )
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~