Path: utzoo!utgpu!news-server.csri.toronto.edu!mailrus!wuarchive!zaphod.mps.ohio-state.edu!uakari.primate.wisc.edu!aplcen!haven!ncifcrf!lhc!usenet
From: usenet@nlm.nih.gov (usenet news poster)
Newsgroups: comp.ai.neural-nets
Subject: Re: Observations on the State of NN theory
Message-ID: <1990Aug22.015905.6922@nlm.nih.gov>
Date: 22 Aug 90 01:59:05 GMT
References: <schraudo.651012988@beowulf> <3430008@hpwrce.HP.COM>
Reply-To: states@tech.NLM.NIH.GOV (David States)
Organization: National Library of Medicine, Bethesda, Md.
Lines: 36

kingsley@hpwrce.HP.COM (Kingsley Morse) ("km>") writes:
Nicol N. Schraudolph ("ns>") writes:

ns> most of the GA/NN research is aimed at
ns> finding a GA (specifically, a genetic representation of NNs) for which
ns> the recombination operator exploits some regularity concerning the basins
ns> of attraction for NN gradient descent.  The two main questions are:
ns> 1) Are there any such regularities in the first place, aside from simple
ns>    invariances such as flipping the sign of all weights?
ns> 2) Can we find genetic encodings and/or recombination operators that
ns>    exploit them?

[...]
km> We know that GAs can evolve true intelligence, because we've evolved to
km> our present human intellect. 

I am not sure that this is strictly correct.  Biological genetics evolved
a neural structure capable of being trained, but I am not aware of any
evidence for genetic type algorithms actually playing a role in biological
learning.  Specifically, somatic cells, such as neurons, do not undergo
recombination.  There are periods of large scale cell death in neurological
development so a "selection of the fittest" parallel algorithm might
be invoked, but that is not a recombination operator.

km> But just knowing that GAs CAN work isn't enough. The question now is:
km> What genetic encoding will allow a large network to stay flexible and
km> be trained with many patterns?

Maybe a better question is: what network structures are trainable?  

Food for thought:  how hard can you expect training to be?  The brain
has on the order of 10^10 neurons each with 10^2 to 10^4 synapses and
fires ~10 times/second.  That adds up to 10^13 - 10^14 connections/second.  
Training a human still takes a few years.

David States