Path: utzoo!utgpu!news-server.csri.toronto.edu!mailrus!uflorida!uakari.primate.wisc.edu!sdd.hp.com!ucsd!orion.oac.uci.edu!eabg020 From: eabg020@orion.oac.uci.edu (Donald Doherty) Newsgroups: comp.ai.neural-nets Subject: Re: Observations on the State of NN theory Message-ID: <26D10159.23454@orion.oac.uci.edu> Date: 21 Aug 90 09:39:37 GMT References: <3430008@hpwrce.HP.COM> Reply-To: eabg020@orion.oac.uci.edu (Donald Doherty) Organization: University of California, Irvine Lines: 39 >... I encoded a neural net in a GA and fine tuned it >with gradient descent, but as the net evolved to be larger and larger, it >wouldn't learn more and more. The aggregate of brain material became >computationally "stiff", in that the elements were too tightly >coupled. > >[...] > > What genetic encoding will allow a large network to stay flexible and >be trained with many patterns? Interesting Kingsley... you have seen a principle exhibited by your electronic network that I would have probably predicted given our knowledge of biological systems. At the risk of being overly simplistic, "primitive" or relatively simple organisms tend to be "hard wired" through genetic determination. There are many instances of studies on insect nervous systems, for example, that demonstrate genetically predetermined progenitor cells and *relatively* straitforward mechanisms that lead to specific connectivity (Bently, for instance, studies such a system in the legs of grasshoppers at UC Berkeley.) On the other hand, it is evident that even in relatively "simple" mammals (i.e. mice and rats) genetic factors in connectivity and nervous function are interweaved in a highly complex manner with "epigenetic" factors. For instance, activity dependent changes in genomic expression is a robust and probably widespread phenomenon in the brain (For instance, see papers by Chris Gall on activity dependent changes in expression of various peptides etc. in the hippocampus and dentate gyrus.) These changes almost certainly result in changes in the activity and processing going on in these structures. Your network exhibits "behaviors" stemming strictly from the genetic algorithm giving what could only be relatively unflexible results. It is evident that in biological systems many orders of magnitude more degrees of freedom result from dynamics not coded for in their DNA.