Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!samsung!rex!ames!dftsrv!hq!lgn!spoffojj From: spoffojj@hq.af.mil (Jason Spofford) Newsgroups: comp.ai.neural-nets Subject: Re: Observations on the State of NN theory Keywords: Genetic Neural Training Pepsi Message-ID: Date: 4 Aug 90 17:36:05 GMT References: <1990Aug3.175023.28210@ariel.unm.edu> Sender: news@hq.af.mil Lines: 47 bill@hooey.unm.edu (william horne) writes: >I don't think GAs have much to offer for learning techniques in networks >which have a good gradient search technique for learning (i.e. >MLPs, recurrent networks, etc...), and especially when these networks >use floating point weight representations. GA's are general purpose algorithms and can't hope to compete with specifically tailored training algorithms. The problem is that a specific training algorithm can only solve a small subset of problems. The GA fits in by solving many problems equally well (or bad .. depending on your perspective). > My experience with >GAs have been that they are terrible at searching the bizarre error surfaces >associated with something like MLPs, in fact they are no better than a >completely random search. This seems to be due to the fact that the bits >in floating point representations are highly correlated with each other. >There are things you can do to avoid this, like Grey coding and not allowing >crossovers in the middle of a 32-bit word, etc... These algorithms seem >to improve the performance of the GA, but not to the point where they are >competitive with a simple gradient search. I am not sure exactly how you are defining random search. Do you mean hoping around in the search space totally at random, keeping the best individual to date? Obviously, GA's are biologically inspired. If one is not a creationist, than one can only attribute the development of NN solutions in living creatures to this process. So there is motivation and justification for researching the topic. You touched upon the most important aspect of research of GA/NN's and that is the genetic representation. The one I use is not the answer but it does provide positive results! My solution to the 9x9 37 character recognition problem took just over one trial per bit of genetic code to solve the problem (much better than random). The nature of the search space the GA must traverse is directly related to the genetic representation. But once a good genetic representation is found, than a myriad of problems can be solved simply by changing the fitness function. Training algorithms can't compete with that type of flexibility, although they may solve their subclass of problems efficiently. -- ---------------------------------------------------------- ) Jason Spofford The LAN Manager ( ) spoffojj.hq.af.mil George Mason Univ. Grad. Stud. ( ----------------------------------------------------------