Path: utzoo!utgpu!news-server.csri.toronto.edu!bonnie.concordia.ca!uunet!mcsun!hp4nl!donau!duteca4.et.tudelft.nl!kooijman From: kooijman@duteca4.et.tudelft.nl (Richard Kooijman) Newsgroups: comp.ai.neural-nets Subject: Re: solving the XOR problem with Rumelhart's neural net Message-ID: <1991Jun25.112337.12048@donau.et.tudelft.nl> Date: 25 Jun 91 11:23:37 GMT References: <1991Jun24.182310.11745@jato.jpl.nasa.gov> Sender: news@donau.et.tudelft.nl (UseNet News System) Organization: Delft University of Technology, Dep. of Electrical engineering Lines: 38 Nntp-Posting-Host: duteca4.et.tudelft.nl ted@aps1.spa.umn.edu (Ted Stockwell) writes: >In article <1991Jun24.182310.11745@jato.jpl.nasa.gov> greg@huey.Jpl.Nasa.GOV (Greg Wanish) writes: >> Has anyone been able to recreate Rumelhart's performance for the XOR >> problem? I am referring to the results published in PDP Vol. 1, >> Chapter 8: "Learning Internal Representations". A student in his lab, >> Yves Chauvin, trains a 3 node net to solve the XOR problem in an >> average of 245 iterations. When I attempt to recreate this result, my >> net solves it in ~2000 iterations. I believe I have used the correct >> input range (0,1) and output values (.1, .9), and I have used correct >> values for learning rate and momentum -- n = .25 and momentum = .9. >> Omitting the momnetum term, did not decrease my iteration time. I >> have used several different sets of weights, and my results have been >> consistently longer than Rumelhart's. >> >> Greg >I attempted to train 1000 nets with weights and bias terms randomly >initialized to values in the range from 0.5 to -0.5 with a learning >rate of 0.25 and a momentum of 0.9 . Training was aborted if it had >not succeeded after 750 passes through the training data. This was >the case for 264 of the attempts. For the networks that completed >training, an average of 499 passes were required, however the number >of passes needed were well distributed from 250 to greater than 700. > In the book, it does not say how Chauvin's networks were initialized, >and it appears that this makes a significant difference. For anyone who is interested, I can train a 2-2-1 network to solve the XOR function in an average of 15 steps. I use neuron output ranges of -1 to 1. Any value beyond that crashes the back-propagation rule but I haven't figured out exactly why. Does anybody know the answer. Richard.