Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!mailrus!cornell!vax5!olky
From: olky@vax5.CIT.CORNELL.EDU
Newsgroups: comp.ai.neural-nets
Subject: Does back-propagation work with a wider dynamic range...?
Keywords: Back-propagation, dynamic range....
Message-ID: <18635@vax5.CIT.CORNELL.EDU>
Date: 18 May 89 15:09:11 GMT
Sender: news@vax5.CIT.CORNELL.EDU
Reply-To: olky@vax5.cit.cornell.edu (M. Kemal Ciliz)
Organization: Cornell Information Technologies, Ithaca NY
Lines: 37


	Hi,

	I am working on various applications of NNs for learning control
	systems, adaptive learning etc. Now, I have a little problem...
	
	I would like to train a feedforward net with input-output patterns
	that have a wider dynamic range. For example the outputs of the net
	will vary , say, between -4.0 ,4.0 or for some cases you don't even
	know the range, because NN will be a part of a dynamic system with
	a certain degree of freedom. 

	So what I did? I modified the activation function for the output
	units and used f(x)=x, also made the necessary changes in the 
	error derivation where you need the derivative of f(x). Anyway
	I got very bad results, huge numbers in the order of billions.

	I used sigmoids at the output layer with arange 0 to 4.0, it didn't
	work...
	Then I tried a symmetric sigmoid with a range -2,2 , again nothing.
	Then I used this limited sigmoid for each layer, no results  again.

	In some papers people claim ,that they trained a net with a linear
	output activation function. How ? 
	I even bought PDP volume III to check their bp.c software. They used
	only the common 0-1 sigmoid. So I was disappointed, and a little 
	bit mad that I spent 30 bucks for it,(since I couldn't get
	reimbursed for this by my advisor..).

	At any rate , CAN smbdy help me with this situation? If anybody
	has worked on problems with wider dynamic ranges, suggestions
	are greatly appreciated...


							Kemal Ciliz
						olky@vax5.cit.cornell.edu
						mkciliz@cmx.npac.syr.edu