Path: utzoo!utgpu!news-server.csri.toronto.edu!rutgers!apple!portal!cup.portal.com!mgj From: mgj@cup.portal.com (Mark Gregory Jurik) Newsgroups: comp.ai.neural-nets Subject: Re: re:optimization question Message-ID: <39417@cup.portal.com> Date: 19 Feb 91 10:38:05 GMT References: Distribution: usa Organization: The Portal System (TM) Lines: 25 About training a network to perform the inverse of another net: Regardless of how a network is trained, if the original network does a many- to-one mapping, then the inverse will have a problem with ambiguous one-to- many mappings. If this is not the case, then in response to your statement: " 3. Use the trained network as the input to another network which has one input and many outputs. The problem with this scenario for back-prop is that back-prop learns by propogating the error from output to input. Thus the error at the single input node would tend to move in very large jumps and convergence seems unlikely. " I suggest you consider using BackPercolation instead of BackPropagation. Perc assigns each adaptive cell an output error that is not proportional to the cell's output error gradient. Training is stable and fast with nets having multiple layers. For example, 6 input parity classification was trained using Perc on a net with 6-6-6-1 configuration. It attained a total squared error < 0.0001 in only 93 epochs. Perc does not experience large jumps with either a large number of rows or a large number of cells per row, or both. I am submitting an announcement about BackPercolation to the neural-net digest. If accepted, it will be posted in a few days, I guess. -Mark Jurik, Jurik Research & Consulting, Aptos CA, 95001