Path: utzoo!utgpu!jarvis.csri.toronto.edu!rutgers!gatech!ncar!boulder!bill
From: bill@boulder.Colorado.EDU
Newsgroups: comp.ai.neural-nets
Subject: Re: Original STEP discussion continued
Message-ID: <11494@boulder.Colorado.EDU>
Date: 8 Sep 89 20:56:45 GMT
References: <1082@rex.cs.tulane.edu>
Sender: news@boulder.Colorado.EDU
Reply-To: bill@synapse.Colorado.EDU (Bill Skaggs)
Organization: University of Colorado, Boulder
Lines: 37

[From Cris Koutsougeras:]
>  So I would relate the concept of a learnable function with the TS in use.
>  I would suggest that a given function is learnable with respect to a given
>  TS if it is the "simplest" function which is correct under TS and that the
>  "simplest" correct function is unique.

  Ah yes, Occam's razor slashes again!  But the whole point, the very crux
of the problem, is that "simplest" is not a well-defined concept.  I will
illustrate.

  There is an old mathematician's joke-paradox-problem, which goes as
follows.  The first five elements of a sequence are 1,2,4,8,16 -- what
is the next element?

  Answer: 31.

  Why?  Well, the way to extend a sequence is to find the simplest function
that is consistent with the numbers you have.  But the simplest sorts of
functions, as every mathematician knows, are polynomials; and the
polynomial (1/24)n^4 - (1/12)n^3 + (11/24)n^2 + (7/12)n + 1  , 
for n = 0,1,2,3,4, is the lowest order one that gives the members of the
sequence.  So to get the next element, just plug in n = 5; if you do,
you get 31.

  You say that there is a simpler function?  I say you're wrong.  The
function 2^n is really quite a complicated one -- it only looks simpler
to you because you're familiar with it.

  Okay?
  Well, you needn't take the example too seriously in order to get the
message, which is that "Simplicity is in the eye of the beholder".  I
realize that not everybody will agree with this notion, and it can be
made to seem extremely counterintuitive:  check out Hofstadter's
'Goedel, Escher, Bach' for an opposing viewpoint.  Still, I think
our attempts to design learning systems are gradually forcing
us to realize that there is just no such thing as a universal
notion of "simplicity".