Path: utzoo!utgpu!news-server.csri.toronto.edu!bonnie.concordia.ca!uunet!cs.utexas.edu!sdd.hp.com!wuarchive!mit-eddie!media-lab!minsky
From: minsky@media-lab.MEDIA.MIT.EDU (Marvin Minsky)
Newsgroups: comp.ai.philosophy
Subject: Re: Minds, machines, and Godel
Summary: Maybe a duplicate.  I got rejection message
Message-ID: <4952@media-lab.MEDIA.MIT.EDU>
Date: 21 Jan 91 06:24:04 GMT
References: <1991Jan18.223602.15474@bronze.ucs.indiana.edu> <28154@cs.yale.edu> <1991Jan19.055638.27731@bronze.ucs.indiana.edu> <28203@cs.yale.edu>
Reply-To: minsky@media-lab.media.mit.edu (Marvin Minsky)
Organization: MIT Media Lab, Cambridge MA
Lines: 97

Any system strong enough to express arithmetic is either incomplete or
inconsistent.

Now what does "expressing arithmetic mean".  Because (for reasons that
I suspect Torkel Franzen could explain better than I can, even if he
might not agree with them) I am wary of attempting to express this
formally.  So let's say, simply, that this refers to systems that can
talk about proofs of theorems in the same systems.  What does
arithmetic have to do with this?  Simply because arithmetic is
*sufficient* for making the required self-referential statements -- as
Godel showed with his numbering scheme.  Is arithmetic *necessary*?  I
dunno.  You can do the same things with Smullyan's lexical tricks with
strings, or with McCarthy's very "natural" methods of using
LISP-quotes.

We all know how inconsistencies arose from self-reference, in the work
of the giants upon whose heads we stand.  Theory of types, and
stratification were attempts to avoid it -- but because we really
*did* want to make those deductions anyway, we had then to find ways
to collapse those infinite hierarchies by inventing various sorts of
closure-convergence gadgets.  In my dim understanding of these things,
the same idea seems to keep recurring, at least in spirit: various
sorts of ultrafilters, directed systems, Dana Scott's gadgets, and so
forth.  But these alternatives to stratifications are not "obviously
consistent", although some have not shown flaws for many years now.

OK.  Now Drew McDermott "suspects that" we differ here:

  "I believe [Minsky] thinks that inconsistency is actually a hallmark
  of  inferential power, whereas I think it's an unavoidable bug."

Well, we actually agree, in that the bug seems unavoidable.  Where do
we differ?  If anywhere, which I doubt, it is that I didn't mean to
argue that inconsistency is a virtue in itself.  However, I am pretty
sure that *self-reference*, or rather, the permission to consider,
conjecture, speculate, -- in short, to think freely and reflectively
-- about what one (that is, the thinking system that one is) is doing
may be an indispensable tool for making good discoveries.  And this
makes it worth the risk of making mistakes.  That's what happened to
Frege, and Russell, and Quine, and all those brave pioneers.  

(Now, each of those attempts was followed by making a new
reformulation, so that we can regard the whole community of Logicians
as generating a Myhill creative set, in which each theory generates a
new one.  Of course, there is no difficulty in making a single Turing
machine that does that sort of thing iteratively -- and it is up to
the external observer to regard it as a logic circuit that generates
symbol strings (a view in which the concept of *true* is
inappropriate) or as a formal system that happens to be either
incomplete or inconsistent, depending on how the observer chooses to
interpret what it's doing.  I tend to regard Penrose in both ways,
form moment to moment; first as a somewhat incoherent generator of
(nice-sounding or well-written) strings that have nothing at all to do
with truth, and then as a logical system which, in the "standard
interpretation" is rich in patent inconsistencies and evident
falsenesses.)

To be sure, perhaps self-reference might turn out to not be
indispensable.  But consider the price.  The computational cost of
self-reference is virtually nil.  Your (thinking) machine has an input
channel and an output channel; all you need is a quotation trick to
create processes that self-refer.  You can make your Spelling Program
check the spelling in its source code.  No trouble there.  You can
make your interpreter interpret itself when applied to itself.  Lots
of trouble there.  You can try to inhibit any process that gets too
close to self-reference -- and then you'll have a machine that won't
be about to think about any Godel theorems at all.


So the price -- not of consistency, but of its consequence -- of not
being permitted to be reflective, referential, or -- may I suggest it
-- of not being permitted to be conscious -- seems too high a price to
pay.  Indeed, the most obnoxious part of Penrose' book, at least to
me, is the muddle he makes of consciousness.  Perhaps his confusion
about the various sense of how machines can be seen as being
constrained by logic is why he insists on treating consciousness as a
primitive mystery, rather than feeling obliged to offer substantial
suggestions about the various phenomena associated with that term.
That may be one reason why he seems to propose that consciousness might
involve some entirely new and undiscovered principle of physics. I
might be wrong, but it seems to me that the most prominent aspect of
consciousness is indeed some capacity for being able to manipulate
descriptions of one's own recent mental (computational) activities.
Is there a hidden thread in his reasoning?  My view is that his book
is an "air sandwich" of the following form.

  "I'll show you machines can't think."
  (Two of hundred pages of irrelevant mathematical autobiography)
  "See, I showed you!"

Where's the beef?  Is it possible that this is his hidden agenda:

 "Machines are logical and consistent."
 "Then they can't be self-referent."
 "But people are.  So they can't be machines."

-- Marvin Minsky