Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Posting-Version: version B 2.10.2 9/17/84; site mhuxt.UUCP
Path: utzoo!watmath!clyde!burl!ulysses!mhuxr!mhuxt!js2j
From: js2j@mhuxt.UUCP (sonntag)
Newsgroups: net.origins
Subject: Re: Now more than ever.  PART II
Message-ID: <880@mhuxt.UUCP>
Date: Mon, 20-May-85 16:08:41 EDT
Article-I.D.: mhuxt.880
Posted: Mon May 20 16:08:41 1985
Date-Received: Tue, 21-May-85 06:57:49 EDT
References: <298@cmu-cs-edu1.ARPA>
Organization: AT&T Bell Laboratories, Murray Hill
Lines: 35

> > { from: miller@uiucdcsb.Uiuc.ARPA (A Ray Miller) }
> > ...
> > Sorry, but thermodynamic models have been shown to hold in other
> > areas as well.  One of the fields which can make use of this know-
> > ledge is information theory.  It is here that we can see the the-
> > oretical grounding for evolution crumble.  For example, Dr. Ian
> > McDowell, an information engineer, wrote:
> >
> >      "Communication engineers faced with the problem of coding
> > and transmitting a maximum of information on a given channel have
> > defined quantitatively the information content of a message.  The
> > amount of information to be supplied to transmit any given message
> > using symbol x where the probability of any symbol occurring is
> > P(x) = H(x) = SIGMA P(x) . log2 P(x) which is the negative of the
> > usual entropy formula of thermodynamics.

    The formula you've given for the information content of a message
seems to have been slightly garbled.  Since I'm one of those 'communication
engineers' you reference, allow me:
    The information content of a symbol is:  -log2 P(x).  Thus if you have
only two symbols, occuring with equal likelyhood, the probability of either 
symbol occuring is .5, and the information content in an occurence of a
symbol is -log2(.5), or one bit.  If you have 16 equally likely symbols, 
an occurance of any symbol carries -log2(1/16) or 4 bits.  The information
carried by a message is simply the sum of the information carried by each
symbol occuring in the message.
    I don't have the definition of entropy at hand right now, though I'm
the one who posted it earlier, but the definition I've seen bears no
resemblance to the basic formulas of information theory.  Could you supply
the 'usual entropy formula of thermodynamics' which you reference please?
-- 
Jeff Sonntag
ihnp4!mhuxt!js2j
    "Time has passed, and now it seems that everybody's having those dreams.
     Everybody sees himself walking around with no one else." - Dylan