From: utzoo!decvax!yale-com!leichter
Newsgroups: net.physics
Title: Re: Physical Entropy vers. Informational Entropy
Article-I.D.: yale-com.27
Posted: Sat Oct  9 11:55:35 1982
Received: Fri Oct 15 01:34:09 1982
References: eisx.411

Yes, physical and informational entropy are closely related, although I don't
know of any single simple argument that shows why; instead, you have to look at
a number of different experiments and see that a relation seems to exist.

Probably the classic experiment is (all of these are though experiments!) is
Maxwell's demon.  Take a box full of gas at a uniform temperature, and divide
it down the middle by a wall with a door in it.  Controlling the door is a
demon who opens it to let slow particles go from left to right, but not right
to left, and fast particles go from left to right, but not right to left.
After a while, the right-hand side is hot and the left hand side is cold, in
apparent contradiction of the second law.  Now, you can make specialized
arguments about particular means by which the demon could measure the speeds
of the particles to reach his decision, and show that employing them would
require him to expend more energy than he can gain from the heat difference;
but you want a general result that tells you no method can work, whether you
have analyzed it or not.  (Otherwise, we have to throw out the second law!)
So... what you notice is that the fundamental thing the demon is doing is
GATHERING INFORMATION:  initially, he only knows the average speed of the
particles in the two halves, but he somehow determines which particles are
fast and which are slow.  (If you take fast and slow to be divided so that
half the particles are in each subdivision on average - i.e. divide the
distribution in half - then what the demon is getting is essentially one
bit of information - which half of the distribution is the particle in?)
Thus, it appears that gaining information is inextricably tied to expending
energy, since the second law tells us that the demon must be expending
energy.  More detailed calculations even tell you how much energy he must
expend to get one bit of information.

There are other thought experiments that lead to the same conclusion.  More
important, they give you the same number for the energy for one bit.  Thus,
it is reasonable to claim that the information quantity in a bit of in-
formation is a real, physical quantity, not a property of a particular
situation.

These are the general arguments.  In fact, the numbers you get for entropy
in physics and information theory turn out exactly the same (in situations
where both have meaning) mainly by definition:  The information theory
quantity called entropy was DEFINED by analogy to the physical one.  What
the above argument shows is that the definition is not arbitrary; it has
physical content.

As to reducing entropy by storing information in a chip:  Of course you do.
Initially, the state of the chip was random; all you could say was that on
average half the bits are on and half are off.  After you store your infor-
mation, you know the exact state of the system.  This doesn't contradict
the second law unless you managed to store that information without expending
any energy!  In fact, thermodynamics gives you a lower bound on how much
energy ANY chip you could build would require to store a bit.

							-- Jerry
						decvax!yale-comix!leichter