Xref: utzoo comp.theory:564 sci.math:10576
Path: utzoo!attcan!uunet!wuarchive!brutus.cs.uiuc.edu!samsung!xylogics!transfer!lectroid!lectroid.sw.stratus.com!cme
From: cme@lectroid.sw.stratus.com
Newsgroups: comp.theory,sci.math
Subject: Definition wanted:  a kind of entropy
Keywords: entropy, information theory
Message-ID: <1059@lectroid.sw.stratus.com>
Date: 10 Apr 90 18:04:18 GMT
Sender: usenet@lectroid.sw.stratus.com
Reply-To: cme@ellisun.sw.stratus.com
Followup-To: sci.math
Organization: Stratus Computer, Software Engineering.
Lines: 24

I know how to compute the bits of entropy of the outputs of a
communications channel given the probability distribution of its
alphabet.  I want to compute bits of information from a different
source.


Let there be a boolean array, f, of length N filled with random bits
and a communications channel carrying symbols, x_{i} in an alphabet of
N characters.  Let there be a machine which takes that channel as
input and outputs y_{i} = f( x_{i} ).

I want to compute the number (fraction) of bits of information about
x_{i} which I learn from y_{i}.  y_{0} gives me 0 bits of information
because it is totally random.  y_{1} gives me a small amount of
information because if y_{1} != y_{0} then we know that
x_{1} != x_{0}.

I don't know how to compute that amount of information.

If I get good direct replies, I'll post them.

Thanks,

	Carl