Xref: utzoo comp.ai:8373 sci.bio:4265 sci.psychology:4062 alt.cyberpunk:5640 Path: utzoo!utgpu!cs.utexas.edu!samsung!usc!zaphod.mps.ohio-state.edu!rpi!bu.edu!purdue!haven!ncifcrf!fcs260c2!toms From: toms@fcs260c2.ncifcrf.gov (Tom Schneider) Newsgroups: comp.ai,sci.bio,sci.psychology,alt.cyberpunk Subject: Re: The Bandwidth of the Brain Message-ID: <2015@fcs280s.ncifcrf.gov> Date: 10 Jan 91 22:11:05 GMT References: <37618@cup.portal.com> <2755@infinet.UUCP> <1991Jan9.150033.14718@cs.umn.edu> Sender: news@ncifcrf.gov Followup-To: comp.ai Organization: NCI Supercomputer Facility, Frederick, MD Lines: 52 In article <1991Jan9.150033.14718@cs.umn.edu> thornley@cs.umn.edu (David H. Thornley) writes: >The difficulty with treating multiple meanings as communication is that >what is happening is not exactly communication. I disagree with this. See below. > The example used earlier >in this thread is the word "nuclear", which calls up meanings including >bombs, reactors, families, and cellular reactions. Unfortunately, since >it calls up *all* of these meanings, it cannot be credited with communicating >any of them, since the actual meaning is unclear without other cues. If >it were possible to send multiple meanings selectively, this would indeed >be an effective bandwidth increase. An analogy in computer communications >would be a garbled message that could be interpreted as "compile nuclear.c" >or "archive nuclear.c" or "mail this to the nuclear group". Aha! You have made an intersting statement! The problem is the confusion in the literature about how to measure information. I follow the early workers, and take the measure to be the decrease in uncertainty of the receiver as the measure of the information gained by the receiver. It's a state function. The uncertainty is Shannon's measure: H = - SUM _{all symbols, i} P_i Log_2 P_i where P_i is the probability of the ith symbol. This is NOT the information! If the communication channel is noisy, the reciever has more uncertainty after receiving a symbol or message than before. This is often forgotten, since we assume that our communications are clear when they are not. For example, if I'm thinking of one of the 4 bases of DNA, your uncertainty is 1 in 4 or log_2 4 = 2 bits. If I say "G" then your uncertainty is zero, and the difference is 2 bits. But suppose I said "G" and you were on a terminal where G and C could not be distinguished. Then your uncertainty would be 1 bit after, NOT ZERO because you would still be uncertain about which base (G or C) it was. So the uncertainty decrease is 2 - 1 = 1, and you have learned only one bit of information (ie, it could not be A or T, has to be G or C). The interesting connection is that if I say the word "nuclear" then, although your uncertainty has decreased, it does not go to zero. There remains an uncertainty as to which meaning should apply. It's interesting that one can usually, but not always, resolve that uncertainty from the context! By the way, if you think we are no longer talking biology, check out our recent paper on Sequence Logos (NAR 18: 6097-6100 (1990)), where the same ideas are used to study binding sites. Tom Schneider National Cancer Institute Laboratory of Mathematical Biology Frederick, Maryland 21702-1201 toms@ncifcrf.gov