Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP Path: utzoo!linus!philabs!cmcl2!harvard!panda!genrad!decvax!decwrl!cooper@pbsvax.dec.com From: cooper@pbsvax.dec.com Newsgroups: net.sci Subject: Parapsychology: S/N Ratios Message-ID: <4029@decwrl.DEC.COM> Date: Thu, 3-Jul-86 13:19:16 EDT Article-I.D.: decwrl.4029 Posted: Thu Jul 3 13:19:16 1986 Date-Received: Sat, 5-Jul-86 04:54:15 EDT Sender: daemon@decwrl.DEC.COM Organization: Digital Equipment Corporation Lines: 145 Bill Jefferys ({allegra,ihnp4}!{ut-sally,noao}!utastro!bill, bill@astro.UTEXAS.EDU) writes: >Experimental data falls into several broad categories. > >In the large majority of cases, statistical treatment of the data is used >to improve signal-to-noise, but the conclusions would be basically >unchanged if they were not used. As detectors become more sensitive, >for example, experiments that used to be very difficult become routine, >and sophisticated statistical processing of the data becomes less >necessary in order to detect the signal. There are two ways that "detectors become more sensitive". One way is that they introduce less noise themselves. For example, IR detectors may be cooled so that their is less internally generated IR. This is certainly an important process but it is not the dominant one. A detector is a device which is used to take an input, extracts the useful part (the signal) and rejects the useless (for the present purpose) part (the noise). It is performing a statistical transformation on the available data. It is no less a statistical transformation because it is performed mechanically by an "analog" computer (in the broadest sense of the term) rather than by explicit calculation. It is no less a statistical transformation because its harder to determine the statistical assumptions, distributions, probabilities, etc. associated with the transformation. The S/N ratio of the phenomena being studied does not decrease, only the S/N ratio at the start of the final stages of processing (those in which people do part or all of the processing) is reduced. The experiments become easier because *more* sophisticated (or at least, more specialized) statistics are being done, and because they are being done mechanically. If there is anything intrinsically wrong with doing statistics on a particular source of data then hiding them won't make it any better. >In many cases, the signal-to-noise is low, and statistics is required to >bring the signal out of the noise. This is where things begin to get >dangerous. The smaller the signal, the more sophisticated the processing, >the more likely it is that small biases in the data (which are unavoidable) >will be amplified and mistaken for signal. Every good experimenter knows >this, and so in well designed experiments, the greatest part of the >effort goes into understanding any independent experimental apparatus. The weakness of the inferences is the sticky point. Under the null hypothesis neither the inferred nor the actual desires should be correlated so the weakness of the inference doesn't interfere with the rejection of the null hypothesis. If we assume that the actual correlation is with subconscious desires, however, a weak inference (weak correlation between actual and inferred desires) would result in a lower measured correlation. This would contribute to the apparent erratic nature of psi, and to the high rate of repeatability failures. This is, of course, only speculation since it is very difficult to test directly, though it does have a fair amount of explanatory power. The attitude of parapsychologists is that there is an unknown source of systematic biases. No known source of such biases, or combination of sources fit its characteristics very well. Maybe someday we'll identify a subtle effect arising from known processes, maybe not. But it is anti-scientific to simply assume the effect arises from known causes or to require full understanding *before* any effort be put into investigating it. Bill Jefferys seems to agree that there are systematic biases. He offers no explanation for them, but simply dismisses them as, es320, p. 119, >13 March 1986). (When Marks is saying anything concrete about parapsychology at all he is talking about repeatability and gross experimental errors. Nowhere can I find any support in his commentary for your interpretation.) You seem to have a rather exaggerated concept of the size of parapsychological experiments. It's true that the average parapsychology experiment uses more data than the average psychology experiment -- but not that much more. A very big parapsychology experiment might involve 20000 trials. More commonly an experiment has 1000 trials. If statistical significance in experiments of that size is digging too deep into the noise level, than we're going to have to throw out many other, more conventional fields: epidemiology, modern crystallography and particle physics come to mind off hand. Some techniques which show great promise (though I think that they are still in need of some maturing) seem to get statistically significant results with only a few dozen trials. We'll have to see how they work out. A good parapsychological experiment is carefully designed to remove or control the known sources of bias. As you say, "The greatest part of the effort goes into understanding and controlling the systematic biases." When such experiments are performed, fairly small but distinct systematic biases are frequently found -- much more frequently than chance would explain. Specifically there is a correlation between the results of a random experiment and the desires of an operator (more accurately, what can be reasonably inferred to be the desires of the operator). The bias remains after changes of target -- when the operator desires another result. Control runs (when there is no operator preference) show no bias. The direction of bias may be positive or negative, but is consistent for a given operator under given conditions, over moderate periods of time. A number of fairly consistent correlations have been demonstrated between various psychological measurements (e.g., beliefs and personality) and the direction of the bias. Others have been demonstrated between other psychological measurements (e.g., moods) and the size of the effect. (The previous description is meant to serve for either ESP or PK experiments. Space doesn't allow discussion of the many techniques which are used to control known sources of statistical bias, check any experimental report in a refereed parapsychology journal). I should comment on my use of the term "desire" in the above description. The actual "desires" of any operator, especially subconscious desires, are unknowable. What we can do is make reasonable inferences about what the operator desires. It is these *inferred* desires which are actually found to correlate with the seemingly independent experimental apparatus. The weakness of the inferences is the sticky point. Under the null hypothesis neither the inferred nor the actual desires should be correlated so the weakness of the inference doesn't interfere with the rejection of the null hypothesis. If we assume that the actual correlation is with subconscious desires, however, a weak inference (weak correlation between actual and inferred desires) would result in a lower measured correlation. This would contribute to the apparent erratic nature of psi, and to the high rate of repeatability failures. This is, of course, only speculation since it is very difficult to test directly, though it does have a fair amount of explanatory power. The attitude of parapsychologists is that there is an unknown source of systematic biases. No known source of such biases, or combination of sources fit its characteristics very well. Maybe someday we'll identify a subtle effect arising from known processes, maybe not. But it is anti-scientific to simply assume the effect arises from known causes or to require full understanding *before* any effort be put into investigating it. Bill Jefferys seems to agree that there are systematic biases. He offers no explanation for them, but simply dismisses them as, essentially, too small to worry about. Too me, that isn't an attitude which is likely to advance scientific understanding very much. Topher Cooper USENET: ...{allegra,decvax,ihnp4,ucbvax}!decwrl!pbsvax.dec.com!cooper INTERNET: cooper%pbsvax.DEC@decwrl.dec.com Disclaimer: This contains my own opinions, and I am solely responsible for them.