Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP Path: utzoo!utgpu!water!watmath!clyde!cbosgd!ihnp4!homxb!mhuxt!mhuxm!mhuxo!ulysses!allegra!alice!jj From: jj@alice.UUCP Newsgroups: rec.audio,sci.physics,sci.electronics Subject: Re: Mercury Filled Speaker Wire Message-ID: <7280@alice.UUCP> Date: Wed, 16-Sep-87 12:43:35 EDT Article-I.D.: alice.7280 Posted: Wed Sep 16 12:43:35 1987 Date-Received: Sat, 19-Sep-87 08:38:51 EDT References: <3827@watdcsu.waterloo.edu> <551@unisoft.UUCP> <3837@watdcsu.waterloo.edu> Organization: AT*T-BL, Murray Hill, Signal Processing Research Department Lines: 92 Xref: utgpu rec.audio:2795 sci.physics:2097 sci.electronics:1188 Summary: sigh! > = Maraldo on double blind testing... The numbers in parenthises are mine, and refer to comments below. > I do not feel that a simple double-blind test can report more accurate > data than an extended listening test. (1) For me, a double-blind test must > be rigorous, extended, and thorough; that is, there must be a large > number of people involved in the evaluation(2), and the test procedure > must be overseen by at least one member of a group of potentially biased > individules(3) (one audio engineer(4), physicist(5), golden ea(6), and stereo > salesman(7)). As each procedure is concluded, the overseeres must > unanimously agree that the procedure was unbiased before enumerating > the resultsa(8). The listening environment must differ between anechoic and > very 'alive'(9), with each group listening in the different situations; > one listening environment does not suffice. (1) You say "feel", but you do not address the many published references that assert you are completely incorrect. Certainly any test must be done in an unbiased manner, and a careful and thorough description of the test conditions should be in the test report, to show beyond any doubt what the test did, and what the test did not, control. (2) There are simple statistical tests for "how many is enough", and good tests report on their statistical significance as a matter of course. (3) It's not clear what having biased individuals "oversee" a test amounts to. If it's censorship of something that, for example, audio salesmen don't want known (the relationship between small loudness differences and 'quality', for example), then you'll get your way, I'm sure. You say "individuals", you then proceed to name: (4) audio engineer - Perhaps reasonable, certainly the equipment must be working correctly, and all equipment in a subjective test should be periodically verified. Perhaps the engineer will also have some knowledge of what the test is about, if so, the bias must be deliberately EXcluded. (5) physicist - I don't know why you say this. Perhaps you mean acoustician, psycho-acoustician, physiological acoustician, or somethiung like that, but what a particle physicist, for example, would do is beyond me. If you mean something more specialized, say it, after all, you appear to be attempting a concise requirements document. (6) Golden ear - Say what? Define "golden ear". If you mean people who appear to have very good hearing, and can detect differences that other people usually can't, I'd rather have them IN the test. I'd prefer to have one or two try out the equipment beforehand, to ensure correct operation, but what's the POINT of having a non-expert verify the experimental design and operation. (7) Stereo salesman. - This is silly, ridiculous, and downright offensive. While there ARE good stereo salesmen, most stereo salesmen (or women, or critters) are simply out to make a buck. Those in "high-end" stores are very prone to flights of superstition, and most insist that "double blind tests are stupid", hence one such individual would never "pass" a double-blind test. Ever. Perhaps that's your hidden agenda? (not a number, because you left out the important person) - An expert in subjective testing, usually a psychologist or acoustician with extensive experience in subjective testing, test design, test administration, and what-have-you. THIS is it MOST important thing to have oversee a test, and if possible, run it. The fact that you don't even mention it speaks very poorly of your attempted defination of a proper subjective test. (8) Why? You should have your "oversight" folks ensure that the test is working correctly, and indeed tests what it attempts to test, as much as that can be shown. That much is commonly done in some circles, I fear that audio testing often isn't one of them. (9) Now, then, this is just plain, ordinary stupid, unless what you're testing is the effects of listening environments on whatever it is that your subjects are supposed to detect. If you're NOT trying to test that effect, then one, perhaps, might want to try to isolate the subject from these confusing and interfering influences, and try to lower the noise-level of the testing process. If you're trying to isolate the differences in (for example) low-level cables, surely you don't want to mask the effects with the room-induced frequency shaping, auditory masking due to reflections and diffraction, energy-storage effects of the room that provide very-long-term self-interaction of a signal, and so on, do you? All those effects are shown, in careful tests, to reduce the sensitivity of a subject to make a RELIABLE decision. I'm NOT going to post the whole damned bibliography for this AGAIN, damnit. -- TEDDY BEARS HAVE *GREEN* EYES! "...Farewell even to our Scottish name, so ..." (ihnp4;allegra;research)!alice!jj Copyright JJ 1987. All rights to mail reserved, USENET redistribution otherwise granted to those who allow free redistritution.