Path: utzoo!utgpu!news-server.csri.toronto.edu!cs.utexas.edu!wuarchive!udel!princeton!pucc!PSYCH@TCSVM From: harnad@phoenix.Princeton.EDU (Stevan Harnad) Newsgroups: sci.psychology.digest Subject: PSYCOLOQUY V2 #1 (Discussion: Consensus Journals - 136 lines) Message-ID: <9101040458.AA10918@reason.Princeton.EDU> Date: 3 Jan 91 23:41:17 GMT Sender: VMNNPOST@pucc.Princeton.EDU (Listserv to Netnews Gateway) Organization: Listserv to Netnews Gateway at pucc.Princeton.EDU Lines: 131 Approved: PSYCH@TCSVM PSYCOLOQUY Thu, 3 Jan 91 Volume 2 : Issue 1 Consensus Journals - David Stodolsky Reply - Gordon Becker ---------------------------------------------------------------------- From: David Stodolsky Subject: Consensus Journals/Stodolsky "Gordon Becker" writes in "Response to D.S. Stodolsky's 'Consensus Journals'" (PSYCOLOQUY V1 #16): "The statistical procedure that Stodolsky proposes for evaluating articles and for selecting new authors would make it even more difficult than it is now to publish articles that deviate from the dominant paradigm." This criticism is not supported and I do not think it can be supported. First, any author can publish directly, independent of the consensus calculation procedure. Second, the consensus calculation supports two consensus positions, thus identifying an opposition and treating it just as it treats the dominant position. Third, the Cultural Consensus model does not take into account any historical information, thus use of this model does not grant any benefit to those with developed reputations. I suggest, however, that the integration of reputation information is appropriate in scientific discourse. The weight reputation is given, however, is not specified in the model. The Cultural Consensus model should be used alone to maximize sensitivity to new paradigms. A major objective of the Consensus Journal process is to overcome the current conservatism of scientific journals. This criticism, and the others in the response, give the impression of misperception of the level of analysis of my proposal. The procedure I suggest operates at the level of statements or paragraphs, not full papers as we see them today. Becker continues: "Stodolsky's method also perpetuates the use of anonymity with all of its disadvantages. Not only does anonymity permit the reviewer to make false and unsupported statements, to be unnecessarily disparaging, personal and disrespectful, all with impunity; it also prevents opening or continuing a fruitful exchange and cooperative effort with helpful reviewers." This question is not treated in the target paper, but the criticism is precisely incorrect. It fails distinguish between anonymity and the use of pseudonyms that corrects the problems mentioned above. Finally, Becker states: "The task of reviewers might then be to verify the accuracy and relevancy of citations rather than the worth of the article itself." This _is_ one task of reviewers of Consensus Journal articles. The response raises some other interesting points for discussion, but the basis for them is apparently a misunderstanding of the objectives and procedures for the Consensus Journal, thus it is hard to offer a reaction to them. I suggest Becker reanalyzes the article and relevant supporting material to determine which, if any, of his criticisms and suggestions are, in fact, relevant. David S. Stodolsky Office: + 45 46 75 77 11 x 21 38 Department of Computer Science Home: + 45 31 95 92 82 Bldg. 20.2, Roskilde University Center Internet: david@ruc.dk Post Box 260, DK-4000 Roskilde, Denmark Fax: + 45 46 75 74 01 ------------------------------ From: Subject: Reply to Stodolsky's reply to my comments on Consensus Journals Reply to Stodolsky's reply to Becker's Comment on D.S. Stodolsky's "Consensus Journals" in PSYCOLOQUY VI #16 I reread Stodolsky's article, as he suggested in his reply to my original comments. He is right in pointing out that I had overlooked the fact that anyone can invite hirself to write an article. This does not invalidate my criticisms, but strengthens them. If anyone can publish anything, anytime, then why go through all the statistical procedures and invitations? What is gained from that compared to simply allowing the reviews to drive the procedure and force bad articles out of print? I also wonder how many articles would be self-invited (as they are today) and how many referee and moderator invitations would be accepted. The _statistical procedure_ that he uses and that I criticized in my first comment still holds: it is conservative in so far as most reviewers are likely to take the mainstream position and reject significant deviations from it. In order to obtain the second modal position there will have to be several other reviewers thinking like the deviant author - the more original, the more deviant the article, the less likely the second (positive) modal position. The distinction that Stodolsky would like to make between "anonymity" and "pseudonyms" is, to use his own expression, "precisely incorrect", since the reviewer remains anonymous to all but the moderator ... which is precisely the same situation we have today for anonymous reviews: The editor knows who the reviewer is but nobody else does. Giving the reviewer a pseudonym doesn't change that. Moreover, contrary to Stodolsky's desire to have the reviews enhance the reputation of the reviewer, the pseudonym enhances the reputation of an anonymous pseudonym whom nobody but the moderator knows? Gordon Becker UNO Omaha NE 68182 becker@UNOMA1.BITNET ------------------------------ PSYCOLOQUY is sponsored by the Science Directorate of the American Psychological Association (202) 955-7653 Co-Editors: (scientific discussion) (professional/clinical discussion) Stevan Harnad Perry London, Dean, Cary Cherniss (Assoc Ed.) Psychology Department Graduate School of Applied Graduate School of Applied Princeton University and Professional Psychology and Professional Psychology Rutgers University Rutgers University Assistant Editors: Malcolm Bauer John Pizutelli Psychology Department Psychology Department Princeton University Rutgers University End of PSYCOLOQUY Digest ******************************