Newsgroups: comp.sources.d Path: utzoo!utgpu!news-server.csri.toronto.edu!rpi!zaphod.mps.ohio-state.edu!think.com!mintaka!bloom-picayune.mit.edu!athena.mit.edu!jik From: jik@athena.mit.edu (Jonathan I. Kamens) Subject: Re: v01INF1: Status - Status of comp.sources.reviewed Message-ID: <1991Apr14.232818.15851@athena.mit.edu> Sender: news@athena.mit.edu (News system) Organization: Massachusetts Institute of Technology References: <16390:Apr1305:56:2091@kramden.acf.nyu.edu> <6787:Apr1420:38:3991@kramden.acf.nyu.edu> Date: Sun, 14 Apr 91 23:28:18 GMT Lines: 152 In article <6787:Apr1420:38:3991@kramden.acf.nyu.edu>, brnstnd@kramden.acf.nyu.edu (Dan Bernstein) writes: |> Face it: this has absolutely nothing to do with journal publication. |> The editor is the author's sole contact; reviewers simply do not contact |> authors. Yet there is not a hint of the traditional anonymity in these |> guidelines. If a journal reviewer decides that there is something ambiguous or unclear in the paper he is reviewing, he may indicate that in his review, and the journal may decide that the ambiguity is severe enough that is has to be corrected (or, of course, they may decide that the paper should be rejected outright; but that isn't really relevant to the current discussion). At that point, the journal will forward the reviewer's comments back to the author, with all names removed (the anonymity that Dan is defending so vociferously), and the author has the opportunity to respond to the reviewer's comments if he feels they are incorrect, or to make corrections if he does not. (Note that you can replace "he" with "he/she" in the entire previous paragraph, and the rest of this posting where appropriate, if you are easily offended by such things.) The way we currently think we'll be doing things in c.s.r, the moderator will give no indication to the submitter of who the reviewers are. They will remain anonymous, unless they choose to contact the submitter directly. If they do not want to reveal themselves, and they need to contact the submitter to ask questions, they can do so through the moderator. Finally, any comments that the reviewers have that they think might help people will be included when the submission is finally posted to c.s.r. Mr. Bernstein appears to have three different problems (that I can see) with this proposed scheme. First of all, he objects to the idea of the reviewers ever contacting the submitter at all, anonymous or not. Second, he objects to the possibility of unanonymous communication between a reviewer and submitter. Third, he objects to the idea of reviewers' comments being published when the submission is posted. I see nothing wrong with reviewers contacting the submitter of a package. It seems no different to me from the forwarding of reviewers' comments back to the author in the case of a real journal, and I *know* that this happens, at least in some cases, because my sister is a biologist who has submitted several papers to several different journals and has, in several different cases, been given copies of reviewers' comments (with names reviewed) and asked to respond to them. The only difference, it seems, is that the c.s.r process will be iterative and responsive (i.e. a dialogue between the reviewers and the submitter), while the journal process is usually one-step (reviewer gets article, reviewer read article, reviewer writes evaluation, author gets copy of evaluation). This strikes me as an improvement over the journal review process, not a flaw. Now, as for the question of anonymity. Dan may turn out to be right -- we may end up deciding that it was a mistake to ever consider allowing reviewers to contact submitters and reveal their identities. However, thus far the only reason Dan has given for espousing this belief is that that's not the way real journals do things. In my opinion, that's not a very persuasive argument. If he would tell us why real journals do reviews anonymously, and then explain why he thinks those reasons apply equally in the case of c.s.r, I might be more convinced. It seems to me that there are several reasons for anonymity in real journals. First of all, you may end up reviewing an article written by a colleague, and you may not want that colleague to find out that it was you who panned his article and caused it to be rejected. Second, the colleague whose article you review today may end up reviewing one of your articles tomorrow, and if you give him a bad review today, he may do the same to you tomorrow as revenge (or, on the other hand, if you give him a good review, he may feel obligated to give you a good review even if you don't deserve it). The world of science is very competitive, and it seems to me that these and all the other reasons for anonymity are attempts to keep that competitiveness from contaminating the review process. I believe that the review process which we will undertake in c.s.r is *not* competitive, and was not meant to be. We do not have 100 programmers out there, all over the world, working on software in the same ground-breaking field, and competing with each other to be the first to publish their programs on the net. For the most part, when a submitter sends us something, he will probably be the only person working on anything like that, or one of the view, and it is very unlikely that he will have to feel a need to compete with other people working on similar projects. Furthermore, our criticisms will, for the most part, be of the "you have to fix these problems before this can be posted" variety, rather than of the "this program is utterly brain-damaged and we see no reason to post it to this newsgroup or any other variety" variety. What all this comes down to is that, in the majority of cases, there is *no need* for anonymity in the c.s.r reviewing process. In cases where there *is* a need for anonymity (for example in the latter case in the last sentence of the previous paragraph), it has been guaranteed, because reviewers need never contact the submitter directly. If Dan has some reason why he believes that anonymity in the c.s.r review process will be harmful despite what I have said above, I would like to hear it. If his only reason remains, "That's not how real journals do it!" then, like I have said, I remain unconvinced. |> > There are currently two or |> > three alternatives to comp.sources, c.s.r is just another option. |> |> That's one of its biggest problems. I've never understood this attitude. Why is giving people more options a problem? |> Yes. I invite you---any of you---to explain what comp.sources.reviewed |> would have done for my latest package, volume24/yabbawhap in c.s.unix. |> Rich made no changes to it, so it's easy competition, right? Surely you |> can point out *some* advantage in your precious ``reviewing'' process. |> All I see is that c.s.reviewed would take weeks where Rich took days. First of all, Dan, as I am sure you are aware, arguments of this sort from the specific to the general hold very little water. You cannot argue that because one package would not have benefitted from c.s.r, no packages can benefit from c.s.r. The fact that you are attempting to do so seems to me to cast doubts on your ability to come up with better arguments against c.s.r. In any case, the c.s.r reviewing process may have found typos in your documentation. It may have found unanticipated OS dependencies that needed to be removed. It may have found bugs you were not aware of. It may have found any number of problems that neither you or I can anticipate. Since you seem to think you know so much about the review process in real journals, you must know that it is not intended only for the reviewers to find the problems that the author knows about; it is intended for the reviewers to find any problems that may be there, including ones that the author may not have anticipated. Therefore, to answer your question -- the c.s.r review process may have found problems that you did not anticipate. Now, you can argue, "But there were none!" but that's like saying that the journal review process is unnecessary because all submitters of papers to journals can guarantee to the journal in advance that their papers have no problems in them. I will give you two counter-examples in response to your one example. The "chemtab" package, as originally posted to comp.sources.unix, had so many problems that I could not install it on my system when I got it. The configuration options didn't, and the compile-time options didn't. I have talked to Rich Salz privately about this, and he has told me, "Chemtab was a mistake." C.s.r would not, in my opinion, have allowed that mistake to happen. After all, I could very well have been one of the reviewers, in which case I would have caught the problems *before* it was posted, in the review phase, rather than *after* it was posted, which is what happened when it was posted to c.s.u.. Another example is my "delete" package. When the whole comp.sources.unix flame war was happening in news.admin, I expounded at length about the problems I had with my package and comp.sources.unix. I have talked about that with Rich Salz too, and he has acknowledged that he made mistakes with my package and apologized (an apology I accepted willingly; I think Rich is a great guy, and I certainly don't think that anything he did was intentional or typical). I truly believe that *none* of the problems I had with my package in c.s.u would have occurred had I posted it to c.s.r. -- Jonathan Kamens USnail: MIT Project Athena 11 Ashford Terrace jik@Athena.MIT.EDU Allston, MA 02134 Office: 617-253-8085 Home: 617-782-0710