How to Publish a Scientific Comment in 1-2-3 Easy Steps
by BilboMichael Behe's latest post at his blog records yet another failed attempt to publish a comment to a paper that supposedly refuted his argument for irreducible complexity. But Behe was quick to note that he's by no means the only one having trouble getting their comments published, and referred readers to this example, by Prof. Rick Trebino. It's a humorous narration of actual events. At the end, Trebino gives his suggestions on how to correct a badly flawed system.
September 17th, 2009 at 12:11 am
For a discussion of this case on ScienceBlogs go to The Saga of the Journal Comment. Rick Trebino makes an appearance in the comments.
Trebino's experience is far from unique. Back in my days as a grad student I co-authored a comment with two of my advisors. The guy whose work we criticized was indisputably wrong (his theory violated a mathematical identity), yet it still took us 11 months to get the comment published alongside a snotty reply.
I have not published a proper comment ever since. It is frankly a waste of time. If someone's theory is wrong at the level of math or logic, it's already dead, whether someone points it out or not. If I have something positive to say beyond mere criticism I write a regular paper.
I sympathize with Trebino. His opponents pointed out what they thought was an ambiguity in FROG, a widely used technique for extracting the shape of short laser pulses developed by Trebino in the 1990s. From his satire it is obvious that he took it personally and wanted to clear his reputation by immediately publishing a comment in the same journal. Well, that process takes a while. A year is not an outrageously long time for publishing a critical comment—a process that usually involves two sets of bruised egos with an editor caught in the cross fire.
On the other hand, I don't take seriously Trebino's pronouncements about fixing the badly broken system and doubt that he will undertake any concrete steps. I agree with the author of the ScienceBlogs post: printing a paper in a peer-reviewed journal, even a prestigious one, does not automatically validate your work. It just means that the reviewers thought it was interesting and valid. Having a paper published merely opens the work to a larger audience.
Comment by olegt — September 17, 2009 @ 12:11 am
September 17th, 2009 at 5:34 pm
Yes, it is outrageous. You're just so used to it taking that long that you don't realize how outrageous it is. There was no good reason to deny Trebino the opportunity to immediately respond to a paper that was criticizing his work. That should be standard. Likewise, there is no good reason to deny Behe the opportunity to respond to all the papers aimed at criticizing his work.
Comment by Bilbo — September 17, 2009 @ 5:34 pm
September 17th, 2009 at 5:40 pm
Bilbo,
You have no idea what you are talking about. The typical time between submission and publication of a regular paper is 3-6 months. When the subject is contentious, the period can stretch to a year. The main cause of delays is a fairly long turnaround time: referees take up to a month to review a paper. We have other things to do besides refereeing, you know.
Comment by olegt — September 17, 2009 @ 5:40 pm
September 17th, 2009 at 6:06 pm
Are you saying that you need to referee the comments and replies, also?
Comment by Bilbo — September 17, 2009 @ 6:06 pm
September 17th, 2009 at 6:11 pm
Like hanging out at Intelligent Design blogs?
Comment by chunkdz — September 17, 2009 @ 6:11 pm
September 17th, 2009 at 7:40 pm
Bilbo wrote:
Comments are typically refereed. Here are, for instance, the editorial policies of Physical Review Letters.
Comment by olegt — September 17, 2009 @ 7:40 pm
September 18th, 2009 at 5:01 pm
From your link, Olegt:
Comments and Replies.— Comment authors are encouraged, but not required, to send their Comment first to the authors of the object Letter for a direct response. The editors usually begin review of a Comment by seeking a reaction from those authors. Possible reactions include:
1. The Comment seems appropriate for publication without a Reply.
2. A Reply to the Comment is submitted for possible simultaneous publication. (Submission of the Reply later in the Comment review process may be reserved as a future option.)
3. The Comment does not seem sufficiently relevant to the Letter or does not appear to be scientifically valid; a detailed discussion is enclosed.
The author of the Letter is not asked to review the Comment as an anonymous referee. In most cases, the editors will consult at least one independent, anonymous referee regarding the suitability for publication of a Comment (and Reply, if any). As with Letters, the editors may be selective in deciding what material is forwarded for consideration.
To be publishable, Comments and Replies must be of interest to our readers and free of detectable errors. If a Comment meets our criteria it will be published whether a Reply is published or not. The editors will not excessively delay the publication of a suitable Comment due to the lack of an adequate Reply. If both are published, the Comment and Reply must appear together, in the same issue. On occasion, a Letter will generate several similar Comments. In these cases, the editors may decide to publish only a subset of the Comments received. Just as Letters are restricted in length, Comments and Replies are restricted to one journal page each.
The Reply is shown to the authors of a Comment prior to publication or as necessary. Substantial revision of a Comment in response to the Reply may be interpreted by the editors as a sign that the Comment was misconceived, and might be cause for rejection. The editors may choose to restrict further modifications of a Comment or Reply at any stage of the review process, taking the version at hand as final.
Only one referee needed. And the rest of the procedure sounds pretty reasonable. I would only suggest that if a paper is aimed at somebody's work, then they have to right to publish a comment.
Comment by Bilbo — September 18, 2009 @ 5:01 pm
September 19th, 2009 at 9:45 am
Bilbo wrote:
Physical Review treats comments as short articles, so publishing one is not a right, it's a privilege. The editorial board sets the guidelines. You have seen them.
Upon receipt of a comment the editor first sends a comment to the author of the original paper. The original author is considered the involved referee. If the involved referee does not contest the comment it gets published. Otherwise the manuscript is sent to at least one other referee, possibly two.
This preprint on the arXiv tells the story of a comment that was submitted to Physical Review B, underwent 6 revisions, and in the end was rejected by the editor. The process took almost a year.
Comment by olegt — September 19, 2009 @ 9:45 am
September 19th, 2009 at 11:51 am
Hi Bilbo,
Yes. My advisor and I tried publishing a comment on a contentious paper and it went to peer review—we could never get beyond a 50-50 split of the two reviewers, so the little paper was never published. The back and forth went on for months.
Comment by KC — September 19, 2009 @ 11:51 am
September 20th, 2009 at 5:01 pm
And I have said, when the paper is attacking somebody's work, they should have a right to comment, not a privelege. Otherwise, we can have situations like Trebino's, where his work is regarded as discredited, and he's never offered an opportunity to defend it. And we can the repetitve situation with Behe, where scientists make a name for themselves by showing how IC can evolve, and Behe is never allowed to show how their studies go askew. Do we want truth or just a game of who gets to publish?
Comment by Bilbo — September 20, 2009 @ 5:01 pm
September 20th, 2009 at 6:33 pm
O, my heart goes out to poor Rick Trebino! He had some fun at the expense of Optics Letters and got the comment published. What a tough predicament.
Journals have standards, Bilbo. If someone points out that my theory is wrong, I can submit a comment rebutting that charge. However, my rebuttal can be plain wrong (the preprint I linked to was). Should it be published just because its author feels like it? I dunno. It's up to the editorial board to decide and most journals send comments to reviewers.
And if a journal refuses to publish your comment it's not the end of the world. You can write a longer paper and publish it in the same journal or elsewhere. Behe's cries prosecution just to score points with the creationist crowd. That's the favorite modus operandi of the Discovery Institute, of which he is a senior fellow. Did you see him compare Blogginhead.tv to Josef Stalin? Such a crybaby.
Comment by olegt — September 20, 2009 @ 6:33 pm
September 21st, 2009 at 4:50 pm
Excuse me? The only way he got it published was by going over the head of the senior editor. You have a perverted idea of what "fun" is.
No, it should be published, because the person who came up with the theory is the one most familiar with it and most capable of defending it. If they are unable to mount a cogent defense, then the reply will reveal it, and the goal of obtaining the truth will be brought closer.
But if they rejected your comment, why accept your longer paper?
No, Behe cries persecution because that's what it is. People keep getting papers published showing that IC is evolvable, because science journals want to prove Behe wrong. When Behe points out that the papers don't prove squat, the journals ignore him. What are they afraid of? The truth?
Comment by Bilbo — September 21, 2009 @ 4:50 pm
September 22nd, 2009 at 12:38 pm
One last point: If allowing comments by the originators of theories to the papers that were attacking them were common practice, then science journals would probably have much less crap to sift through. The potential authors of papers would realize that not only do must they survive the peer review process, they must also deal with the people whose theory they are attacking, a much more formidable foe to overcome.
For example, if Trebino's critics knew that Trebino would automatically be given a chance for rebuttal, would they have ever tried to get their piece of crap published? Would Behe's critics be much more careful about the crap they offer up as supposed refutations of irreducible complexity?
It's obvious that allowing comments in these situations would make science journals jobs easier, not harder.
Comment by Bilbo — September 22, 2009 @ 12:38 pm