 |
|
Tuesday, April 11, 2006
previous entry | main | next entry | TrackBack (0)
Horror stories about anonmous peer review
Henry Farrell links to a Chronicle of Higher Education story by Jeffrey Young about how Microsoft Word's tags have eroded anonymity in peer review. Henry adds: Word documents preserve a lot of metadata, including, very often, the author’s name – so that if you submit your review via a Word email attachment (as many journals ask you to these days), and the journal forwards the review unchanged to the article’s author, he or she can figure out who you are without having to play the usual guessing game. I’ve been aware of this for a couple of years (I carefully strip all data before sending reviews out, just in case) – but I suspect that many academics aren’t (some of them may not even realize that Word collates this data automatically).I've been outed once as a reviewer after I rejected a piece, but it was not due to anything as high-tech as MS Word metadata. I faxed the journal -- which shall remain nameless -- my review. The journal then faxed it to the paper-writer -- who shall also remain nameless. The problem was that the journal's fax to the writer contained my department's fax number and identification -- and from there it was pretty damn easy to identify the referee. Here's a link for potential referees about how to stay anonymous if you electronically submit your referee reports. posted by Dan on 04.11.06 at 11:33 PMComments: Any horror stories out their about reviewers who don't use Spellcheck? posted by: Zathras on 04.11.06 at 11:33 PM [permalink]The biggest problem with reviewers --by far --is them not actually doing the review, or getting around to it after being chased four times. A fairly well known scholar in my field always identifies his reviews and insists the identification remain with the review. He believes it leads to a more honest, thoughtful and conscientious review. Frankly I think he is right, and I can't really see a downside to identifying yourself. Sure, academia is full of fragile egos, but it seems to me that anonymity is a often a shield for poor work. On the other hand, submitted papers should be kept anonymous to avoid favoritism (or its oppisite). posted by: Mitchell Young on 04.11.06 at 11:33 PM [permalink]If a reviewer wants, googling an article's title will often reveal the article posted on the author's website, where it was given at a conference, etc. Just another way things aren't as anon as they may appear. posted by: Dan on 04.11.06 at 11:33 PM [permalink]"Collated"?! posted by: P. Froward on 04.11.06 at 11:33 PM [permalink]Even less technical methods of anonymization are not foolproof. Some years ago, I was sent a referee's report from a Dutch journal which was a xerox of what the referee had sent them, with his signature whited out. All I had to do was turn it over and look at the back while holding it up to a bright light: the name was clearly visible. I was also surprised to see the report refer to me by name. Strict double-blind refereeing seems to be a mostly Anglo-Saxon thing. At least in my field (Classics), continental journals often use single-blind refereeing or even let the editor pick and choose whichever submissions he likes. Even before the web, double-blind refereeing was never foolproof. I once asked an eminent classicist who had devoted his entire career to one of the major Greek authors (Homer, Plato, Thucydides, Euripides, one of those guys) how often he knew the name of the supposedly anonymous authors he was reviewing. He said about 50% of the time. After twenty years spent cultivating the same corner of the classical garden, he knew the stylistic tics and ideological approaches of many of his colleagues well enough that he couldn't help knowing who he was reviewing a lot of the time, even when he hadn't seen previous versions of their work at conventions and such. Conclusion: double-blind refereeing is good, but far from foolproof, and the system still depends on the basic fairness of the referees. posted by: Dr. Weevil on 04.11.06 at 11:33 PM [permalink]Post a Comment: |
  |