Friday, June 10, 2005


Peer Review, Part 2: Actual Responses from Actual Peers!

Yesterday’s admittedly scattershot rant attracted some really thoughtful and intelligent commentary (thereby implicitly invalidating my thesis). Maybe peer review works best in the blogosphere?

Ianqui made an excellent point about the turnaround time on academic journal articles. I’ve heard stories of frustrated authors mailing birthday cards to articles they’ve submitted. Back in the day, typesetting needs dictated long lead times. Obviously, this is no longer the case, and in the age of email and blogs, the only lags should be caused by readers not getting around to reading.

I suspect that part of the problem is a lack of incentive. If you’re a reader for a journal, is good performance rewarded? (That’s a symptom of a larger issue in academia, which I think can be traced to its aristocratic pretensions: letters of recommendation. Is there a less useful, less rewarded, less practical holdover than these? As with reading journal submissions, writing letters of recommendation is taken as part of the noblesse oblige of academia, thoughtful nuggets of wisdom to be tossed off between swigs of sherry, yet entire careers hinge on them. To call this system ‘unsustainable’ would be too generous.)

Doc turned one of my examples around on me, pointing out (correctly) that most of the substance of Freakonomics originated in peer reviewed journals. Good point. Slogging through the peer reviewed journals in my own discipline (which isn’t economics), I’ll just say that Leavitt strikes me as exceptional.

Doc also pointed out, correctly, that blind peer review can prevent favoritism and the dominance of Names. The downside of that, though, is that it also seems to prevent anything risky, interdisciplinary, or terribly interesting. By homogenizing the product, we can diversify the producers; Henry Ford figured that out 100 years ago. Is that really a worthy goal?

Single-editor journals can be maddeningly idiosyncratic, but that’s part of their utility. As with blogs, they reflect the editorial tastes of actual people. They create space for risk-taking.

Finally, the divine Aunt B (whose blog, tiny cat pants, has quickly become a favorite) made a point about authorial position that I had never thought of in quite that way. She noted that, in writing for a peer-reviewed journal, you’re assuming that you’re writing for the three people in the field who know more about your work than you do – in other words, you’re writing like a student, trying to curry approval. Hence the forced prose, pedantic footnoting, bootlicking, etc. When you assume instead that you know more than your audience does, you tone down the pedantry, adopt clearer prose, get to the point, and produce something more readable. Maybe part of the reason so much academic writing is so painfully bad is that we’ve adopted a cultural practice based on a sort of professional regression. Rather than owning the authority of knowledge, we hide behind footnotes, just like the nervous students we once were.

It may just be me, but it’s hard to convey just how brilliant an insight that is. Compare the prose style and authorial positioning of a Michael Lewis, Malcolm Gladwell, or Barbara Ehrenreich to a typical academic journal article. The former could be charged with arrogance, but damn, they’re readable. And read. Unlike journals.

Aunt B’s observation may give me a way out of the (admittedly stupid) dilemma in which I trapped myself. In criticizing peer review, I’m obviously vulnerable to the question “as opposed to what?”. Maybe the problem is instead the criteria the peers use. Ianqui, Doc, and Aunt B gave me some damn good peer review, unpaid, on the same day the entry was posted. Thank you all for proving me at least partially wrong. Maybe there’s hope...

Dean Dad: I did recommend that you don't get me started, didn't I? Well, I won't hijack your comments, but will give you links to a few of my posts on the glacial pace of peer reviewing. You can decide whether or not you want to read :)

This Can't End Well (read the links for "this" and "fiasco" for more background on this issue)

Publication is FUN! (The link for "Sunday night rejection" is actually the end of the story to "This Can't End Well".)
The letter of recommendation in all its forms may just push me out of academia entirely. I hate writing them so, so much. They are typically speaking such B.S., yet, as you mentioned, entire careers hang on them.

I know this was an aside on your discussion of the peer review journal (my record was 2 1/2 years before hearing anything from a journal. I'm in the Humanities), but just hearing the words "letter of recommendation" set me off. I've written four this week and it's summer.
OK, I'm officially shocked. In my field (I'm Dean Dad's biologist correspondent), we generally get decisions on MSes within three months. Some journals are slower, but they are working to speed the process up since authors are reluctant to submit to journals with long lead times (especially if you have a competitor breathing down your neck!)

I am currently reviewing a MS, and the monitoring editor asked me to get the review back in three weeks. My own last paper (which went in press on Monday) was seriously "hung" during the submission process; the Associate Editor had trouble finding reviewers, and apologized for the delay. Even so, it was submitted late November, first decision late March (resubmit with minor revisions), resubbed late May, accepted two weeks later.

Incidentally, I have found the reviews on my last several papers to be quite helpful on balance: non-attitudinal comments that helped us strengthen the work. That's what peer-review is supposed to do. I don't work in the humanities, so I wonder if the process works differently there.
Wow, Dad, thanks for the high compliments. You've hit on one of my favorite things to spout off at the mouth about.

I have to say that I'm really glad that we now have a space like this--the relative anonimity of of the internet--to really talk about this stuff, to hash over the role of peer review, and to talk openly about our experiences. I don't know what will come of it, but it feels important that we can even do it.
The academic field that seems to me the farthest in left field (and I don't mean that I like their politics) is LAW. I just found out a couple of years ago that for academic Law journals the reviewers are law students. How are law students in a position to judge professors' work? Aren't they too busy trying to get in the top 10 percent of their class so that they can make mega bucks? Can anyone explain this system to me? The first time a law prof client explained the system to me I got an empathy headache just imagining the injustice of the field of justice.
Work at home opportunites For people who are smart enough to go after there own dreams

Work from home opportunites

Women who have children will love it...
Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?