Friday, June 10, 2005

Peer Review, Part 2: Actual Responses from Actual Peers!

Yesterday’s admittedly scattershot rant attracted some really thoughtful and intelligent commentary (thereby implicitly invalidating my thesis). Maybe peer review works best in the blogosphere?

Ianqui made an excellent point about the turnaround time on academic journal articles. I’ve heard stories of frustrated authors mailing birthday cards to articles they’ve submitted. Back in the day, typesetting needs dictated long lead times. Obviously, this is no longer the case, and in the age of email and blogs, the only lags should be caused by readers not getting around to reading.

I suspect that part of the problem is a lack of incentive. If you’re a reader for a journal, is good performance rewarded? (That’s a symptom of a larger issue in academia, which I think can be traced to its aristocratic pretensions: letters of recommendation. Is there a less useful, less rewarded, less practical holdover than these? As with reading journal submissions, writing letters of recommendation is taken as part of the noblesse oblige of academia, thoughtful nuggets of wisdom to be tossed off between swigs of sherry, yet entire careers hinge on them. To call this system ‘unsustainable’ would be too generous.)

Doc turned one of my examples around on me, pointing out (correctly) that most of the substance of Freakonomics originated in peer reviewed journals. Good point. Slogging through the peer reviewed journals in my own discipline (which isn’t economics), I’ll just say that Leavitt strikes me as exceptional.

Doc also pointed out, correctly, that blind peer review can prevent favoritism and the dominance of Names. The downside of that, though, is that it also seems to prevent anything risky, interdisciplinary, or terribly interesting. By homogenizing the product, we can diversify the producers; Henry Ford figured that out 100 years ago. Is that really a worthy goal?

Single-editor journals can be maddeningly idiosyncratic, but that’s part of their utility. As with blogs, they reflect the editorial tastes of actual people. They create space for risk-taking.

Finally, the divine Aunt B (whose blog, tiny cat pants, has quickly become a favorite) made a point about authorial position that I had never thought of in quite that way. She noted that, in writing for a peer-reviewed journal, you’re assuming that you’re writing for the three people in the field who know more about your work than you do – in other words, you’re writing like a student, trying to curry approval. Hence the forced prose, pedantic footnoting, bootlicking, etc. When you assume instead that you know more than your audience does, you tone down the pedantry, adopt clearer prose, get to the point, and produce something more readable. Maybe part of the reason so much academic writing is so painfully bad is that we’ve adopted a cultural practice based on a sort of professional regression. Rather than owning the authority of knowledge, we hide behind footnotes, just like the nervous students we once were.

It may just be me, but it’s hard to convey just how brilliant an insight that is. Compare the prose style and authorial positioning of a Michael Lewis, Malcolm Gladwell, or Barbara Ehrenreich to a typical academic journal article. The former could be charged with arrogance, but damn, they’re readable. And read. Unlike journals.

Aunt B’s observation may give me a way out of the (admittedly stupid) dilemma in which I trapped myself. In criticizing peer review, I’m obviously vulnerable to the question “as opposed to what?”. Maybe the problem is instead the criteria the peers use. Ianqui, Doc, and Aunt B gave me some damn good peer review, unpaid, on the same day the entry was posted. Thank you all for proving me at least partially wrong. Maybe there’s hope...