Thursday, November 03, 2011

 

Trading Papers

(Sorry for the unscheduled down time. Mother Nature decided to knock out electricity for five days, just for the sheer fun of it. I wrote this just before the power went out.)

What if Professors Smith and Jones swapped papers for a semester? I’d be intrigued to hear from anyone who has actually tried this.

Anyone who has taught courses in which grading relies on judgment knows the delicate balance of both encouraging and judging students. You try to set the student up to succeed, but the result is dreadful; now you have to be the bearer of bad news. Since students don’t always understand the basis of the judgment, especially in the heat of embarrassment, it’s easy for them to default to some not-very-flattering assumptions about the instructor. Suddenly, you’ve got a psychologically fraught situation that does not lend itself to good teaching.

So a thought: what if Professors Smith and Jones traded papers for a semester? Obviously, I’m assuming that they’re teaching different sections of the same course, with enrollments close enough to equal that their workloads would not meaningfully change. If I grade your 25 papers and you grade my 25 papers, the workload adjustment is pretty much a wash.

The upside, I think, would be that the role of ‘coach’ and the role of ‘judge’ would be clearly separated. Now it’s not “try to psych out the teacher;” it’s “you and me against the guy behind the curtain.” With the roles more clearly demarcated, the instructor would be free to position herself as the student’s ally, which, in fact, she is.

It could conceivably make for more consistent grading, too. It’s easier to be objective when you don’t know the student. (At least, it’s less likely that personal likes and dislikes will enter into the judgment unconsciously.)

The major objections I’ve heard have been twofold, but neither strikes me as terribly compelling.

The first is the task of coordination. Yes, there would have to be some planning and communication between the instructors to keep things aligned. But this strikes me as the kind of thing that gets less true as you do it more. The first time out, I’d guess that the costs of coordination would be non-trivial, but by the fourth or fifth, they should be pretty minor.

The second is that the instructor would not get as complete a picture of student performance as she would if she read the papers herself.

There’s some truth to that, though it doesn’t strike me as a deal-breaker. I’m thinking that in a class with, say, four papers over the course of the semester, maybe the swap occurs in the final two. The professor gets to prep the students for the objective, outside judge. You’d still get a sense of who was who, but having that outside person come in later could help with the psychological dynamics of the class. That would be especially true if the grades on the later papers counted more heavily.

As with so many back-of-the-envelope ideas, the devil is in the details. So I’d like to hear from any of my wise and worldly readers who have actually tried this or something like it. Did it help? Did it harm? Is there a trick to getting it right?

Comments:
You've missed the most compelling reason:

Outside of things like math, grading represents a subjective determination of what people want. Smart students spend the semester attempting to understand their professor's subjective preferences so that they can get a better grade: Do they like serial commas? What sort of intro do they advocate? Is there a favorite term, or word, that they like to see? Can they mention things which demonstrate--in a way that only the teacher would recognize--their attendance in class? Can they reference things which would make it clear--in a way that only the teacher would know--that the student did extra research on a subject?

Grading exchanges work fine in math and science, where there is only one "right" answer. (they might also work for music theory, certain history courses, and so on.) If you want paper-swapping for English, though, you would need to have a direct instruction model of teaching. (there's a continuum between math and english.) Otherwise you're discarding much of the value of different teaching styles
 
This reminds me of the STEM strategy where all students in all sections of Chem 101 (or some other STEM intro class) take the same midterm exam at the same time. Everyone teaching Chem 101 (or, at least all the TAs) pitches in with the grading.

It's also like the situation where the professor lectures and the TAs do all the grading.

Neither of these has seemed to foster much additional collegiality in these intro courses.
 
Welcome back. I figured your absence was due to buying pumpkins rather than a generator for Halloween.

The first anonymous should have identified hir professional area. There may only be one "right" answer to a math or physics problem, but any non-trivial problem on a free-response exam requires assigning partial credit. Should the student who hit the wrong key on the final calculation lose the same 20% of the exam grade as one who started with the wrong physical principle? No. Are there problems where any wrong answer is totally wrong? Yes.

(FYI, even a well-designed multiple choice exam can separate the two, because it will have a choice for the wrong approach but won't cover random key errors. The latter student will recompute and get it right.)

What science and math profs have done, often as a result of communal grading, is negotiate a grading key (what is apparently now called a "rubric") before splitting up the grading work. Even when each person does all of the grading of a single problem, you want similar problems graded on the same scale.

In my experience, the grading scheme is settled before the exam is designed. The two cannot be separated in science or math. Only amateurs write an exam that has to be curved after the fact.

Also in my experience, there is quite broad agreement, collegiality as it were, on what the outcomes should be and how to assess it in various science and math fields. This probably is the result of talking to each other.
 
I like how this makes me wonder how much I would teach differently if I knew my students' essays were going to be graded by someone else.

The only drawback I can see is that it becomes more challenging with the degree that one incorporates discussion in class. I include a lot of open-ended discussion about primary sources, in which students can draw out a variety of equally valuable themes. If different sections take these discussions in different directions, as they'd be likely to, you'd need to be very careful about aligning expectations between different instructors so that they're not expecting precisely the same content.
 
You are lucky to have power, we are still in the dark at home (though not in the office). Nothing like getting dressed by the light of one's cell phone.
 
I've never tried this specifically, but as a TA in history I frequently had to give my professor a representative distribution of essays that I had already assessed but hadn't marked up, to make sure that his/her grading and mine were comparable. (Even though I did all the grading, it was a consistency/fairness check, and it had the placating effect on students that you mention, because there was a second opinion involved in the process. It made it much harder for them to argue bias/misunderstanding/etc.)

However, employing this with two fully autonomous instructors may be untenable in some disciplines. Even though they are teaching the same class, they may have different ideas about what the purpose of the essays is. For instance, if one thinks the primary purpose is help students develop good argumentative prose and the other thinks the primary purpose is to demonstrate knowledge of content, that will be problematic. I can only see this working if both professors share the same assumptions about the objectives of the assessments in question (good luck with that). Essentially, they would have to share a rubrik as well, because of the "partial credit" problem that CCPhysicist discusses. Some departments encourage this kind of standardization, but my impression is that most don't.
 
Texas Tech's composition program enacted such a system in about 2006. They separated the tasks of teaching and grading completely. Here's a Chronicle article about it: http://chronicle.com/article/A-New-Way-to-Grade/28153/
I don't know if they've continued it. As a grad student in rhet/comp at the time, I disagreed with it for the reason that it seemed to buy into the assumption that students or anyone can write successfully for a general audience and that such a goal is a useful and learning-filled endeavor.
 
My department pools (final exam/ essay) grading for all courses, and everyone is responsible for an equal share.

In addition to avoiding the ``what do I have in my pocket'' aspect of final writing, it means that faculty are much less resistant to teaching large intro classes and there is less of a fight for the right to teach senior seminars since the quantity and quality of grading is the same for both (1/nth of the total department grading) and some people both enjoy and are good at giving big lectures while others are better at leading discussions.
 
My department is so small we don't have multiple sections of anything. Each professor teaches in their own specialty area or a particular section of the general curriculum.

Even when we teach the same course, we teach very different material. My "Approaches to European History" includes sections on the Renaissance and the history of crime. My colleague teaches the same course but uses neoclassical and modern topics to illustrate the points and inculcate the same skills I teach with my very different sources.

This drives my social science colleagues right round the bend. They can't understand how you can teach the same course in such radically different ways.
 
I do this with my TAs: They rotate the grading -- their idea, not mine. It has had a range of advantages, including making the amount of grading fairer. (This semester, they have even played with who grades what percent of each assignment, to fit their own schedules better.)

The only negative has been TA fights over specific grades, which I've had to fix each time. (The student goes to their own TA when unhappy, and if the TA kinda agrees, she brings it to me.) I see an advantage in this as well: The student's own TA winds up being the defender and I wind up being the judge. It brings to light more grading issues that I would likely miss otherwise.
 
I TA'd a class that worked kind of like what DD's Grad School Friend mentioned. There were three large lab reports and three sections of the class, each with a TA. So one TA graded the first report from all three sections (and then had to do no more grading for the rest of the semester), the second TA did the next report, and so on. It sucked when it was your turn, but it definitely made the grades more fair across sections.

We did, however, go on field trips with all three sections together, so there was at least some chance for the students to meet all three TAs.
 
I've done this, and when it works, it's terrific. But it's an easy scapegoat for dissatisfaction when something else isn't working. So I think to do it well, you need to do it systematically and make it normal, as far as disciplinary specialisation allows. Tough road.
 
I always cringe when I see comments like 'Grading exchanges work fine in math and science, where there is only one "right" answer.' The assumption for science teachers is that we all just give multiple choice tests and they are so easy to grade. I doubt that everyone in my science department would assess the answers on my non-multiple choice science exams in exactly the same way.

I'm not sure why this assumption seems to be in place, but I needed to comment.
 
End-of-the-semester portfolio assessments in English comp classes work just dandy. The classroom teacher becomes a kind of coach, conjuring up the spectre of a gang of really picky English teachers who're going to be reviewing students' work.

Portfolios also ensure that everyone's doing (very broadly) the same thing in the classroom: You can't get away with assigning just two papers over the course of a semester.

And I was wondering where you went, DD. Out here in sunny SoCal (it was in the 70s and 80s last week), I'd forgotten all about your early snowstorm and blackout.

--Philip
 
Wouldn't this violate FERPA?
 
it is relative. it is relative, depending on your view of point of view you choose. Nothing that can not be resolved, right or wrong way we take, you just choose which one according to your interests.
 
Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?