Monday, November 12, 2007

What Kind of Grader Are You?

A left coast correspondent writes (this is a long one):

We have had a kerfluffal blow up today at my CC. This morning our VP
of Student Services sent out the following email:

"I wanted you to be aware that we recently received a request for our
grading records under the Public Records Act. We have secured
direction both from our legal counsel as well as the [California]
Chancellor's Office as to our mandate to comply with the Act. The
requestor asked for the grades assigned by every faculty member,
sorted by term, from Fall 2003 to present, to include faculty name,
course name and number, and total grade distribution.

We sent the grade records to the requestor yesterday. They include
all evaluative grades (A-F) assigned by each faculty member, along
with non-evaluative grades such as "I" (incomplete), "W" (withdrawal),
"MW" (military withdrawal), "IP" (in progress) and "RD" (report
delayed) for fall 2003 through summer 2007. No student identification
information of any kind was included in the file.

While we are aware that this type of request has been made to many
other colleges and universities around the state, we do no know what
the requestor intends to do with information. I understand that this
might be very concerning to faculty and I want to assure you that we
sent only the information that we had legal obligation to provide.
The Academic Senate will be discussing the issue at an upcoming
meeting. I urge you to join the discussion and/or contact me if you
have any further questions or concerns."

Last year's President of Academic Senate posted this reply:

"Hi Everyone,

Actually, the Academic Senate has known about this for awhile and the
discussions have appeared in our minutes. This issue has also been
discussed at strategic council. The information, as far as I
understand, was not private information. It did not include any
identifying student information. The information released was really
information about us as instructors. How many A's do I give in my
Philosophy 6 class versus how many A's does Dr. H*** give in her
classes? What are the grade distributions? Do I give more A's with few
F's or am I a professor that has earned the name of "C Minus T***?"
It's that sort of information that was released. All data that
identified students had to be stripped out before the requested
information was released. My understanding also is that advising the
Academic Senate of this information is really a courtesy so that we
can let everyone know what is happening and have discussions about the
meaning of grades, the difficulty of assignments, grading rubrics,
etc. for the various instructors in our departments. In other words,
if a student signs up for my Philosophy 6 class and another student
signs up for Dr. H***'s Philosophy 6 class, is it fair that I give
three scantron exams while she requires three 20 page papers? (This is
a fictitious example. J ) What if she gives mostly C's with very few
A's while I give all A's? How do we work together collaboratively, as
a department, to ensure some kind of uniformity of assignments and

Legally, there is nothing we can do about the release of the
information—again, as I understand it. This does however offer us an
invitation to discuss, in our departments, the meanings of grades and
the rigor of our assessments.

I don't' mean this message to minimize the apprehension caused by this
announcement. I am apprehensive and expect that my students will soon
tell me that a new website has listed me under the "sucks badly"
category. However, until the law changes, we may want to focus our
attention on how we can improve instruction across the curriculum.

Nope, no hot button issues here!

Back in the day, professors used to post student grades on their office doors at the end of the semester. Enterprising students could use that to suss out who graded 'tough' and who was easy. Not that I ever dreamed of doing such a thing.

Now, of course, FERPA prevents that. But Open Records Laws, on the other hand, treat aggregate grades (as opposed to individual ones) as public records, open to public scrutiny. So professors (at public colleges and universities, anyway) are in the position of having to guard individual grades closely, while having years' worth of data posted on the internet. It's not a contradiction, strictly speaking, but it's certainly an odd juxtaposition.

There have always been student grapevines about which professors are easier than others. That's not new at all. I recall being warned by dormmates at Snooty Liberal Arts College not to take a particular professor who was famous for giving nothing but B-minuses to all and sundry. (It was a point of pride that I took him and did better than that.) The difference is that now the grapevine will have access to actual data.

From a dean's perspective, there's actually something useful in knowing – with data – that Prof Jones grades much more easily than the rest of the Basketweaving department. It gives some context for the student evaluations. If a professor grades unusually easily, I'm inclined to discount positive student evaluations. If a professor is known for strictness, I'm inclined to cut some slack on student evaluations. Of course, if a professor can't even buy love with easy grading, then I can be pretty confident that there's an issue. And a professor who grades tough but still gets glowing reports is probably doing something right.

All of that said, I'd be wary of putting data out there that isn't normed by class. In other words, the grade distribution in a particular class may reflect the professor's grading, or it may reflect the location of that class in the curriculum. Remedial classes, for example, almost always have much higher 'fail' rates than upper-level electives, regardless of who teaches them. So a professor who teaches a lot of remedial and first-semester classes will look like a tougher grader, all else being equal, than a professor who teaches mostly courses for majors. (By the time you get to grad school, grading is pretty much reduced to 'A' or 'Not A.') Telling me that there's a higher fail rate in remedial math than in calculus doesn't tell me anything about the instructors or the relative rigor of the courses; it tells me that the only students who take calculus here are students who really mean it.

At Proprietary U, one of my least favorite tasks as an administrator was to come down hard on faculty whose drop/fail rates were “too high.” (I don't usually recommend foot-dragging, but when it came to that, I foot-dragged like it was going out of style, until I found another job.) The justification, to the extent there was one, was that professors were supposed to find ways to reach even the more challenging students. In practice, of course, it resulted in lots of extra credit and some very creative curving. I considered it a stupid and offensive application of data that, treated differently, could have been useful.

In the age of the internet, certain kinds of discretion and/or secrecy just aren't viable anymore. If the data will escape, I think the burden on higher ed is to come up with ways to frame it productively. Let the colleges beat the profiteers to the punch, and put the data out there in ways that reflect what we know to be true. I'd suggest lumping several years' worth of data into a single report, and reflecting standard deviations from course means, rather than raw grades as such. If there's a particular professor who is consistently, conspicuously above or below her peers who teach the same classes, then I know I have something to examine. (Higher grades could reflect easier grading, better teaching, or the luck of the draw.) As any experienced teacher can tell you, some classes are stronger than others, which is why I'd put out rolling averages that encompass several years at a pop. Looking at one of my sections doesn't tell you much; looking at every section I taught over several years just might.

In a perfect world, a professor who fell unknowingly into 'outlier' territory would take being singled out as a wake-up call. Of course, in a perfect world, students would be motivated solely by love of learning, nobody would need remediation, and it would only rain at night. More likely, I would expect to see considerable defensiveness, attempts at blame-shifting, flat-out denial, and the usual huffing and puffing. Still, I don't have a conceptual problem with holding professors to account for how they perform the grading aspect of their jobs, since I (and most students) consider it part of teaching. I'm just concerned that if we don't take ownership of this in a thoughtful way, others will, and they'll do it in the stupid and thoughtless ways we rightly fear.

Wise and worldly readers – what do you think?

Have a question? Ask the Administrator at deandad (at) gmail (dot) com.