Sunday, June 18, 2017

 

The Evaluation Conundrum


How representative a class do you give when you’re being observed by a dean?

I’m guessing that for many people, it’s sort of like how they drive when there’s a cop in the rear-view.  It’s not faking, exactly, but there’s an element of wanting to put on a good show.  

In the case of the cop, that’s harmless enough, and probably inevitable.  In the case of teaching, though, that’s kind of a problem.  The only people who see the truth of long-term performance are students, and they often have agendas -- conscious or unconscious -- of their own.

That’s why I was so taken with this piece in IHE last week.  Alison Cook-Sather, a professor at Bryn Mawr and Haverford, used grant funding to develop a system in which students who aren’t taking a given class are hired to sit in on it and provide constructive, non-evaluative feedback to the professor on a regular basis.  

It’s a nifty idea for several reasons.  For one, it separates feedback-for-improvement from feedback-for-accountability.  They really need to be separate, and in practice, that can only work when they’re done by separate people.  Having students provide feedback that never makes it to the dean can help the faculty improve before a problem becomes a performance issue.  

Even better, the constructive feedback is from a student’s-eye view, and not based on a single day.  As hard as a peer or a dean might try to emulate a student’s view, it’s hard not to bring experience and training to what you see.  That can be a good thing, but it’s not the same thing.  Many years ago I observed a professor that most students considered a hateful jerk, and came away wondering why; he had a sly sense of humor that I really enjoyed.  They may have sensed that they weren’t getting a joke, or there may have been something else.  A student observer might have had a better shot at ferreting out the real issue.  To this day, I don’t know what it was.

I also like that it employs students in a way that treats their student status as an asset.  It suggests that they have something useful to say, and offers a venue in which they can say it.  

Too many colleges have some version of feedback-for-accountability, but no formal mechanism for feedback-for-improvement.  Instead, they try to get a single system to serve both purposes.  In trying to serve two purposes, it does justice to neither.  If I’m teaching, and I suspect that a shaky performance on observation day will have negative professional consequences for me, I’m likely to stick to the hits on observation day.  That’s individually rational, but collectively destructive.  I’m unlikely to break the cycle myself, though, because if I offer a warts-and-all performance and nobody else does, I look like a problem.  But splitting the roles between two people allows for both functions to be served fully.

I’ve long held that we could and should try something similar in the classroom with grading.  If we separate teaching from grading -- as in, swapping papers -- then the psychological dynamic between professor and student becomes simpler.  Instead of the fraught and complicated “I’m helping you, but I’m also judging you,” it becomes “I’m helping you; that guy behind the curtain is judging you.  It’s you and me against him.”  Athletics often work like that, with either the clock or the other team serving as the de facto grader.  

At Brookdale we’ve established a cadre of faculty who are “on call” to provide non-evaluative observations and feedback for other faculty.  Uptake so far has been modest, but I remain convinced that there’s merit in the model.  The observers are sworn to secrecy about who they observe and what they see; all I ask at the end of the semester is a total number.  Anecdotally, the observers report that the post-class conversations are often quite good, since there’s nothing stopping them from telling the truth.  And from an administrative perspective, I’d much rather have faculty performing well than performing badly for a whole bunch of reasons.  If the occasional tune-up helps someone who’s struggling to get back on track, everybody wins.

Kudos to Prof. Cook-Sather for finding a way to take that model and expand it to include students.  It’s a terrific idea, and the concept -- if not the funding -- is easy to replicate.  I tip my cap.



Comments:
Up front I will say that I dislike the scheduled visit because the class gets tailored to the evaluation. I know of one instance where a major problem was missed for years because of that. I don't care when the dean visits as long as it isn't an exam day, but you might need to schedule around a quiz so you don't arrive until it is over or can leave before it starts.

What they observed and you do all rings true to me. I wish my college did either of those! I think I've mentioned that an "open classroom" (for visiting faculty) was the flavor of the month a few (more than a few?) years ago at my CC. A colleague and I used it, just among ourselves (not even included in our annual account of service to the college), and we both found it helpful. However, overall, I get more useful feedback from students through some informal anonymous mechanisms within the course. If you can get written free-response feedback from students to you college evaluation from a good sample, you can also learn what you need to know.

I like your approach, but faculty look at a class as faculty. I might notice a few things related to how the lesson was approached, but I mostly notice what students are doing during class as a measure of engagement and whether I can see what is on the screen or can understand what is being said. Students are much more likely to pick up on something that we don't notice, both good and bad. That said, both of those evaluation processes have the advantage of reasonably prompt feedback. A dean's evaluation at the end of the semester is pretty much useless.
 
Harry Brighouse on Crooked Timber wrote about doing hiring a student to observe his classes and give feedback. He found it very helpful as well.

http://crookedtimber.org/2017/01/18/a-neglected-equity-issue-on-college-campuses-instructional-quality/

SamChevre
 
Sorry, I posted the wrong link above: here's the correct one.

http://crookedtimber.org/2013/05/28/employing-a-student-to-criticize-my-teaching/

Key paragraph: I have no proof that the students learned any more in that class than in other classes. I am certain that having her as a teacher-critic on the spot as it were made my teaching better during the semesters in which I hired her, but also that my teaching has improved, generally, as a result.

SamChevre
 
Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?