Tuesday, August 01, 2006
Ask the Administrator: Observing Outside Your Discipline
At a CC, how are teaching observations vs. Student
feedback used in tenure evaluations? How do you (if
you do) handle observations of folks not in your
I'm thinking of my dean observing me teaching
statistics (I also taught Spanish, but no one ever
observed me doing that). She would come in and take
notes, which would, somehow, be used someway... She
didn't know stats to save her life. Two semesters
later I was tutoring her through the same level course
that she was taking elsewhere.
Basically: Most of us (ya'll) were faculty in a
discipline. In your academic training you probably
got a 1-credit seminar on how to teach your
discipline. You got some on-the-job mentoring, but
most of us (ya'll) never read education research. How
would you know what's good practice and what's not?
And, how do you determine if your faculty are enacting
I left the CC before ever having a tenure-meeting, so
never did find out how that file was used.
I've never taken seriously those little course-eval
forms. I only know a few students who ever did.
Oddly though, I prided myself on getting high marks.
You’re overly generous in your assumptions about teacher training. I still remember the sum total of my training before being thrown in front of an intimidatingly-large class: the prof for whom I was t.a.’ing, an extremely famous personage many people have heard of, declared in his typical Olympian fashion, “you’ll be fine.”
Thanks. I wouldn’t have thought of that, since I am a complete idiot.
Even a one-credit seminar would have been more substantive than that.
The heart of your question, though, is about observing classes outside one’s own discipline. I do this all the time, since my own discipline is a very small percentage of my administrative jurisdiction. Some of the other fields are close enough that I don’t feel out of my depth, especially since the classes are all at the 100 or 200 level, but some of the fields are simply beyond anything I’ve done before. (Example: I’ve observed Intro to Italian. My only Italian comes from menus and the florid cursing of friends’ grandparents.) And no, I don’t have formal training in education research. That’s not unusual for someone in my position. In fact, the higher up the food chain you go, the more it will be true that you’ll be responsible for disciplines of which you know little or nothing. No college president is master of every discipline at the college, and it would be unreasonable to expect otherwise.
In a way, a certain lack of familiarity can help. Since I don’t know enough to fill in the gaps myself, I can approximate the perspective of a student fairly well. Can I follow what’s going on? If yes, and if the students are clearly getting it, then I assume that all is well. In my own discipline, I have the handicap of being able to follow what’s going on even if the teacher is flopping, so it might be harder to notice a partial or confusing explanation.
(It’s the same reason that the best players don’t often make the best coaches. Ted Williams was a lousy manager, since he couldn’t understand how anybody couldn’t hit. Someone who knows what it is to struggle can make a better instructor.)
A former colleague of mine used to say of teaching, “stand on your head and spit wooden nickels if it helps.” There’s something to that; either a class works or it doesn’t, whether it’s done in my style or someone else’s. I’ve seen instructors use approaches that I would never use, but they’ve worked, and that’s what matters. I don’t pretend to have a monopoly on pedagogical wisdom.
I’ve used my own observations as reality checks. In my experience, student evaluations are usually closer to the mark than most of us would like to admit, but they can be swayed by accents, humor, and grading. (Students will forgive almost anything other than an accent.) Department chair evaluations are always, without exception, glowing, which makes them useless. My own visit to a class helps me place the other inputs into perspective. At my previous school, I twice had the experience of visiting classes that always got low scores from students, only to come away highly impressed with the teaching: in one case, he was a hard grader, and in the other, she had an accent.
Has anybody been denied tenure based solely on my observation? Nope. It’s one input among others, which is probably about right. I’d hate to give it up, since then I’d be entirely at the mercy of people who have clear self-interest in particular outcomes, but it’s certainly not decisive in itself.
Have a question? Ask the Administrator at ccdean (at) myway (dot) com.
The forms used to capture the evaluation information were generic and I'm sure that my dean would have seen things differently had she been a philosopher. Overall, she was looking at how I interacted with the class, not whether or not my information was accurate.
I'm not saying this out of bitterness; I haven't yet recieved my evaluations from my first time teaching. I'm also not saying we should junk the evaluations; there's nothing to replace them. But I still feel twitchy about them.
That's what kills me about adjunct work. It's not even a matter of "getting fired"; it's that any random thing, from a student comment to a desire to hire someone new, can mean they simply don't have classes for you this semester. Try again next time!
This is to say, I get the feeling that my evaluations from students, when glowing, mean nothing. No one comes around and says, "AWB, you're doing a bang-up job, clearly," but if there's something "bad" there, I'm in danger of losing my job altogether.
If you mean that student evaluations are not useful as a guide to improve one's teaching, then yes this is often the case. Even when a student tries to help, the forms rarely leave them enough room to say anything useful. (I once got the comment: "Sometimes you go too fast, other times too slow". True, no doubt, but not helpful.) An exception would be when you are doing something wildly wrong about pacing, workload, etc.
However, as measure of general student satisfaction, I presume that they're fairly accurate. Of course, "good teaching" is only one component that goes into that, see e.g. the Texas study which showed that how attractive the instructor is can have a marked affect on evaluation scores.
Dean Dad is certainly correct about the accent thing. I read about a study where they recorded a white Midwestern prof lecturing, and then played back the tape to two different student audiences. For one group, there was a picture of the original professor next to the tape recorder; for the other group, there was a picture of someone who was originally from Asia. The second group gave the lecturer much lower evaluations, even complaining about his "accent"...
Re: observing outside of your discipline- I, like all administrators, trained as a specialist. MOST of our observations are from outside the discipline. However, since my primary discipline is in the arts, I am asked at interviews and functions 'how I think I can help an academic instructor to improve their teaching.' My reply has been 'perhaps my perspective makes me a better evaluator, since my subject area teach in a much different fashion. My methods of assessment and teaching style may offer a new slate of ideas to the instructor.
The bottom line in all of this is that the administrator must be interested in improving instruction, and the instructor must be aware and comfortable with that from the outset. Otherwise, the process is faulty and unsuccessful.
Which suggests two possibilities: people are no good at evaluating, or they're so good that they can glean a lot from the still picture. Depending on which aspect is being evaluated, I lean towards both.
I also read about studies trying to determine how much it helps for the teacher to be good-looking. It does; and it matters more for men!
Evals from students and colleagues are a sore spot for me because my colleagues at mostly white university often looked solely at the eval numbers when making their conclusions. And when I was a professor of color at mostly white institution, those deeply felt biases about race, gender, and sexuality among the students emerged pretty quickly.
But DD is also right about tenure and teaching evals. No one ever lost tenure (that I know of) based exclusively on poor evals. Bad reviews are usually a cover for something else.
And no, student evaluations aren't worth a rat's ass for tenure, unless they're awful. Then, they CAN be the kiss of death.