Tuesday, August 01, 2006

 

Ask the Administrator: Observing Outside Your Discipline

A gratifyingly prolific correspondent asks:


At a CC, how are teaching observations vs. Student
feedback used in tenure evaluations? How do you (if
you do) handle observations of folks not in your
discipline?


I'm thinking of my dean observing me teaching
statistics (I also taught Spanish, but no one ever
observed me doing that). She would come in and take
notes, which would, somehow, be used someway... She
didn't know stats to save her life. Two semesters
later I was tutoring her through the same level course
that she was taking elsewhere.

Basically: Most of us (ya'll) were faculty in a
discipline. In your academic training you probably
got a 1-credit seminar on how to teach your
discipline. You got some on-the-job mentoring, but
most of us (ya'll) never read education research. How
would you know what's good practice and what's not?
And, how do you determine if your faculty are enacting
it?

I left the CC before ever having a tenure-meeting, so
never did find out how that file was used.

I've never taken seriously those little course-eval
forms. I only know a few students who ever did.
Oddly though, I prided myself on getting high marks.


You’re overly generous in your assumptions about teacher training. I still remember the sum total of my training before being thrown in front of an intimidatingly-large class: the prof for whom I was t.a.’ing, an extremely famous personage many people have heard of, declared in his typical Olympian fashion, “you’ll be fine.”

Thanks. I wouldn’t have thought of that, since I am a complete idiot.

Even a one-credit seminar would have been more substantive than that.

The heart of your question, though, is about observing classes outside one’s own discipline. I do this all the time, since my own discipline is a very small percentage of my administrative jurisdiction. Some of the other fields are close enough that I don’t feel out of my depth, especially since the classes are all at the 100 or 200 level, but some of the fields are simply beyond anything I’ve done before. (Example: I’ve observed Intro to Italian. My only Italian comes from menus and the florid cursing of friends’ grandparents.) And no, I don’t have formal training in education research. That’s not unusual for someone in my position. In fact, the higher up the food chain you go, the more it will be true that you’ll be responsible for disciplines of which you know little or nothing. No college president is master of every discipline at the college, and it would be unreasonable to expect otherwise.

In a way, a certain lack of familiarity can help. Since I don’t know enough to fill in the gaps myself, I can approximate the perspective of a student fairly well. Can I follow what’s going on? If yes, and if the students are clearly getting it, then I assume that all is well. In my own discipline, I have the handicap of being able to follow what’s going on even if the teacher is flopping, so it might be harder to notice a partial or confusing explanation.

(It’s the same reason that the best players don’t often make the best coaches. Ted Williams was a lousy manager, since he couldn’t understand how anybody couldn’t hit. Someone who knows what it is to struggle can make a better instructor.)

A former colleague of mine used to say of teaching, “stand on your head and spit wooden nickels if it helps.” There’s something to that; either a class works or it doesn’t, whether it’s done in my style or someone else’s. I’ve seen instructors use approaches that I would never use, but they’ve worked, and that’s what matters. I don’t pretend to have a monopoly on pedagogical wisdom.

I’ve used my own observations as reality checks. In my experience, student evaluations are usually closer to the mark than most of us would like to admit, but they can be swayed by accents, humor, and grading. (Students will forgive almost anything other than an accent.) Department chair evaluations are always, without exception, glowing, which makes them useless. My own visit to a class helps me place the other inputs into perspective. At my previous school, I twice had the experience of visiting classes that always got low scores from students, only to come away highly impressed with the teaching: in one case, he was a hard grader, and in the other, she had an accent.

Has anybody been denied tenure based solely on my observation? Nope. It’s one input among others, which is probably about right. I’d hate to give it up, since then I’d be entirely at the mercy of people who have clear self-interest in particular outcomes, but it’s certainly not decisive in itself.

Have a question? Ask the Administrator at ccdean (at) myway (dot) com.

Comments:
Having just got tenure at a CC, I can tell you that the process itself was kind of mysterious -- but that my classroom evaluations were always followed-up by a meeting with the observer to discuss how the class went.

The forms used to capture the evaluation information were generic and I'm sure that my dean would have seen things differently had she been a philosopher. Overall, she was looking at how I interacted with the class, not whether or not my information was accurate.
 
I still don't understand how student evaluations are useful. I'm intelligent-enough to be in a PhD program, and only in my last class as a student did I have a good enough understanding of how the professor was actually teaching to say anything remotely useful on an evaluation.
I'm not saying this out of bitterness; I haven't yet recieved my evaluations from my first time teaching. I'm also not saying we should junk the evaluations; there's nothing to replace them. But I still feel twitchy about them.
 
Recently, my boyfriend lost an adjunct job teaching an unpopular Writing for Business class at a fine arts college. He was there for ten years, always finding new ways to make the class more worth his students' time, doing resumes and mock-interviews, turning it into a real get-a-job workshop. His faculty reviews ranged from good to excellent. But last year, a student, upset about getting a B, wrote a comment that said he often arrived late for class. (He had been late once due to a family emergency.) The school said they were "extremely concerned," but wouldn't meet with him when he offered to defend himself. They said their schedule was too tight this year and maybe they'd have a class for him. They didn't, and they probably won't again.

That's what kills me about adjunct work. It's not even a matter of "getting fired"; it's that any random thing, from a student comment to a desire to hire someone new, can mean they simply don't have classes for you this semester. Try again next time!

This is to say, I get the feeling that my evaluations from students, when glowing, mean nothing. No one comes around and says, "AWB, you're doing a bang-up job, clearly," but if there's something "bad" there, I'm in danger of losing my job altogether.
 
I still don't understand how student evaluations are useful. I'm intelligent-enough to be in a PhD program, and only in my last class as a student did I have a good enough understanding of how the professor was actually teaching to say anything remotely useful on an evaluation.

If you mean that student evaluations are not useful as a guide to improve one's teaching, then yes this is often the case. Even when a student tries to help, the forms rarely leave them enough room to say anything useful. (I once got the comment: "Sometimes you go too fast, other times too slow". True, no doubt, but not helpful.) An exception would be when you are doing something wildly wrong about pacing, workload, etc.

However, as measure of general student satisfaction, I presume that they're fairly accurate. Of course, "good teaching" is only one component that goes into that, see e.g. the Texas study which showed that how attractive the instructor is can have a marked affect on evaluation scores.

Dean Dad is certainly correct about the accent thing. I read about a study where they recorded a white Midwestern prof lecturing, and then played back the tape to two different student audiences. For one group, there was a picture of the original professor next to the tape recorder; for the other group, there was a picture of someone who was originally from Asia. The second group gave the lecturer much lower evaluations, even complaining about his "accent"...
 
Re: student evaluations- often they are not helpful, but a saavy adminsitrator takes them for what they are- a potential warning of a problem they may not have seen. Unfortunately, there are many adminitrators less willing to do the legwork and more interested in 'gotcha' things, and they are taken at face value to the detriment of dedicated adjuncts etc.

Re: observing outside of your discipline- I, like all administrators, trained as a specialist. MOST of our observations are from outside the discipline. However, since my primary discipline is in the arts, I am asked at interviews and functions 'how I think I can help an academic instructor to improve their teaching.' My reply has been 'perhaps my perspective makes me a better evaluator, since my subject area teach in a much different fashion. My methods of assessment and teaching style may offer a new slate of ideas to the instructor.

The bottom line in all of this is that the administrator must be interested in improving instruction, and the instructor must be aware and comfortable with that from the outset. Otherwise, the process is faulty and unsuccessful.
 
I've heard about studies looking at the correlations between student evaluations vs. evaluations based on watching a video of the teacher for five minutes vs. those based on a still picture of the teacher. Apparently, the correlations are very high.

Which suggests two possibilities: people are no good at evaluating, or they're so good that they can glean a lot from the still picture. Depending on which aspect is being evaluated, I lean towards both.

I also read about studies trying to determine how much it helps for the teacher to be good-looking. It does; and it matters more for men!
 
The issue of accents, and other "othering" characteristics that set the professor apart from the students, is what makes me cringe when thinking of student evals. Though that's not the focus of this discussion, I often wonder if a colleague can pick up on the discontent among dissatisfied students.

Evals from students and colleagues are a sore spot for me because my colleagues at mostly white university often looked solely at the eval numbers when making their conclusions. And when I was a professor of color at mostly white institution, those deeply felt biases about race, gender, and sexuality among the students emerged pretty quickly.

But DD is also right about tenure and teaching evals. No one ever lost tenure (that I know of) based exclusively on poor evals. Bad reviews are usually a cover for something else.
 
I find that the accent thing can go the other way as well. Although I would love to credit my brilliant and insightful lectures, I suspect my "cool" Australian accent contributes more than a little to my glowing student evaluations...
 
As a tenured prof at "Fallen Ivy University," I have ONLY been evaluated by students. Not a single colleague has checked-in on my teaching.

And no, student evaluations aren't worth a rat's ass for tenure, unless they're awful. Then, they CAN be the kiss of death.
 
I've taught now for two years and NEVER been observed by my program coordinator (although she was well-intentioned, it never happened). Because I teach human sexuality, the content of my student evaluations makes me VERY nervous! For example, if they're freaked out by the concept of homosexuality, the fact that a whole chapter is dedicated to sexual orientation is going to make them VERY miserable and they'll blame me.
 
Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?