Sunday, July 12, 2015

Knowing Why They Know


Maybe political theorists just have a certain way of thinking.  That might explain it.

Betsy Barre, the Assistant Director of the Center for Teaching Excellence at Rice (and a fellow political theorist by training), fired off a set of tweets this weekend that made a lot of sense to me.  She was addressing the disconnect between relatively small gender effects in quantitative student evaluations and the relatively large gender effects in qualitative feedback.  In other words, while the “scores” of male and female professors aren’t very different, the comments they get are.  As Barre put it (quoting tweets):

  • my completely unscientific guess, given what I have learned from the literature, is that students are relatively good at…
  • identifying teaching that helps them learn.  They’re not so great at understanding “why” that teacher helped them learn…
  • so in the qualitative comments, they’re reaching for things to explain their intuition.  And when doing this…
  • they likely think in gendered ways (in both directions).  This would explain why numbers seem less gendered, at least.

Unscientifically, but with many years of experience of faculty evaluation (and of teaching), I think Barre is onto something.  

Last week, in the post about tolerance for ambiguity, I mentioned watching students in a writing class grasp for words to explain a hunch.  Once they found a cliche, they grabbed it and held on tight; even if it didn’t really work, it offered a sense of familiarity.  It came closer to feeling like an answer than anything else they had at the ready, so they used it.  My job as a teacher was to help them get past that response, and to develop the skill of articulating their own answers.  

But teaching evaluations are supposed to be uncoached, by definition.  (I once received one in which a well-meaning student offered that the class “helped me write more clearer.”  Great…)  In the comments section, students can easily fall back on long-ingrained habits.  

The good news, to the extent that there is good news, is that the unconsciously loaded language doesn’t always go very deep.  When asked to express the same ideas in a different form, such as numerically, the differences shrink notably.  

From a faculty perspective, of course, the larger worry may be less what students write and more how it’s read.  Here, too, Barre got it right: a single, isolated comment about almost anything should probably be disregarded, but if the same issue comes up over and over again, from both good students and bad, there may be something to it.  If I saw one or two students complain about too much reading, I’d write it off to “college is supposed to be hard.”  If a dozen students complained about the professor chronically showing up a half-hour late for class, I’d follow up.

I know it’s an article of faith among many faculty that student evaluations are meaningless at best, but I’ve been struck over the years that they’re usually in the ballpark.  They’re imperfect, of course, and I’d oppose any move to take them as gospel.  But if you observe enough professors, and read enough evaluations, you’ll notice they tend to get it broadly right.  If the sample size is decently large, the wisdom of crowds seems to kick in.  Yes, hot-button social issues can produce some jaw-dropping comments, but they’re less common than you might expect.  

Barre’s juxtaposition of quantitative and qualitative feedback suggests that some of what comes across as biased or inappropriate judgment is, in fact, an artifact of weak writing skills.  From the perspective of over a decade of administrative experience, I think she’s right.  

None of that is meant to discredit or discount claims for social justice.  It’s just to say that when we read student comments, we have to remember that we’re reading student writers.  Student writers tend to fall into certain traps.  They know what they want to convey, but they don’t always know why they know, or how to explain it in language that people with graduate degrees would consider appropriate.  When they shift to another register -- numbers -- they become more surefooted.

Thank you, Professor Barre, for connecting the dots.  And sorry for the cliche.