On Monday I had a heartening discussion with a professor on campus about some promising practices to improve student retention that he had seen discussed at a recent conference. I don’t want to betray any confidences, so I’ll just say that they sound potentially fruitful, but also labor-intensive. Absent a visit from the money fairy, if we were to try them, they’d have to be phased in slowly.
Which raises a question that I’m sure some wise and worldly readers have addressed at one point or another.
A slow phase-in means that some students will receive a benefit that others won’t. And, bless them, students talk to each other. I’d bet that some students who don’t get the benefit will hear from others who did, and will start asking questions. “How come my friend got this and I didn’t?”
It’s a fair question, at one level, but it also threatens to throw a monkey wrench into an experiment.
The student grapevine is a mixed blessing. In some areas, I’m absolutely counting on it. For instance, once OER adoption hits critical mass, I wouldn’t be surprised to see some students confront faculty who aren’t using it to ask why not. At the very least, I’d expect to see students vote with their feet and populate sections with OER before filling other sections of the same course, thereby creating a sort of market pressure to keep costs down. If you believe, as I do, that textbook costs are a barrier for many students, this kind of pressure can be positive.
In some areas, it’s both good and bad, but a fact of life. Students have discussed favorite and least favorite professors forever. There’s a well known website that starts with “rate…” that can be counted on to start faculty grumbling at the mention of its name, but the kind of information the website provides isn’t new. The innovation is that what had been a sort of oral tradition has been written down and disseminated, but without the editorial care that we like to think usually attends publication.
When it comes to on-campus experiments, though, the grapevine can be a real problem. Even answering the question of why a given student isn’t getting a benefit can contaminate the sample. But stonewalling doesn’t make the question go away; it just opens up a vacuum in which other, more sinister or baroque, explanations can thrive. People will try to make sense of the world given what they have to work with. As a kid, I was perplexed that the ancients would pick out five stars in the sky and declare confidently that it was an octopus playing a harp. That kind of filling in of the blanks happens when blanks are left open.
I can’t be the first person to face this. So, wise and worldly readers, I look to you. Have you seen reasonably elegant ways to do control-group experiments with interventions on campus that don’t fall apart when someone from the group left out asks why?
