Tuesday, April 26, 2011

 

Ask the Administrator: Incentives for Course Evaluations

In a timely note, a regular correspondent writes:

My college is on a big push to do course evals... we do them at midterm and the end of the course. So about every 7 weeks. Understandably students get burned out on all that evaluating and drag their feet about completing them, so we receive several "tips" a week about how to get students to do so, such as this (it's all cookie cutter stuff fromonlinecourseevaluations.com, no problem sharing them):


Getting Response Rates Tip 4:

Have you thought about offering incentives? Some faculty do not like reminding students because they feel like they are nagging their students. Group incentives are a great alternative. This allows the students to push their classmates to complete their evaluations. You may use something similar to the following:

'If this class gets an 80% response rate by the end of the evaluation, I will allow one 3x5 index card of notes to be used during your final.'

'If this class gets a 70% I will remove the lowest quiz grade.'

Also this from a youtube video were were encouraged to watch:

"I decided to tell students that if they filled out their course evaluations they would get a 1 to 2 bonus on their final score".

Is it just me or is all of that pretty unethical? I find the last part beyond the pale... to suggest we give points on the final grade for something that really has nothing to do with learning or coursework? What if that was the difference between failing and passing? That would mean we were handing out college credit for busy work, basically.

The real question is what should I do? My instinct is to just suck it up, ignore the bad advice and move on. On the other hand I feel like they need to be told this is not OK. They bought into this eval website and are just repeating their bad advice... maybe someone just needs to call them on it. Suggestions. from your wise and worldly readers?



It’s a fine line between rewarding participation and rewarding a positive response. I wouldn’t be surprised to find that some students simply assume the latter.

I’ll share that I am not -- at all -- a fan of “extra credit.” It introduces too much noise into grading -- a noisy enough process already -- and often rewards the wrong things. “Collective” extra credit is that much worse, since it puts pressure on students to put pressure on other students. It goes well beyond the standard issues involved with “group work,” since it involves the entire class. (Of course, from my side of the desk, the most offensive extra credit is the kind that’s offered ad hoc to one or a few students and not to the rest. If some of the un-offered students are in protected classes, the legal issues are staggering.)

As with any kind of poll, the people most strongly motivated to give feedback are the ones with the most strongly held, and therefore usually the most extreme, opinions. In some ways, those can be the least valuable. Nearly any teacher worth her salt has had a student who thought the sun rose and set on her, and another who thought she smelled vaguely of sulfur. What I want to know is what the overall tendency is, and for what reasons.

I’ve never heard of midterm evaluations that were shared with administration. I’ve heard of professors doing their own midterm evaluations just to find out where the students were struggling, and then using that feedback to make adjustments. That seems reasonable to me, though I rarely did it myself. But the standard number-two-pencil evaluations that get tabulated? Not at midterm...

I wonder if the issue isn’t so much student attitudes per se, but rather a ridiculous situation that would naturally give rise to student cynicism. Even after all these years in administration -- in the words of the old Westerberg lyric, you can count the rings around my eyes -- I can’t imagine a productive administrative use of midterm evaluations. It seems like that would be the conversation to have. End of semester, I understand, as do most students. But midterm just seems like overkill. After a few years with multiple classes per term, I could easily imagine students getting a little tired of it. I would.

But maybe that’s me. Contexts differ, and some people out there have probably found ways to make this work (and/or to make it useful). Wise and worldly readers, what do you think? Is there an effective, yet ethical, way to get students to indulge all those evals? Or is the entire enterprise misguided? Or, both?

Good luck. Sorry I don’t have the magic bullet on this one.

Have a question? Ask the Administrator at deandad (at) gmail (dot) com.

Comments:
I agree that 'collective' extra credit is a bizarre concept. However, I do not have a problem with giving students a small amount of credit for participating in an eval. At our college, we use a standardized instrument (http://www.theideacenter.org/node/5) that allows us to assess various parts of a course, in addition to evaluating the instructor. As such, we consider it a valid use of students' time and part of the learning experience. We have also discovered that the response rate in online classes is horrendous, and the only way to get a meaningful response rate it to offer a little credit. We don't call it 'extra' credit, it is just an assignment, like others in the class.
 
Make a real effort to explain just how these evaluations (or the CCSSEs or whatever survey you're giving them) will be used. Explain that you actually read the evaluations, perhaps even provide specific examples of feedback that you've incorporated into your teaching. I've found that students will take surveys more seriously when they are given a reason to believe it is meaningful. When the instructor shows little interest in the process, students can tell and it shows in their responses (or lack thereof).
 
Course related incentives? Ugh. Just give them cookies. It's not expensive, and an Obvious Bribe has the advantage of being Obvious.
 
I agree with Becca.

Cookies, cupcakes, chocolate dipped fruit or candy as a treat before every test and sporadically for turning in some assignment works very well for me.

Their brains need the extra sugar for the test. If it is a long test (40 minutes or >), they will need more than one.
 
All of our CC classes have an online companion site. When it is time for evalulations students complete them through this site and cannot access the online course materials (aka grades) until they finish the assessment.

Certainly some students just skip through it, but once there on the survey most take it seriously.
 
Why even worry about it? "The college"--meaning who?--"is on a big push to do do course evals."

Not all students are participating. That should be that. Why waste time nagging, or worse, bribing, students to get questionable data?


As DD points out, an important problem with this kind of data is that only those with strong feelings are likely to respond.

--Philip
 
The correct response is food. You don't need to do anything else other than say, "I would like for everyone to complete these forms. Here is some food."
 
Also, don't try to fight the pretend data. We're doing nothing but pretend data, until we stop promoting idiots.
 
Our institution has removed evals from the classroom completely. Now students are goaded (through email, one presumes) to complete the eval online. Not clear yet if this will be tied to anything, grades or otherwise. Hard to imagine it will. Also I'm not sure what prompted the move to online evals--likely a good deal of faculty either weren't administering them or were just going through the motions. That said, the return on the investment strikes me too as questionable.
 
A side note to your side note about extra credit, CCD. You mentioned you do not like XC becasue it introduces "noise" into "a noisy enough process already".
For me, the idea that the process is noisy is the very reason *to* be comfortable with XC.
The point is this. We can not hope to capture our students learning perfectly. No measurement system is that good (in the behavioral realms at least; now if you're in something realtively easy to measure like physics that's a different story, but behavioral areas like understanding learning are much more difficult than physics). Acknowledging that our measurements aren't perfect I allow limited opportunities for my students to show alternative ways he/she has learned the material.
 
Bizarre!

1. It is trey weird that the administration collects midterm examinations. That strikes me as pretty poorly thought-out.

2. Response rate should not be held to be the problem of the instructor. (Yet I realize that at many schools, an instructor who gets poor response rates will be held responsible, and this fact will be viewed as a negative when it comes time for tenure or promotion.) There are very simple ways to address response rate: you administer the survey in class at the end of the last class, and you establish a practice of always doing so. Or, if it is absolutely necessary to conduct surveys online, you must use coercion: e.g., the student's grades are not released until they complete the survey. If the administration refuses to take those steps, well, that's the administration's fault. At my school, we use in-class surveys, and it works pretty well: it gets good response rates. The students who don't respond are mostly those who don't show up at class at all, and frankly, I'm not sure those students' opinions should be included in a survey anyway, if they can't be bothered to show up to class on the one day when surveys are administered.

Anyway if the administration sets up a poorly-thought-out system that pressures instructors unfairly into gaining good response rates, then I don't think it is out-of-bounds for instructors to offer modest incentives to students, if it comports with their own morals and does not violate university rules. If that feels like an undesirable outcome, we should look to the folks who set up unwise rules that put instructors in an unreasonable situation. (Let me guess: I'll guess the survey system was not passed with the participation and consent of the university's faculty, and I suspect shared governance is weak at that school. Am I right?)

P.S. Personally, I would not give extra credit on exams for returning surveys; it does not comport with my own sense of morals. But I wouldn't look down on anyone else who did so.
 
Gary, DD made the sound point that extra credit "rewards the wrong things", a point that I agree with completely. You failed to explain why adding more noise will improve the assessment of what students have learned.

I wonder if Engineering Prof just figured out that extra credit in college classes explains some of the math skillz you see today.
 
DD, my dean uses food to reward participation at faculty meetings. Works well.
 
The incentives seem crazy to me. First, getting students to pressure each other to fill out a survey that is mainly for the teachers / school's benefit is unethical. (It's generally not allowed when you are doing research surveys.) Though you could argue that the course evals will eventually help other students, there's no excuse such as "we're getting the students to get a better education"-- the evals aren't helping the current students and aren't meeting educational goals for current students. Second, the bribes mentioned might make students feel guilty about giving bad evaluations. The website is promoting practices that are likely to affect the evaluations, thus undermining their reliability and utility. It would be much better if the school itself were putting the pressure on in an anonymous way-- like the technique mentioned here with grades only being viewable after going through the course eval.
 
Cool! Author the best! Thank you!
resume editing
 
CCPhysicist -- Oh, I agree we do not want to reward the "wrong things" (e.g., I think rewarding evaluation completion, since it has nothing to do with the content of my classes, is rewarding the "wrong thing"). Perhaps my point is too tangential to this particular post/thread, but if I read "noise" to be contamination/deficiency/less than perfectly valid grading, then I accept that it exists and accept that my grading schema will never be perfect (no one's will). I offer *limited* opportunities for my students to show me they can do, or know the "right things". It puts some power in their hands (I think they tend to cede way too much power to me anyway, but then I'm your fairly typical lefty anti-authoritarian faculty type). It also provides them some autonomy/control (which humans appreciate).
How do I know what's the "right thing"? I've been teaching most of my stuff for 12 years now. I have gained some intuition in recognizing students' meeting course objectives. You'll have to trust my validity on this becasue I'm not going to explain my course objectives here nor how I come to this intuitive understanding (intuition, by its very nature is difficult to explain)
 
Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?