Thursday, November 15, 2007
Death Valley
Although the very thought of it makes some academics blanch, I'm beginning to think that 'evidence-based management' could be really useful in solving some nagging academic problems.
As I understand it -- and I'm no expert -- 'evidence-based management' takes as a given that it's appropriate to look at statistical patterns that have emerged over time, and to use those as reality checks for future decisions. It's particularly helpful in testing long-held assumptions for which we somehow keep noticing exceptions.
Most colleges of any size have someone in the back corners of the administration whose title is something like "Institutional Researcher." (The really big places have entire offices devoted to IR.) It's a sort of locally-applied social science. I think I've found something to ask my local IR guru to check out.
It's an article of faith among certain faculty that certain time slots for class meetings generate wildly higher attrition than others. ("Death Valley" is the local term.) The late afternoon weekday sections of required Gen Ed courses -- English Composition, say -- are held to be the last to fill, and the first to shrink as students just vanish. The "Prime Time" sections -- basically late morning to early afternoon, Monday through Thursday -- fill first, and usually finish the semester with almost as many students as they started.
So my questions for the IR guru, and for faithful readers who may know of research in this area:
1. Do Death Valley classes actually have higher drop rates? If so, does the effect disappear for upper-level courses? (If it does, I could see a pragmatic argument for running Gen Eds in prime time, and upper-level courses in Death Valley. That's pretty close to what Proprietary U used to do.)
2. If the drop rates are actually higher, is that really a function of the well-documented "last in, first out" rule of registration? (That is, the students who decide to sign up for classes the day before the semester starts have much higher drop/fail rates than the students who sign up months in advance. This is true, presumably, for many reasons.) Or do the students who sign up for Death Valley early also have higher drop rates?
3. Can we predict, with some level of specificity, the degree to which we could expect higher drop rates? Do those students usually come back, or are they usually lost for good? (Do they disappear at higher rates than other students who drop?)
4. If the answer to 3 is an actual number -- or at least a relatively modest numerical range -- could we do a little cost-benefit action to see if adding 'spillover' sections in Death Valley actually makes fiscal sense over time? (If that strikes you as cold, do the analysis in terms of future academic success.) Would we be better off, in the aggregate, simply abandoning Death Valley and going to waitlists and/or more online and/or hybrid offerings, or is Death Valley the least bad option?
5. Other than registering later, do Death Valley students differ in meaningful ways from Prime Time students? Are any higher dropout numbers actually consistent with those factors, rather than the time slot per se? (If so, then moving them around wouldn't really solve anything.) Or are these really the same students, just with funny schedules?
6. Do the instructors who typically teach the Death Valley sections -- and it's usually the same cohort from year to year -- have higher drop rates in Prime Time? Is this an instructor effect written onto a timeslot?
Wise and worldly readers -- I'm asking for your help in a couple of ways.
1. Do you know of any good research on this? It isn't the area of my scholarly training, and I'm just a wee bit busy.
2. Can you think of other questions I should add to the list?
Thanks for your help! My IR guru will likely have me killed, but evidence-based answers to these questions might actually be useful, which is sort of the point.
My department frequently pesters the Powers-that-Be asking for the data so that we can do these types of analyses on our own (for free!), but the PTB claim that they can not get the data out of the computer system. (Not a FERPA or policy objection -- a technical one.)
If your IR guru can't devote time to your project, you can probably get someone in math or econ to do it in exchange for a very small bribe.
We, as a faculty, learned recently that all sorts of institutional data we thought were lost in the entrails of the system are actually available for internal use by planners (albeit maybe only recentlly) and will shortly be available to all faculty on our portal, and mining the full data set for cross-course correlations (passing as a function of grades in pre-req courses) will be supported.
It can be done. What has been done is already interesting.
DD: That is a great idea. Please let us know what you learn. One thing we know is that the true Death Valley is a section created to handle demand (opened up on the last day of registration), regardless of the quality of the instructor. Predicting enrollment so the right number of sections are available is a big effort here.
: 5:00 AM Since the difference between your gen ed courses and your upper level courses is that there are a bazillion sections of gen ed courses whereas with upper level ones, you have to take them when they're available... student behavior in the gen ed courses would seem to be a good indicator of student preferences in scheduling.
If you take the courses that there are few of, that students have to take when they are offered, and put them in the least convenient time slot for your students, it is entirely possible you are creating serious barriers to completing the degree for some students.
After all it's unlikely that all your students are saying to themselves, "I really hate 3:30 in the afternoon," but it may be things more like having to pick up their children from day care, work schedules, etc. that make those time slots unworkable for them. It seems unkind then to take advantage of the fact that some courses are ONLY offered in one time slot and therefore are a take-it-or-leave-it deal.
: 8:43 AM That said, there are a few things that influence enrollment against Death Valley Syndrome (DVS).
1. the professor is a star/has a cult of personality on campus.
2. the class is required of all majors.
3. the topic is sexy enough and/or is so infrequently offered, you take it when you can get it.
Can you schedule your "stars" at certain times?
: 12:32 PM Not that it is relevant to your potential data research Dead Dad (of which I would love to hear the results), but those death valley classes were both last minute additions.
: 5:08 PM I'm teaching two sections in Death Valley of, you guessed it, English Comp. It's a required class and the most basic of the writing classes in the English Department.
My drop rate this semester has been insane. Out of 50 students total I'm left with 22. I definitely think there is (in general) something different about these students. The good students are great and the rest are struggling. So I, too, would be interested to know the answers to your questions. I'd love to get some facts about drop rates at my institution for different times. I've taught the class earlier, and less people drop, but the pass rate isn't much better.
Thanks for asking those questions.
This semester I taught a class in the morning after the professor had a schedule conflict and couldn't do it. I think the morning students are more focused or partly asleep. The drop out rate was about 10%. I am talking 8:00 AM. The aft. ones seem to be more sociable. Drop out rate-50%.
I think the aft. classes are usually overflows, so many of the students in them enroll at the last moment. To me that means they may not have well developed organizational or planning skills, which may be the cause of the high drop out rate. They just do not plan their assignment working times well or something along those lines and pretty soon they are overwhelmed. Maybe you could print a little booklet on organizing time.
I would like to see the results of the data, too.
: 6:09 PM From departmental experience, I've seen more effect from the "last scheduled, first dropped" sections AND students than it being the timeslot. But it would be nice to see more data and figure out if that's more than a few odd one-offs of my own experience talking.
I made sure to point that out to the prof as we really didn't like overloading each section by ten students to "do these kids a favor." Now I'm not really likely to let students crash full sections at all.
I was scheduled to teach an honors section which didn't fill with honors students (another problem, another post..). So, in the last few weeks of registration, it opened up -- in prime time (T/Th 9:30).
This is the WORST class I've ever taught. Their drop rate is pretty high and their failure rate will be much higher than any class I've taught in 'Death Valley'.
I'm doing the same material on the same schedule as my evening section, and the difference between classes is shocking.
I've also taught amazing sections of this course in the 9:30 time slot. Those sections fill up quickly, so I doubt the difference is me.
1) Classes scheduled in DV will fill last.
2) The "golden" classes will fill and get hefty wait lists
3) Drop rates seem no different unless...
4) As someone noted above, these classes are added at the last minute.
The DV idea is one faculty are fond of, as it lets them go home early. I've just started looking at Saturday courses and they seem to *not* be DV classes - our low level transfer ones all fill and overflow. Hard to say what this means, since we've offered so few of them, but I'd like to see a graduated expansion to figure it out..
<< Home
