Tuesday, May 14, 2013


Inigo Montoya and Graduation Rates

Last weekend, fed up with being the only card-carrying Gen X’er in America who hadn’t seen The Princess Bride, I watched it with The Girl.  We were both charmed by it, predictably enough, and I finally found the source of a number of lines I’ve heard repeatedly over the years.  (“Inconceivable!”)  My favorite, from Inigo Montoya, only came once: “You keep using that word.  I do not think it means what you think it means.”

I was reminded of that line in starting Jeff Selingo’s new book, College (Un)bound.  I haven’t finished it yet, so I’ll save a full review for a later post.  But Selingo’s opening anecdote includes a common, and fundamental, misuse of a statistic that leads well-meaning people to faulty conclusions.

From page viii of the Introduction:

What Dietz failed to examine was Fairleigh Dickinson’s graduation rate.  In 2006, only 38 percent of its students graduated within six years, a rate well below all the other schools she had considered...Though Fairleigh Dickinson was giving Dietz a boatload of money, her chances of emerging at the other end with a degree were pretty dismal.

Smart people keep using the graduation rate that way.  I do not think it means what they think it means.  And I won’t even go into differing goals, part-time attendance, or the well-known flaws of the IPEDS definition of a graduation rate.  Let’s pretend those don’t matter.  There’s an even more basic flaw at the heart of the argument.

A college’s graduation rate is not evenly distributed among its students.  

For example, on most campuses, gender is a consistently strong predictor of success: female students outperform male students in every racial category, and sometimes across categories.  (On my own campus, in any given semester, Latina women and white men are neck-and-neck for pass rates in developmental math.)  Students who arrive without any developmental placements graduate at notably higher rates than do students who need remediation.  Students who take Honors courses graduate at higher rates than students who don’t.  Students in tight cohort programs graduate at higher rates than students in larger programs with less regimentation.  Students either right out of high school, or over thirty, do better than students in their early twenties.  (That one is a head-scratcher for me, but there it is.)  

If we wanted to improve our graduation rates, and didn’t want to change how we do business internally, we could just tweak our outreach and admissions to skew our population to more 18 year old white women who score “college ready” on placement.  They do amazingly well here.  And we’d tell 25 year old men who need remediation that gee, we seem to have lost their paperwork.  

We don’t, since that would fly in the face of the open-door mission.  But selective colleges do that all the time, by design.  They actually brag about it.

A college’s graduation rate reflects, in considerable part, the profile of its student body.  The Harvards of the word screen out anybody who looks high-risk, and they get results consistent with that.  

Last week I took issue with a piece by Michael Petrilli in which he claimed that colleges could save a ton of money -- Selingo’s “boatload” -- by just excluding students who weren’t likely to succeed.  I argued that we don’t have the epistemological foundation to do that.  In other words, we don’t know in advance who will make it and who won’t.  We have probabilities, but we can’t move with confidence from probabilities to individuals.  Therefore, the most ethical approach is to treat everyone as potentially successful, and then let them tell us with their own performance what they’ve got.

So in saying that the composition of a student body will affect its graduation rate in predictable ways, am I contradicting myself?

Nope.  The key is in recognizing the difference between aggregate probabilities and individual cases.  It’s pretty clear in the national research, for example, that young men of color graduate at lower rates than do upper-income young white women.  But that tells you nothing about what any given student will do.  If an upper-income young white women attends a college with a more diverse student population than her other options, its graduation rate may well be lower, but her chances should be unaffected.  The college’s overall graduation rate does not equal her personal chances of graduating.  

In saying that, I’m not at all dismissing the obligation to do a better job by all students.  That’s a given, and one that I and my colleagues focus on relentlessly within the budgets we have.  And yes, a graduation rate well above or below what local demographics would lead you to predict can reveal something.  But when you look at state-by-state comparisons of community college graduation rates, it’s hard not to notice that states in which students have relatively few options tend to have much higher rates than states with robust private college sectors.  That’s because in states with few options, more of the high achievers attend the community colleges.  That goes beyond any given campus, or even any given state.

Disclosing graduation rates to the public may serve some sort of purpose, if we can get the usual IPEDS flaws under control.  (The current measure would do no good at all.)  But to assume that a graduation rate equates to a given student’s chance of success is simply false.  It does not mean what many people think it means.  And if we act as if it does, we will systematically punish the colleges that reach out to the people who need college the most.

I'd be interested in comparing your random 25 y.o. male to a veteran of the same age. More interesting would be to dis-aggregate the Marines from the rest.

That said, interesting question about mid 20s after controlling for other variables but you can't try some ed innovation if there isn't any theory to test.
Excellent post. Might it make sense to link the low grad rates of 25-year old males with the need for more "stackable" curriculum? Most of these 25-year old men are working, but not earning much, working irregular schedules, and are whipsawed by the 15-week commitment that balloons into a years-long commitment if they are seeking a traditional 2-year degree.
Could there be a factor of "intent" with the early 20's folks? I'd guess (based on anecdata) that most people going back to school in their 30s are more motivated to succeed, both internally and externally - they're more likely to have families to support, and they've had more time to mature. People in their early 20s may be there because they have to do *something* but aren't as invested in a particular outcome, and might coast no matter what they're doing.
On the one hand, yes, of course, obviously the statistical graduation rate isn't the same as an individual's chances of graduating.

On the other hand, I do think graduation rate matters, inasmuch as the amount of slack in the system matters. When stuff goes wrong, if you get sick, or your childcare arrangements fall through, or your car breaks down... all else equal, an instructor that doesn't have a bunch of other students who aren't completing their assignments on time can afford to be more lenient.

This is why, in the K-12 world, I don't buy the "a smart kid will be fine anywhere" line of reasoning. Having to go through metal detectors, being treated as suspect by default, and simply having to be really needy to be the teacher's first priority... all of those things can take a toll.

And that's ignoring any more direct value-added from fellow student impacts.
Sometimes students start college. Life happens, even good things, and they quit because they are over 18 yo no matter what their parents want them to do.

More life happens and at 25 they decide maybe a college degree is a good idea after all. So they finish their degree (having to take some classes again because of the length of time between enrollments). I think that should count as graduating...just longer than 5 years after starting.

It happens...

"If we wanted to improve our graduation rates, and didn’t want to change how we do business internally, we could just tweak our outreach and admissions to skew our population to more 18 year old white women who score “college ready” on placement. They do amazingly well here. And we’d tell 25 year old men who need remediation that gee, we seem to have lost their paperwork."

I believe the appropriate quote is "I have a confession to make: I am not left handed."

I really need to rewatch that movie.
1) "Disclosing graduation rates to the public may serve some sort of purpose"

I hate this kind of attitude. Disclosing graduation rates, and every other piece of data you collect, is your obligation to the public and should always be your default position.

2) I started at my local CC at 23. I had to take two semesters of remedial math. I went part time for a while. I transferred to a Universtiy of where I graduated Magna. It took me 6 1/2 years, so I don't think I'd count as part of the graduation rates for my CC. I can see how that would make an administrator tear his or hear hair out in frustration.

3) Can't believe you'd never seen the Princess Bride. WTF?
Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?