Last weekend, fed up with being the only card-carrying Gen X’er in America who hadn’t seen The Princess Bride, I watched it with The Girl. We were both charmed by it, predictably enough, and I finally found the source of a number of lines I’ve heard repeatedly over the years. (“Inconceivable!”) My favorite, from Inigo Montoya, only came once: “You keep using that word. I do not think it means what you think it means.”
I was reminded of that line in starting Jeff Selingo’s new book, College (Un)bound. I haven’t finished it yet, so I’ll save a full review for a later post. But Selingo’s opening anecdote includes a common, and fundamental, misuse of a statistic that leads well-meaning people to faulty conclusions.
From page viii of the Introduction:
What Dietz failed to examine was Fairleigh Dickinson’s graduation rate. In 2006, only 38 percent of its students graduated within six years, a rate well below all the other schools she had considered...Though Fairleigh Dickinson was giving Dietz a boatload of money, her chances of emerging at the other end with a degree were pretty dismal.
Smart people keep using the graduation rate that way. I do not think it means what they think it means. And I won’t even go into differing goals, part-time attendance, or the well-known flaws of the IPEDS definition of a graduation rate. Let’s pretend those don’t matter. There’s an even more basic flaw at the heart of the argument.
A college’s graduation rate is not evenly distributed among its students.
For example, on most campuses, gender is a consistently strong predictor of success: female students outperform male students in every racial category, and sometimes across categories. (On my own campus, in any given semester, Latina women and white men are neck-and-neck for pass rates in developmental math.) Students who arrive without any developmental placements graduate at notably higher rates than do students who need remediation. Students who take Honors courses graduate at higher rates than students who don’t. Students in tight cohort programs graduate at higher rates than students in larger programs with less regimentation. Students either right out of high school, or over thirty, do better than students in their early twenties. (That one is a head-scratcher for me, but there it is.)
If we wanted to improve our graduation rates, and didn’t want to change how we do business internally, we could just tweak our outreach and admissions to skew our population to more 18 year old white women who score “college ready” on placement. They do amazingly well here. And we’d tell 25 year old men who need remediation that gee, we seem to have lost their paperwork.
We don’t, since that would fly in the face of the open-door mission. But selective colleges do that all the time, by design. They actually brag about it.
A college’s graduation rate reflects, in considerable part, the profile of its student body. The Harvards of the word screen out anybody who looks high-risk, and they get results consistent with that.
Last week I took issue with a piece by Michael Petrilli in which he claimed that colleges could save a ton of money -- Selingo’s “boatload” -- by just excluding students who weren’t likely to succeed. I argued that we don’t have the epistemological foundation to do that. In other words, we don’t know in advance who will make it and who won’t. We have probabilities, but we can’t move with confidence from probabilities to individuals. Therefore, the most ethical approach is to treat everyone as potentially successful, and then let them tell us with their own performance what they’ve got.
So in saying that the composition of a student body will affect its graduation rate in predictable ways, am I contradicting myself?
Nope. The key is in recognizing the difference between aggregate probabilities and individual cases. It’s pretty clear in the national research, for example, that young men of color graduate at lower rates than do upper-income young white women. But that tells you nothing about what any given student will do. If an upper-income young white women attends a college with a more diverse student population than her other options, its graduation rate may well be lower, but her chances should be unaffected. The college’s overall graduation rate does not equal her personal chances of graduating.
In saying that, I’m not at all dismissing the obligation to do a better job by all students. That’s a given, and one that I and my colleagues focus on relentlessly within the budgets we have. And yes, a graduation rate well above or below what local demographics would lead you to predict can reveal something. But when you look at state-by-state comparisons of community college graduation rates, it’s hard not to notice that states in which students have relatively few options tend to have much higher rates than states with robust private college sectors. That’s because in states with few options, more of the high achievers attend the community colleges. That goes beyond any given campus, or even any given state.
Disclosing graduation rates to the public may serve some sort of purpose, if we can get the usual IPEDS flaws under control. (The current measure would do no good at all.) But to assume that a graduation rate equates to a given student’s chance of success is simply false. It does not mean what many people think it means. And if we act as if it does, we will systematically punish the colleges that reach out to the people who need college the most.
