Sunday, September 23, 2018

The Best Facts

The best facts, as a writer, are the ones that contradict a widely-held bit of dogma.  They call into question things that you didn’t know were questionable. (I’ve heard the same said of science.  The breakthrough moments aren’t marked by someone proclaiming “Eureka!” They’re when someone notices a result, raises an eyebrow, and mutters “That’s weird.”)  A recent Washington Post column offered a great fact.

It was about the relative predictive value of standardized test scores, as opposed to grades.  

That narrative usually goes in one of a few ways.  There’s the “standardized tests are evil” line, which is well-worn in academic circles.  It typically points to the low predictive value of test scores for anything other than subsequent test scores.  It also often points to cultural bias in tests, and/or to racial or economic gaps in scores. Counter to that is the “level playing field” line, which holds that for all of the flaws of standardized tests, they at least allow students from schools that aren’t as well known to show what they can do.  More recently, the “multi-factor placement” line has become popular. It holds that some combination of the two is more effective than either by itself. It comes in “inclusive” and “exclusive” flavors. The “inclusive” flavor says that a student who passes any one of several factors should be allowed to bypass remediation; the idea is to encourage more students to take college-level courses upfront, the better to improve graduation rates.  The “exclusive” one is the version used by exclusive colleges, in which they look ten different ways to find flaws.

The article, which draws on a study by Seth Gershenson, suggests that the “inclusive” version of multi-factor placement may fall victim to an insidious flaw: grade inflation in affluent schools.

When I’ve talked about using high school GPA for course placement, as opposed to Accuplacer scores, one of the usual objections I hear is that a “B+” from some districts is not the same as a “B+” from others.  The assumption is typically that the lower-income schools have easier grading curves, so the students towards the top in a lower-income school may not be at the same level as one towards the top of a higher-income school.  

This article and study suggest that it’s the other way around.  Grade inflation is more pronounced at more affluent schools.

That’s actually consistent with studies of grade inflation in college.  Grade inflation may be rampant at Ivies and similar places, but it’s virtually unknown among community colleges.  The reasons may be similar. In both cases, socially-driven senses of entitlement, sometimes combined with real or threatened parental action, can nudge grades upward.  Where those are less common, so is grade inflation.

This may be where the “standardized test as level playing field” argument gains some traction.  They can provide a reality check on the seemingly-stratospheric GPA’s of students from more affluent places.  If one were cynically inclined, one might also wonder if this is part of why some elite prep schools are dropping AP classes.  A reality check can’t help elites; it can only hurt them. Chad’s parents may be able to nudge the B to a B+, but they can’t nudge a 4 to a 5.  What would Legacy Prep gain by taking the risk?

The inclusive version of multi-factor placement assumes that past grades are a reliable guide to future grades.  It makes sense only as long as the two institutions giving grades are playing by the same set of rules. But it seems that some of the more affluent ones may be bending the rules, whether consciously or not.  A reality check may be in order.

I like this fact a lot, because it flips an existing narrative on its head.  The best facts do.