Sunday, April 22, 2012
Placing Thousands of Students Quickly
It isn’t as straightforward as it sounds.
Most community colleges require entering students to take standardized placement tests in math and English. If the student scores below a certain cutoff, s/he is shunted into developmental courses. Depending on how far below the cutoff she scores, she may be shunted into two or three semesters’ worth of developmental coursework, nearly doubling the anticipated time to graduation. (Of course, graduation rates of students who start out at the lowest levels of developmental coursework are far lower than students who don’t.)
In many cases, the students are referred directly to testing upon initial admission, with no opportunity to review for the test. My college, like many, uses a test mandated by the state, so we don’t have the option of changing it or disregarding it.
It’s frustrating. Last year, when we started looking at restructuring (shortening) the developmental math sequence, one math professor here looked at student performance in existing developmental classes and compared it with placement test results. He found no correlation. In other words, the test scores offered absolutely no predictive value.
Yet we’re still required to use them.
It’s easy to condemn placement tests. They carry all of the flaws of any high-stakes standardized test, and they don’t even help in the aggregate.
But condemning the tests doesn’t solve the underlying problem. When you have thousands of new students showing up in a compressed timeframe, ranging in age from fresh out of high school to retirement, and you need to place them all quickly, what do you do?
Small, selective places have the option of doing granular reviews of high school grades, and/or of simply turning away students who aren’t prepared to jump right in to college level math. That’s fine for them, but it doesn’t work for a larger, open-admissions setting. We don’t have the staffing to do that, and even if we did, it’s not clear that it would make sense for older students. (I last took advanced math in the 1980’s. Drop an exam from that class in front of me now, cold, and I wouldn’t have a clue what to do.)
Alternately, we could allow students to select their own classes. (There are times when I lean this way myself.) The danger there, though, is that students will badly overestimate their own abilities and quickly wash out of college-level classes. In the meantime, though, they will have taken seats that could have gone to students who might have succeeded. The libertarian ideal of “let them fail” falsely assumes that the cost of failure accrues only to the student; unfortunately, the student who took up a seat deprived another student of that seat. Given a scarcity of seats, we have a responsibility to allocate them as wisely as we can. (One could also argue that “let them fail” represents a waste of financial aid, which is largely tax-funded.)
There’s also the annoying political reality that “let them fail” would lead, in the short term, to even higher attrition rates. In an era in which attrition is assumed to be the college’s fault, that would amount to institutional suicide.
In the short term, the easiest and most prudent approach is probably the small-bore solution of finding a test that actually tells you something, preferably with students getting an opportunity to review ahead of time. The more radical solutions of embedded remediation or just letting them fail would either take years to develop, as in the former, or require a political sea change, as in the latter.
Is there a better way? Wise and worldly readers, have you seen (or come up with) a reasonably fast and efficient way to place thousands of students at the right level in a short time?
I'm glad to see you have done some internal data analysis, which was to be my suggestion, so I have to ask if your state based its "cut" scores on research like that or just made them up. A similar question applies to students passing the next class when (A) placed into it or (B) passing the developmental class. That would be where HS transcripts might come into play if you have them electronically so they can be part of your data analysis. It would be great if you could believe the grades based on a standard "Regents"-lite type of final exam.
I say this because: The only plural of anecdote I can offer is that the actual name of a class appears to mean nothing in my state. I've advised students who were currently taking (and, reportedly passing) a class called "pre-calc" in HS (meaning they took the college placement test about the same time they took the final exam in June) and failed the algebra placement. When I write out a simple 3x+b=c problem and ask to solve for x, they can't do it.
That means to me that our test was correct. Is it possible this student has completely forgotten everything in just a week? Yes, I think it could be. And I think that could be why the exams are ineffective. There is, of course, another explanation, which might correlate with NCLB results for that school. One way to investigate the cause is to give the placement test much earlier, during the school year. You might try that if you draw a lot of your students from a few high schools.
I'm aware of the Quantway approach, which blends a semester of developmental work with a semester of college-level math into a 2-semester sequence for students not going on to trig or calculus. What I know about its results is too early to evaluate, but this is an area where 50% would look good!
I'd like to see each question on the traditional, computer-scored, multiple-choice placement exam paired with a second question that asks:
A math class that taught me how to solve problems like this would be:
A. Much too easy.
B. A little bit too easy.
C. The right level for me.
D. A little bit too hard.
E. Much too hard
That way you could compare what the students know with what they think they know.
There's some research out of the UofOregon that shows that placement tests over-predict male's performance and under-predict the performance of females.
Could you write a nice linear equation where each student's Math class grades, SAT, placement test grades, and gender were dropped into the equation and poof! out comes a 'placement score' or is even that too labor intensive?
I could imagine that the admissions people could help by adding a couple of lines to a file, then the placement people add in the placement test score, and have the equation built into the database?
If you can make it advisory - come up with your own internal metric that does a better job (hey, it sounds like ANYTHING would be better) and use that for actual placement. If that knots some panties at the state capitol then use the internal metric only when people score below some threshold on the state-mandated test. (though, if the test shows no correlation to outcome, even this would be arbitrary)
I think a chance to review for students who have been out of high school a few years would be good.
If a student gets an opportunity to review a day or more before the test and doesn't, but then goes on to score at a developmental level, I consider that a valid placement. I think such a student won't devote the time and effort needed to do well in a Freshman level class, so the placement in developmental is valid.
BTW, the real problem on the math side is that they can't use a calculator on the placement test. This is strange, because at my college they CAN use a calculator on all of the K-12 tests in my state and for all college math classes until they get to certain calculus classes.
Dean Dad, could you clarify what your prof analyzed? How do you evaluate the placement test if all students who fail it are forced into developmental classes? Was it a comparison of fail rates among those who passed and those who took a class?
I suppose one could look at pass rates in whatever class students place into. In a roomful of students taking the same course (whether a remedial course or a more standard college-level course), some will have higher placement test scores than others. (Unless you have so many placement levels that the scores in every class are in a very narrow band.) See if those scores predict pass rates, grades, or whatever within the class that they are placed into. If the test is useless for distinguishing the A students from the F students in, say, pre-calc, and they're also useless for distinguishing the A students from the F students in Calc I, it is plausible that they are useless for deciding who needs pre-calc before calc.
My problem with the entire process is that we all know the cutoff is not a sharp one for predicting calc I based on pre-calc grades, so I would imagine it cannot be sharp for a placement test. That is why I like the idea of a formula with some other factor (like HS grades) for students on or near the borderline.
a) true facility with fractions, particularly addition and subtraction, and
b) true facility with negative numbers, particularly addition and subtraction.
A quick test that focused exclusively on these two skill regions would almost certainly give you far more predictive power regarding which students really belong in remedial classes versus which students are likely to succeed in introductory algebra.
Not that any test could be worse than what you're using now, evidently.