Tuesday, September 30, 2014
A Question for the Statisticians
It’s tempting to just throw up one’s hands and declare the entire enterprise a fool’s errand, but politically, that’s a non-starter. If there will be “accountability,” how should we count?
...but all of this kind of IS a fool's errand. Living and dying by the numbers is silly when the numbers will never capture the full picture.
And how sad it is that college's only purpose is now to get a job. What ever happened to learning for the sake of learning and being a better citizen. John Dewey must be rolling in his grave.
Should we punish an elementary school because its grads don't make much money in 6th grade? A HS because its grads make very little while in college?
An AA degree (and dual-enrollment credits) should be measured by looking at the odds of succeeding at the transfer institution and earnings 5 and 15 years after the final degree is earned. Those metrics (success and income) should also count for/against the transfer school, which should be penalized for the failure of AA transfer students just as it should be for any student who has to transfer to a CC before returning to graduate.
PS to comment above:
IMO, there is a difference between learning for its own sake (a life of the mind) and Dewey's view of educating the mind so everyone can help run the nation. Education still has practical ends, but ones that require more than trade school just as they require more than being merely an "educated" voter who can't hold a job.
@Morgan Price - Agreed, especially your question at the end. Perhaps this is what DD means when he quotes "In theory, there's data".
If different graduate streams make different amounts of money, then separate the streams and show them separately. In other words, show what transfer students ultimately make, what direct graduates/workforce entry make, and what "life happens" people make 2 years out. Some of those numbers won't be glamorous, but then your work changes to explaining why some grads make more than others. If including transfers 2 years out drags your ratings down, while including them 20 years out lifts your ratings, then you either have to accept the distortion (good and bad), or present them as their own dataset.
An alternate way of presenting the data is to present the data as X years into the workforce, rather than X years post-graduation. That would get around your transfer-issue quite cleanly. Is that politically do-able?
When management develops a nearly Pythagorean obsession with numbers, this can lead to some very perverse incentives. Lest they get the Feds or their accreditors angry with them, or if they let their ratings slip, management now needs to make sure that the college or university “meet its numbers”. College and university managers become as narrowly focused on short-term results as corporate executives trying to satisfy analysts’ expectations for the next quarter’s balance sheet. Any faculty member or employee who does something (or fails to do something) that is perceived as preventing the university from meeting its numbers is in danger of being dinged on their next performance review. Will the dean get canned if the college has too low a graduation rate? Will the college president be in danger of being dumped by the trustees if the college slides downward in the ratings? If the gainful employment numbers look bad, will the Feds come down hard? If an assistant professor fails too many students, could they be in danger of failing to get tenure? Will a part-timer who fails too many students get thrown under the bus? Since meeting their numbers is now so important, there is a temptation for management to fake the numbers or make up the results, just to keep them out of danger. The system will encourage everyone to lie to everyone else, all the way up the chain.
Perhaps most important is the danger that a college or university that is too strictly managed by a set of numbers can lose sight of its reason for being there in the first place. The mission on the college or university is now no longer to increase knowledge or to educate the next generation of citizens in a democracy, it is now strictly to meet its numbers.
I think it's fair to limit your income reporting to people who are full-time employed, as long as you're also including a metric that tells your data consumers how big a chunk of your data is being excluded.
You could argue against that of course. People in grad school earn less, but so do people doing internships or taking jobs with significant "dues-paying" periods.
Under the college ratings system being proposed by the Obama administration (as well as possibly upcoming gainful employment regulations), the true purpose of a college education seems now to be for graduating students to be able to obtain good jobs which provide enough income, presumably to ensure that college loans can be repaid.
Now , it has certainly been true that the ability to obtain a good, well-paying job upon graduation has always been an important part of a college education. But until very recently, this was never the most important reason for going to college. More important was the opportunity to obtain a liberal education so that one gained exposure to the most important aspects of our culture so that one could become a more productive and better-informed citizen in a democracy. An important product of a successful college education was the development of intellectual curiosity, along with the ability to learn new things, so that one could make accommodations as the economy and society in general undergo changes. Best to teach a graduate how to learn—a curriculum too narrowly focused on teaching skills that are now in demand on the current job market runs the risk of teaching skills that may well be obsolete in only a couple of years.
But back in the day, a graduate could usually count on getting a decent job upon completing their education, one which would a sufficiently high salary so that they could live a middle-class lifestyle, to perhaps buy a house, start a family, and take a vacation now and then. But things are now quite different—the job market has tightened considerably and a college education is no longer a guarantee of being to live a middle-class lifestyle. Employers are now a lot more fussy, and demand a lot of experience in the particular projects that they are working on before they hire anyone. The want an employee to be able to jump right in, without needing a lot of training.
So the pressure for colleges to train their graduates to be able to survive in the current job market has increased greatly in recent years. It now appears that colleges will be judged on how much money their graduates make immediately upon graduation. But how does one measure this? Does a liberal arts graduate who is now flipping burgers at Wendy’s count the same as a STEM graduate who is now in medical school? How do you compare a collage graduate who immediately obtains a job at a Wall Street hedge fund with a graduate who enrolls in Harvard Medical School—does the first one count favorably in the rankings, whereas the second one counts against the college? Does a community college get punished when too many of their students transfer to four-year colleges? Will a graduate who is taking an unpaid internship count against the college in the rankings system? Will the college get dinged if too many of their graduates enroll in graduate school and who are currently making very little money? A meaningful rankings system based on graduate employment needs to able to look further down the road—more significant would be for a college to look at how well their graduates are doing 10 to 15 years down the road.
As I mentioned earlier, a college rankings or ratings system based on the amount of money that their graduates make could lead to a whole bunch of very perverse incentives. Colleges will be pressured to eliminate programs and majors that graduate a whole bunch of students who do not immediately make a lot of money. There will be pressure to introduce courses and majors designed to teach skills that are now in demand on the current job market. College presidents, deans, and vice presidents will rise or fall based on how well the college does on these gainful employment scores. More seriously, colleges may be tempted to fake the statistics or turn in fake numbers, just to avoid getting into serious trouble.