Tuesday, December 13, 2005

 

How Do You Know a Good College When You See One?

“There are market tests for institutional performance.” – Stephen Karlson, Cold Spring Shops

Well, yes and no.

How do you measure how well a college is doing its job?

I read two thoughtful pieces today that directly conflicted on this question. One was the entry over at Cold Spring Shops, quoted above. It was a piece on the long-term need for colleges to maintain high standards so employers will continue to value the degrees they award. The other was Jim Collins’ recent article/monograph (it’s an article, but he sells it as a monograph – who says business writers aren’t sharp?), “Good to Great and the Social Sector.” There, Collins suggests that the standard measure of institutional success for businesses – money – doesn’t work for nonprofits, since that isn’t their mission. Collins instead settles on reputation as a sign of success, which strikes me as shaky. Reputation is, at best, a trailing indicator.

Neither quite seems to get it right, but I don’t have a well-thought-out alternative at hand, either.

The academic prestige hierarchy (a version of Collins’ ‘reputation’) is almost purely based on inputs, rather than outputs. Pour lots of research money and some high-SAT undergrads into a university, add a good football team, and voila! Selectivity of admissions correlates very strongly with academic prestige, regardless of what the teachers in the classrooms actually do. As a former professor of mine put it, “we don’t often turn a silk purse into a sow’s ear.” Give me a college full of valedictorians, and I’ll give you a great job placement record, even if the undergrads spend all four years majoring in beer. I’d wager any sum you please that the best community college in America has a higher attrition rate than the worst Ivy.

(Actually, that suggests a good exercise: Quick, name the five best community colleges in America! If you have no idea, you see the problem.)

Employers’ reactions – what I take to be CSS’s preferred measure – are also fickle. I saw that directly at my previous school. During the late-90's boom, we couldn’t produce graduates fast enough; one of the major drivers of attrition then was employers poaching students before they graduated! When the bubble burst, so did our placement record. The faculty did not meaningfully change in two years, but the market did.

More to the point, employers’ reactions are often based on watered-down versions of the academic prestige hierarchy anyway, laced with personal ties. It would be lovely to assume that employers keep close tabs on the outcomes assessments at the colleges and universities and hired accordingly, but most don’t. Most employers don’t have access to that kind of information (keep in mind, most employers are small, distracted, and busy). Yes, a few elite technical programs in very specific fields may command specific respect based on quality, but those are very much in the minority. Outside of that rarified stratum, one degree largely resembles another. (If that were not the case, the University of Phoenix could not have grown as quickly as it has.)

So, how do you know if a given college is ‘good’?

I have my own biases, but that’s all they are. I prefer schools that know their own mission to those that don’t; community colleges, potted Ivies, and R1's, as different as they are, at least know their missions. The Midtier States of the world often don’t. I prefer schools that don’t limit themselves demographically, but I freely concede that, say, Wellesley probably offers a better history major than Midtier State.

Others have their own biases: some swear by women’s colleges, others use football prowess as an indicator. Some prefer all-undergrad campuses, others use graduate program prominence as a proxy indicator for the quality of undergraduate teaching (a colossal mistake, imho).

I’ve been thinking about this lately as I’ve struggled with ways to improve the reputation of my own cc. It’s pretty well-respected in its community already, which is great, but there’s always room to do better. The problem is that I honestly don’t know where reputations come from.

Vox Blogosphere, Vox Dei: how do you know a good college when you see one?

Comments:
As you pointed out, there are a varied number of indicators. Many students and their families chose primarily by cost and, in the case of colleges like yours and mine (regional state university), they are usually pleasantly surprised and only later realize the actual teaching and personal interest given to them as individuals. They weren't merely something that kept someone from researching and/or were not primarily taught by someone other than the "big names" touted by R1s.

The reputation of our particular program is well-received in the field although students toward the end of the program (as they enter more "professional practice" related classes) complain that we are expecting way too much. Only later when they themselves graduate do they thank us for preparing them for the real world. And since those evaluations come after the university's official student measurement instrument, the numbers don't correlate with any of this.

There's also a fine line between providing someone with an education and vocational training. Both have value but serious discussions need to take place and revisited to keep the official goals and outcomes as well as the students in mind.
 
couple of candidates:
Foothill College
Santa Rosa Junior College
 
What a GREAT question. I wish I knew that answer, but I think you're right in saying that colleges that know their mission are likely to do it better. I disagree, though, that mid-tier state universities are less likely to know their missions.

If you take out competition between schools, and look at a single school, how do you know it's a good one? Does assessment tell you that?

If so, what kinds of assessment "tools" actually give you useful information? Do students understand portfolios well enough at most schools to make them really worthwhile?

Kelly above makes the point that only some time after graduation do most of her students realize how well they've been prepared. That suggests that student evaluations and exit interviews aren't necessarily useful.

But would a five or ten year after survey be useful? Would people who weren't well-served be surveyed? (If someone falls off the radar, they aren't going to get the survey maybe, or aren't going to take time to respond? If someone's been especially successful, they may attribute that success to the school instead of to some other factor.)

Dean Dad's right that even simple graduation rates can be misleading, too.

How would I tell?

*Mission statement
*Visit campus and look at how well the facilities (library, layout, labs, whatever) seem likely to serve the mission.
*Unofficially talk to people lower on the "food chain" about the school. (If a junior librarian or math adjunct has a sense of the school's purpose and how it's getting there, then the sense of purpose is pervasive.)
*Look in the dorm areas on Friday and Monday morning (if there are dorms).

Well, that would be a start, anyway...
 
I wouldn't expect many people to name the five best community colleges in the United States, because community colleges don't really operate on a national level. I certainly can name the five best community college districts in my own state, where there are over fifty. Well, perhaps it would be more accurate to say "I have an opinion on which five are the best" because, as you point out, there's no objective measurement of "best".

I think the problem is that there may be a different "best" college or university for each student. As you point out, the measurement of "best" colleges often has more to do with inflow than outflow, but many times it seems that the student's experience has more to do with his or her effort, enthusiasm and level of preparedness than with the quality of instruction.

Signed,
I'm accounting as fast as I can!
 
I suspect a lot of people will say: "Look at the quality of the graduates." Which, of course, may reflect (as DD already noted) admissions policies more than institutional quality.

Another lot of people will say "Look at the quality of the faculty." Defined how? In research terms? Then a fair number of quite good liberal arts colleges, state university regional campuses, and community colleges will get hit.

"Quality" is, unfortunately, a very slippery concept, even in the production of goods and services (quick: Which is a higher quality computer, a Mac or a Dell? Why?). It may be like Justice Powell's definition of pornography.

The various magazine (US NEws, BusinessWeek, etc.) rankings do add some value, believe it or not, because they synthesize a fair amount of information that many people don't have access to. BusinessWeek is especially good on variabtions between higher-tier business schools.

And reputation is a lagging indicator, but still has some value. For mid-tier liberal arts colleges, for regional campuses of state universities, and for community colleges, reputation is more local than regional or national. Quick, what's the best small private liberal arts college in Wisconsin?

What, you expect me to know?
 
For what it's worth, the dogma here in Canada is that you can receive a very good education at any of the universities.

I recognise that this is dogma, tied to our universalist public university education, multiculturalist, and social(ist?) policies. I recognise that it has limitations in appliction... whihc is why students will move from one university to another, or drop out (but they may do those things for reasons not related directly to the education).

at the same time that I recognise the dogma, I want to point out that there is value in asking the question a little differently. Thus, we are encouraged as students seeking a univeristy to attend not to ask which university is "best", but which one is best suited to my interests and needs.

Even the money-raking special issue of the MACLEAN'S UNIVERSITIES magazine every year (an issue aimed at baby boomers witha consumer mentality and looking for bang-for-buck education) points out that students have to know themselves well in order to know which school will serve them best.

Yes, the rest of the world is only aware of Queen's, Toronto and McGill... but, in fact, the country has many, many excellent schools that know their missions and serve their populations well. Check out UNBC -- geared toward providing university education to Northern and/or Native populations... opened in the early 90's -- already granted fine recognition for its programme delivery.

Check out Lakehead, reknowned for its forestry programme -- very useful in Northern Ontario.

Nippising provides a Teacher's College in North-Central Ontario -- cricual for the region.

These are schools that few outside Canada would know of, and certainly you don't send your kids there to become astro-physicists, but these schools don't claim to be suited to such a purpose.

One caution... I learned recently that although there are universities in Canada that are designed to provide Christian-based schooling, few are accredited, and students can find that their Bachelors' degrees from such plaes are not recognized.
 
One test of an educational institution's quality is the success of its graduates. The test of whether the graduates credit their educational experience for their success is whether they donate to the school after graduation. Not merely the presence of some celebrity alumni and some big donors, but the routine participation rate among all alumni in fundraising drives.
 
There is actually one book that takes on (part) of this question, _Colleges That Change Lives_ (can't remember the author). He surveys about 40 liberal arts colleges--not the most selective ones--and points out some measures that show these schools seem to *add* something--they're not just good because the entering students were good. These colleges produced more PhDs per capita, for instance, than large universities whose entering students had similar SATs, etc. My favorite fact--Cornell College in Iowa, a tiny liberal arts school, had a higher percentage of alumns in Whos Who than Cornell U. All of which is to say it may be tough to measure what a college adds, but it shouldn't be impossible.
 
I'm a little perplexed about the attention to mission statements. I haven't read any that were specific enough to be meaningful. It's just "we're committed to excellence in teaching, research, and outreach" type of boilerplate. But maybe I'm not that well trained to translate it.
 
Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?