First, if you've ever wanted a sense of academic hiring, read Profgrrrrl's post. Now. Slowly.
It's all true.
Worse, it doesn't stop at the department level.
Let's say, for the sake of argument, that your dean/vp didn't just fall off the turnip truck. Any chances that s/he might be wise to some of these factors? (Hint: Yes.)
Facts of life like these are why I have this much patience for the argument that the academic hiring market is some sort of meritocracy. It just isn't. That becomes much more true in the disciplines in which it's common to get hundreds of applications per position. After an initial screen for bright-line qualifications, you'll still have dozens of people who are fully qualified, many of whom will have strong letters, academic pedigrees, and experience. That's where things start to get, if not random, then at least situation-specific.
(And that's before even counting recessions. Is hiring down this year because candidates suddenly got worse? Nope.)
I've been on any number of searches in which I've met extraordinary candidates who did everything right and still didn't get hired. Sometimes it comes down to niches. Smith may be a better hitter than Jones, but if Smith is a first baseman and Jones a shortstop, and I'm already set at first base, I'm going with Jones. Substitute teaching specialties for positions, and you get the idea. That's the non-sinister meaning of 'fit.' Departments usually hire because they have holes; the exact shape of the hole is specific to that situation. If this year's hole is different than last year's, then this year's winner will be different.
I've also seen committees try to rig the outcome by putting forward the one person they really want, and two obvious sacrificial lambs. I put a stop to that by threatening to hire one of the lambs. My position is that anyone on the finalist list is, by definition, fair game. That may sound sinister, but I see it as preserving real openness. If the fix is in anyway, why bother running an open search at all? Of course, good luck defending yourself in court when a rejected applicant from a protected class claims discrimination. Although forcing openness may look like administrative meddling, I'd argue that it actually offers the possibility of fairness to all applicants, which can only benefit the college in the long run.
The more difficult case is the committee member who feels threatened in her niche. I've seen a few iterations of this. One is the senior professor who doesn't want to give up a pet course, so he systematically tanks anyone capable of teaching it. Another is the queen bee who simply refuses to hire any women younger than herself. (I know it's an ugly stereotype, but I've seen it in action.) Since no candidate is perfect, it's always possible to find a flaw if you want to badly enough.
Most of these are symptomatic of the vagaries of luck, circumstance, and what Kant called the crooked timber of humanity. My sense is that good admins need to do what they can to preserve real openness of process, and to challenge what seem like arbitrary reasons. But as long as the demand for slots so drastically exceeds the supply, some wonderful people are going to be shut out for what seem like silly reasons. Common decency suggests that we shouldn't add insult to injury by telling those left out that they just weren't good enough.
