Monday, December 07, 2015


That’s Not How It Looks from Here

On Monday, USA Today ran a piece about Marco Rubio’s desire to “change accreditation rules to let more vocational schools and online universities take advantage of the roughly $130 billion a year in federal loans and grants -- but only if they meet certain benchmarks tied to student outcomes and debt repayment.”  (The article commits the common journalistic sin of conflating “online” with “for-profit,” but that’s another issue.)  The article goes on to cite Amy Laitinen, from New America, saying that moving accreditation from counting books in libraries to looking at student outcomes would be a welcome change.

And I thought, hmm.  That’s not at all the impression I got at Middle States.

If anything, the discussion at Middle States was split between assuring compliance with Federal mandates -- as opposed to their own -- and documenting efforts and continuous institutional self-improvement.  In fact, some of the panels were specifically about ways to reduce the strain of continued process improvement.  The single most interesting concurrent panel I attended, which was on Friday morning, was titled “Making Outcomes Assessment Sustainable.”  The presentation, mostly by Bret Bennington from Hofstra, took as given the idea that we’re already putting in tremendous amounts of work on outcomes assessment.  (Bennington’s solution came uncomfortably close to collapsing assessment into grading, but that’s another issue.)

If it were primarily about counting books in the library or the number of Ph.D.’s on the faculty, it wouldn’t be so hard.  Counting books is easy.  Counting degrees is easy.  Counting student learning is much, much harder.  But that’s what we’re charged with doing, and have been for some time.

It’s not just a Middle States thing, either.  The same was true with NEASC, the New England association.  

It’s almost as if the policy folk and the folks on campuses are talking right past each other.  The wonks assume that accreditation is still all about inputs, as if it were still the 1990’s.  Meanwhile, accreditors are pushing outcomes assessment so hard that institutions are struggling to keep up.  What’s the disconnect?

I’m guessing it’s a combination of factors.  One is simple distance: unless you’re actually in the process, it’s easy not to see what’s going on.  If Senator Rubio or the folks from New America would like to drop by to see our assessment protocols, I’d be happy to show them.  The protocols are about outcomes, not inputs, and they’re far more nuanced that such critiques suggest.

Second, I suspect, is that we’re using the same words to mean different things.  When academics talk about student outcomes, we usually refer to some sort of demonstrated learning.  That could mean looking at “artifacts” of student work, or juried performances, or presentations.  The idea is to identify the areas in which students are falling short of the desired outcomes, and then changing something in the curriculum or delivery to improve it.  

But in the policy world, when they say “outcomes,” they tend to mean degrees and salaries.  Learning is assumed, much as in the input-based model.  Or it’s simply dismissed as irrelevant.

Finally, in some cases, there’s another agenda entirely at hand.  The article notes that Senator Rubio made special pleas for “leniency” for Corinthian Colleges as they came under scrutiny, and notes too that Rubio wants to expand the reach of for-profit higher ed.  Given the for-profit sector’s track record so far -- its outcomes, if you prefer -- that could be a tough sell.  But he’s not looking at that.  His goal is based in an ideological position that holds “market good, state bad,” regardless of outcomes.  In that case, replying with a nuanced reading of the success we’ve had with students in the ALP would miss the point.  In this context, “Outcomes” aren’t really outcomes.  They’re an excuse for another agenda altogether, which needs to be responded to accordingly.

As longtime readers know, my view of for-profits is more agnostic than most; I’m willing to entertain the idea that they can do certain things quite well, having seen it done. I’m not theologically opposed, even though many of my colleagues are.  But if we want to make an argument from outcomes, we have to look at the ones that actually exist.  I don’t know how many books we have in the library, off the top of my head, but I know what we’re doing to assure that students are learning.  That should be the point.  Increasing scrutiny on us while begging leniency for Corinthian doesn’t look like outcomes assessment from here.

I'd assume that the Senator only knows what someone told him, although those remarks do raise the question of whether he did proper outcomes assessment for the undergrad classes he taught at FIU and participated in the continuous improvement process. Anyone know where FIU is in the SACS reaffirmation cycle? That said, he has always pushed vouchers so children could go to unassessed private schools rather than heavily assessed and tested public schools, so those views are no surprise. Read his bio.

On your main points, I would love to see a series of essays on those sessions you attended. It sounds like Middle States (and NEASC?) have been doing this longer than my region, where we didn't start doing outcomes assessment until the midway point of our previous cycle. We came through that without any trouble at all, but the extra work is really hard on our younger faculty and any who took it seriously enough to do more than the most minimal assessment. Specifically:

1) "some of the panels were specifically about ways to reduce the strain of continued process improvement" (How long has Middle States been at it, and how much of that strain is on IR and how much is on the faculty?)

2) " “Making Outcomes Assessment Sustainable.” The presentation ... took as given the idea that we’re already putting in tremendous amounts of work on outcomes assessment." (Is that the case, in your opinion, and how does that effort change as you go from year 0 to year 5 to year 10 and 15? Since the full reaffirmation cycle is 10 years, some colleges will be in year 10 when others are just starting, so your mileage may vary a lot across the region.)

3) "Bennington’s solution came uncomfortably close to collapsing assessment into grading, but that’s another issue." (That would be the issue I'm most interested in, because to be sustained and sustainable, it has to become as regularized as regular grading but more standardized in the rubric that gets used.)

IMHO, as valuable as I have found outcomes assessment to be as regards gaining insight into what my students have learned, it is almost meaningless if done during or at the end of the current semester. They haven't learned anything unless they can do it next year (or after transfer from a CC in our case or after taking a job for AS degrees and 4-year degrees), but we have absolutely no mechanisms for evaluating anything after they leave our class.
+1 on the need for more sustainable program-level outcomes assessment. We are under pressure from NEASC to show 100% program compliance in their next visit. The main faculty complaint I hear is about this being the nth request for more to be done, without less of anything else being done.
Alas, ate up my closing remark (shame on me for not previewing, and don't tell my students ...)

(end of stereotypical faculty rant)
Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?