Thursday, January 23, 2014

Thoughts on the Experimental Site Authority Concept Paper

Sometimes, it’s worth reading the whole thing.  As they say on the Supreme Court: concur in part, dissent in part.

A consortium of seventeen colleges and universities has submitted a concept paper to the Department of Education, petitioning for “experimental site authority” for their campuses to keep financial aid eligibility while moving to competency-based education.  (Hat-tip to Amy Laitinen, from the New America Foundation, for calling attention to it on Twitter.)  

Yes, the buzzwords were strong in that one.  Essentially, the colleges are asking for special permission to waive certain rules and regulations that normally govern financial aid eligibility.  As written, the rules are largely about time: hours per week in class, semesters or quarters with quotas of credits earned in each (“satisfactory academic progress”), and the like.  The colleges would rather move to “direct assessment” of student learning.  The idea is to measure what they’ve learned, rather than how long it took them to learn it.  Doing that requires new definitions of, or approaches to, “satisfactory academic progress,” continuous attendance, and so forth.

The idea behind moving to competency-based measurements makes sense.  On an intuitive level, measuring learning is more to the point than measuring time spent trying.  If you’re a quick study, it’s hard to justify making you sit and wait for others to catch up.  And if we’re ever going to get a serious handle on budgets, we’re going to have to get Baumol’s Cost Disease under control.  By definition, that cannot happen as long as we measure our product in units of time.  Competency-based measures allow the possibility of finally achieving actual productivity gains, using the term ‘productivity’ in the Econ 101 sense.

All of that said, though, the concept paper is well worth a close read for the odd little asides and compromises it includes.  It’s claiming both more and less than it should.  Some highlights:

- The report correctly notes that while colleges now are theoretically allowed to use “direct assessment,” many of the regulations in Title IV are written in ways that prevent it for all practical purposes.  Moreover, colleges are required to be either entirely time-based or entirely competency-based; they can’t mix and match.  In practical terms, that means that a college that wanted to try a competency-based approach would have to redo every single back-office function before starting.  (This is probably why SNHU had to establish an entirely separate division, College for America, before making the leap.)  That’s a hell of a commitment for an institution with students currently enrolled.  The report asks for the rules to be rewritten to allow colleges to try a competency-based approach in individual programs, rather than across the board.  This strikes me as an excellent idea.  Get some proof of concept before trying to renegotiate faculty union contracts, for example.

- Since it’s difficult to measure “satisfactory academic progress” in a given semester if semesters don’t exist, the report offers alternatives.  The first, which seems reasonable to me, is to look at time in the context of overall degree completion, and to pro-rate the number of competencies (with some wiggle room) so that a student needs to be on pace to finish within 150 percent of normative degree time.  The second is radical, and I can’t decide if it’s brilliant or preposterous.  It’s to pay out financial aid only after competencies have been demonstrated.  The idea there is that if aid is disbursed only when students are actually learning something, then the amount of time it takes them to learn it becomes irrelevant.

Yes and no.  Yes, it solves the SAP problem with admirable elegance.  But no, it doesn’t solve the cost problem.  If aid isn’t disbursed until after the fact, how are the upfront costs covered?  Colleges incur costs from day one, and asking them to “eat the cost” of that first semester-equivalent isn’t terribly realistic.  Many colleges are running close to the bone as it is.

The “learn now, pay later” model is also analytically distinct from a competency-based approach.  In theory, we could apply “learn now, pay later” to traditional semesters.  But we don’t, and there’s an obvious reason for that.  That obvious reason would still hold under the alternative approach.  

The paper further suggests allowing Title IV student aid only for “direct costs,” such as tuition and books, in the name of reducing student borrowing.  (Current rules allow for some estimation of living expenses, as well.)  The merits of that suggestion are arguable -- I’d argue that moving living expenses off-books doesn’t make them go away, and would instead just drive students to private lenders -- but more importantly, it’s separable from the competency-based vision.  The only reason I can imagine that it’s included is to run up the score.

Of course, “direct costs” are harder to figure when each student moves at a different pace.  If a college charges by the competency -- similar to charging by the credit, now -- then there’s a question to be raised about students who show up already testing out of some.  Do you charge them for testing out?  (To be fair, CLEP exam fees aren’t covered by financial aid now.)  I prefer the subscription model that CfA and some others use, in which you pay x dollars for y amount of time, and use an “all you can eat” approach during that time.  That gets around the “where did you learn that?” problem cleanly, although it reintroduces time-based measures through the back door.  

The weirdest moment in the report came on page 20.  I can’t really do it justice in paraphrase, so here it is in all its glory:

Federal aid policies discourage students in competency-based degree programs from taking courses offered in the traditional credit-hour format.  Some students in competency-based programs might do better in traditional credit-based courses in certain subject areas.  For example, students who struggle with mathematics might thrive in credit-based courses in which there is significantly more direct contact with instructors.  Allowing students to choose the instructional or learning modality that best suits their learning styles could reduce the amount of repeated course-taking and also shorten time to degree and save money. (emphasis added)

Wow.  That one contains multitudes.

From a community college perspective, my first thought is that students who struggle with math aren’t the exception.  They’re the majority.  If a competency-based approach disadvantages those students, then it’s inappropriate for us.

Why would it disadvantage them?  Apparently, a competency-based approach features “significantly” less direct student contact with instructors.

I’m not sure why that has to be true.  But if it does, it’s pretty damning.

The assertion flies in the face of what I’ve seen on the ground.  This semester we’re rolling out our new self-paced developmental math course, which features both faculty and on-site tutors to help students as they confront new (or persistent) hurdles.  We’re hoping to recoup the extra cost through improved student completion over time.  It may or may not work, but it’s worth trying.  (We’re working with developmental because the “transfer” issue is off the table at that level.)  It’s not a pure competency-based approach, since the course is still scheduled into semesters, but a student could conceivably get through two or three semesters’ worth of material in one.  

The report mentions in passing some dramatic changes to the faculty role.  On page 11, it notes matter-of-factly that “[s]ome institutions separate subject matter-expert faculty who design programs and assessments from student-mentor faculty, who serve as the primary contacts with students.  In addition, some programs have additional student supports and faculty who solely handle grading and assessments.”  

The faculty role gets unbundled, with a production model that winds up looking very much like my graduate program did back in the 90’s. And that model didn’t require a competency-based approach.  In this model, t.a.’s have a different name, but are otherwise recognizable.  In the new model, most faculty would regress (or be regressed) to the t.a. duties they performed in grad school.  

I understand the usefulness of presenting a concept in its purest form; you can always take less than you’re granted, but you can’t take more.  Might as well ask for it all upfront, and make the necessary compromises later.  I get that.  And if that’s what this is -- an opening salvo -- then many of my concerns are moot.

But it also raises hackles that don’t necessarily need to be raised.  A competency-based approach could conceivably work in a number of different ways.  

The paper is also quiet on some of the concerns that led to a new focus on seat time over the last ten years or so.  In the 2000’s, a number of for-profits took some pretty implausible liberties with seat time and credits in order to maximize student loan income while minimizing labor costs.  Requiring a minimum number of hours of seat time may be asinine in certain ways, but it’s also, at least in part, a response to some very real abuses.  It’s not clear from this paper what would prevent a recrudescence of those abuses.  

I offer these thoughts as a sympathetic critic, as one who wants the idea to work.  It will work best if it’s less theoretically stringent and more cognizant of facts on the ground.  I hope the experimental site authority gets approved, precisely so the idea could be refined through contact with those facts.