Sunday, April 28, 2013


“Fail Fast:” Fumbling Towards a Theory

“Fail Fast” is a mantra among entrepreneurs.  It means that if it becomes clear that a given project isn’t working, the best move is to pull the plug quickly so you don’t lose more time you could have spent on something else.  It’s based on minimizing opportunity cost, and it assumes a certain amount of failure as a feature of the system.

In public higher ed, at least on the teaching side, we seem to take the opposite tack.  Any failure at all is dangerous to admit, so it’s politically better to let a substandard program limp along than to be the bad guy who actually pulls the plug.  We start things slowly -- “pilot” is the term of art -- and then scale them up (or not) based only partly on results.  To make matters worse, many states now are basing significant portions of their appropriations on “performance,” as measured by numerical goals for, say, graduation rates.  In that setting, fast failure can quickly become permanent, since this year’s drop saps the resources that could have gone to trying something new next year.

Given very different culture, rules, and incentives, it’s not surprising that higher ed is as methodologically conservative as it is.  (“Politically” is another matter.)  If change is taken as either a threat or an implied indictment, rather than an opportunity, then a downward spiral -- first slow, but gradually getting faster --  is the predictable outcome.  As the good folks at Kodak and Blockbuster can attest, the world moves on whether you give it permission or not.  Failing to adapt doesn’t stop change.

My own view is that we need to distinguish mission from form, and not continue the mistake of conflating the two.  When the form itself becomes the mission -- when maintaining a traditional structure takes precedence over actually achieving a social goal -- then we’ve lost our way.  Form should be subordinate to mission; if a form that once helped no longer does, then it needs to be changed.  If renting movies for home viewing is the goal, then filling stores with dvd’s made a great deal of sense in 2000 and no sense at all in 2013.  If providing high-quality, accessible education is the goal, then we need to think seriously about the forms best suited to do that now and in the near future.  To dust off a wonderful old churchly word, subordinating mission to form is idolatry.

in a culture prone to idolatry, “fail fast” sounds heretical.  (Yes, the religious language is intentional -- see this post from a few weeks ago.)  Admittedly, “fast” is a relative term; given how long it takes to get results, even a determined reformer would need a few years to determine whether something was working or not.  But in academic time, two or three years is quick.  

Many colleges handle the dilemma by bifurcating the organization.  They’ll have a “continuing education” side that’s flexible, responsive, and frankly utilitarian, and a “traditional” side that is none of those things.  (In many cases, profits from the secular side offset the losses on the traditional side.)  The traditional side generates prestige, and the continuing ed side generates income.  Each enables the other.  Without income, the prestige side would go out of business.  Without prestige, the income-generating side would have a harder time.  

But I’m not sure that bifurcation is a long-term solution.  It feels more like buying time.

I’m fumbling towards a theory, but I’m not there yet.  Wise and worldly readers, what would “fail fast” look like in an academic context?

At my college, "fail fast" is equivalent to "data driven" since that is how it happens. However, it is seldom the case that failures are public. You have to be paying attention, asking "what happened to ..." to realize that some prominent initiative failed.

On your main point, did you notice this article? Not a very big sample, but the observation that students prefer on-line classes for topics they don't want to learn (easy ones they think they already know or can pick up without being bored in class) and face-to-face for ones they think they need to learn (lab sciences) or consider personally interesting.
CCPhysicist, your link reminds me of my only experience in an online class -- in a cutting edge 1990's high school. "Health" was a required class, and the "cool kids" all had to sit there and listen to the teacher awkwardly cover the sex-ed material, but the nerds knew there was a new trial "online" option, and got to flip past photos of venereal disease outbreaks on a computer screen, much faster. It wasn't exactly meant to be an academically challenging course, and I think I got an A without even reading half the material. But I very much "preferred online for a topic I didn't want to learn (and thought I already knew or could pick up.)"

On the other hand, I dropped out of the "distance learning" class they offered for German 4. I really did want to learn that, but the Skype-like video system made interaction really hard, and the other two kids who were taking the class from my campus dropped out first...
Your comments on "fail fast" raised the question, in my mind, of "whose failure are we talking about?"

You were talking about failure at the systems level, meaning I think, does this type of higher ed work -- or does this curriculum or delivery method work? You could also ask, does this major work? does this faculty member "work," in the sense of does s/he help the students learn what they want/need to learn, in the context of one or another curriculum or format. You could also ask does this student "work," in the sense of is this student making adequate progress. And on to, does this financial aid system work? all of these components would have different time lines by which to measure whether something they are doing is "failing fast," or more likely in higher ed, "failing slowly."
"Fail fast," or, usually, "fail fast, fail cheap" may have some application for higher ed. What it usually means is that the best way of finding out if something works is not by endlessly researching the topic, but by putting it out there and seeing if it succeeds, or partially succeeds, and then building on that. The idea being that an entrepreneur may have a good idea, but probably doesn't have all of the details right, and the best way to get the details right is to try the thing out and see how it goes, and then adjust from there. (The alternative being to conduct more and more research to try and get the details right, which will likely take longer and which may not lead to any better results.)

It's basically what FDR did with a lot of the new deal, where his idea was to try a lot of different things and see what worked - but the main thing was to do *something.* (Not typical then or now, of course.)

One problem I see with doing that in higher ed is that you can't - in a lot of contexts - just completely yank the rug out from underneath them. Students who are halfway through a major in, say, central asian studies, need to be able to finish the major even if the program isn't working out. Students who enrolled because of an online program should be able to expect that it will continue for a reasonable period of time. It's not the same as trying out a free website on pet care.

So I kind of think that the bifurcation between continuing ed and traditional ed is a good one: continuing ed students don't have the settled expectations of the traditional students and are fine if a lot of courses are offered as one-offs. Perhaps the missing piece is taking successful experiments from continuing ed and transplanting them to traditional ed?

@MKS - I think that when distance learning does work, it works in courses that can be taught as lectures. I don't think it is a good fit at all for "skills" courses which tend to have a lot of one-on-one contact during each period. That's probably an idea that should "fail fast," if it isn't already.

Software engineering is another area where "fail fast" makes a lot of sense. 30% of large software projects outright fail. Can you identify the failures when you've only spent 50% of the budget or do you identify the failure when you've gone 50% over budget? The difference in cost is obviously huge.

The big difference between higher ed and software engineering is the expectations of the employees. In software engineering, you expect things to be fluid. Even if your employer goes under, getting another job is not a big deal. In higher education, outside of the rare 1-year visiting position, you expect to get hired for the long haul. Sure, tenure is never guaranteed, and at certain institutions, it is downright rare for your first job, but stability is generally expected.
Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?