Monday, December 14, 2015
Innovation and Scale
That article was about bad science, terrifyingly bad science that would not be tolerated in other life-and-death professions like aircraft and car design. It is called "lying" when someone says that their test of the new MOPP protocol showed that new protocol did not work when they did not follow the MOPP protocol. Period, end of story. That group showed that their own innovative protocol, derived from a successful one, failed. Sayting otherwise was bad science then, while today it would likely be called fraud.
That medical doctors, who are not real doctors, would be ignorant of good scientific practice is not at all surprising to me. They are not scientists. Gladwell should have pointed that out. Innovation is useful when the results are reported honestly. That is what science is about.
Putting science into practice is technology. Ideally, technologists stay within established safety factors, but even there you continue to learn when it turns out those safe boundaries are not where you thought they were. Freeway design today is different from what freeway design 60 years ago. What that article exposes is a system where there are no building codes and no process to improve them.
Learning from innovation is what teaching is about, but it only works if you assess -- as honestly as possible -- whether something is working or not. That does not happen when each prof is on an island with no point of comparison such as group assessment via a common rubric of student performance on an assignment. I think that is what you are talking about. But of equal relevance is when someone makes a half-hearted try at applying an innovative, research-backed teaching approach and finds it does not work. You and I have to watch for that sort of confirmation bias. But it could be that the innovation is only good at doing one thing, and that is improving student performance on one, narrow, assessment tool. We all have to watch for that as well.
And, finally, it could be that the new technique does not scale. I know of one instance where a new approach worked great, but failed as soon as a different prof took over, despite doing the exact same things. On paper, at least. There are artisans out there who do more than lay the bricks according to plan.
Medicine started to become something like scientific in the 1950s, but medical doctors are still not trained as scientists, though they should be. As CCPhysicist points out, those doctors should have been found guilty of malpractice and disbarred. A double-blind trial of their protocol against MOPP would have shown immediately that their freewheeling gay experimenting with the lives of their patients, was unsupportable.
In education, just look at the CC (Common/Corporate Core) fiasco - wild experimenting on students, based on no evidence at all. There should be some culpability here, but instead David Coleman waltzes off to make millions as CEO at the College Board. Bill Gates goes on speculating on what might work in education, conducting his grand experiments on thousands of unfortunate subjects.
Also note that Gladwell is utterly unreliable - good at storytelling, not so good with facts and the scientific method.