Thursday, February 03, 2011
Thoughts on "Academically Adrift"
My first observation, which was largely ignored in the initial wave of reports, is that the sample they used only included four-year colleges. Community colleges were not included in the sample. Based on the rest of the findings, I doubt that we’ve cracked the secret code, but it’s certainly a glaring oversight for a study of the first two years of college.
That said, I’m pretty conflicted in my responses overall. It’s an impressive piece of analysis, certainly. The data work took some doing, and the prose is evenhanded and relatively clear by social science standards. (Social scientists aren’t known for our limpid prose, as a breed.) I’m just not sure how useful it is.
- I wasn’t shocked to see that math, science, humanities, and social science majors tend to show the greatest gains in critical thinking, as compared to students in business, health, computer science, or communications. The authors were gracious enough not to go into too much detail there, and it would be politically unwise for me not to heed their example, but it was hard not to notice.
- Also not shocking: students who are assigned more reading and writing get better at reading and writing. In a related story, bears tend to crap in the woods.
- Group study was no more conducive to improved critical thinking skills than socializing. Individual study correlated strongly to improved skills; group study did not. This certainly fits my own personal preference and intuition, but it’s nice to see empirical confirmation.
- “Student engagement,” as measured by NSSE (and presumably CCSSE), correlates to retention, but not to increased learning. Fraternities and sororities lead to higher student satisfaction, but lower learning. Again, not shocking, but nice to see confirmed.
- Instructor expectations matter. Students who have more professors with high expectations learn more than students who don’t. Given that students will often go out of their way to seek out ‘easy’ professors and avoid ‘hard’ ones, this suggests a dilemma.
- Federal research funding dwarfs federal funding for improving instruction -- say, FIPSE -- by several orders of magnitude. Incentives matter.
- What’s good for retention may or may not be good for learning. Students can stick around for years without learning much, depending on what they’re doing. Arum and Roksa note that learning communities are positively correlated with retention in the national literature, for example, but there is no evidence one way or the other of their effects on learning.
- Reflecting on my time at Snooty Liberal Arts College, I could see why its students would do markedly well on tests of critical thinking. It had no ‘business’ or ‘communications’ majors; it had very selective admissions and therefore a strong ‘peer culture,’ and it lacked frats. My cc also lacks frats, but the other components don’t really carry over.
- ‘Peer culture’ is huge. If you run with a crowd of high achievers, you will adapt to it; if you run with a crowd of hard partiers, you will adapt to that. In an open-admissions institution, this presents a substantial challenge. (Some peer cultures are trickier than others. Coming from a public high school in a middle-class suburb, it took me a semester to raise my game when I got to SLAC. I didn’t know that the prep school kids affect insouciance in public while studying like crazy behind closed doors.)
- On-campus employment helps, if it’s up to ten hours a week. Off-campus employment hurts. Interestingly, grants help but loans hurt.
- Many students see college (and here the fact that it’s a sample of four-year colleges may matter) as primarily a social experience. It’s a chance to get away from Mom and Dad, to make new friends, to explore lifestyle options, and to get a credential. If that’s your orientation, then ‘learning’ is fine, as long as it doesn’t require time and effort. In that climate, lone instructors who raise academic expectations may pay a price in student anger.
At my cc and at most that I’ve seen, dorms don’t exist, and the whole “college experience” is pretty attenuated. (We don’t have climbing walls, a football team, fraternities, or even a quad.) If football Saturdays are your idea of college, you don’t come here. That said, though, it’s still very much the case that academics are often only one priority among many in students’ lives. As our student body gets progressively younger and more ‘traditional,’ some of the quirks of 18 year olds will probably become more relevant here.
- To their credit, Arum and Roksa note that making sustained and significant progress on student critical thinking skills would require fundamental realignments of incentives across the entire structure of higher ed. They seem a little too quick, in my estimation, to assume that “employers” want critical thinking skills -- at the entry level, in my observation, they’re much more focused on enthusiasm than on analytical prowess -- but that just makes matters worse.
And the incentives point is what’s ultimately so frustrating about the book. Yes, it would be lovely if students naturally clustered into the liberal arts, where virtuous and civic-minded professors larded their plates with ample helpings of robust reading and writing assignments. In the settings where that actually happens, measured learning outcomes are strong. But when you have open-door admissions and low per-student funding, getting there from here would require changes of staggering magnitude. Funding mechanisms would have to change; national markets would have to change; collective bargaining agreements would have to change; longtime readers can guess the rest...
Still, it’s a reminder of some of the right questions, and it sheds useful light in some corners. Maybe expanding the “individual quiet study” area in the library should take precedence over the “group study” section; I can do that. Maybe a little more skepticism towards “student support” offices, as against direct instruction, is in order; that may work.
Now if I could just get the voters to do something about that funding...
Back when I was a CC student (I transferred to a 4-year this year, so it wasn't long ago), it seemed like all the teachers got the OPPOSITE memo. Every science teacher I had stressed the importance of study groups (one prof even forced us to form study groups! I'm pretty sure that a lot of those groups never actually met...). At least 1/3 of the time in my humanities and social science classes were spent discussing things in small groups.
And you know why it's no better than socializing? Because it mostly IS socializing!
In lab groups, group projects, and group discussions, one of two things happens:
1) students pick their own groups and they self-select. This is great for Team Competent but you wind up with A LOT of groups where the blind are leading the blind. It seems like the majority of study groups end up this way because most students only join study groups because they're doing poorly on their own.
2) students are put into groups by the teacher and/or just group with those in closest proximity.
I'm happy to help my peers and I'll tutor anyone or edit any any paper if the student approaches me independently. But I'm not going to be the annoying nag in a discussion group who says "Sorry! You guys are disinterested and want to socialize but I'm going to force you to discuss the readings that you didn't read!"
When professors try to socially engineer groups so that they contain students of mixed competence or dedication, it does not make the poor students improve. It's just the opposite. The good students notice that nobody else is invested in doing a good job and they wind up doing all the work! In groups that are fixed for the course of a semester (e.g. lab groups), the poor students quickly realize that they can completely slack off and the work still gets done. Then they slack off even more. If the professor isn't paying attention, it can SEEM like the poor students do better in a group, but that's just because the good students do 90% of their work for them. When students are evaluated as a group, slackers get grades they don't earn and dedicated students often have little choice but to let their slacker peers take advantage of them. Poor students who AREN'T slackers don't benefit because usually they're too worried about 'dragging the group down' to make much contribution.
Plus, teachers often make these groups way too big to be effective. Two-person groups can work; 6-person groups can't (this was one of the nastier effects of budget cuts on my California CC last year -- bio and chem profs suddenly couldn't afford sufficient supplies and we were forced to work in HUGE lab groups. My physiology lab had 8-10 people in a group! In o-chem, where we were supposed to do most experiments alone, we sometimes had to work as 6-12 people as a bench because the supplies were expensive. In a group of 10, 7 or 8 people's roles are going to be trivial or nonexistent)
I'm sorry for posting something of a rant, but I know a lot of professors and adjuncts read your blog and it's nice to finally have confirmation on something that I've observed and experienced so many times!
Dean Dad, does your CC have a tutoring center over which you hold any authority? I worked at my CC's tutoring center & I have a lot of ideas about how to make them more effective. I'm just a student, not a professor or anything, but if you're interested, I'd be happy to email you. -Ashley
1) It improves participation in discussion and attentiveness when we're NOT doing small group activities.
2) It allows students to work together to learn how to do difficult things, giving them a practice run for when they will be expected to do those things on their own.
When I assign small group stuff, I give students very specific directions so that all of them have to participate, and I typically connect that work to independent writing and then full-class discussion. Sure, if you say "get into a group of six and discuss the reading assignment" that is likely not to produce anything of value, but there is group work and then there is group work. And learning to work productively with other people is an important - and necessary - skill.
As with all pedagogical techniques, I'd say that there isn't a one-size-fits-all model, and I'd say that any technique in excess is not going to have a positive outcome. But to say "group work is bad," just as to say "all lecturing is bad," is painting with too broad a brush, I think.
@DD - Thanks for this review! I've been interested in the book but I have no time for reading right now that isn't for research or teaching.
1. Is that book really a one-day snow-day read?
2. Funding interests me. How does your per-student funding for lower division classes like English comp, history, and "college" algebra compare to the per-student funding for lower division classes at a middling State uni in Your State?
3. How do you create incentives for new (not yet tenured) and adjunct faculty to hold students to high standards when they live in fear of student evaluations? (And I write from a unit where our Dean clearly places an emphasis on learning over popularity.)
Maybe they were measuring the wrong thing with these students. Might the students have needed to develop other more basic skills first, before they were ready to develop those higher order skills? If this measure was looking at the move from level 3 to level 4 and these kids spent their first year of college going from zero to 3 that would show up as no improvement but it would still be the foundational work that needed to be done before the student could develop higher order skills.
Students who took courses that required lots of reading (more than 40 pages a week) and writing (more than 20 pages in a semester) showed higher rates of learning.
This could be because of self selection – the potential for improvement might be better with these students and they may (correctly) sense that they are ready for challenges that other students avoid in an effort to preserve their GPA.
Interesting - I'll have to find this and read it.
It shouldn't be hard to find them. The students have already sniffed them out. Ask around, or take a peek at the course selections of the students that are just barely getting by, and not because of any external problems such as poor preparation. You'll find the really easy courses there.
However, overwhelmingly, I'm discouraged by my communication colleagues to push the students. "Giving them exams makes the class too hard. Why are you pushing theory on them? Theory isn't important." (yeah, ok.) I realize 5 schools isn't a huge sample size, but I know I'm not alone in this experience. Because communication(s) is supposed to "be easy," those of us who actually try to demand more, incorporate critical thinking, theory and application all together, get told we're doing it wrong. The big problem here is that it's other professors who are discouraging me. My students? Have yet to complain. In fact several have gone so far as to say "you're the only one I had to work for."
Also, while higher-order learning is important, the acquisition of a knowledge base is important too. As DD pointed out (but was loathe to pursue), disciplines such as computer science and business did not show as much of an increase in higher-order learning, but I bet those students learned a lot about their field of interest. Maybe that should not be considered enough, but I also wouldn't simply discount it as, 'not learning'.
The tests are measuring the wrong thing. Focusing on academics, students do not sign up to learn critical thinking skills in college, students sign up to learn whatever is in their major. I didn't improve my critical thinking skills in my years at Flagship Small State U. Instead, I learned a lot about economics and mathematics, which is what I went there to do.
I've already blogged about this book based on reports and other analysis (see link from my name), where one issue is the specific measurement used. I'd guess that I would not have improved on it because my HS comp class was so strong on critical thinking that I should have hit the top score as a freshman. The instrument used is also not at all like the kind of vague problem that engineers, algorithm designers, or business professionals have to deal with. Whether it separates a Frito-eating code monkey from a skilled software designer is an interesting question!
It's probably hard for voters to see that your budget is tight. What do the local senior citizens see when they go to the CC for a basketball game, a play, a speaker that is in town, or to vote? Everything looks fine: the lights are on, the building is heated and nothing seems amiss. The painful cuts are probably happening in the classrooms.
So, I wonder, why not turn the thermostat down to 60 degrees? It's an obvious and immediate sign of the unfortunate austerity measures that have been forced upon the college.
Critical thinking is a useful measure for liberal arts students, and not for others. Not surprising, then, that those are the students for whom you saw improved results in critical thinking.
Which explains much about why union voters or hunters vote a straight-party ticket?
If I'm at the hospital, I don't care about the nurse anesthetist's ability to think critically; I care about her ability to administer the anesthesia without killing me. CS grads aren't hired for critical thinking abilities; they are hired for the ability to write code or do whatever other CS related skill their job requires.
Health, science, CS, ... and to some extent foreign languages ... are skills-based areas, where you spend much of your undergrad career trying to master and apply a certain skillset. Learning calculus is hard for a lot of students...and having done so will probably not increase your reasoning ability much. But you will know calculus.
My undergrad degree was liberal arts, and while I'm sure I learned some critical thinking/reasoning skills in undergrad, it wasn't really until grad school that I felt that I *really* had good critical thinking skills.
So I'm not sure that measuring the reasoning skills of 20-year-old sophomores really tells us very much.
No, that's not the point I was trying to make...and I hope that my health workers have reasoning ability, too.
But the reality of health education, especially at the beginning, is that there is a *lot* of memorization and a lot of very practical hands-on exercises in giving shots, taking blood, and dealing with other bodily functions. Essentially, nursing students are crammed with facts and skills, but there's not a lot of *analysis* involved. The students have to understand a lot of basic facts before they know enough to do any kind of real analysis in their field. (And even for doctors, real analysis, I'm told, doesn't start until they've graduated from med school and started doing their residency and working with real patients.)
Compare this to an English class. The students already (one hopes) know how to read, and have already (again one hopes) been exposed to literature and even some interpretation in high school. The class is *all about* reasoning and analysis because (much more so than nursing students, at any rate) the students already know the basic facts necessary to do an analysis.
But much of this misses a broader point: selection bias.
The authors used a one-dimensional test and the reports only talk about IMPROVEMENT rather than some "absolute" (subjective) level of critical thinking broken down by major, so we have no way of knowing if students in one area are already skilled at the specific skill they were measuring while others developed that skill in their college classes.
Further, they weren't asking you to apply your knowledge of chemistry and math to decide if the dose prescribed by the doctor might kill a child. One group of majors would not improve at all on that sort of test in 4 years of college, while another would improve a lot.
Mind you, I have found online discussion groups (in hybrid classes - so tied to assessment) extremely valuable. I'm assuming the transparency of online postings makes us all accountable and thus raises the stakes.