Monday, May 01, 2017


Online Testing

Is there an elegant way to administer exams in online courses?

I have no evidence that cheating is greater in online courses than in face-to-face, but I probably wouldn’t.  Intuitively, it seems like it certainly could be.  Anyone old enough to remember the New Yorker cartoon about “on the internet, nobody knows you’re a dog” will know why.  And some faculty still refuse to teach online for that very reason.

Like many colleges, we have a fair number of online courses that require a student to show up physically somewhere to take exams.  That usually means the campus testing center, though I’m told we have informal reciprocity with campus testing centers around the country, so a student taking a class from, say, Ohio could use the testing center at her local community college.  The military also has versions of testing centers that we rely on for proctoring for students who are deployed.

Students often object, though, saying that the whole point of taking an online class is to avoid having to come physically to campus.  I understand the objection, and at some level, agree with it. The problem is that elegant alternatives are hard to find.

We have a lockdown browser that faculty can use to ensure that a student can’t have anything else open on the computer while taking the test.  That works really well if the student only has access to one screen.  But in a time of smartphones and tablets, we can’t assume that.  Even if the computer is locked down, the phone may be right there.  

I’ve heard of systems with webcams that can look around the room where the student is taking the test.  The idea is to ferret out any second screens, books, notes, or other contraband.  But taken far enough to inspire confidence, it borders on creepy.  Assuming that many students are taking online courses and exams from home, that brings a level of surveillance that I’m not comfortable embracing.  It also presumes the presence of people on the other end who are looking at what the webcam is catching, which has to be one of the dullest jobs on the planet.

To some degree, replacing traditional exams with other sorts of assignments may hold the promise of both solving the cheating problem and being more pedagogically interesting or sound.  Depending on the subject matter, idiosyncratic assignments can sometimes be the way to go.  But sometimes they aren’t, and even when they are, they’re often much more labor-intensive to develop.  Sometimes, tests serve a purpose.

Wise and worldly readers, have you seen a non-creepy (or at least minimally creepy) way to give exams online that makes cheating difficult enough that faculty can have confidence in it?

For my online literature classes, I set up a timed exam on Blackboard that consists of several short essay questions. I haven't experienced any problems with cheating, that I know of. My online composition classes don't have anything except short quizzes (not worth the time it would take to cheat) and various drafts of papers, some short papers, some longer papers. Other subjects rightfully have other concerns. Ultimately, I recognize that my job is more to support the students who want to learn, less about catching or punishing the slackers who might try to cheat.
We only use proctored exams for distance ed classes, although faculty can always do whatever they want. On-line exams use a lockdown browser in a proctored environment where there are videos that can be viewed as needed and where no phone or other device is allowed. Remote locations are supposed to follow the same protocol.

The weak link is always the ID check. Is that your student or a stooge? That can be a problem at a testing center, and is always a danger when no one can see the student taking the test.

The goal is to match what is supposed to happen in the classroom. I'll grant you that there are faculty who walk around their exam room helping students with problems and answering non-trivial questions while "proctoring". (I have that on trustworthy authority from a faculty member who oversees common finals for a particular course.) After all, it helps with success. Some even leave the room. And if you give an exam that is open book and open notes, having an open cell phone is just a tiny extra step. Soon that would be necessary for an open book test, with OER textbooks that are on your phone or laptop.

Favorite example from the ancient past: One prof let students bring anything they wanted. Some had a half-dozen books, including giant integral tables. The smart ones brought a single sheet of paper that had all the key things in one easy-to-find place. After all, it was a timed exam!
You could always use this as an opportunity to design more realistic exams. People have been warning for decades that traditional closed-books exams test the skill of writing exams, not a skill that you can use outside the classroom. In almost any real-world situation, you can check external sources, and the real challenge is getting the right answer fast enough. Traditional North American exams are easier to write, but we teachers are the ones getting paid the big bucks.

Its certainly true that in an "open devices" exam, some students will send the questions to a third party to write the answers ... but students have been hiring stooges to write exams for them as long as there have been written exams. Math courses often have a "show your work" policy, and that makes it hard to get just the answers off software or a buddy on Skype.
I've been teaching online for about six years now, and I no longer give exams. I use other assessment tools like research proposals and problem-based assignments. I find that these methods are a more cognitively engaging and valid measures of student learning. Most exams (at least in intro courses) test little more than recognition memory (a low-level cognitive skill). In my view, the sooner we set aside the misconception that closed book exams are an essential tool to measure student learning, the better!
"...idiosyncratic assignments can sometimes be the way to go. But sometimes they aren’t, and even when they are, they’re often much more labor-intensive to develop."

And to grade.
Tests that force students to think and analyze make it much harder to cheat. Tests that require recitation of basic facts make it easy to cheat. ID is a problem in online classes, but it's also a problem in on-campus classes. I've had to show ID for a few proctored exams, but I never had to show ID to attend a class and take exams in that class so there's no reason (other than being poor) that I couldn't have paid someone to take the class for me.

Using cameras to proctor exams is creepy as hell. A friend of mine did his MS through Western Governor's University and they video-proctored his exams; he had to turn the camera and show them the the rest of the room before starting the exam.
My rule is only low-stakes summative assessment. For high stakes, only formative assessment. But in my field (ethnomusicology, which hybridizes humanities and social sciences) that's relatively easy to do--it would be a lot harder with natural science disciplines.

Incidentally, keep in mind that all the surveillance in the world won't prevent students from cheating the old-fashioned way if you don't change things up regularly. We just had a little scandal at my university with a long-term (multi-year) cheating ring, students sharing the answers to the tests for online classes. The news wasn't widely shared, I think in part because the classes in question are major money makers for the departments concerned. That's another reason I avoid high-stakes summative assessment, even in face-to-face classes--the temptation to cheat is just too strong.
I teach 7th through 12th grade math in an online program, and I've changed to making them take their assessments in person on paper this year. Our previous system was to make them take the final exams in a proctored computer lab but the rest of their exams on their own, and I was finding that some students were Googling their way through the course and bombing the proctored final.

Also, it's really hard to have them show their work on an online test, because typesetting math is obnoxious and it's hard to draw diagrams and such. I'm getting higher quality answers out of both the kinds of students who might otherwise cheat and the kinds of students who definitely would not by making them take paper tests rather than online ones.

I have a designated exam week about once a month where all of my classes come in and take their next test, which means the tests aren't at natural chapter breaks in most of the classes, but it's still worth it. I offer tests on a drop-in basis during various hours that week.

I'm really getting more benefit from the "paper" aspect than the "proctored" aspect, I think. It's surprisingly hard to get students to send me high-quality scans of problems they do at home, so the tests have become one of the best times to get a good look at their processes and explanations.
Anonymous@3:32PM -

That was really interesting, and now you have me wondering if NOT testing at chapter breaks is helping your students learn. There are only a few times when I break the "unit" mindset on hour exams, and it seems to reduce the compartmentalization of what will be forgotten in the next few days.

Regarding scanned homework, I routinely accept HW from f2f students via cell-phone picture. I have to print them to grade them, but they are generally more than adequate. I've reached the point where I say "photo or scan" when asked about alternative submissions.

I used to teach in-person high school math from a spiraling curriculum (the book series is called College Preparatory Mathematics), and we were told to deliberately put questions from previous units on test all year long (the test bank was even written that way, with, say, the chapter 6 test bank having a section of "chapter 6" questions and a section of questions about things from chapters 1-5 to test after chapter 6). The homework would be a mixture of things from previous lessons rather than more practice on the current lesson most of the time, too. It did lead to a lot less compartmentalization on the part of the students. There were a lot of good things about it, but it was hard to pull out specific topics when you needed to for one reason or another or to flex the books around anything else you were also trying to do as a school or program (I always wanted to do a co-taught Geometry and Art 2 period block class, and those books wouldn't have had the flexibility to put the topics in different orders to help make it fit a combined curriculum).

Your students must have better cameras or photography skills than mine. I usually can't make out much detail from their cell phone photos. It may be that 7th graders are not given as nice of phones as college students buy (which is probably wise).
Or just get rid of testing which only finds out if a students can, via short-term memory, regurgitate back "facts".

Testing doe NOT prove knowledge or wheteher somoen has learned material.
Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?