Dr. Helaine Alessio, FACSM is a Professor and Chair of the Department of Kinesiology, Nutrition, and Health at Miami University and is a past President of the MWACSM and a fellow of the American College of Sports Medicine. She teaches Exercise Science-related courses and has received university commendations for her teaching. She has been funded by NIH, private foundations, and corporations to support research, teaching, and service projects. She has published 2 books, 13 book chapters, and 56 journal articles, as well as national and international peer-reviewed blogs, infographics, and NPR broadcasts. She is listed in the top 2% of Exercise Scientists cited in the world by Stanford University researchers. Her work on academic integrity includes co-editing a special edition of a journal on the topic that was the most widely published for the Journal of Excellence in College Teaching.
Episode Description
Remember the pressure of taking a test when the teacher was glaring at you, daring your eyes to roam? Cheating was not a temptation unless you were desperate and willing to risk the ever-present teacher catching you. However, the offering of online classes exploded in recent years. So what happens when you’re testing during online classes that’s the focus of this episode of Stats+Stories with guest Dr. Helaine Alessio.
+Full Transcript
John Bailer
Remember the pressure of taking a test when the teacher was glaring at you, daring your eyes to roam? Cheating was not a temptation unless you were desperate and willing to risk the ever present teacher catching you. But the offering of online classes exploded over recent years, and pandemic restrictions were a catalyst for even greater development. So what happens when you're doing testing and online classes? Today's episode of Stats and Short Stories will feature a conversation about cheating in virtual classes. I'm John Bailer. Stats and Stories is a production of Miami University's departments of statistics and media, journalism and film, as well as the American Statistical Association. Rosemary Pennington, Professor of Journalism, joins me in the studio along with our guest, Dr. Helanie Alessio, Professor and Chair of the Department of Kinesiology, Nutrition and Health at Miami University and past president of the Midwest Regional Chapter of the American College of Medicine. Helanie, so great to see you and have you on here.
Helanie Alessio
Thank you very much.
John Bailer
You know, for those listening I will confess, I've known Helanie forever. I mean, we've talked about stats and research questions in the kids, the kiddie pool, with our kids. Yeah. So this is, this is such a delight and a treat to have her joining us today.
Helanie Alessio
That is true. That's the first time I ever thought about how this is going back 25 years, 30 years ago, about power analysis.
John Bailer
Oh, the power pool. But today, we're actually talking about something different. We're going to talk about the kind of issue that you got very interested in. And that is, how do we monitor that someone is doing the work that they should be doing and getting credit for an assessment when you're doing online classes? Can you talk about the origin story here? How did you first get interested in this question?
Helanie Alessio
Sure. So first, the interesting part of this is when I was reading a little bit more about it. The bottom line is: everybody cheats. It's a terrible thing to say. But it's everywhere. It's not just in the classroom. Companies cheat. We learned about it in West Virginia, the review of Volkswagens miles per gallon for, you know, certain diesel fuel, it was a bunch of students that found out that the numbers that were being provided by the company were wrong. And it wasn't just an honest error. It was cheating there. And they're not just cheating for money. They're cheating for prestige. They cheat in Harvard, they cheat in West Point, they cheat at the academies. So it's, it's very ubiquitous, it's happening everywhere. And it was interesting to me when we were switching over to an online course delivery, and an online test delivery, that I just sensed that there was an uptick in cheating compared with face to face.
Rosemary Pennington
So what exactly did you and your co-authors do to study this?
Helanie Alessio
So we had an opportunity to compare test scores in nine different sections of a class that was being taught online. And some of the tests were being offered without proctoring. And some were, and this was the first, as John would explain, natural design. We didn't identify any sections to be proctored or not. It was just that we knew that some were proctored, and some weren't. When we compared the test scores, we saw a 17 point difference in the average test score.
Rosemary Pennington
And in the proctored classes, it was 70 points lower than the non proctored.
Helanie Alessio Correct, correct. Yeah.
John Bailer
So one aspect of that, so this was a terminology class, right? The medical terminology class, if you're doing this and so maybe people might not know what exactly the setting up proctoring software does in a virtual class.
Helanie Alessio
So when you set up proctoring software, you are asking the student to number one, confirm that they are who they are, by holding up their picture ID next to their face on the screen in real time. So that's one thing that the instructor then can check. Is it the same person? Secondly, if you're asked to do a room scan, now in 2022, there was a law that came by in Ohio that said you can't do that. You can't do a room scan. So that's something that's changed, but nevertheless, you can audio and videotape the student while they're taking the test. And that's what happens. So while the student is taking the test, this is where AI comes in. There are some motions and movements that are flagged.
John Bailer
Yeah, so like in movements, if you are looking down maybe to look at a document you're not supposed to have, if you're taking a test where it's supposed to be just your memory, or looking off to look to see if a person might be in the room with you as an example, or?
Helanie Alessio
Correct. So if someone walks behind a person taking a test, because a lot of students take it in their dorm room or in their home apartment, that's flagged. So those are the types of things that are flagged, and they come up with an integrity score. And they're flagged with both numbers. So numerically, there'll be flagged up, let's say, 30% of the test, there has been some suspicious activity, and then they'll literally red flag it, or orange flag it or green flag it. And then this was the part where it's important to understand that good professors don't just let AI or let the scoring, let the proctoring AI make the final call. The good proctoring software will take you to those points where the suspicious behavior occurred. See, and then the professor can see ah, there was a person walking behind the student, but really, it was benign. Or, yeah, the students' eyes were leaving the screen. But it didn't appear to be long enough in their judgment for them to be looking something up.
Rosemary Pennington
Yeah, I mean, I was really struck when I was reading through the article, just how significant the difference was between the non-proctored and the proctored exams. And it made me think, you know, as you are aware, like there is this debate around proctoring software, because some of the software doesn't recognize particular skin colors in certain ways, and can flag things in ways that are problematic. And then you have the problem of some professors who maybe don't do the due diligence of checking. So like, for me reading through this, it sort of became this moment of like, this seems important. And like we should recognize that this does have a significant impact on academic integrity. But how do we have this conversation thoughtfully in a way that also figures out, like, manages some of these other things, right? Like, how? That's what I keep getting stuck on. Like, how do you advocate for these things, when it seems like there have been some problems with it? But your data, and other research, seems pretty striking that if you want to control for academic integrity, and an online space, something like this is necessary.
Helanie Alessio
Yeah, I don't have all the answers. But it does require due diligence of the professor, for sure. And that's really the way it should be even in a regular classroom. So in some ways, the same type of diligence that you use walking up and down the aisles and not being in the front of the room, let's say on your computer, that type of due diligence has to occur. And then if you see someone checking their phone, and then following up, like, for the actual evidence that it wasn't just a text from home, it was actually them looking something up. So it's due diligence that tries to mitigate the false positives.
John Bailer
Well, it seems that the presence of some of these tools is to try to put immediately kind of, you know, speed trap ahead, this kind of warning that, you know, you're being pre-emptive in doing this. And, you know, it was really interesting, because I had the pleasure to collaborate with you on some of this work, and some of my students did many years ago. And I was struck by not only the fact that there was a shift in the grade distribution between the students that were using monitoring software versus those that didn't have the proctoring software in place, but also the time that students were spending to take the exam, and that was also a signal of something. So can you talk about how for a longer time in a proctored setting was telling a story as well?
Helanie Alessio
Sure, so the time was, if there was a 60 minute time limit or even a 45 minute time limit, the percentage of time spent taking the test for those who were not proctored was always close to 100%. So they use the entire 60 minutes or the entire 45 minutes. And then for the proctored students, they were 20 minutes, 30 minutes max and half the time. So it implies but again, we don't have the positive evidence, but it implies that if you're not being proctored, then you have the time and the opportunity to look up the answers.
John Bailer
You know, the one thing that I found really fascinating were some of the stories that you told about people before they realized that before they started the test, they were identifying to the camera that they were there, and their behavior was being recorded. Even before that, there were some rules of unusual things that occurred. Are you comfortable sharing any of those?
Helanie Alessio
I'll share a few. And you know what? This happened when online testing was really just starting at Miami. So everybody, professors and students, were not very savvy. And I think that I'm not making an excuse, but that might lead to the lack of due diligence on the professor's part. But on the students' parts, I don't think they realized when they started with the ID and the face recognition, to affirm that they were there. They were the students. The cameras kept rolling. And so that's when we heard conversations. We heard conversations, where one other student came into the room and said, What are you doing? I'm taking a test. Oh, should I leave the room? Oh, no, professors never check the video. So you don't have to work. And then another one, a student had a sticky tack pad with sticky notes, sticky notes, all over their computer screen so that when they did the computer scan, you know, they wouldn't pick it up. And then a student came in and said, What did you do to your computer? And the student literally said, oh, yeah, those are sticky notes. I'm taking a test. Oops. So you know, and that's something that we couldn't see when we reviewed the film. But the student admitted that, right. And then another one was about buying beer and drugs after the test. That was another one that we had a couple general counsel. Oops.
Rosemary Pennington
I was thinking about, like, I taught an online course a couple of years ago for a summer term. And it was the first time I'd ever used proctoring software. And I was really shocked by how much was captured. I think these students must have taken online classes before because they were sort of, you know, very about the book, but I just was like, Oh, this is recording everything that's happening. And I was not surprised students are unaware of what's going on. I guess I wonder if you mentioned the fact that, you know, there's this law in Ohio, where those room scans can no longer take place. Given the sort of work that you've done, are there other things that you imagine are going to be landmines moving forward with this or other places that you think educators should be thinking about in relation to these proctoring software's either best use are things that they need to be concerned about that could be problems in the future?
Helanie Alessio
So, I mean, what you want to do is be the least intrusive, and as fair as you can be. But you mentioned something before, John. And I want to point this out, whatever proctoring software you use, the fact that you're using proctoring software changes the whole mindset of the student, because they know they're being watched. And the analogous thing to that is, if a faculty member would give an exam, and they told the student in advance, look, I have to run an errand across campus. I'll be right back. It changes. So by just having an, I'm not saying bad proctoring software, but by just saying you're going to be proctored, you're going to be watched, changes everything. And there is research that says faculty should develop trust with their students and develop a code of ethics or an honor code. I'm not dismissing any of that. But the research teams show when the stakes are high. And that's another thing that we've reported in our research. The students most likely to cheat, we actually identified, are the ones where the stakes were highest. So they were not only the biology majors, but the biology majors that decided to go pre-med.
John Bailer
Yeah, the low grade has more dire consequences, or at least perceived to have more dire consequences. Well, I'm afraid that's all the time we have for this episode of Stats of Short Stories. Thank you so much for joining us
Helanie Alessio You're very welcome.
John Bailer Stats and Stories is a partnership between Miami University’s Departments of Statistics, and Media, Journalism and Film, and the American Statistical Association. You can follow us on Twitter, Apple podcasts, or other places you can find podcasts. If you’d like to share your thoughts on the program send your email to statsandstories@miamioh.edu or check us out at statsandstories.net, and be sure to listen for future editions of Stats and Stories, where we discuss the statistics behind the stories and the stories behind the statistics.