Conner Jackson is a Research Instructor in the Department of Biostatistics and Informatics in the Colorado School of Public Health. He serves as the chair of the Education Committee for the Center for Innovative Design and Analysis and teaches a 6-week short course about statistics and data science to bench scientists. His research focuses on the analysis of correlated data, largely in the context of infectious diseases.
Episode Description
Meteorologists go to school to be able to predict the weather accurately, but for some people, weather prediction is a hobby. Maybe they have a trick knee that hurts when it rains or perhaps they know when a storm is coming by how the birds at their feeders are behaving. Some lucky folks have pets that can help them figure out what the weather is going to do and that’s the focus of this episode of Stats and Stories with guest Connor Jackson.
+Timestamps
- How did this come about? (1:40)
- How did you frame this project? (4:40)
- How do you know, he's predicting rain? (8:29)
- So how did it work out? (11:40)
- Conner’s writing style (15:24)
- Has this influenced your other work? (20:42)
+Full Transcript
Rosemary Pennington
Meteorologists go to school to be able to predict the weather accurately. But for some people weather prediction is a hobby. Maybe they have a trick knee that hurts when it rains. Or perhaps they know when a storm is coming by how the birds at their feeders are behaving. Some lucky folks, though, have pets that can help them figure out what the weather is going to do. And that's the focus of this episode of Stats and Stories where we explore the statistics behind the stories and the stories behind the statistics. I'm Rosemary Pennington. Stats and Stories is a production of Miami University's Department of Statistics and Media, Journalism and Film, as well as the American Statistical Association. Joining me is regular panelist John Bailer, Chair of Miami Statistics Department. Our guest today is Conner Jackson. Jackson is a research instructor in the Department of Biostatistics and Informatics at the Colorado School of Public Health. He serves as the chair of the Education Committee for the Center for Innovative Design and Analysis, and teaches a six week short course about statistics and data science to bench scientist. His research focuses on the analysis of correlated data largely in the context of infectious diseases. He also authored an article in 2021, for significance about Pietro[ the weather tortoise, the article led to Jackson winning the 2021 statistical Excellence Award for early career writing, Conner, thank you so much for joining us today.
Conner Jackson
Thank you for having me.
Rosemary Pennington
So I'm going to ask the obvious question, how does an infectious disease guy come to write about a weather predicting tortoise?
Conner Jackson
It's a great question. So it turned out that sort of in the middle of the pandemic, we were all at home, we were all on Zoom, sort of an off the cuff comment from one of my co workers just basically, oh, it snowed yesterday. And my pet tortoise said that it would and I was like, Wait, excuse me. And it was funny, you know, I mean, I work at a center that's almost exclusively statisticians in it. My first thought was like, we have to look into this, like, we have the tools like we are all quantitatively minded people. So we like, not that I was, am the sort of cynical monster that is convinced that, oh, there's no way that's true. Like, this would be super interesting. So that came up and actually had known about the Significance writing competition, my Mary, who is the one who told me about it, who's the CO Director of our Center, kind of had planted that seed. And then so sort of in the background, I was thinking about what to write about. And then this came up, and I was like, This just seems so. So different. So novel. You know, obviously, I'm not a tortoise expert. So yeah, I was just like, let's do it. Why not?
Rosemary Pennington
That's so funny. I mean, obviously, like, you know, I think many of us have probably heard stories like that. But I can't, like my brain just would not have been like, Oh, yes, let's test that. So it's so interesting that that's exactly like you're like, Okay, let's figure out if this is true or not. Or how true is it?
Conner Jackson
Yeah, it's funny. I think that it might have been a function of the fact that it wasn't taking classes at the time. So maybe I was like, actively looking for something to do. So yeah, it's definitely, I worked sort of full time while I was finishing my master's degree. And now I'm working full time and working on my PhD. So I don't know, for whatever reason, maybe that extra spare time was enough for me to just stare at a tortoise security camera footage for 1000s of hours.
John Bailer
Well, you know, I mean, I have to ask, you know, as a biostat guy embedded in this process, did you get informed consent from Pietro?
Conner Jackson
It's a good point. We didn't put any disclaimers on that. Yeah, we didn't get the IRB involved. Yeah. It's, it's funny. Yeah, it's, it's, it was very invasive, right. We had 24 hour security camera footage on hand. So he was a good sport about it. It was fun. So it turns out that my coworker actually has two tortoises. So one of them is Pietro. They're in separate cages, because otherwise, I don't know how I would tell the difference. Pietro is excellent at predicting the weather. And then the other one is basically might as well be a flip of a coin. So yeah, it's funny that, you know, for whatever reason, he has this innate ability.
John Bailer
So that really begs the question. So, so, take us through some of the thinking of how we look at it, that Pietro you know, kind of prognostication patterns in a systematic way. I mean, How did you frame it and how did you explore it?
Conner Jackson
Yeah, I think I think for me, where I started was What does accuracy mean? Especially in the context of weather, right, because, you know, it's what weather or how the weather might impact your day to day life, obviously, is going to depend on different people. So I think in the article there is a sort of talk about, like, if I'm just going to work, I don't really care if it's raining, I'll just bring an umbrella. And so I think that for me, trying to contextualize and think about, like, when does weather prediction matter most to me, and that was sort of where I got the idea, like, Oh, I'm planning a barbecue. So I think that was where I started. And then from there, we still don't know, you know, what is accuracy? Like, what is this sufficient prediction for the weather. And obviously, we could get into more detail, we could think about snow versus rain. And Pietro himself seems to be good at predicting precipitation. So here in Colorado, when we were looking at the weather, it was April, so it snowed, and it rained, and it didn't seem to matter to him, which, which it was, so that was interesting and by itself. But yeah, I think that figuring out what accuracy meant was where we started. And so that was where I started kind of asking coworkers, like, you know, how, how accurate Do you feel like your weather person needs to be? And maybe it's sort of like, I feel like the weather in Colorado is pretty unpredictable. And I think it's probably unpredictable. In a lot of places. There's this sort of idea that, you know, like how accurate truly is the weather, if it can change quickly in five minutes, and especially growing up here? We would get afternoon rain storms in the summer, constantly. And so, you know, I think trying to structure that and figure out a way to come up with a testable question was very important. And so we sort of asked our co workers like what's, what's a reasonable accuracy, right? I mean, 50% is a flip of a coin. So you'd obviously want it to be better than that. But 100% seems kind of unreasonable. So I just pulled people and it was funny to get a kind of a, I got a very wide range of people, it was like some people would expect that their weather person would be over 90% accurate as far as predicting precipitation, which seems high to me, especially in Colorado, and then others were a bit more forgiving. So maybe they were related to a meteorologist or something I'm not sure. So so yeah. Once we sort of established, like, what, what does accuracy mean, in the context of this kind of question? I think from there, you know, and I sort of mentioned this in the article as well. It's like being a statisticians. Oftentimes, we have this desire to like, overcomplicate things, and because we're just interested in the statistics, but oftentimes, it's easier just to go with something simple that answers your question at hand and marry my kind of reading partner, she, she was like, well, maybe we could do this kind of machine learning problem. Maybe Pietro gets better with more information. And I was like, that would be very interesting, but maybe outside of the scope of this kind of part time project that I'm doing so. So yeah, that was kind of where it started. And, you know, once we established what accuracy meant to us, and once we kind of established what our testable hypothesis was, we, you know, we had a barometer, we had a hypothesis, and then from there, we just, we started looking at the videos.
Rosemary Pennington
So what were you watching for when you were trying to figure out if Pietro is predicting the weather? He's obviously not standing at a green screen pointing at cold systems coming through? Like, how do you know, he's predicting precipitation versus a sunny day?
Conner Jackson
Yeah, it's, it's a good point, right? Because, you know, in working as a consulting statistician, you kind of have this format that you follow. When you meet an investigator, right? You kind of say, like, well, so what are you trying? What are you trying to do here? Tell me about, tell me about what your problem is. And so I kind of approach that with my coworker who's not a statistician, right? I was like, you know, so. So tell me a little bit about Petro. And what she was saying was that, when he thinks that it's going to rain, he hides under this sort of upturned Red Bull in his cage. And when he thinks that it's gonna be really nice, he's out and about because, you know, the weather's gonna be nice, even though he's indoors, you know, it doesn't matter. And then there's this sort of third option where he's, like, he's kind of out but not necessarily so much as like kind of about in his cage, there's this log that he kind of hangs out on. And so from there, we were basically like, okay, so it seems like a simple way to go would be is he or is he not under the Red Bull? Does it or does it not rain? And so that was kind of where we started and so it just so happened that Melissa, my coworker who owns Pietro, her, her parents had like the security camera footage or the security camera. And so she set it up. And it was just so fun like it would turn to black and white at nighttime. And, you know, we had to figure out logistically how to get like, like tons and tons of hours of video transferred over to me, and, and then from there, you know, I just started watching, you know, day one, I just started watching and I pulled the weather data from, you know, close to where, where she lived. And that was the easy part. And then and then for Pietro is actually sitting down and saying, Okay, well, this is where he is, he's not in his Red Bull. The thing that I was worried about was like, Well, you know, if he sort of skirts around the side, is it possible that I'll miss something. But what I learned very quickly is that tortoises don't move very fast. So I was able to kind of get a sense of like, okay, well, well, you know, he's sort of headed towards the red ball, I discovered very quickly that I could kind of speed it up four times on the video and get through the videos a little bit faster. So the 1000 hours of video is true, but there was a lot of fast forwarding involved. And, and also, I mean, that was the other thing, too, it's like, sort of biologically, he wants to go under the red ball, because he thinks it's gonna rain, he wants to protect himself. And so if it snows for an extended period of time, he'll just stay under there. So it was easy enough for me to kind of skip ahead and see like, Okay, well, nothing has changed. So yeah, it's not quite as daunting as perhaps it sounds. But it was definitely interesting to kind of get a sense of what, uh, what his behaviors were like.
Rosemary Pennington
You're listening to Stats and Stories. And today we're talking with Conner Jackson, of the Colorado School of Public Health, about Pietro, the weather tortoise.
John Bailer
So I'm still trying to picture the comment that Rosemary made of thinking about this tortoise in front of a green screen doing the weather. I'm thinking that's gonna be a really slow segment.
Conner Jackson
Yes, absolutely.
John Bailer
So you're going through, you know, 250 hours at speed, you know, or the 1000 hours plus? And, you know, I did love that. I think that the writing was really nice throughout this piece. And I yeah, I love that, you know, they may not move quickly enough to trigger motion activated cameras. Yeah, that's, yeah, I was chuckling as I was reading through it. So congrats. Kudos on a job well done. And on bringing a smile as I was going through it, you know, so can you talk a little bit about some of the endpoints. So you've, you kind of told us the, you know, the 80, this 84% was kind of this, this value that you kind of narrowed in on as the target for success? Okay. Can you do the big reveal? Now, Conner? You know, how? And then we can get, as part of that reveal? We can maybe dig in a little bit on how you came to it?
Conner Jackson
Of course, yeah. So yeah, the question that I always get when I'm telling the story about Pietro is, well, how did he do? What, of course, is He? Is he good? And what I tell people this Yeah, he was actually pretty good. So we landed on an accuracy of about 70%. Which, again, is better than a coin flip. And essentially, what we do is we kind of just establish a confidence interval around our accuracy measurement that we had come up with, and I think the average that we got was like, 84%. So he was just under that kind of upper limit for our confidence interval that we made for our accuracy prediction. And so what I say is that he was not statistically significant from, from our kind of accuracy, baseline that we had set up from the beginning. And then the nice thing about this kind of a setup like this, right, is we have accuracies. But we can set up a very nice two by two table and look at things like sensitivity and specificity. And so that was kind of an obvious next step for us to actually start to unpack a little bit more about what our answers mean. I mean, I think, with accuracy, it's sort of like a big picture, like, okay, overall, like, how good was he predicting when it would rain? How good is he predicting, you know, what it would rain, but when you can actually kind of take that two by two matrix or the confusion matrix. It's, you get a little bit more information. And so I think that's where it got interesting, where we could actually look at the sensitivity and the specificity and essentially, what we find is that he was very, very good at predicting when, you know, essentially, we determined that his sensitivity was equal to exactly one. So essentially, what that means is that there were no days in which he predicted no rain, and a raid anyway. And so I think that, what we were kind of able to figure out is like, you can kind of use Pietro as a barometer for how you're predicting your barbecue, or how to decide whether or not you want to cancel your barbecue. Because he's very good at predicting the rain. If he's kind of out and about in his cage, you can say with some confidence, at least 100% of the time for our data. It probably won't rain for your barbecue, which is awesome. But what we found out was sort of on the opposite side, oftentimes, he would be under his red ball, and it didn't rain, right? So that would be essentially a prediction in which you might want to cancel your barbecue because you think it might rain, but it would have been perfectly fine. So it kind of goes back to that kind of sensitivity and specificity trade off question.
Rosemary Pennington
I love the title of your article, Pietro, the weather tortoise and the pursuit of soggy bond prevention. It's really cute in sort of, you know, intention capturing and then you have as the last line, where you're talking about how you know, the phrase sun's out guns out doesn't really appeal to you. And then you write when PAH row says, Hi, your buns will be dry, which was just a delightful little turn of phrase to and I'm a journalism professor. And so my husband mocks me mercilessly because once upon a time, I was writing about a food bank in West Virginia. I feel ashamed saying this, but the lead for my story was like Old Mother Hubbard, West Virginia's cupboards are bare, because I couldn't figure out a lead. And I was like, just wrote it and just threw it away. And the assistant news director left it in the new story. And so it's what my husband makes fun of me for constantly. So I appreciate like little, you know, cute turns of phrase. And I wonder how you approach the writing of this? Right? So you're writing about this weather tortoise, you talk about the stats that went into it. And I thought the discussion of how you figured out accuracy was really useful. And I can imagine this being a really useful article in a stats class or even, you know, a data journalism class where you talk through, like, how do we understand accuracy when we're doing our work? And so I wonder how you approached the sort of tone and the flavor you were going to give this as you were writing it up?
Conner Jackson
Yeah, absolutely. So I think that the title took a few days for me to kind of figure it out. And I think that something that I at least tried to approach writing with is to not take it too seriously. I, one of the things that, especially with and I call it the confusion matrix, right, like a lot of status sessions will know what I'm talking about. That sort of two by two table, thinking about concepts of like true negatives, type one error re or, or never intuitive to me. And I think that approaching it in a way that sort of fun, hopefully will help kind of people interact with it. And even today, I think about okay, well, it's true negative what Pietro was doing, just because for me, it helps me contextualize those things. And so we've actually started incorporating that article into some curriculum at our school, which is really cool. But yeah, I mean, as far as to kind of answer your question, I think that for me, the guiding principle was to not take it too seriously. I felt like with the pandemic, things felt really heavy. And understandably so. And I knew that for the essay contest, a lot of people would want to talk about COVID, because it was such a salient part of our day to day lives. But for me, I just think that the way I would describe it is that I take what I do very seriously, but I don't take myself very seriously. And that was kind of the sentiment that I wanted to use for the article. So I think sort of the way that it developed, at least for the title was, I said, I don't know if either you're familiar with Oh, Brother, Where Art Thou the Coen Brothers movie. So in that movie, there's the soggy bottom boys, the name of the band that they're in, and actually my best friend and I won our high school talent competition singing that song. And so that name has always been in the back of my mind. And so is this sort of kind of iterative approach, right? Like I knew that if I wanted to do well, in this competition, I needed to have like an eye catching kind of title. Right. I wanted to have something that was felt so sort of novels sort of interesting. And so I think I probably I got to soggy buns first. And then from there, I was like, Okay, well, how do I want to think about like, something like preventing, I was trying to figure out like the best like verbs to use, and I wanted somewhat of an alliteration between, I ended up with pursuit and prevention. And my wife and I, we went out to breakfast, and I was just like, I was probably not a very good partner, because I was just like, all I could think about was just like, okay, like, how do I want to phrase this, and we ended up, I remember that. The second I came up with the title, we were in the car, we were leaving the restaurant. And it's like, this is it, this isn't gonna be good. So So yeah, I think that, as I was writing it, I wanted that kind of light hearted feel to it, just because the absurdity of writing an essay about a weather predicting tortoise is, is absurd. And so I wanted to kind of lean into that a little bit. As far as the last line. I don't know, I don't really remember how I came up with it. I just I, you know, it's sort of, as you're saying, right, like, as statisticians we sort of step through a problem, we we start by, okay, well, what is our problem? And then we think about what a reasonable approach might be to answering that problem, and then you get your solution. And then and then you're like, Okay, well, how do I translate this into something interesting? And so when I figured out like, Okay, well, the, the storyline here is, yes, he's somewhat accurate, but the bigger storyline is, he's very, very good at predicting, you know, when it's actually like, when you should be okay with your barbecue, when you'd be likely okay to have your barbecue, because he's out and about in his cage. And so I just, I sort of, like really kind of meditated on that for a while. And it just, it just struck me so. So yeah, I think that allowing, at least for this context, right, like I, you know, I'm writing a few manuscripts now, I don't exactly approach it in the same way. But it was, it was fun to kind of just be silly. And, you know, acknowledge the absurdity of it all.
John Bailer
You know, I really like the use of this and explaining these ideas that are essentially conditional probability. And what are you conditioning on and all of these concepts that that many struggle with? I mean, and, and I, I find that that I could, that this is the kind of piece that I'll recommend to some of our intro classes to consider when they're talking about, you know, this conditional probability stuff. And so I, I congratulate you on that. I think that's really a neat, neat resource. The other thing that you said that I think is a lesson that's a good takeaway, is that we think a lot about crafting analyses and describing what we do with care and to engage. But I think that the complement to that is also crafting the way we write about what we do, you know, that there's this, there is this, this, this incredibly important accompaniment to the analyses that are done. And if you do things, just just this nice kind of struggle and focus on trying to craft the words that that really can win the day. I mean, but not just not just awards and significance for this kind of writing, but, but also just in terms of engaging readers and even a more complicated story that a research paper might involve. So I'm just curious, have you seen this type of process that you've gone through start to influence your writing now, and some of the ways that you, you do some of your, your other scientific work?
Conner Jackson
I think that now I have a much greater appreciation for the big picture. I think that now thinking about and I've always, I've always been really passionate about teaching statistics and making science accessible in particular. And so I think that's always been a common through line for me. But really, a lot of what this has taught me is just really about being confident about what I have to share with people. And I think that it also is an important illustration that you shouldn't be afraid to take a risk. Ray and I think that perhaps making an effort to flip something upside down or to think about it in an abstract way, is perhaps maybe worth spending a little bit more time on. So I wish I had more specific examples, but I do think that what this has taught me is one thinking about the bigger picture is very important, but also being willing to kind of step back and approach problems in sort of an abstract way, I think has been an important lesson. I'm actually working on another essay for significance now. Because now I've got the writing bug. So maybe that's been pretty influential as well.
John Bailer
Well, I have to tell you, you know, you certainly see disclosure statements and all sorts of things you read. But this may well be the first time that I've seen that, that you know, i Turtle a tortoise pardon me. Pardon the tortoise. Sorry, Pedro, have just, you know, disclose that there's no competing interest. But I think that the request to kind of solicit other funding, say from the kale lovers of America, I thought that was kind of almost crossing a line. So I just didn't know about that. That just commercialism that pH or was was engaged in, you know, I'm just not sure that that is fair.
Conner Jackson
You know, I've heard from Lissa, the coworker who owns Pietro, it sounds like this has really gone to his head. It's, it sounds like because, you know, he has an Instagram page. And, you know, it sounds like, you know, she's saying he's demanding only the best vegetable. So that's actually how I came up with the comment about, you know, Pietro would love to be a part of the kale lovers of America, because she was like, Well, now, Pietro is demanding this, you know, the best kale that I have, and all this stuff, and he's being really picky. So that was sort of how that ended up there. But yeah, it's funny how, you know, with this disclosure statement, I was thinking, Well, okay, well, I don't really have necessarily anything to add here. So I might as well, you know, that's something sort of one less piece to it.
Rosemary Pennington
Well, that's all the time we have for this episode of Stats and Stories. Conner, it's been a real joy talking to you today about Pietro.
John Bailer Thanks, Conner.
Conner Jackson Of course. Thank you.
Rosemary Pennington
Stats and Stories is a partnership between Miami University’s Departments of Statistics, and Media, Journalism and Film, and the American Statistical Association. You can follow us on Twitter, Apple podcasts, or other places you can find podcasts. If you’d like to share your thoughts on the program send your email to statsandstories@miamioh.edu or check us out at statsandstories.net, and be sure to listen for future editions of Stats and Stories, where we discuss the statistics behind the stories and the stories behind the statistics.