Ashley Mullan is a PhD student and research assistant in Vanderbilt University's Department of biostatistics. Currently, Mullan works on a team focusing on the care children receive in Tennessee's child welfare and juvenile justice systems. She's also interested in pop culture, and in her spare time, analyzes her own consumption of popular media. That led Mullan to author a Significance article on The Kiss Effect, the impact of a "will they won't they?" couple's first kiss on a TV show's ratings.
Episode Description
Television is filled with "will they or won't they" couples. Friends had Ross and Rachel. Parks and Rec had Leslie and Ben. The Gilmore Girls, had Lorelei and Luke. But what happens after the couple's kiss? Do we keep watching? One statistician dug into the data behind the kiss effect, and that's the focus of this episode of Stats and Stories with guest Ashley Mullan.
Timestamps
What Is the Kiss Effect & How the Project Started (1:45)
Data-Set Deep Dive (5:41)
When Do Kisses Happen? Decades & Timing (7:14)
The Curious Case of the 1990s (8:58)
Genres, Nerdy Shows & Dataset Variety (10:30)
Anticipation, Narrative Satisfaction & the Zeigarnik Effect (13:12)
Genre, Story Structure & Why Classification Is Hard (15:14)
Beyond the Kiss: Marriage and Other Milestones (16:58)
Applying the Method to Taylor Swift & Media Events (19:26)
Transcript
Rosemary Pennington
Television is filled with will-they-or-won’t-they couples. Friends had Ross and Rachel. Parks and Rec had Leslie and Ben. The Gilmore Girls had Lorelai and Luke. But what happens after the couple’s kiss? Do we keep watching? One statistician dug into the data behind the kiss effect, and that’s the focus of this episode of Stats and Stories, where we explore the statistics behind the stories and the stories behind the statistics. I’m Rosemary Pennington. Stats and Stories is a production of the American Statistical Association in partnership with Miami University’s Departments of Statistics and Media, Journalism, and Film. Joining me is regular panelist John Bailer, emeritus professor of statistics at Miami University. Our guest today is Ashley Mullan, a PhD student and research assistant in Vanderbilt University’s Department of Biostatistics. Currently, Mullan works on a team focusing on the care children receive in Tennessee’s child welfare and juvenile justice systems. She’s also interested in pop culture and, in her spare time, analyzes her own consumption of popular media. That led Mullan to author a Significance article on the kiss effect—the impact of a “will-they-won’t-they” couple’s first kiss on a TV show’s ratings. Ashley, thank you so much for joining us today. Ashley Mullan: Thank you for having me. So glad to be here. I’m just going to ask you; how do you operationalize the kiss effect in your work? What is it, and why did you get interested in this thing?
Ashley Mullan
So, like most projects I do on the side, this did, in fact, start with an argument. I was a second-year master’s student in the Department of Statistics at Wake Forest University, and I was arguing with my advisor, who was also a fan of New Girl like me, about how when I was watching New Girl over COVID and I got to season two, episode 15—the last nine minutes when Nick and Jess first kissed on the screen—I watched probably two seasons in one sitting immediately after that. Oh gosh, I’m excited. And she looks at me like I’m a little bit crazy and goes, “Are you kidding? That’s when you people get bored.” And we’re arguing about this, and we’re sitting there. I’m like, wait a second—we’re statisticians. We can do something about this. Fast forward to a couple of months later. Actually, funny story: one of us had COVID and had nothing better to do than sit there and collect TV ratings from IMDb. And we’re sitting there, and we’re just collecting TV shows, and we’re googling the internet: “will-they-won’t-they” couples? And we had just a random list of BuzzFeed lists of couples that just show up in pop culture. Side note, this is not something that’s new. This shows up as early as Pride and Prejudice. I think it came out in 1813, and it’s still going on. I just watched People We Meet on Vacation a couple of weeks ago, and that’s another classic example of this—the adaptation from the Emily Henry book. But we’re sitting there and we’re collecting data and collecting data, and we’re like, okay, this is a great chance to test out a new method that I’ve been kind of learning about for another project called interrupted time series, where basically what you look at is you have a measurement that you take over a period of time, and there’s an event sometime in the middle that you look at: do trends of some measurement over time directly change as a result of that event? So, you’ve got the before period and the after period, and you can kind of look at what is the generic trend looking like before the event. You can look at what is going on afterward. You can quantify the level of the jump in that measurement, and you can also quantify the change to the trend. And we thought this is exactly a perfect kind of setup to look at this kind of thing, so we decided to put together ratings from a bunch of these couples. We were also tracking how popular the couples that we were looking at were by mentions in all of these BuzzFeed articles. And we sit there. We tabulate, we rank them all up, and we go, okay, anybody that lands in the top 20 can go into our statistical model. And because I’m nosy and I like certain shows from that list, we ended up doing the top 20. I’d seen probably about five or six of those shows. We said, okay, let’s fit this with both a couple in a show like Nick and Jess from New Girl. And just in general, we aggregate all these shows together and take a look at that effect. And spoiler alert, I was actually kind of really disappointed. The optimist in me was like, okay, everybody else is going to think like me, and let’s kind of binge the show immediately after the couple kiss on screen. But it kind of didn’t work out that way, and I was a little disappointed. But stats don’t lie.
John Bailer
Yeah, for the longtime listeners to the show, I’d like people to note that Rosemary just asked three questions the first time, and she gives me such grief about asking two questions sometimes. Rosemary Pennington
But I just want that to be noted. I was just very excited about this episode! John Bailer
Well, I just get very excited too, Rosemary. So, this was a fun article to read. I’d like to know a little bit more about the data set. Because, you know, I think for me, one of the things I kept doing was considering shows that I really liked and followed. So, The Big Bang Theory, The Mentalist, you know, Bones—all of these were shows that had kind of a buildup to a relationship, some with very different patterns of what happened afterward. You said IMDb was where you got information about each of the series, the shows you were following, and the seasons of those series. You also extracted data from Wikipedia as well. Is that right?
Ashley Mullan
So, there’s a couple of different measurements that we track for each show. The main one that we’re looking at—the outcome in the analysis—is IMDb ratings, so you can get those on an episode level, which we hand compiled for the entire run of those shows in the top 20 couples, all the way through 2023, which was when we started this project. Some shows were still running, like Grey’s Anatomy in the list, and I think they’re still running. But one thing that we did with that is we looked at where the show was on its run at that point to see where the kiss landed. Because it’s so not fair to compare something like Grey’s Anatomy to another one that was on the list that I’d seen, which was Shadowhunters, where it was three seasons versus Grey’s Anatomy, 20 and counting. I would look at where the kiss was relative to the run of the show, rather than, “Oh, it’s in season four,” versus season four of a show that only had five seasons, if that makes sense. And the other thing that we grabbed from Wikipedia was when did the show start, to see if it’s shows that started around the same era, kind of following a similar pattern. And the ones that ended up in the analysis—we had a couple from the ’80s, a couple from the ’90s, a bunch starting in the aughts, and a bunch starting in the 2010s. And we specifically defined it to be when the show started. Because some, like take your Grey’s Anatomy, for example, run for 20 years and just cross time jumps. And what was pretty interesting was that when we bracketed them by decade of the show starting, we saw where the kisses were coming was pretty consistent. The median kiss time was when there was over two-thirds of the show left to go for shows in the ’80s, the aughts, and the 2010s. But that wasn’t true in the ’90s. So, I don’t know if it was known on that, but it was kind of interesting what I found when I was kind of taking a dive into the data there.
Rosemary Pennington
What happened? As a child of the ’90s and someone who consumed many of those ’90s shows—what was different about the ’90s?
Ashley Mullan
I don’t know. To be fair, I’d said I’d seen maybe five or six of the 20 shows that I analyzed. Most of the ones that I’d seen were starting in kind of the “aughts” or the 2010s. So, I’m not as familiar with the whole plot line to the degree that I’m familiar with New Girl, and I can tell you what episode and what minute that kiss happens. That just means I have things to add to my watch list.
John Bailer
So, one example—in looking at one of the graphs that you had—is what percentage of the series remained after the kiss occurred in this “will-they-won’t-they” story. And, you know, you had sort of the general takeaway. One of the outcomes was that most shows had about plus or minus 75% of the series remaining after that first kiss. But there were a couple. There was one in the ’90s that had only like 20% or less of the series remaining. I was curious, do you know what that outlier was?
Ashley Mullan
I didn’t check on that one. I was actually more interested in the ones that had them early and then another one later. So, the one that popped up that I noticed in the data set was New Girl again, because we have Nick and Jess and we have Cece and Schmidt. And those two kisses are in the same show, but they’re different storylines, so they’re popping up in different kinds of senses, if that makes sense.
John Bailer
So what genres are represented in your data set? I’ve already confessed to some of my loves of shows like The Big Bang Theory and The Mentalist. Bones is sort of different—those are two very different genres. I will confess that New Girl’s not on my radar. So, I don’t know. You like nerdy science shows. I like nerdy science shows. But The Mentalist and Bones are—well, I guess they’re—well, okay, yeah. But, you know, what’s the collection of genres that you had represented in this data set?
Ashley Mullan
So, we’ve got a bit of variety. I’m personally a sitcom, rom-com type of person. I love The Big Bang Theory. I’ve got a little Lego set or bobblehead next to my TV that just sits there, whatever you’re watching. We’ve got lots of rom-coms. You’ve got sitcoms. The Mentalist shows up in there, which I would, I guess, categorize as more of a procedural-type show. And then we’ve got a couple of those sci-fi ones in there too, as well. So, we span a wide variety. If romance shows up in the show, it’s probably on the internet and probably made its way into our data set in some way. And for folks that are interested in trying this out at home, we actually do have a Figshare where we make all this data publicly available. If you want to poke around and take a look at some of those outliers yourself, if you’ve seen them.
Rosemary Pennington
When you were answering a question earlier, you said you were somewhat disappointed after you had crunched the numbers.
Ashley Mullan
I was badly disappointed because I wanted other people to be like me, and when they saw that, they just watched the rest of the show. So, I felt less like it was a “me” thing and more of an “everybody” thing. But I was surprised, because we fit models on both an individual-show level. So, like, I fit a New Girl model, because that’s what I talk about. My phone lock screen has pictures of Mac Miller in it. And we fit one to Friends, because I haven’t seen it all the way through, but I’ve seen enough of it to know exactly what’s going on there as well. And then we switched them all together, and it looks kind of different. Those were three pretty good case studies to look at. So, when we looked at New Girl, it looked—in the pictures in the article, you can see—it looks like a dramatic drop once that kiss happens in season two, but it’s actually much less of an effect than you would think. It looks like a drop, but when you account for the error there that would be a confidence interval for all the stats fans on here—it crosses zero, which means that you can’t be sure that the effect that you’re seeing is really there. And in Friends, the trend levels out a little bit, but again, that confidence interval is all over the place. But the more data you get, the better of an answer you can get to a question. So, once you zoom all the way out to look at all the top 20 couples together, we start to see a little bit of things making more sense, but the effect gets smaller. We’re more sure about it, but there’s less of it. So, if it is there, it’s tiny, and we’re not quite so sure of the small amount of data that we can get on each show.
Rosemary Pennington
So, it was interesting. As I was reading this, I was thinking about—this is about a film—but the 2005 version of Pride and Prejudice, right? Which is a beautiful film for those of you who have not seen it. But there was a bit of controversy because, at the original ending of the film, it ends on—and I’m not—I mean, Pride and Prejudice is very old. I hope I’m not spoiling this for people—but it ends on Elizabeth Bennet leaving her father’s study, having gotten permission to marry Darcy. And that’s where it ends, right? We don’t ever see—and they never really kiss in the film, right? And then they had to add a new ending for the American version of the film that included Darcy and Elizabeth kissing at Pemberley, because test audiences found it unsatisfying that they never kissed. And so, as I was reading through this, it was interesting for me to think about the anticipation that is so important in these “will-they-won’t-they” couples, but how you really need that kiss to be able to feel like that narrative is complete. But it’s interesting to see that maybe there was a decline post-kiss, but it certainly didn’t increase people’s interest in the couple post-kiss. It’s sort of suggested by your data that it maybe didn’t have much of an effect, potentially.
Ashley Mullan
So, to be super clear, I’m the one that gets excited and I watch after the kiss. The data is saying, “Eh, maybe that’s just a me thing,” which was kind of sad. But you’re absolutely on the ball to think about this anticipation thing. I was thinking about, okay, why am I like this? And I started reading from our friends in psychology, and I came across something called the Zeigarnik effect, which I kind of summarized as people tend to remember unfinished things better than finished things. So, in terms of the kiss effect and sitcoms and rom-coms, what I take that as is people are going to be more excited about the part of the show pre-kiss, because they’re there for the buildup rather than what comes next. I’m personally there for what comes next. But it sounds like, after my analysis, that it might just be a me thing.
Rosemary Pennington
You’re listening to Stats and Stories, and we’re talking about the kiss effect with Ashley Mullan.
John Bailer
Yeah, there’s part of me that would love to see the effects of the different genres on this trajectory, because I wonder if there’s another component—other dimensions of story—that might confound this. So, for example, the Bones story—someone who’s a forensic pathologist and an FBI agent are the leads in this—the fact that they’re still solving mysteries throughout, and that the relationship was kind of in parallel, maybe the pattern of behavior wouldn’t change that much in terms of the audience watching. Maybe that wouldn’t be as impactful. What do you think? Is that crazy?
Ashley Mullan
I love thinking about genre as a way to think about what’s going to happen because of the genre. There was a project I worked on a couple years ago where I was trying to use genre to learn about more metadata-type things about a song. And the thing that I ran into there, which I think would transfer over here in the TV world as well, is that it’s so hard to define a genre. So, for example, if we’re talking The Big Bang Theory, you can think, okay, that’s a nerdy science show. It’s also a sitcom. What bucket are you going to throw it in? And I think it even depends on the part of the show. You introduce Amy Farrah Fowler, then you’ve got that whole line that goes on. And then you start in the beginning, and it’s just the boys doing physics, which is a totally different representation of the genre. So, I’m not sure how I’d even begin to categorize there, to be fair.
Rosemary Pennington
You know, one other thing that I was thinking about, which is sort of related to the idea of a kiss effect, is that there’s a lot of buildup in these shows, particularly in rom-coms, of the “will-they-won’t-they”. Then they get married, and then what happens after they get married? And as I was reading this, I was wondering: would there be potentially a real drop-off after they’re married? Like, even after a first kiss, there’s still the possibility that they won’t stick together, and there’s a little bit of drama around the narrative. But once they’re married, that feels like sort of the kiss of death to the “will-they-won’t-they” thing. And I wonder what that might potentially do to viewership down the road.
Ashley Mullan
It’s so funny you say that because we were thinking about how to collect data to answer this question. We were arguing a bit back and forth about what metric we want to use to define a couple. And we have to be consistent across the entire data set and use the same thing for every show. The kiss one was common enough and easy enough to define that you could say, okay, this happens. I’ll take New Girl again—season two, episode 15, last nine minutes—easy to document. Marriage would be another fun one to talk about. I’d have to make sure that we have enough shows where the characters get married to have enough to throw it into the model there. But there’s probably a million different ways that you can think about how to measure “will-they-won’t-they”.
John Bailer
I was curious about the shows that aren’t like this, and sort of what would happen if I looked at some other series that didn’t have this tension. Does it have stronger showings up through about 25% of the way through and then have kind of a decline? What’s the life course of most series, including those that are not these “will-they-won’t-they” tension-component series?
Ashley Mullan
I was thinking about, if I did this again, I might want to try it with books. The limitation—I was actually going back and forth about this when we were writing the article—was that we did this analysis at one point in time, and some of the shows were still running. So, take Grey’s Anatomy. It’s still going. And you get more and more data whenever you do the analysis. The nice thing about a book is that when it’s published, it’s finished—you know the last page of the book. And you could say, oh, the kiss happens on page 127 out of 500. You give it a little bit more standardized metric. And I’m curious if that would do something similar as well. But that just means I’ve got to go get some more data.
John Bailer
As I said, the books are finished, unless they’re one of these long-running series like George Martin. We’re still waiting for the end of some of those.
Ashley Mullan
I’d have to go for standalones, probably.
John Bailer
Do you have any examples of shows that differed from the general average pattern—where there was kind of an uptick, like you were hoping to see? Did you do any exploration of individual series and say, yeah, well, this was the average trend of the decline post-first kiss—were there any that had kind of a little bit of that takeoff?
Ashley Mullan
Only two that I looked at just on their own, and that was New Girl and Friends. And I noticed that the point estimate—or that specific one that you get from the data without accounting for wiggle room in error in ratings—was a little bit more dramatic in Friends than New Girl. But also, Friends had a little bit more to work with than New Girl. So, it’s always a function of how much data you have and how big that trend is going to be. The longer a show runs, I probably should have done Grey’s Anatomy. I keep going back to that one just because it’s got a monster of episodes to work with. The more you get, the more you can get a better idea of what’s going on there with those ratings trends, because they start to stabilize a little bit more as there are more episodes to work with.
Rosemary Pennington
If you had a chance to talk with someone who was developing a show with a “will-they-won’t-they”, what advice would you give them based on the research you did about when is the right time to have the will-they happen?
Ashley Mullan
That depends. Am I making the show for myself, or am I making the show based on data? Because if it was just for me, then I’d tell them, let’s go early and let’s see the plot line afterward.
Rosemary Pennington
If it was going to be appealing to a general audience based on the trends you saw in your data.
Ashley Mullan
Unfortunately, I do have to go with the masses here. Granted, the effects were not super well defined, just because those estimates of how much wiggle room we have were all over the place. But unfortunately, I would tell them to hold off and leave it toward the end as a series kind of finale, almost.
John Bailer
So, I’m curious, as you’ve done this and explored this question, what might be next for you? What if you were going to do additional analyses with these data or do some further analyses with other data? Like you mentioned books—what would be next?
Ashley Mullan
The reason I kind of like this analysis is because it’s pretty simple to implement code-wise. And if you can get data that lends itself to that kind of problem, you can run it through a similar programming script and get a similar answer. I did another project kind of like this, where Taylor Swift just dropped an album a couple of months ago, The Life of a Show Girl. And part of her marketing scheme was that she went on her then-boyfriend, now-fiancé’s podcast to talk about it. And I did an analysis of whether Google search trends of this new album were going up or down after this podcast appearance. And that was a pretty cool thing to look at, too. It was, again, less pronounced than I thought it would have been. But we definitely saw a spike right around then, and that was really cool to look at. So going on podcasts works, apparently.
Rosemary Pennington
So, you mentioned this work on the Taylor Swift album. Are there other pop culture things that you’re thinking about exploring or excited to explore?
Ashley Mullan
Statistically, those were the two I was super excited about at the moment. And now that I’ve been getting the chance to talk a little bit more about this article, I really kind of want to go compile some more book data. But I love doing just kind of little one-off projects on pop culture things as I see them. For example, I’m a big fan of the LinkedIn games. I now have a spreadsheet tracking some of my friends’ performance on these games—who’s winning and who gets ultimate bragging rights for which game. So that’s pretty fun. I have it all set to run in a pipeline. So, I click one button, and it tells me, am I winning yet?
John Bailer
So, what’s next for you in terms of your profession? I mean, certainly it’s not all New Girl all the time. So, you’re in the midst of lots of interesting research projects. What kind of stuff are you working on now as part of your work?
Ashley Mullan
So, you all got to talk a little bit about what I do now as a member of the analytics team with Vanderbilt, talking about how we work with our partners in the Department of Children’s Services and government. We get to look at some of their data and help them see trends about what’s going on with kids in our area to help them target resources to better serve them. We look at patterns of kids coming in and out of custody and other things like that and say, okay, here is where we’re seeing patterns of events happening with kids in similar geographic regions. This is where you might want to target your resources and other things like that. And we help make a lot of visuals and tables for them to have as a reference when they’re making policy about what’s going on in their region.
Rosemary Pennington
Well, that’s all the time we have for this episode of Stats and Stories. Ashley, thank you so much for joining us today!
Ashley Mullan
Thank you for having me. This has been so much fun.
Rosemary Pennington
Stats and Stories is a partnership between the American Statistical Association and Miami University’s Departments of Statistics and Media, Journalism, and Film. You can follow us on Spotify, Apple Podcasts, or other places where you find podcasts. If you’d like to share your thoughts on the program, send your email to statstories@amstat.org or check us out at statsandstories.net. Be sure to listen for future editions of Stats and Stories, where we discuss the statistics behind the stories and the stories behind the statistics.