Chris Dalla Riva is an analyst for the music streaming service Audiomack by day while spending his nights writing and recording music and writing about music for his newsletter Can’t Get Much Higher.
Episode Description
The Billboard Hot 100 has been ranking the week's most popular music since 1958. The first song to top the chart was Ricky Nelson's Poor Little Fool. The most recent song to do so is Taylor Swift's The Fate of Ophelia. A lot has changed in the music industry between those two songs, not only in the types of songs that top the charts, but also in how they're promoted and how they're determined. A new book explores the statistics behind the Hot 100, and it's the focus of this episode of Stats + Stories with guest Chris Dalla Riva.
Timestamps
History and methodology behind the Hot 100 and Chris's book (3:09)
Defining musical eras and chapter methodology (5:37)
Rating process and role of different raters (8:51)
Death narratives in early chart-topping songs (10:24)
Subjectivity in rating music; notable song highlights and lowlights (14:02)
Data visualization: industry revenue and sonic similarity trends (17:59)
Changes in Hot 100 metrics: sales to streaming (22:12)
Demographic shifts: gender, race, and representation (24:27)
Musical truths from analyzing pop charts (27:14)
Reflections on music evolution and recurring trends (29:25)
Full Transcript
Rosemary Pennington
The Billboard Hot 100 has been ranking the week's most popular music since 1958. The first song to top the chart was Ricky Nelson's Poor Little Fool. The most recent song to do so is Taylor Swift's The Fate of Ophelia. A lot has changed in the music industry between those two songs, not only in the types of songs that top the charts, but also in how they're promoted and how they're determined.
A new book explores the statistics behind the Hot 100, and it's the focus of this episode of Stats + Stories, where we explore the statistics behind the stories and the stories behind the statistics. I'm Rosemary Pennington. Stats + Stories is a production of the American Statistical Association in partnership with Miami University's departments of statistics and media, journalism, and film.
Joining me, as always, is regular panelist John Bailer, emeritus professor of statistics at Miami University. Our guest today is Chris Dalla Riva. Dalla Riva is a senior product manager at Audiomack, where he focuses on data analytics and personalization. He also writes the newsletter Can’t Get Much Higher that explores the intersection of stats and music.
Dalla Riva’s work has been featured by outlets such as The Economist, Business Insider, NPR, as well as Stats + Stories. He's back today to talk about his new book, Uncharted Territory: What the Numbers Tell Us About the Biggest Hit Songs and Ourselves. Chris, welcome back to the show.
Chris Dalla Riva
Thanks for having me. I'm excited to get into it again today.
Rosemary Pennington
What made you decide to write the book?
Chris Dalla Riva
I didn’t set out to write a book. It was a weird, circuitous journey where, out of college, I was working in economic consulting, so I was already in the stats world, but I've always been involved in music. I've always played in bands. In high school, I'd write and record songs and make people suffer through them by burning them to CDs.
At the time, like I said, I was working in consulting, and I just came up with this scheme or this idea to listen to every number one hit for really no reason in particular. I was just like, “Oh, you know, it'll be nice at the end of the day. I'll sit down and listen to one #1 song. Maybe my songwriting will become better.”
But because I have some data experience—I was working in a data field at the time, and I eventually started working in the music industry with data—as I would go along, I would track tons of facts and figures about these songs. I started noticing some patterns, and I felt compelled to write about them.
And then there's a long, weird journey about how the book actually got published. But yeah, that's how it started. I was just listening to these songs for fun, and because I have a data side, I started collecting information, noticed some trends, and wanted to write about them.
John Bailer
So how many songs did you end up listening to, and how many years did this span?
Chris Dalla Riva
Yeah, so Billboard has been around since the late 1800s, and it actually, as the name suggests, started out as a magazine that would track bill postings or what was being advertised on literal billboards.
In the early 1900s, they started getting more into covering celebrity gossip. And then, I think by the 1930s, it was pretty exclusively focused on music, and they would occasionally roll out charts of what was popular, because ultimately, it became an industry publication. So, you know, you think people working in the music industry want to know what's selling.
They had a bunch of different charts throughout the 1950s, and then in 1958 they launched the Hot 100. The goal was to combine all sources of listening. So back then it was sales, radio, and jukeboxes to try to figure out what was the true most popular song in a given week in America.
The first number one hit was Ricky Nelson's Poor Little Fool on August 4, 1958, and there has been a new chart published every week. Most songs are number one for a couple of weeks. So, there have been, I think at this point, about 1,200 number one hits. I, of course, had to stop listening at some point to publish the book, though it gets close to the present.
So I was a little bit under 1,200, and I was doing one song a day. If you do the math, it takes over three years to get through all of those songs. I was pretty consistent. I'm sure I missed a day here and there, but it took a little over three years of listening, and then actually getting it published took many more years.
John Bailer
Well, first I want to say how much I've enjoyed reading your book. It's been a lot of fun. I like even the way that, you know, you've framed this. It seems—so the way that it looks like—you have this history, culture, technology component, then highlights, lowlights, argument stoppers, odds and ends, and then “everything else.”
So you package that in each of the chapters, and the chapters were eras. How did you decide on eras, and then, ultimately, what variables did you have to add to be able to think about highlights, lowlights, and argument starters?
Chris Dalla Riva
I'll start with the eras question, where I just call each chapter an era of music. I talk about the arbitrary nature to the way I define these eras because I was ultimately working on a book. I would stop listening and start writing, or I’d define the end of an era when I thought that I had enough material to write about.
So there was a utilitarian nature to how I was defining eras. But something I talk about in the book is that we often think of music as decades—like ’60s music or ’70s music. But of course, history doesn't stop and start on January 1 at the beginning of a new decade.
So I didn't want to adhere to strict timelines like that. I wanted to have flexibility to be like, “All right, there are musical trends that started in 1968 and ran through 1974,” or what have you. So I was really using my ears there and the idea that I had to stop and write every once in a while and say, “Okay, this is the first ‘era’ that I'm going to write about.”
So there's some internal logic there. There is some arbitrariness, but I think no matter where you start and stop, there's always going to be an arbitrary nature.
As you note, at the end of every chapter—or at the end of each era—I would mark the best and worst songs from that era, along with argument starters, or things people disagreed upon, and some other songs that I wanted to highlight.
The way I did this was not super sophisticated. I had three people listen to every song, and I would make them rate it out of 10. I would average the ratings, and whatever was the highest, whatever was the lowest, would get highlighted at the end of each chapter. Whichever had the greatest variation in ratings I would call an “argument starter,” and I would talk about that.
The reason I did this was, first, I wanted to highlight that there is a subjective nature to what we think about music. But I think it's very important for us to try to quantify our feelings, because even if you don't believe that there is a true way to define what's great and what's not, everyone has opinions on which songs they like and which songs they don't. I think it's fun to talk about those things.
Often, even when I disagree with people, it makes the music come to life when you start talking about why you disagree or trying to understand what your feelings are. So I wanted to embed that into the end of each chapter, because it's a little fun, but also because that was initially how this data started.
Initially, I was tracking just how people I was asking were rating these songs, and then slowly it blew out into all these other things I was tracking. But because that was the beginning, I thought it was a fun thing to tack on to the end of each chapter. And I thought it sort of centered things where you knew you could go to the end of every chapter and see a couple of songs, and maybe listen to them if you were unfamiliar.
Rosemary Pennington
You mentioned in the book that two of the raters were consistent throughout this project, but then the third you were rotating through folks. Why did you decide to do that?
Chris Dalla Riva
Good question. The two consistent raters were me and a friend of mine who I always talk music with. I had mentioned that I was going to listen to all these songs—I was probably only a couple in—and he was like, “Oh, I'll do it with you.”
So it just started as a friendly thing. Kim and I were just listening to these songs. We'd text about them, we would rate them and track our ratings, but we have pretty similar taste. Of course, we would disagree about certain songs, but I just thought it would be nice to get a third perspective to try to balance things out a little more.
Of course, I could have surveyed 100 people if I had the time or resources, but after a certain point, I didn't want to make it seem like I was actually uncovering some platonic truth about these songs. At the end of the day, there is a subjective nature to doing this.
I tried to invite people for that third rating who were cognizant of what was going on at the time whichever songs we were listening to were popular. So, you know, people of my parents’ generation when I was going through the ’80s, some older cousins when I was going through the ’90s, some of my coworkers when I was going through the 2000s.
I think it added a little more depth and perspective than if it were just me and my friend who have somewhat similar taste. But yeah, that's sort of the reason I chose to go about it that way.
Rosemary Pennington
I have a question about, I think, the earliest narrative era that you discuss. So in each chapter, you talk about these narratives that often appear in some of the songs that are charting in your era.
I was struck because, when I was a kid, we would take these long car rides and my dad would lead us in sing-alongs—what else can you do with three little kids trapped in a car for four hours, right? And one of the songs we would sing was Tom Dooley. Every single car ride, we loved Tom Dooley. We would sing it every single time.
I was much older when I realized what a morbid, strange song that was to be something you sing together as a family.
In your discussion of that earliest era—that first era of the Hot 100—you note that there are all of these narratives around death that are showing up in these songs. What was going on?
Chris Dalla Riva
Yeah, that was actually what inspired me to start writing. I got through about 50 songs and thought, “Why are there so many songs about people dying?” This seems like a weird topic—for the top of the pop chart, it just doesn't seem like a popular topic.
I dwell on that in the first chapter, and I think there are a couple of things going on here. First, I think it's important to note that those songs, like Tom Dooley, are sort of old folk songs, and there is, going back hundreds of years, musical and literary traditions of these really gruesome narratives.
Think about Shakespeare—there's gruesome death and lovers dying. So in that sense, there is a continuation.
But I think there is something very specific to that era. Many of these songs—they call them teenage tragedy songs—where it's like two teens in love, and one dies tragically, often in a car crash.
When you think of this phenomenon around teenagers, it makes a little bit more sense, because at that time, teenagers were an emerging demographic, which is weird from our current perspective. Previously, it was really like you were a child and then you were an adult. Of course, people lived through ages 13 to 19, but that wasn't really thought of as a distinct demographic period.
With the rise of compulsory education and more money after World War II, you start getting teenagers: this group of people who are in school all day together, and culture generates around them.
It is a grim topic, but it is often focused on teens. So I think in that sense, that's connected to the rise of this demographic.
But there is a grimness in a lot of things that went on in the early part of the 20th century—multiple world wars, pandemics, the Great Depression—and at the same time, you saw lots of young artists and actors dying pretty young.
So I think it's a confluence of factors that come together to make this weird, depressing trend that, to our contemporary ears, seems odd. But, as you're saying, people would sing along to these songs, and I think that often also illustrates that for a lot of people, it's about melody and it's not as much about what you're actually singing.
There are multiple times, I'm sure you've experienced in your life, where you're singing along to a song and then you're like, “Huh, that's sort of a weird lyric.”
Rosemary Pennington
You're listening to Stats + Stories, and we're talking about the data behind some of your favorite—or at least some of the top hit—songs with Chris Dalla Riva.
John Bailer
Yeah, you know, as you're talking about this, I was thinking about my kids. My daughter loved “These Eyes” by The Guess Who—that was the song that really resonated. And in terms of the song “Beast of Burden,” you would think, “What's that?” Well, my son said, “Don't leave your pizza burning.”
So I think the lyrics didn't matter so much as the tune that was there.
Now, one thing I love about what you've provided as a resource is the data set of all of your ratings and songs. So naturally, I had to go in and use it and read it.
I went in, and one of the things I thought I would do is look at the number of times all three of the raters agreed, and which songs were kind of the stars of the lowlights and the stars of the highlights.
I thought that, you know, the ones that you all rated 10s—there were some really, really great ones there. I mean, “Me and Bobby McGee,” but also “Stayin’ Alive” were top there.
But there were only two that made it to the consistently hated end. One was “The Streak,” and the second one was “Itsy Bitsy Teenie Weenie Yellow Polka Dot Bikini.” Although I suspect it wasn’t far behind that you had “Torn Between Two Lovers” and “Alone Again (Naturally).”
I found that affirming. I thought, “Okay, I had some validation for my own biases.”
Can you talk about those where you joined? I mean, only 86 out of 1,177, which kind of reinforces your comment about how subjective it is appreciating music.
Chris Dalla Riva
The directions I would give people when I told them to rate songs were: listen to it at least twice, even if you're familiar with the song. Just be fair in your rating, in the sense that if you don't like Bon Jovi, don't just automatically give every number one hit by Bon Jovi a one. Try to be somewhat fair about what you're doing.
I think you see that with something like “Stayin’ Alive” by the Bee Gees. If you put it on—it's a song everyone's heard a gazillion times, to the point where it's basically impossible to rate some of these songs or to listen with fresh ears. But if you sit down and listen to something like that, the whole song works in a million different ways. It's got a great bass line. It's an infectious song. I think everyone really connected with that and was able to step back and be like, “All right, this is an incredible composition.”
In the same sense, we would occasionally get, on the other end of the spectrum, songs where we'd be like, “Wow, this is horrendous.”
I do feel like there was some bias on everyone's part against what I'll call novelty songs. “The Streak” is a good example, and I think “Itsy Bitsy Teenie Weenie Yellow Polka Dot Bikini,” in that they're sort of built around a joke of sorts. “The Streak” is about streaking, which was weirdly a trend in the 1970s.
So it's like, were we too harsh on something like that, when it was clearly made as a joke? Maybe. But I would never try to subject someone to listening to “The Streak” for pleasure.
Rosemary Pennington
What a strange way to put that.
John Bailer
You know, one thing that was a lot of fun in going through the book were all the figures you used to help describe some of the data patterns that you saw.
A couple that I thought were interesting—you also did the same chart with different eras, ending them a couple of times. Two times you did that: one was with the revenue of the music industry, and one was sonic similarity. They were the two cases where you ended up showing where the music industry was at one point, and then later on there was an uptick in some of the revenue that was being generated.
And with sonic similarity, there were other similar trends. Can you talk a little bit about the idea of music industry revenue over the eras that you observed?
Chris Dalla Riva
Yeah, that's one of the most fascinating charts. All the charts were made by my friend Kaylee Nerney—the book would not be the same without her. That data on revenue comes from the RIAA, which is the Recording Industry Association of America.
I love looking at that chart because it shows revenue by all different formats over the years. You can see the rise and fall of the cassette, the rise and fall of the CD, and the rise of streaming.
What's interesting is that from the 1980s up to around 2000, music industry revenues go through the roof, and it's mostly because of the CD. Basically, you had to rebuy your whole music collection if you owned it on cassette or vinyl as a CD.
This was one of the music industry's great tricks: every couple of years they would roll out a new format, and people would have to rebuy the same music over and over again. At the same time, CDs were very expensive—to the point where you were probably paying more for a single CD than you would pay for an entire month of your music streaming subscription now.
You multiply that out and you see how the industry was making so much money. At the same time, there was also some price fixing going on in the CD market—I mention this in passing—so prices were sort of illegally high, and there was a settlement in the early 2000s around this.
The industry was making tons and tons of money. This is the era of flying someone out on a private jet just to come meet someone at the label when you clearly did not need the private jet ride.
But right around 2000, as it reaches its all-time peak, revenues collapse. The reason for this is this company caerlled Napster rolled out, which, if you're of age, you'll certainly remember. Napster was a file-sharing service. You could effectively download music illegally, and naturally this became very, very popular.
Part of the reason I think it became popular was because CDs were so expensive. Back in the day, you could buy a single song, but in the CD era labels stopped doing that. So if you wanted one song by—I don't know why I always use this example—the Barenaked Ladies had a song in the late ’90s called “One Week” that was very popular. But if it wasn't released as a single, you had to buy the whole album just to get access to that song.
Suddenly, this company Napster comes along where it's like, “Oh, I could just grab a single song.” Of course, you're not paying, so that was a different issue in itself. But I think the music industry sort of backed itself into a corner here, where they were really taking advantage of listeners to some degree.
Revenues collapse. The industry tries to fight this by rolling out digital download stores like iTunes, but still, revenues collapse for about 10–15 years, and it only starts to recover when streaming rolls out. There’s tons of controversy around how artists are paid in the streaming world, but at the very least, with the rise of streaming, there is money to be made in music again in a way that there really wasn't in the 2000s.
I love that chart because you can see the history of the last 50 years in it, and it's nice to look at it over time.
Rosemary Pennington
That dovetails really neatly into the question that I was trying to ask before I got sidetracked, because you mentioned earlier how the Hot 100 was determined very early on, right?
But we're in this moment where the technology is all different. How we're using and consuming media is different. How we're buying it is different. I pay for an Apple Music subscription and buy albums for fun, to collect them—not the way I would have bought CDs in high school to have the music.
So how are they determining what the Hot 100 is now, given this media landscape?
Chris Dalla Riva
I think this is actually an interesting statistical question, because the Hot 100 is the same in name, but it's really not measuring the same thing as it was back in the day.
You were mostly measuring purchases. Radio was in there too, but for the most part you can think of it as: we're measuring sales data. What are people leaving their house and going to buy?
Music is still purchased today, as you just mentioned, and there's even been a resurgence of vinyl, but I think it's more not for listening and more as something to own. They do surveys of young people buying vinyl, and some of them don't even own record players. It's just something to put on the wall.
But the chart is mostly determined now by streaming, and streaming is not based on purchases. It's based on actual consumption.
If you wanted consistent data over the last 60 years, it would not be “How many people purchased ‘I Want to Hold Your Hand’ by the Beatles?” It would be “How many people went home and how many times did they actually listen to that record?”
Billboard does its best to try to ultimately figure out what's the most popular song in a given week, but it's measuring it in completely different ways these days. Now it's mostly based on consumption, whereas in the past it was based on sales.
As I'm sure you know from your history buying CDs, I'm sure there were CDs you purchased that you listened to one time, or you didn't even listen to at all. Whereas now that would not count towards the chart because it's all about listening, whereas back then, once you walked out of the record store, no one knew what you did with your music.
John Bailer
I really enjoyed seeing how the construction of music has changed—who was writing music has changed over time. You have some descriptions of the percentage of number one hits by a Black solo artist, or a group with at least one Black member.
You also have other figures describing the number of number one hits with at least one woman songwriter. There have been dramatic changes over time.
Chris Dalla Riva
Yeah. I think this was something I wanted to make sure the book got at, because I think when people hear you're writing about music—especially popular music—it's like, “Well, you know, pop songs are silly little things you hear on the radio. They're playing in the background of the supermarket,” whatever.
What always fascinates me is that popular music usually is illustrative of much larger trends. You're getting at demographic trends, which I think are important.
In the 1960s, of course, you have the civil rights movement, and right around the same time, you start to see more Black artists cross over to the pop charts in the US. A lot of that at the time was connected to the rise of Motown.
I think people underrate the importance, in race relations, of more Black artists or Black celebrities being present in the American consciousness.
By the ’90s, the charts are often dominated—if you're just breaking things down Black versus white—you see more Black artists dominating the charts. I try to keep demographics simple, and I acknowledge in the book that I can't keep track of every small difference in racial composition or gender composition.
But in broad strokes, by the ’90s, the charts are dominatefd by Black artists.
Women: at this point, in terms of gender on the artist side, there's pretty much parity between men and women as artists. There is less parity when you get into production and songwriting, and if you go even more technical, those are still fields heavily dominated by men.
That's something I also wanted to highlight. Of course, there's been tremendous social progress since the start of the Hot 100, but you still see women seem to be systemically excluded to some degree in certain places. Songwriting and production come to mind.
So I wanted to get at how popular song again is illustrative of larger, seemingly more important things related to gender and racial equality and stuff like that
John Bailer
You finish your book with some musical truths that you gathered along the way.
You listened to almost 1,200 songs, spent three-plus years listening, countless more years writing, and you derived some musical truths, including:
Popular music is neither created nor destroyed, it evolves.
Let enough time pass and musical ideas that are out of fashion will become fashionable again.
If somebody derides a musical element or tradition as “not real music,” they inevitably like a song that utilizes that element or some variation of it.
I thought it was really interesting to think that you boiled down your experience in this process to these observations.
What was the biggest surprise in this project? Maybe it's reflected in one of these truths, or maybe it's reflected in something else.
Chris Dalla Riva
I came to appreciate music much more through this journey and realized tons of biases that I had in my music listening and how I thought about music.
I tried to boil ways that I changed down to those three truths at the end. I can give you an example of each.
When I say music evolves—it doesn't start and stop—that goes back to that idea of musical eras. On December 31, 1979, everyone wasn't like, “All right, the ’70s are over. Let's start something new.” There's a continuation on January 1, 1980, and we should be aware that things change slowly over time. It's very, very rare that someone flips a switch and everyone's behavior changes.
The second one about musical ideas being recycled over time is: people usually look back to the past for inspiration, and sometimes the past becomes cool again. I feel like right now the 1980s are really in vogue. There's a very synthy sound in popular music, and we've seen that idea come up again.
Earlier, we were talking about teenage tragedy songs. We have not seen the teenage tragedy song come back, but there's been a grimness in popular song over the last decade that I think harkens back to that era.
People say the past doesn't repeat itself, but it rhymes. That was something I observed again and again throughout the journey.
The last one—if you try to say something's not “real music,” usually you like something that utilizes whatever that element is—was something I came to appreciate again and again.
An example I talk about is, in the ’60s, it was a big thing: “Oh, you know, The Monkees—we can't take them as seriously because they were made for a television show and they don't write their own music.” But there were tons of artists who didn't write their own music whom people take just as seriously. I point to Frank Sinatra as a good example. He was not a composer, but most people aren't listening to Frank Sinatra and saying, “Oh, this isn't serious music because he didn't compose it.”
I would see this again and again throughout the decades. When new musical technology emerges, there's often some backlash against it, and then it becomes accepted. You realize that art is tremendously varied, and the artistic process is going to use whatever is available to it.
We shouldn't immediately say, “Oh, that's not right because it doesn't involve an electric guitar.” Decades before, there was probably some debate about electric guitars and how they were not the right way to make music.
So I learned to keep these things in mind because you realize how wide and varied the musical world is.
Rosemary Pennington
Well, I'm afraid that's all the time we have for this episode of Stats + Stories. Chris, thank you so much for joining us today.
Chris Dalla Riva
Thank you.
Rosemary Pennington
Stats + Stories is a partnership between the American Statistical Association and Miami University's departments of statistics and media, journalism, and film. You can listen to us on Spotify, Apple Podcasts, or other places you can find podcasts. If you'd like to share your thoughts on our program, send your email to statsandstories@amstat.org or check us out at statsandstories.net. ffeBe sure to listen for future editions of Stats + Stories, where we discuss the statistics behind the stories and the stories behind the statistics.