Alexandra Freeman is the Executive Director of the Winton Centre for Risk and Evidence Communication, a role she took up in 2016. She previously spent 16 years working for the BBC, primarily a producer and director for BBC Science. Alexandra is passionate about bringing science to the widest possible audience. Along with working in television she has also helped develop content for computer games, social media and websites, as well as formal learning resource
Episode Description
Scientific publications drive science, well that's stating the obvious, isn't it, but the form and way it's processed is historic. It's historic in the sense that there are journals that are gatekeepers, their editors that are sending out submissions to reviewers, who are then providing comments, and then it's almost that it's set in stone and locked in place for the future. But there are challenges to this and one of the challenges is the focus of this episode of Stats+Short Stories with guest Alexandra Freeman.
+Full Transcript
John Bailer: Scientific publications drive science, well that's stating the obvious, isn't it, but the form and way it's processed is historic it's historic in the sense that there are journals that are gatekeepers, they are their editors that are sending out submissions to reviewers who are then providing comments and then it's almost that it's set in stone and locked in place for the future. But there are challenges to this and one of the challenges is being embedded in an idea that Alexandra Freeman, the executive director of the wind the Center for risk and evidence communication at the University of Cambridge has been considering. And, well, occupy may be, you know, attractive and we've seen them gamble and we've seen them do all sorts of things. We also are now finding out that octopus is going to be the label of a proposal that Alex has for thinking about research publications in the future. And Alex I wanted to just have a chance to have a short conversation with you about kind of what it is that you're responding to and what is kind of the vision that you might have about publications into the future.
Alexandra Freeman: Yeah, well I think when I came back to academia at the end of 2016 I've been working in the media for all my career of 17 years. And when I was working in the media, I was judged on you know how much people liked my work and how many people watched it or read it and forwarded it to friends, and all of that seemed quite appropriate because I worked in an entertainment industry, even though I was trying to make factual programs. But when I came back to academia, I realized that this is exactly how scientists are judged on their work you know they've got citations and they they talk about how to tell a story through that data and what story that papers are telling and, and they're under all exactly the same kind of incentive structure as I was when I was in the media. And this didn't seem right. I mean, I've heard about things like the replication crisis, but I hadn't really deeply understood how much it was driven by the incentive culture within science publication, and how much you know trying to make a story that would make a really well read paper or popular paper, kind of drove people to what are called questionable research practices. And so I really thought about this a bit further and I thought well, journals are kind of doing two jobs at once. One is that as you say they're trying to be the primary research record almost like a patent office for ideas and work so that you get it out there, you get your name on it it says so and so did this work in this year at this point, and you know, all credit to them. But on the other side, they're trying to do what maybe they were originally founded to do which is disseminate information to practitioners to doctors to engineers to pharmaceutical companies about the work that people are doing with insights and, you know, General science interest. And that is a much more kind of, you know, journalistic thing you want to review the evidence as a whole, you want it to be easy to read you want it to be you know referenced but you want it to be, you know, a good read and and accessible. And those two things are fighting each other more and more, because research these days is incredibly detailed you know to be reproducible, the method could be you know multiple pages of technical information about exactly what cell lines you used and what temperatures everything was done out and yeah real nitty gritty, and then the data sets you want to make those available so that people can reuse them you want to look at the analytical code so that you can rerun it and see what's going on and maybe reanalyze it. All of these things do not lend themselves to five pages of paper in a PDF for, you know, something that is easily read by a jobbing Doctor Who wants to know what the latest research tells them about something they can use in their clinical practice. So they're really adults, and it feels like the journalistic side and the dissemination side is driving away from the sort of patent office primary research record side. And that is, I think, driving so much of the problems with reproducibility and, you know, the sorts of things that we're aware of within science. And so my idea for octopus was to separate the two journals carry on concentrating on dissemination and aggregation and bringing all the evidence together in an evidence synthesis way. And the primary research records so where you get your research out there with your sort of name and date on it and in all the detail that others would need to be able to understand it and follow it again is in a different structure. And that's what octopus sets out to be. And instead of publishing you know waiting years to complete all your research and publishing it all in one whacking go whilst you're in the meantime terrified that others are going to scoop you in octopus you publish in smaller units, which means that you know you can publish your hypothesis, you can publish your methods you can publish your data you can publish your analysis, and other people can build on that. And I think there's a whole range of advantages to doing it this way. But the main thing is that it completely changes the way that you think about an approach research, and I hope changes the incentive structure to one that allows us to judge science as being good science, and not just a good story.
Bailer: So, let me follow up. So as you were talking about that the the smaller units are. Are those the eight legs?
Freeman: Types of publication Yes. And I think this is also for my psychology background so they have quite a lot of, you know, they've been decentralized nervous system so it's not all in some central brain there's eight legs that have independence.
Bailer: They can also squeeze through really tight spaces and move in different location I mean so there's there's I like the imagery here, so I I'm curious about how you see the relationship between, you know this this publication model that's embedded in octopus. And, and kind of traditional journals. Do you see that there being a connection, do you see it being formal Do you see it being informal Do you see it, that in some ways what you're envisioning with octopus is a deconstruct not deconstruction but a partitioning of some of the things that you see an archive kind of publications, it's not it's more than what you're talking about is more than a preprint service than that.
Freeman: Yeah, but I think the the popularity of preprint service, especially during the pandemic. you know that really brought home to so many people, the need for change for the publication level because the need for speed, you know, for one thing, but it's much more than that because as you say, on archive and all the preprint servers you're essentially just putting your papers there. well so this is much more of a deconstruction of the whole research process. So I see it as being separate from the journal system, not trying to replace the journal system in totality, but to to partition and separate the two roles that I think journals currently do. So I think journals will have to evolve to reflect the research that's been published in octopus. But that will allow them to shed some of the really difficult and expensive parts of their business which is getting peer review, you know, which is every journal editor will say that peer review is the really slow and painful bit of it.
Bailer: So that'll be embedded in part as people are responding to the various stages that are in this
Freeman: Absolutely, so peer review happens in octopus as an open post publication peer review system and the eighth type of publication is a review, so that you get as much credit for writing a good review as you do for writing a good piece of any other part of the research process because writing a critique of somebody else's work is a skill in itself, and needs to be done very well. So I think, you know, incentivizing it and recognizing those skills will mean that it is something that is not done quickly and in a hurry outside of your working hours to the, to the demands of the journal, it's done as a part of your work.
Bailer: So can you just describe kind of the reaction that some of the reactions that you've had to this, this proposal
Freeman: It has been really positive. You know it's a really quite radically different way of thinking about the publication process, and it takes some people a little while to get their head around you know it's different from just being a preprint server. And this is how the peer review bit would work and, but generally people see it. Once they get the concept as being really potentially exciting.
Bailer: That's a great, great place to end Alex. I want to say that's all the time we have for this episode of stats and short stories and Alex thank you so much for being here.
Freeman: Thank you very much.
Bailer: Stats and Stories is a partnership between Miami University’s departments of Statistics and Media, Journalism and Film, and the American Statistical Association. You can follow us on Twitter, Apple podcasts, or other places where you find podcasts. If you’d like to share your thoughts on the program send your email to statsandstories@miamioh.edu or check us out at statsandstories.net, and be sure to listen for future editions of Stats and Stories where we explore the statistics behind the stories behind the statistics.