Dr. Stephen Ruberg was in the pharma industry for 38 years where he worked in all phases of drug development and commercialization – from R&D to Business Analytics. In his last 10 years at Lilly, he formed the Advanced Analytics Hub for which he was the Scientific Leader and ultimately the Distinguished Research Fellow. He retired from Lilly at the end of 2017. Since his retirement, he has formed his own consulting company, Analytix Thinking, which is dedicated to teaching good statistical principles and consulting on analytical strategies for organizations.
+ Full Transcript
Rosemary Pennington: When people think of what it takes to get a new drug approved, they likely think of the various trials a drug has to be run through, then the production and marketing of a drug once it’s ready for release. What they might not consider fully is the scope and impact of statistics in the production and manufacturing of drugs in the pharmaceutical industry. That’s the focus of this episode of Stats and Stories where we explore the statistics behind the stories and the stories behind the statistics. I’m Rosemary Pennington. Stats and Stories is a production of Miami University’s Departments of Statistics and Media, Journalism and Film, and the American Statistical Association. Joining me in the studio are regular panelists John Bailer, Chair of Miami’s Statistics Department and Richard Campbell former Chair of Media, Journalism and Film. Our guest today is Steven Ruberg. Ruberg was involved in the drug industry for almost 40 years working in all phases of development and marketing. At his last ten years at Eli Lilly he formed the advanced analytics hub for which he was the scientific leader, and eventually distinguished research Fellow. He now runs analytics thinking which he founded to promote good statistical principles and to help organizations come up with analytical strategies. Steve thank you so much for being here today.
Steve Ruberg: Rosemary it’s a real pleasure to be here. looking forward to the conversation.
Pennington: Great so I guess my first question for you is just how you got involved in- and I guess as a statistician in manufacturing of drugs.
Ruberg: Yeah so I didn’t know I wanted to do this I actually went to Miami University to get a master’s degree in statistics.
Bailer: Oh yeah baby.
[Laughter]
Ruberg: No let me take that back. I actually went to Miami University to get a masters degree in mathematics, but when some astute professors noticed that I appeared to be more interested in applied mathematics they suggested taking some courses in statistics, which I loved and converted to a masters degree in statistics which I got, and I was just totally captivated by the whole field of statistics and applying quantitative thinking to real world problems and I went on to get a PhD in biostatistics. I went to work for a small pharmaceutical company in Ohio because that’s the job offer I got and [Laughter] and interesting and I thought it was good, I like biology, you know I got this PhD in biostatistics from the University in Cincinnati, so I was hanging around the medical school and doing stuff related to that. So, it was kind of a natural transition, but I can tell you it didn’t take too long for me to realize or to feel that I was part of a noble cause. That I was using my quantitative skills to help scientists and physicians figure out how to cure disease or how to relieve symptoms or how to improve quality of life, and that felt very good and rewarding and enriching to me, and over the course of my 38-39 years now hanging around the pharmaceutical industry I’m very proud of the contributions that I have made, and I have seen many statisticians that play a very impactful role in bringing new drugs to the market that help cure or ameliorate disease. So that’s what kept me engaged and really totally motivated to press on in the field of statistics and drug development.
Bailer: So, are there particular diseases or concerns that you focused on in your early work that really led you to this thought or this realixation that you were a part of this noble cause as you describe it?
Ruberg: You know, I think I just saw the pharma company that I work for which merged with some others and kind of grew into a really big pharma company, I just saw people in all different aspects on this. I spent some early years in research, and I was so thoroughly impressed with the depth of knowledge that these people had about cellular mechanisms of the human body, and how passionate they were about trying to tear apart a cell and figure out what’s going wrong in this cell, what proteins have been mis formed or aren’t being produced so that this cell can operate in a reasonable way. And I saw the same things with clinicians who were trying to figure out what’s the right dosage for these patients or what are the right patient characteristics for this drug that might be best for a person to take this drug etc.. So everywhere I looked I saw people passionate, committed, really smart, and had invested their entire lives and careers into this scientific enterprise, or maybe even one specific niche. The beauty for me as a statistician is data is everywhere and I got to play, as some folks have said as a statistician I got to play in a lot of people’s sand boxes, and so I got to learn a lot of biology and chemistry and medicine and that satisfied my inherent need of curiosity to learn more, and I just really enjoyed learning more statistics, learning more methods, doing more research on more statistical methods or algorithms or approaches, while also learning more about these other scientific disciplines, and how I could be more part of the team that would bring a new drug to the market. So, John the beauty of- to answer your question is no I felt as a statistician I wasn’t pigeonholed into one area. I’ve worked in allergy and oncology, diabetes, neuroscience, Alzheimer’s and cardiovascular disease and on and on and on. So, it’s been a really fun and exciting way to apply my inherent interests in quantitative things.
Richard Campbell: Very good. Steve this is Richard, can you talk a little bit about as a statistician when you worked for Eli Lilly, what were your biggest challenges? What does a statistician have to do to explain what it is that you do to get people to understand data and the numbers that you work with?
Ruberg: Good question. So, in some respects because regulatory agencies like the Food and Drug Administration and you can imagine there are international counterparts in Europe and Japan and China and other places in the world, because they require clinical trials that are adequately designed and sufficient sample size and analyzed appropriately etc., statisticians are viewed as a necessary and important part of clinical development. But even then, I would say one of the hardest problems, maybe the hardest problem human beings face, is inference. That is, we observe things in the natural world, and we try to infer what is the true natural phenomenon going on behind it. That is, we observe patients have certain responses to a drug and we try to infer does this drug really work or not? Or am I just seeing some random chance or spurious favorable outcomes in a few patients, or whatnot. So I think that inference is a very difficult thing conceptually, and I think that’s probably the hardest thing to communicate to others who aren’t trained in statistics, and I guess I’d even say it’s hard for me who spent a lifelong career in statistics to try to figure out what is true based on this limited view of mother nature that I have, whether it’s 100 patients in this clinical trial or one experiment done in rats in this laboratory, what general inferences can I make? And it’s a very difficult problem. We all have human biases, there’s random variation in nature, there’s assumptions and analyses and models, and so trying to pull that all together in a way that you can help scientists understand I’m here to help, but because statisticians deal with uncertainty and variability and probability, I may not give you a yes or no answer that you’re looking for, I may say something like the probability that this drug at this dose works in these patients is maybe 67%, do we want to do more research on this or do we want to call it quits? Well that involves a bunch of other business decisions etc., but that I think is the hardest thing to communicate- is this notion of inference and the inherent uncertainty that comes with it and that we deal with probabilities and not certainty. So that’s where I think scientists and clinicians who appreciate statistics, to a certain level, enjoyed working with me and other statisticians because they said you guys think differently than the rest of us, and we need that different kind of thinking, but boy it’s hard to think like you guys or could you just tell me yes or no? [Laughter] And it’s like golly, I wish the world were as easy as black and white and yes and no, but we don’t deal with zeroes and ones, we deal on the continuum of zero to one, in terms of probabilities.
Bailer: So, Steve you mentioned the idea of clinical trials, I was wondering if you could just sketch out the workflow from the discovery of a promising compound to the investigation of that compound, to finally marketing in this pharmaceutical compound?
Ruberg: I’ll give you a general overview, but I hope along the way I can give you a couple specifics statistical stories that I think are very fascinating and compelling about the value of statistics in drug development. So basically, in the research arena scientists are looking for what are known as targets for disease. So, a patient has Alzheimer’s disease and the scientists are trying to figure out what is it that’s going on in the human brain that has gone awry so that a person has cognitive decline and functional decline? And so, they’re looking at the neurons and synapses in the brain to say what’s gone wrong, and if you can identify what’s gone wrong, that’s called a target. We say we think this is what’s gone wrong in this cell that’s our target, we have to repair that or change that or regulate it with something along the way. The next step is to design a molecule that interferes with that target in the way you’d like. So, let’s say that target is something to do with the production of a protein and your cells aren’t producing enough or too much of that protein. The scientific community says let’s find a molecule or an antibody or something that will go in and stop the overproduction of that protein or will enhance the production of that protein whatever the case may be and that’s called a lead compound. Generally, there are a number of lead compounds that are generated because biology is very complex and it’s not always sure which one might be best. You then proceed to testing those things in animals and animal models to see if they actually do what you think they should do. And if they’re tested in animals and they seem to be producing more of this protein in the rat brain or the dog brain, and you think that’s relevant to humans, then you go on and do safety testing in animals, so-called toxicology work, to make sure that this thing for all intents and purposes looks like it’s safe. So, years of toxicology researching to make sure that this thing doesn’t do anything bad in animals, so therefore now we think it’s safe to take in humans. And so the first trials in humans that are done are safety trials, very small numbers of people; 6-8 people get a single dose of the drug, and they’re watched for a week or two to make sure nothing goes haywire and that the drug is safe, and you gradually increase doses in subsequent cohorts of patients, and once you determine it’s safe and say gee it’s safe up to 200mg a day, then you begin efficacy testing in phase two trials where you start to test on people with the disease, say Alzheimer’s disease, and you might try 25-50-100-200 milligrams of the drug, and you might study it for six months or a year or some period of time. And you do all that testing to say does the drug look like it’s really having an effect? And if it looks like it’s having a positive effect and it looks like it’s still safe, these trials tend to be hundreds of patients for longer periods of time, then you go to phase three confirmatory trials where you say alright we’re going to do a very large long term trial, we’ve picked the dose that works, we know the kind of patients we want to enroll in our trial, and you do this very extensive, very long term international clinical trial- usually involves dozens of countries hundreds of doctors and investigators from around the world and at the end of that trial you conclude whether you’ve generated, in regulatory language, they say substantial evidence of efficacy and safety of this drug as it is intended to be used in the intended population for Alzheimer’s disease etc.. Then you submit it to a regulatory agency or multiple regulatory agencies all over the world you submit all your data your preclinical data, tox data, clinical trial data and they review all that stuff and essentially say yes it is approved for marketing, or no it’s not. If it’s approved for marketing the company is in the background investing lots of money to build manufacturing facilities to produce this, so there’s a lot of upfront investment, and when they get the approval they manufacture the drug and they put the sales force out to start informing doctors around the country and around the world to say here’s our new drug, it’s used, here’s the doses, here’s the patients etc.
Pennington: You’re listening to Stats and Stories and today we are talking statistics and pharmaceuticals with Steven Ruberg of Analytics Thinking.
Bailer: Steve that was a great summary. That was really nicely done.
Campbell: So, it’s complicated right?
Bailer: You know, the thing that is always striking to me when thinking about this is just the pyramid that you have here of starting with so many potential lead compounds, and ultimately what ends up coming to market so two questions: One is what fraction of these lead compounds get approval and make it to market? And what is the cost of bringing something from promising lead compound to market pharmaceutically.
Ruberg: Yeah so, from the lead compound area I would say the numbers I’ve seen are anywhere from 1 in 1,000 to 1 in 10,000 lead compounds actually make it all the way to the market as a drug. The process for discerning all that and finding that one compound and getting it to the market- average numbers anywhere from 9 years on the low side to 12 or 13 years. And the cost can be as much as two and a half to three billion dollars of investment to make that all happen. Now of course there’s exceptions where the scientific world is advancing rapidly and there have been breakthrough therapies that have been accelerated for rare diseases, and gene therapies that have gotten through the process in a shorter period of time, maybe five or six years, maybe with as little as a one billion dollar investment but still the time scales and the dollar amounts are quite considerable.
Campbell: So, one of the ways the general public learns about this very complicated process is through journalism. So, and they often- you know the kind of process you described is complicated, and what journalists usually focus on is the more dramatic parts of it, and if you could talk a little but about what journalists could do better? Because I’m thinking the general public today probably thinks- when they think big pharma they think opioid crisis, and all of those stories that have come out in recent years, and you’re reding the newspaper stories on your work, and the work going on, what do you think that the general public needs to know that they’re not getting from journalists on a daily basis?
Ruberg: It’s a very interesting question because I just heard an interview this morning listening to National Public Radio of drug pricing. It’s a very big issue, and this person said it costs a trivial amount of money to manufacture such and such a pill, you know? And yet a pharmaceutical company will charge $500 for a month’s supply of maybe 60 pills, where you maybe take this pill twice a day or something like that, and it literally only costs them 3 or 4 or 5 cents to take that pill, one pill. They say that’s outrageous, that’s crazy and the pricing markup and they’re gouging the public. Well, without getting into all the details and the nuances of pricing and that stuff and the complexities, you know I think what the public needs to know you’re not paying for the manufacturing of that pill, you’re paying for the ten or twelve years of research that went into that pill. There’s a lot of knowledge and intelligence into how that chemical is constructed in that pill, and how the safety of it was documented etc. etc. and the effectiveness, so what you’re really paying for when you buy a bottle of that medication is the billions of dollars and the decade of research and all the smart intelligent people who worked very very hard to bring that deep expertise to bear in terms of a medicine that helps you with your disease. And so, I think that’s one of the things that I see it played out once in a while, but it’s frequently dismissed by people. Yeah yeah yeah, it’s expensive to develop a drug and all that kind of stuff, but I still want it for a dollar a day to treat this very complicated disease called diabetes, or breast cancer, or whatever the case may be. So anyway I think people hear that but it’s still there’s still an emotional aspect- what I just gave you is a very intellectual argument, the emotional argument is man, I struggling to make ends meet with my family and now I have to pay a lot of money for my drug.
Bailer: So in one of your roles when you were working in the industry you were in charge of all things analytical at a large pharmaceutical company, and that spans the clinical trials and preclinical research that you talked about earlier, can you talk about some of the other areas where statistical insights or careful study was required for everything from manufacturing to marketing to post market surveillance all the other thigs that might have been in the scope or practice of your office?
Ruberg: Yeah John, I’m going to give you-you mentioned manufacturing, so I’m going to give you a manufacturing story here very quickly. So, some drugs are dosed based on body weight. These are injectable drugs, especially in the oncology and cancer treatment areas, and so everyone gets a different dose depending on how much you weigh, and the question is well how do you do that? Well what we do is make vials of the solution of the drug say a 10 mg vial, so here’s a small vial with some fluid and it’s got 10miligrams of the drug in it. So let’s say a small person needs to get this drug, say they only need 4 milligrams, what happened is you extract the 4 milligrams from this vial and then since you’ve punctured the seal, the 6 milligrams cannot be used again because of all sorts of concerns about sterility and everything else. And so, 6 milligrams get wasted. Let’s say a larger person needs 11 milligrams, what they have to do is take two vials and take 10 milligrams out of one and only one out of the next vial, and nine milligrams is unused from the second vial. Now, oncology drugs are very expensive. And in fact, the excess has to be destroyed usually by special means to inactivate them, so that’s expensive. So traditionally to reduce waste, a company might create a smaller vial and say let’s use a 5-milligram vial, that will help us reduce waste. But if a patient needs 6 milligrams, they get one 5 milligram vial and the other 5 milligram vial they get only one milligram and there’s still 4 milligrams of waste. So, the question is what is the optimal set of vial dosage strengths that you can use to minimize waste, while also minimizing the number of dosage vials and minimizing the number of vials used for any given patient, right? You can’t just say well let’s make one milligram vials, and then a person who needs 13 milligrams just gets 13 vials. In a hospital, in a pharmacy, in a practical setting that will never work. Usually there is a constraint of a maximum of four vials per patient. So anyway, so suppose you have a drug that has to cover a range of 2 milligrams to 15 milligrams. That’s kind of peoples normal body weights from the smallest female maybe to the largest male gets – well with a little bit of thought you can probably understand that 10 milligrams and 5 milligrams are very bad choices because that combination are multiples of each other, and in fact if you go through a formal optimization process you can find that actually a 3 milligram vial and a 4 milligram vial works best because they can cover almost any of those doses with virtually no waste whatsoever. So if you need 2 milligrams, you get a 3 milligram vial and only 1 milligram waste but a 3 milligram or 4 milligram, you can see the combinations of 3 or 4 milligrams gives you no waste, it’s less than 4 vials per patient etc.. That’s an example where some folks in my group came in and said hey why are we doing all these dosages in multiples of each other? And we went through a formal mathematical optimization process, the objective function to minimize waste constraints, there’s also manufacturing constraints so we had to build in certain other aspects to this, but we did an optimization across all of this, and Lilly started producing vials that had these really odd dosage numbers, you know like a 7 milligram vial and a 23 milligram vial for stuff like that, and it reduced waste considerably. The bottom line as one of our manufacturing people told us that over the course of 5-10 years that this drug would be on the market the amount of waste we would minimize would probably accumulate somewhere in the vicinity of 200-300 million dollars.
All: Wow.
Ruberg: Right? So I can tell you that story caught the attention of our C.E.O. and some others in the company said we want the statisticians involved any time we’re developing a new drug that is an injectable where we have these options of different vial dosage strengths, we want the statisticians involved in the conversation in helping to truly mathematical optimization that minimizes waste for all parties involved. Lower cost for us, lower cost for the patient, lower waste for the hospital, it has to be destroyed etc. everybody wins. It’s a great example of not just statistical thinking but quantitative thinking.
Campbell: Steve I was interested in you know you talked about the processes as the drug is going to market and I know Lilly was one of the very few companies that was involved in non-opioid pain medication studies, and I wonder if you had any or were doing any of that at Lilly, and you know where that research stands today- the non-opioid pain medications?
Ruberg: Yes its important research are given the recent opioid crisis and yes I was still at Lilly we some of that research was going on, and to be honest I’ve actually done some consulting with Lilly who has a joint venture with Pfizer, but I can’t give you any insights.
Campbell: I thought I’d try.
Bailer: For future episodes Steve.
Ruberg: I can only tell you what’s in the public domain some trials have shown success in some pain areas. They’re studying back pain knee pain lower back pain post-surgical pain some of these other areas there have been some successes and some not as much, so the jury I think is still out and people are trying to figure out what is it about these new mechanisms of these new molecules besides using opioids that could help manage pain but ill tell you if a company could unlock that key it would be an important addition to the medical arm of treating pain.
Pennington: Well Steve that’s all the time we have for this episode of Stats and Stories, thank you so much for being here today.
Ruberg: Thank you for having me I’ve enjoyed the chat and keep up the great work with Stats and Stories, I find it interesting and invaluable.
Pennington: Thank you very much. Stats and Stories is a production of the Miami university’s Departments of Statistics and Media, Journalism and Film, and the American Statistical Association. You can follow us on Twitter or Apple podcasts or other places you can find podcasts. To share your thoughts on the program send your email to statsandstories@miamioh.edu or check us out on statsandstories.net and be sure to listen for future editions of Stats and Stories where we discuss the statistics behind the stories and the stories behind the statistics.