Dr. Dooti Roy is a people leader, global product owner and a methodology statistician at Boehringer Ingelheim (she didn’t give me where she worked in her bio so she might not want this) who enjoys developing/deploying innovative clinical research and statistical visualization tools with expertise in creating and leading dynamic cross-functional collaborations to efficiently solve complex problems. She is currently focused on research and methodological applications of Bayesian statistics, artificial intelligence and machine learning on clinical efficacy analyses, patient adherence, and dose-finding. She is passionate about promoting diversity and inclusion, mentoring, cross-cultural collaborations, and competent leadership development. She unwinds with painting, reading, traveling and heavy metal.
Suzanne Thornton professor of Statistics at Swarthmore College, a liberal arts undergraduate-only institution. As an educator, she strives to teach students to understand statistics as the language of science and prepare them to become stewards of the discipline. In 2020 she chaired an ASA presidential working group on LGBTQ+ representation and inclusion in the discipline and earlier this year, she was appointed to a three year term to serve on the National Advisory Committee for the US Census.
Episode Description
Measurement accuracy is something all quantitative researchers strive for, as you want to make sure you're measuring what you want to be measuring. When it comes to gathering gender and sex data, though measurements are complicated, beyond simply teasing apart sex and gender, there's also the imperative to ensure the language and measurement tools researchers use are inclusive of all experiences. That's the focus of this episode of stats and stories with guests Dooti Roy and Suzanne Thornton.
+Full Transcript
Rosemary Pennington
Measurement accuracy is something all quantitative researchers strive for as you want to make sure you're measuring what you want to be measuring. When it comes to gathering gender and sex data, though measurements complicated, beyond simply teasing apart sex and gender, there's also the imperative to ensure the language and measurement tools researchers use are inclusive of all experiences. That's the focus of this episode of stats and stories where we explore the statistics behind the stories and the stories behind the statistics. I'm Rosemary Pennington. Stats and stories is a production of Miami University's departments of statistics and media journalism and film, as well as the American Statistical Association. Joining me is regular panelist John Bailer emeritus professor of statistics at Miami University. Our guests today are Dooti Roy and Suzanne Thornton.
Roy is Senior Principal therapeutic area methodology statistician at Boehringer Ingelheim. She's currently focused on research and methodological applications of Bayesian statistics, artificial intelligence and machine learning on clinical efficacy analyses, patient adherence and dose finding. Thornton is professor of statistics at Swarthmore College. As an educator, she strives to teach students to understand stats as the language of science, and prepare them to become stewards of the discipline. In 2020, she chaired and as a presidential Working Group on LGBTQ+ representation and inclusion in the discipline. And earlier this year, she was appointed to a three year term to serve on the National Advisory Committee for the US Census. They are also both two of the co authors of an article for significance magazine about best possible statistical practices for sex and gender data. Thank you both so much for joining us today.
Dooti Roy
Thank you so much for having us.
Rosemary Pennington
I just to get us started, what prompted this article?
Suzanne Thornton
So what what prompted this article for from my perspective, was conversations with other statisticians were the catalyst that the put this into action, but I had a an experiences as a identify as a queer woman. And in graduate school, I came out as a queer woman. And as I started to learn more about LGBTQ+ history, and vocabulary, and culture, and experiences, I am also learning my training and theoretical statistics. And so these questions were sort of coinciding in my, in my mind is questions about fundamentally, how do we draw conclusions about unknowns from data of what is observable. At the same time, I'm also learning about different ways of human expression, different experiences, and the language that's associated with that. And I saw a fundamental disconnect in the way that we talk about gender and sex is very kind of offhandedly, in many statistics, examples, problems, textbook problems, and actually in applied research as well,
John Bailer
You know, it's interesting, because this is often the, this was the starting place for the simple dichotomous example. Right?. And all right, you know, and all these introductory books from from years gone by. So perhaps it would be helpful to just just step back and talk a little bit about when, when you're talking about about gender, or about sex data, it could put this into a little bit of context, particularly in a context of not only just social construct, but also in terms of kind of gradient of response.
Dooti Roy
And that's right. And this is also very interesting from my side, because while Suzy was discussing her reasons, I was also thinking, what are what are my reasons and something which I want to start with is, you know, my work is in the world of drug development. So I work in a pharmaceutical company. And for every trial, that we build and run for patients, we are collecting a number of information about these individuals, and one of them is sex. And it is I columns, it's male or female. And I was working on trials for mental health patients. And it is a known fact that some of our mental health patients are at more risk of some adverse events, especially for if they're on the transgender spectrum. And yet here I was a statistician during this trial wondering how on earth would I know this? Because there is no way of capturing the data. And this is this real example that I can provide something which science faces because we hear from transgender patients, their healthcare experience, their journey, and it's harrowing. There is no gender inclusive language that are taught to our clinicians or to To the investigators or site personnel, they handle this patients, they don't know how to talk to them. Forget about recording their data accurately. And this is important, right? Because people who are at higher safety risk get treated differently, they are given extra care. But you would need to know that first before you go there, right. And this really was another pointer for me to think, well, we are the data people, can we do more? push for more. And and this really got me started. And there Susie, Susie was there. And then we connected, we talked and we thought, what could we do? And here we are.
Suzanne Thornton
And I think also in response to your question about, you know, what are what are some of the different measurements that we typically use and categorizations that we are typically interested in? So So typically, and even very prominently today, we see research or studies that treat sex and gender as the same thing. Yeah. And so we use the term male, to mean maybe masculine, we don't define what is meant by masculine or feminine and, and we use these terms interchangeably. For me, I personally like to use the word queer. And maybe I should say something a little bit about that I like this word, I use it lovingly, and very generally, to refer to anything that deviates specifically from an assumed cisgender heterosexual norm. And so to me, this is a very ambiguous and incompetent compensate loving words. And it also, it's something I came to use to describe my own identity and experiences as, as I started to learn more about LGBTQ+ spectrum, that it encapsulates much more than sexuality and sexual attraction. It's also more than reproduction, reproductive abilities and functioning. And it's more also about individual senses of self, and expression. And it's just such a gigantic spectrum, a multi dimensional eye opening kind of experience to to get to know more people. And so you get to have chosen family who are transgender, or who are intersex, or gender non conforming, and then you get to run into collaborators like Dooti Roy, Steven Perry, who is a consistent statistical consultant at Cornell, you know, he was approaching us saying that there were people visiting him for consulting experiences, and they wanted to be inclusive. They, they recognized diversity and human experiences and expressions, but didn't know how. And so that's really is what got this work started.
Rosemary Pennington
As I was reading the article I was thinking about so for my dissertation research, I part of my work was a survey that I designed. And I spent a lot of time sort of thinking about how to frame the question, because I separate sex and gender in my survey, but thinking through like how, what categories and this was in 2014. So it's, you know, well before the conversation you're having now, but thinking through, like, how do I frame this in a way that is useful to what I need to understand, but it's also inclusive, because, you know, there's always that tension with survey research is, if you frame a question, the in in a way that's off putting to a respondent, they won't participate, right, as we're trying to create this atmosphere where as many people will participate as possible. It just sort of made me think back to that struggle I had of trying to figure out, how do I measure? How do I get what I want, and also sort of make sure everyone feels like they are included somewhere? And what I what I have here in this tool?
Suzanne Thornton
I think that that's a very common experience for for modern day researchers. And that's why we wanted to publish this article and try to disseminate it as widely as possible is to say that there are there are ways to do that, statistically, what sound ways to be inclusive and respectful of different experiences and identities.
Dooti Roy
Yeah, I mean, maybe maybe I can add here again, going back to the problem that I had at hand. And one way we thought about solving this issue simply introducing a field called gender identity. So we had sex at birth and we had gender identity and and if you now combine both of these, you now have a tool to identify transgender subjects and and thinking about this in clinical research, you're really focused on why really you need this data. And I think this is something we talk a lot about in our people that really, really think hard. What is that you really want to know? It's not just a question you likely throw out a patient or or an individual in a survey but really thinking the why behind it, especially the science why why is this answer should help you and therefore formulate a strategy. Exactly. How do you extract the information? In my experience, building trials now for more than seven years, what I see is these patients when they come to us healthcare experts are a doctor, they're vulnerable. They want to be respected, they want to be treated, but they also want to be heard. And hearing from a doctor, oh, I don't really know how this works for you. Because there is no data. It's not always a reassuring. And this we heard over and over again from patient surveys, and there is really a need for us to step in and change the story here.
John Bailer
I think that's a really powerful statement to say, how does this answer help you? How does knowledge of this have what what is this? What do you really want to know from measuring this variable? And, and I thought that in your paper, you talk about this, this kind of the different aspects of sex and gender identity. And you had mentioned a couple of being sex recorded at birth and gender identity, but then also primary and secondary sex characteristics. And, you know, I found that just by by enumerating kind of these, the categories that were reflected there, it's kind of it's sort of called helping you focus helping a researcher or others just thinking about about these types of data to focus on what kind of questions you really want to answer. So could you give us kind of like, you know, sort of a little bit more of an example maybe of a bad way that that's that that a question like this could be framed as well as then a more productive way? A question like this could be framed?
Suzanne Thornton
Yeah, absolutely. And I can maybe also plug another paper I'm working on. So I'm publishing in the JSM proceedings, where I presented here are some common ways in which people and try to include both gender identity and sex information. And and how do they compare to say, you know, the AASA ethical guidelines for statistical practice? And then I propose a solution as well. So I think that there are several different types of ways of accounting for gender or, and or sex information in a study, I think there are attempts that try to be inclusive, but they actually aren't. An example of this, in my opinion, is having this kind of tutor two step questioning procedure, but, but the responses are invalidating to certain identities. So for instance, one question could be what is your sex at birth, male or female? And maybe you want to include intersex? And then another question, what is your gender identity? But when respondents are only given, you know, choose one of the following. And the options are, man, woman, transgender, non binary, and then add in whatever you'd like, that's not really inclusive, because if you take the perspective, let's say I'm a transgender woman, I know how to answer what my sex assigned at birth was. That that is a clearly defined question. When I when it comes to the question, what is my gender identity? I identify as a woman? I definitely want to answer that question if I identify as a woman, but there's also this response here, just trans. And so it's, it's, you know, there's a bit of confusion there are you so so I think there are there are those questions that are questioning procedures that try to be inclusive, but but actually end up not really being inclusive, there are those that try to be inclusive, but then aren't reproducible, or you're going to run into small sample size problems that you didn't anticipate? Because you realize that a lot of this comes to design, as well, statistical design, but some attempts to be inclusive, but end up not being reproducible include open ended questioning, right? So what is your gender identity, fill in the blank with your own words? That is definitely inclusive and respectful of anybody's identity. But that doesn't mean that people are answering under the same operating definitions of common terms. And so statistically, you're not getting reproducible, kind of accurate measurements necessarily. And then, of course, there are, you know, this research that does not attempt at all to be inclusive, either because, you know, researchers are ignorant of or ignore the issue of conflating gender and sex or, you know, there's also a very strong poll just to stick with tradition to treat gender and sex interchangeably as it has been in western science and medicine for so long. But at the end of the day, I think that this this example of how do we ask relevant and inclusive kind of questions, it kind of gets down to a failure to critically assess what is our variable of interest? What are our variable choices? And how do we define meaningful levels within say, a categorical variable? So in my my JSM proceedings paper, the working title of which is ethical considerations for data involving human gender and sex variables. I do propose another questioning method, and I'll let Dooti talk about the one that was appropriate for her and her work and in a clinical context, but but not all research is clinical right? Not all research needs to know about reproductive of Oregon's or capabilities. So in my JSN proceedings paper, I propose a sort of select all that apply question format that I think is worth considering, and could be very powerful across many different applications.
Rosemary Pennington
You're listening to stats and stories. And today we're talking to Suzanne Thornton and Dooti Roy about inclusive measurement of sex and gender data. Suzanne, you mentioned that sort of one of the one of the roadblocks to having more inclusive measurements has has been, in part, this clinging to tradition, right, where this is how we've measured things in the past. We can we can, you know, compare this over time, like, you know, it's, it's, it's tried and true. So I wonder what has been the response to the significance article and the work that you've been doing in relation to this, because again, when I was doing this dissertation research in 2014, the reason I mentioned it is that the conversation I was having was something that I think a lot of people who I was connected to was not were not having they were not thinking about you, why are you worried about measuring it that way? This is how we always measure it. Right. And so it does feel like the fact that you have this article out of seems like an opening that maybe wasn't there?
Suzanne Thornton
Yeah, I definitely think that, as in general, sort of public health, sex ed, and sort of general societal trends towards trying to accept and understand different human experiences, I think, definitely makes this a different conversation than it would have been 510 years ago, the response that I've received has been largely very interested and very excited and curious, you know, I this is something I try to incorporate in my classroom discussions. When we talk about categorical data analysis, I, I talk about how, you know, we as the researchers, we choose what are the levels, and many, if not most categorical variables that we analyze and how that subjectivity might influence our conclusions, our statistical conclusions. And I think this gender and sex conflation is a good example of this. And I have had maybe a little bit of pushback from some students who, you know, maybe aren't expecting to, to question the status quo in in a statistics class, but largely from the statistical community, I've had responses that are very enthusiastic and are really relieved to see this issue being presented and openly discussed in a search for better science and, and better treatment and respect for our study subjects.
Dooti Roy
So again, going back to my world of drug, drug development, one of the one of the responses is always been encouraging and positive and saying, Oh, you guys did this, this is awesome. What remains the common standard conversation, and I hear this a lot from my colleagues is oh, you know, it's so hard to change the standard. So drug development is very, very standardized, especially when it comes to data collection. And this is like, sanitized by this body called cedars. And, and it is apparently very difficult to influence and, you know, have changed, have them change some of these ways of collecting data, introducing new fields, this is not easy. So the example that I was providing earlier, I was able to convince my chief medical officer of the of the company to let me do this for all trials that the company is doing. And I was super happy for maybe a couple of months until I realized, well, there is a long way ahead. It's just about one company. What about rest, or whatever the rest of the rest of the trials. And so one thing that really came out of this was finding another really interested collaborator from Roche and and Godwin and I, we now have moved this conversation more central, for example, hosting roundtables in key conferences. So just last week, I was in DC, hosting this roundtable in the ACA bar from regulatory workshop, which happens once a year and is mostly well attended by regulators as well as industry folks. And there was a really interested bunch of people from FDA from different other responses. And we were having this conversation and believe this is not the see this came up again, people were like, This is so hard to change the standards. So what is very clear from my perspective, as this this conversation needs to go on and in a meaningful manner, and there is a lot that we can do to change the story where this will need work.
John Bailer
You know, I found that your consideration and practical guidelines were really interesting to just to read them out, again, identifying relevant information, the senator inclusivity and respect and protect the participant in the data. And as you as you all have been talking about this today, I find myself thinking, I can't imagine the FDA would not be interested. I mean, can you imagine, you know, you're collecting data that's, that's just inherently noisy because of measurement error, or non response, or that are collecting data or not collecting information that might help you identify particular strain data that are at much higher risk of some adverse outcome, it seems that there's a strong story to be told for why this type of better measurement is good. You know, can you kind of just weigh in a little bit on this? Am I Am I reading this? Right?
Dooti Roy
You are You are absolutely reading this right. I think we saw the interest from the regulators, we saw they were sitting at the table willing to discuss with us, I think it really depends on how flexible we all are about changing guidelines. And when it comes to guidelines, people really take them as they're written in stone. And but the problem is, is were developed 50 years ago, where this conversation about these minority populations didn't exist much. But now they do. And we want to think about them. And we want to think about them as patients who need our help and support. And then therefore, I personally feel that no, they shouldn't not be thought about as written in stone, we should reevaluate whether they're applicable today or not. And I know this probably applies to many of the conversations that comes up. But that's really where we need to discuss more and and, you know, have a dialogue, which can be ongoing about what really is needed. So really, from my perspective, after this, I came back from the workshop, I was like, Yeah, we really need to just find somebody and see next, find whoever is sitting in the committee and put that person on the table and start kind with conversing with them, because it has to start somewhere. And then that's really the intent from from our end, to move the needle.
Suzanne Thornton
And I want to take a moment here, if we can, and just reflect on what Dooti and Godwin Young are doing. And people like Dooti are doing that. So this is where the rubber meets the road, they are actually trying to change conventions in the places where they work. I'm I have a very different sort of more like General, I guess, abstract kind of perspective, as an academic, I'm thinking about, you know, kind of not just clinical settings, but also demographic settings, and so on and so forth. And trying to make sense of this from a theoretical world. But the kind of work that Dooti has been doing and is continuing to do, I think is the hardest kind of work. And it is so incredibly encouraging to me to have the opportunity to work with her and other people who are committed to doing better science, being better researchers, and changing things that are so challenging and deeply ingrained into our entire culture. So
Dooti Roy
Suzy thank you, you inspire me every day to so so we are more or less on the same page here.
Suzanne Thornton
Oh, thanks Dooti.
John Bailer
So Dooti I just your your response about this sort of background organization and kind of the way things have been done are questions about race and ethnicity. Now the same as they were 30 years ago,
Dooti Roy
they have significantly changed, but there are definite gaps. Actually, this is this is interesting that you bring this up when I was trying to have this internal conversation within my company, I still hadn't found my guts to talk to the chief medical officer because he's a big guy. And I was kind of like talking to my peers. And very my immediate supervisor said What should we do? And I heard this story. Oh, you know, it took like 10 years to change the risk conversation. What do you expect? And and that was somebody's way of telling me being trying to be helpful, but also keeping me grounded and I was like, I'm just not having it. I don't have 10 years.
Rosemary Pennington
I wonder what advice you have Suzanne's because you are teaching stats and it sounds like trying to push this. What advice do you have for statistics educators or people who might teach against survey methodology? Again, I never took I took a stats class in my in my PhD, but never in a stats program. What advice would you have for these educators who are trying to help students think about these measurement issues in these particular ways, which again, for some of them coming out of high school where they would have had a stats class or things like this, it wouldn't have been something they probably tussled with quite so much. To me,
Suzanne Thornton
this is an I am at a liberal arts college. And so I think this is a very sort of liberal, artsy way of answering your question, but to me this is this is sort of an example of where statistics intersects with other disciplines and other fields of study. And that includes history, sociology, psychology, biology, you know, the whole the whole world that we could study. So I think as an as an educator, there's and as a statistics educator in particular, there's a bit of a balance that I feel we have to weigh between showing the power of quantitative analysis and statistical conclusions, gathering data in a kind of controlled way talking about what information we can and conclusions we can draw that are supported by observable data. But there's also a balance with kind of reality, which is, this does not answer questions with absolute certainty. It does not. There are assumptions that we make every step of the way in doing any sort of quantitative analysis, even from just assessment items, right on a quiz or something. So there are assumptions about what is measurable, how that measurement might be reflected how we could represent it numerically, there are assumptions every step of the way. And so it's less about kind of convincing students that that Oh, I have an answer. I have the way to do things, I have the best practices and more about what kind of questions do I need to be able to answer right about what I want my data to be describing? What are the important features of the data? What are important observable features? And how do I describe that in a coherent way? So it's, I It's this kind of balance, I suppose. And also, this is kind of one of the reasons why the the title of our article is, is not best practices for gender and sex data. But it's towards best practices, because our methods, our statistical methods of analysis, computational approaches, they evolve with society. The same thing about what kind of data we can collect, what kind of variables we might be interested in, this evolves, and grows as our society changes. And so there's not necessarily going to be a best absolute way to do things all the time as, as we statisticians know all too well.
Rosemary Pennington
Well, that's all the time we have for this episode of stats and stories. Suzanna Dooti thank you so much for joining us today.
Dooti Roy
Thank you. Thank you for having us.
Suzanne Thornton
Very much appreciate this opportunity.
Rosemary Pennington
Stats and Stories is a partnership between Miami University’s Departments of Statistics, and Media, Journalism and Film, and the American Statistical Association. You can follow us on Twitter, Apple podcasts, or other places you can find podcasts. If you’d like to share your thoughts on the program send your email to statsandstories@miamioh.edu or check us out at statsandstories.net, and be sure to listen for future editions of Stats and Stories, where we discuss the statistics behind the stories and the stories behind the statistics.