Matt Fox (00:00): I'm going to strongly disagree that they're cute. Strongly. I did not find...if they were breeding those for cuteness... Jessica Leibler (00:07): They're very slobbery. Matt Fox (00:09): ...they failed in my view. Jessica Leibler (00:13): Welcome once again to Free Associations from the Boston University School of Public Health: the public health and medical journal club podcast for anyone as confused by the latest health study as I am by the phenomenon of used sneakers. Matt Fox (00:29): This is a phenomenon? Jessica Leibler (00:31): Maybe it's not a new phenomenon, but I just spent a week with my nephew in Miami who's obsessed with the used sneaker market from especially like Nike Air Jordans going back into the eighties and nineties. Matt Fox (00:47): Just so I understand. To own them or to wear them? Jessica Leibler (00:50): Both. I think primarily to own them, but this is a whole...no, no, because I always felt like for me, used... Matt Fox (00:56): Footwear and underwear, I'm not going used. Jessica Leibler (00:57): ...footwear was not something that I wanted to delve into for myself, or people in my family, but it's a very popular thing. Maybe it's like it's just teenagers. No, you two are looking at me kind of like no thanks. Matt Fox (01:09): My kids, they do used things but not sneakers. Allegra Gordon (01:14): I'm here for it. I love a vintage object. I'm here for it. Matt Fox (01:16): Even footwear? No footwear. No footwear. Jessica Leibler (01:20): This is distinct from the market of... Matt Fox (01:23): People's feet are gross. Jessica Leibler (01:23): ...used footwear by Shaquille O'Neal where there's an enormous sneaker and he wore it in a game and he bled in it. Something like that and people purchase this and they're like, wow, this is amazing. It's just so random, used sneakers, and it's a question of how good do they look? How white are they? Do they have scuffs? And then there's a whole subculture. You guys are thrilled by this. I know. Of people trying to detect whether the sneaker is actually like an antique real sneaker or if it's like a knockoff. Matt Fox (01:51): Like the antiques road show of footwear? Jessica Leibler (01:54): Exactly. Exactly. Matt Fox (01:56): Intersting. I am disturbed. Allegra Gordon (01:57): I feel like there would also be an aroma element. I don't know. Right? Like in addition to... Jessica Leibler (02:00): A 30-year-old sneaker Allegra Gordon (02:01): ...the authenticity of the smell. Jessica Leibler (02:03): Right. You could really... Matt Fox (02:04): I don't want to be on that panel. Jessica Leibler (02:05): You don't want to be on that panel. Matt Fox (02:07): At all. Nope. Jessica Leibler (02:08): On that review panel. No thanks. Matt Fox (02:09): Nope. Nope. Jessica Leibler (02:10): All righty then. I'm Jessica Liebler from the Department of Environmental Health at the Boston University School of Public Health, here with Dr. Matt Fox from the Departments of Epidemiology and Global Health, also at the Boston University School of Public Health. Hi Matt. Matt Fox (02:25): Good to be here. Good to not be hosting. And are you going to note that we're being videoed for the first time ever? Jessica Leibler (02:30): We are. I'm not sure if people are going to all be experiencing us totally multimedia, but we are being videoed for our whole episode, which is very exciting slash nerve wracking. Matt Fox (02:42): Very disturbing. Jessica Leibler (02:42): A little disturbing and a little disturbing. We are also joined by our returning and amazing guest host, Dr. Allegra Gordon from the Department of Community Health Sciences here at BUSPH. Welcome, Allegra. Happy to have you back. Allegra Gordon (02:54): So glad to be back. Thank you both. Matt Fox (02:55): Returning champion. Jessica Leibler (02:56): Returning champion. As a reminder to our listeners, please give us a rating on Apple Podcasts, Spotify, or any of the major podcast sites where you find us to help public health and epi people find us themselves and also optimally to pad our egos so we feel good enough about ourselves to keep this going. Matt Fox (03:12): That's really the main reason. Jessica Leibler (03:13): That's the main reason. We love a positive review, what can we say? Now onto the show. Today in our first segment, our journal club segment, we're going to look at a study on racialized tweets over time in the United States. In the second part of our podcast, our deep dive segment, we'll talk about the effect of remote work on collaboration and innovation in research and in our amazing and amusing segment, we'll get into some things that have made us laugh out loud lately. So into segment one, we're going to look at an article that evaluated tweets in the United States over a 10 year period to consider cultural racism. It was published in the journal Epidemiology in January, 2024 very recently, and the study was entitled A Decade of Tweets Visualizing Racial Sentiment towards Minority Groups in the United States Between 2011 and 2021 by first author Thu Nguyen of the University of Maryland School of Public Health. I didn't see any headlines on this one. So Matt, let me start by asking you to describe what this study was about and what they found. Matt Fox (04:15): Yeah, okay. So a super interesting study. It probably does not need to be said that social media has changed the way that people interact and communicate. And because of that, there's lots of information now that is publicly available that people can use to analyze changes in things like perceptions of things over time. And a number of researchers have gotten into this sort of scraping data from Twitter. Twitter is the most common one because Twitter is a public access and at least for a long period of time, Twitter had a way that you could actually download parts of all the tweets in the world kind of thing. And this study, they wanted to see if they could use Twitter to measure area level racial sentiment as an indicator of cultural racism. And so effectively they set out to look at geographic differences in changes in sentiment towards different racial groups in the United States from 2011 to 21. (05:12): And they're looking at cultural racism, and I'll read this to you because this comes directly from the paper. They say cultural racism is the infusion of the ideology of racial hierarchy into the values, language, imagery, symbols, and unstated assumptions of the larger society. And research has shown that regional differences in self-reported experiences of racial discrimination exists. And so these researchers were interested in looking at these differences across the United States as well as changes in those differences over time. This was a descriptive analysis and I just want to emphasize that because we often look at things in which there's a causal implication and there are some things in this that they sort of attribute maybe causally, but generally speaking this is a descriptive study looking at temporal and geographic trends and online discussions referencing different racial and ethnic communities in the United States. They did this by taking data from Twitter's API. I never remember what does API stand for. Nick, you probably know what's an API? (06:15): It's something that allows two systems to talk to each other. Jessica Leibler (06:18): Thank you, Nick. Matt Fox (06:18): Effectively, it's a way to access the tweets and download them. And my understanding is Twitter has put restrictions on this now, but at the time you could get large samples of tweets and so they took a random 1% sample of publicly available tweets from January 1st, 2011 to December 31st, 2021, limited to English language tweets and limited to those that came from the United States and specifically ones that they knew came from the United States because they had state information associated with them. So they ended up with over 55 million tweets from about 3.7 million users who had location and they had 90 race-related keywords that they were using to try and pull out the tweets that they considered to be part of the dataset that was useful for analyzing changes in these trends over time. And then they did sentiment analysis, and I'll say some of the terms that they used, but I think for most people these won't be all that important or meaningful, but they used supported vector machines, a supervised machine learning model for sentiment classification. (07:26): So you've got a very fancy computerized model that is "reading" the tweets and then analyzing the sentiments in them. Then they did a number of things to try and train this machine to try and be able to pick out which were the ones that were associated with the things they wanted and then to be able to code them as either positive, negative or neutral. So they did things like trained sentiment classifiers on a combination of pre-labeled data sources like Sentiment140, Kaggle and Sanders, I don't know what those are, but Kaggle and Sanders makes me think of Colonel Sanders and then I get hungry. Jessica Leibler (08:01): It's very sophisticated. Matt Fox (08:01): They took a whole bunch of tweets and labeled them themselves as positive, negative and related them to these things to sort of train this machine learning algorithm to allow it to go off and do its own classification. (08:15): And I mention all these different things, not because I think that anybody's going to go and look them up, but because just to note that they didn't just sort of say, we have this tool, it does some classification. They worked really hard to try and make sure this machine learning algorithm knew what it was doing, but even then it's very hard to get these things to be perfect. So you're looking for very good. A number of other things like frequency-inverse document frequency, which I don't even know what that is. They compare their model performance with other widely used models and things like that, and they use fivefold cross validation. Again, all these details that I think you don't need to understand this, but I say it just to emphasize the fact that they just sort of pluck a model off the shelf and say, have at it and then we'll just trust whatever it gives us is good information. (09:02): Then they sort of started to do temporal and geospatial analysis. So looking at changes over time, looking at changes in what's happening across the United States, ran a number of descriptive analyses looking at those temporal and geographic trends in online discussions referencing different racial and ethnic communities. And what they found was that overall there was an increase in negative sentiment tweets referencing racial and ethnic groups over the decade followed by a slight decline in the final years. Southern states tended to have higher proportions of negative sentiment tweets with Alabama, Arizona, Georgia, Louisiana and Mississippi all in the top 10 states. Twitter volume and changes in Twitter policies seemed to impact the findings and so they couldn't really say much about the entire United States, but they could sort of say what changes were going on across the different regions. They found that tweets referencing black Americans were the most frequent with increased references to other minoritized groups over time. They saw unique patterns for the different groups with spikes in sentiment aligning with notable societal events like the Black Lives Matter protests or the Atlanta Spa shootings. Negative sentiments towards Middle Easterners increased following the San Bernardino shootings and the "Muslim ban" in the United States when Trump came into power and tried to prevent Muslims from entering the United States. States that experienced the greatest change towards more negative racial sentiment included Idaho, Utah, Wyoming, Vermont, and Maine. So overall they found there were these notable changes over time and sentiment that they could correlate with specific states, but that overall the general trend was an increase in negative sentiments towards minoritized groups over time. Jessica Leibler (10:44): Thank you, Matt. That was a really comprehensive review of this paper. I would note something and then I'm going to ask Allegra your thoughts on this one. One of the things the authors did here is that they also presented this analysis almost as a methods paper and they really walked through in substantial detail how they culled this information from these Twitter sources, including how they employed graduate students in the kind of validation approaches and in informing the machine learning models. And then they kind of applied their approach towards this analysis of racialized tweets over this 10 year period. So they had a lot of methodology in the manuscript and then this particular analysis. Allegra, what were your take home points? What were the big things that jumped out to you with this one? Allegra Gordon (11:27): Yeah, so a couple of things. Well one framing thing that I just want to layer in is that they noted that this paper, this sort of case study demonstration was exclusively focused on negative sentiment, particularly towards minoritized groups and that they referenced another paper where they looked at positive sentiment. So we don't have that information here. So just to note, our discussion is focused on the negative sentiment because that's their focus here, but they allowed those to be sort of orthogonal constructs I think, that both might be occurring, but here they're focused on negative sentiment. So the first thing that stood out to me was that I really appreciated it as a methods paper and that they kept an eye towards trying to make it more accessible to researchers who might not be as well versed in all of these models. They mentioned at points that they were applying sentiment analysis in part because it didn't necessarily require the kind of computational intensity, the big computers that many might not have access to. (12:27): It might be more accessible as a method. So I appreciated that orientation towards opening up the field and trying to bring more folks into it and demonstrating the potential utility of this and especially in social epidemiology. And that links to my second highlight that I really appreciated their overall framing and what they are setting out to do here because it is something that I think is a deep need in social epidemiology and public health generally. They talk about at the start the critical need to measure cultural racism as they define it or think about how we assess racism and racial bias at a population level or at higher levels of the social ecology, which that's been a methodological challenge for some time and it really requires creativity and interest in thinking about new data sources and tools. And so that's what they're demonstrating is sort of let's think about how we might apply this in some ways population level, even though it is in no way a probability sample, data source to thinking about measuring this really sticky and critically important set of structural forces. (13:37): So I really liked that the aim that was key. And then two other things I'll highlight that I thought were real strengths. One is the rigor. You noted the training data. Both of you noted the training data and their mention of using human coders, graduate students, as a component of the training data. I found that crucial. We know that bias is built in to many of the existing algorithms, and so the fact that they brought that in was crucial to me. I almost wished all of their training data could have been based on human coding and that probably wasn't possible. They wanted a larger scale and probably in these kinds of models you need much higher quantity of data than is possible. Matt Fox (14:19): Hard to code 4 million tweets by hand. Yep. Allegra Gordon (14:22): Yeah, or even the training data was like, I don't remember, 20,000 something. Matt Fox (14:26): Yeah, it was in the thousands, yep. Allegra Gordon (14:26): Even that. Matt Fox (14:27): Even that's hard. Allegra Gordon (14:28): They coded something like 2000, it was admirable. But when I think about our interfacing with these systems as somebody who doesn't do this kind of analysis, I want other researchers to be as attentive as they were to building in sort of the human lens, making sure that humans were defining the terms as much as possible, although they couldn't overcome all limitations, which I think we'll get into. And then the second strength I thought was their ability to disaggregate by race/ethnicity across multiple different racial ethnic groups, which can be a limitation in our typical epi samples when oftentimes groups will get lumped together related to power or without real justification. And with a data set of this size, they were able to focus on, in this case negative racial sentiment related to specific groups in a way that I think is really important as they draw attention to it by thinking through some of the different historical moments that might be very specifically related to the experiences of folks in different groups. (15:35): So those are the two or three, I dunno however many highlights I just said. Jessica Leibler (15:39): Thank you. No, I've lost count, but those are really, really interesting thoughts and I agree with so much of what you said about this paper. I feel like I also appreciated the methodological lens and the level of detail that they went into. I think my quibbles were more with the interpretation of some of their findings, the alignment with some of their findings with current events and the bigger picture inference of what you get from this sort of analysis. I wasn't exactly sure I always agreed with their big picture analysis because I feel like when you think about who's on Twitter, or X as it's...who are on these social... Matt Fox (16:18): I refuse to call it X. Jessica Leibler (16:19): We can call it Twitter. That's fine because I'm still in the 1990s anyway, so I think, right, so who are on these platforms and what segment of the population do they really represent? (16:32): They represent a vocal and probably a little more strongly feeling representing group in the population. And so they might in some ways be more reflective of extremes, especially people who might be tweeting racialized content. These folks are more likely to be more extreme in their beliefs than the general public who's probably not on Twitter. And there's the two stages of are you on the platform and then are you tweeting about this content? And so the group of people who are tweeting about this content to me seemed to be a fairly more likely to be more extreme segment of the population. And so extrapolating that finding to cultural racism, I wasn't exactly sure that you could say this is kind of the cultural construct of the United States. This seems to be a wing or segments of the population who are having these feelings. (17:28): From a methodological standpoint one of the gaps that I saw was I would've liked to have seen some sort of committee decide the racialized words that included people in some of these groups who were being commented on. And the author mentioned there were a couple of graduate students who kind of engaged in this analysis, but I would've liked a larger process to figure out what are those words? And they talked through how some of the words, they used an example of the N word that in different applications could be a derogatory term or could be a cultural reference of strength and of community. And I think that was just one of many terms and they didn't go too far into depth about the variation in some of these terms or how they might be used by ingroups or outgroups, and I found some of them were very limited, so I would've appreciated a little more engagement on the terms that they looked at, even though it's hard to quibble with a list of 90, they had a substantial list to start with. Matt Fox (18:33): Okay, so I...longtime listers of this show will know that I always write down my priors before I read a study. This is a descriptive study so I can't say...my prior isn't, "is there an effect, isn't there an effect?" My prior is really around whether or not I think they can do what they say they're going to do, which is describe these trends over time. And I go into this very, very skeptical. It has nothing to do with the authors, it has nothing to do with the methods. I just go into this very skeptical because we've done some data linkage problems, so not exactly the level of what they're doing where they're actually trying to code sentiment. We're just trying to link information from two data sources, information in lab data sets where you're trying to say, is this the same person over time? And information gets input in different ways, so names get misspelled, they get first name gets put in for last name. People have two different names that they use, so sometimes you get one, sometimes you get the other. (19:26): And you have all this information and you're trying to link it together and say, "this is the same person," and we had to go through these same exercises of hand coding a bunch of them and training this thing to be able to figure it out. And I am shocked by the number of times people disagree on whether or not this is the same person and that is so much simpler than this, so my bar is going to be really high that you can do this well. They moved me, they moved me in the direction of believing that they are closer to being able to do this than I was before I read the study, but I still harbor that same skepticism because I think this is really hard to do. (20:02): The other thing is, Jess, you mentioned the generalizability. I mean I think we can say with some confidence that there is changes in what's going on in the Twitter community, but as you say, who does that generalize to and how much is the algorithm itself pushing people into more extreme comments because that's what the algorithm does? So we don't know that, but what I don't see here is any sense for how is everything else changing over time? Not that that would negate their findings anyway, but my sense is Twitter is just getting more negative across the board whether it has to do with race or just politics or whether or not we like the Teletubbies, it's just gotten angrier and more negative. And so it would be interesting to see how things correlate with other changes in sentiment over time because I think that actually would tell you a fair bit. So my struggles are around the ability to correctly classify these, I mean they give this example of that the algorithm can conflate something like I hate X group with hatred of X group is bad and that would be seen both as negative when in fact one is clearly positive. That obviously is a problem because now you're getting the exact opposite inference from those. Over time, with large numbers, I'd like to think that it's getting more right than wrong, but I just don't know. Jessica Leibler (21:31): Right. I mean I think to your point about negativity growing in general, to me that was the biggest and maybe most interesting take home from this analysis altogether. If you jumped in at all to some of their, I think it might've been in some of the supplements, they also showed anti-white commentary and anti-white tweets and how those increased also over this 10 year period basically at the same clip as the anti-black commentary. So I saw that and I was looking for some digestion or interpretation of that in the paper that seemed to me to be, yes, there are increased negative comments towards many groups, but there was just if anti-white commentary was maybe the referent category, it didn't seem that there was disproportionate anti- other minority groups commentary and I didn't see them kind of contend with that. That seemed to be just a reflection of racialized tweets and increased animosity and negativity on the platform. Not to diminish the absolute increase across all of these other groups, but it wasn't always clear that there was a relative increase comparing, for example, just anti-black sentiment and anti-white sentiment and I couldn't quite wrap my head around that. They didn't delve into that in the papers, but the data, they presented the data in the supplement. Allegra Gordon (22:59): To that point, so perhaps there's the relative question, but also this method allowed them to document some absolute differences, which also could be really valuable in terms of thinking about the pervasiveness of negative sentiment and thinking about who's most impacted for us as it relates to health inequities. One thing I'm just reflecting on both of your points and one of the things that occurred to me, and I wonder if it would help a little bit with the thinking about this, if there was a little bit more precision in their sort of main construct. It seems to me that what they're assessing is expressed racial sentiment and that maybe talking about it that way could be helpful in overcoming sort of a block. I'm thinking about, oh, is it growing over time? Is it not? To me that is a meaningful construct to assess because increased expressed racism is going to have a disparate toll on minoritized groups and racially minoritized groups, and that seems relevant to me as a potential indicator that could factor into some of our more social ecological studies. Would that help a little bit if there was a little more precision around that language? Matt Fox (24:11): Well, I guess I would say, I mean the answer is yes. I think I want to emphasize, I mean I don't dispute their general trend findings here. I believe them and I think they've done a really good job. My skepticism is around can we say exactly how much of an increase there has been or how much do specific states have the biggest changes, those kinds of things, and what is its sort of interpretation in the larger context mean, but I don't dispute their...I think they've done from what I can make of it, a really good job of what they have done and they should be applauded for that. I don't know that we're at the development of these methods that I would be super confident in saying those numbers that I see are the right numbers. Allegra Gordon (25:03): Yeah, the other thing that occurred to me reading it, and they talk about this as a limitation, that there's an inherent data limitation here, which is a real challenge, which is that they don't, aside from location, they don't really have sociodemographic characteristics of the content producer of any of these tweets. And so the tweets, to the point about whether the machine can get right, whether it's negative sentiment or not also, it's de-contextualized, so we don't know whether there's negative racial sentiment coming from a within group perspective or from an outgroup perspective, which have very different meanings. Jessica Leibler (25:38): Right, right. I thought one of the more interesting applications of this analysis too was when they did the county level data over time to show the proportion of tweets from a given county. This also might've been in the supplements that had racial negativity, and the question would be what would you do with that information from a public health standpoint if you identified, and the authors are very clear throughout the paper that being in environments of racism is really detrimental to people's health. And so that was where towards the end as I was looking at some of their maps, which I thought were really lovely and well done and interesting, what would you do with this information from a public health intervention perspective if you knew that there were certain states or certain really small geographic areas where there was a lot of negative sentiment that was emanating on this platform, how could we use that in a public health context? Allegra Gordon (26:34): That's a nice point about their maps too. For anyone listening who's interested in learning more, they did a really nice job in terms of dissemination and how we share out our work of building an entire web-based tool where people can look through the maps that they generated at multiple geographic levels, which it's rare that a project is able to provide that level of access to the data. So I thought that was really special. Jessica Leibler (26:58): I also loved the video. I don't know if either of you saw the video that they included as one of their supplement. It was the first author, it was three minutes and she just did a brief overview of the paper. I thought it was brilliant. I was like, I'm going to do this going forward. Matt Fox (27:10): Oh, wow. I missed that and I would like to go back and see that. What a great idea. Jessica Leibler (27:12): It was terrific. She had slides and she just annotated and she's like, "Hello, this is our study. This is what we did." Matt Fox (27:17): Oh, what a great idea. Jessica Leibler (27:18): I thought that was a really nice idea, and I do think that this is a huge data source, and so starting to add to the literature on how to use it is really valuable from an epidemiologic perspective. Matt Fox (27:28): Agreed. Jessica Leibler (27:28): Okay. Allegra Gordon (27:28): I know we have to move on, but we also have to say bittersweetly that this is an amazing data source, and as you pointed out, it is no longer possible to collect data the way they did because of the change in hands of Twitter, now X, Jessica Leibler (27:41): Right, right. Allegra Gordon (27:41): And not only the removing of the research API and now putting it behind a paywall and really limiting how much, so now researchers can only access a really small number of tweets per month, relatively speaking, and I just saw a description of a survey of researchers who do social media research who are really feeling the chilling impact not just of that incredible loss. The demographics of the platform have changed, whereas it used to be a huge common...public common. It is not. And we have a very litigious CEO who is actually suing organizations that are researching the rise in hate on Twitter, now X. So that to me, as somebody who is interested in social media research as a public health...as an angle of our work is very sad, and I think researchers are now having to be creative to figure out where they can access similar snapshots of public sentiment and exposure. It's just a huge loss for us. Jessica Leibler (28:41): We had similar experiences in the post-COVID, late-COVID with mobility data, with cell phone mobility data where a lot of it was very publicly available through Google and through other sources for research. And then there was a closing of some of those data sources, I think for concerns about liability, which from their perspective makes sense, but is a real loss as you think about the power of some of that data. Matt Fox (29:05): Huge loss. Jessica Leibler (29:06): All right, let's pivot a second. Matt is getting anxious. Let's pivot. Matt Fox (29:10): I'm anxious, am I? Jessica Leibler (29:10): Let's pivot into second two. Matt Fox (29:13): I'm not used to being in this seat, that's all. Jessica Leibler (29:15): I know. He's going to be fine. Don't worry folks. Matt Fox (29:18): I'm going to be fine. Jessica Leibler (29:18): In our second segment, we're going to talk about a piece that was published in nature entitled Remote Collaboration Fuses Fewer Breakthrough Ideas by First Author Yiling Lin of the University of Pittsburgh. These authors considered the purported fact, I'm saying purported because this was new to me, but I guess this is fact that while research teams and collaborations are increasingly broad and even global through technology and possibility for remote work rate of new discoveries and innovation is declining. They discuss what they call the "recombinant growth theory," which posits that when there are more and different minds together, innovation should naturally follow and indicate that despite this expansion of collaboration, this actually has not happened. Their overall claim is that remote collaboration is detrimental to the generation and inception of new ideas when knowledge is limited, but that remote work does in fact speed and enhance productivity in work products once an idea is on the table and is in place. Matt, what do you think about this? You look like you have something to say about this one. Matt Fox (30:18): I don't know. I mean, so this fits with a general sentiment that I have heard many times that if you want to be productive, you should work alone, stay home and work there. If you want to generate new ideas and be creative, you should be in an office surrounded by other smart people and be talking to them and communicating. I've heard that. I haven't heard it backed up with a ton of data, but I've heard that sentiment and it generally fits with my experiences that I can stay home and work by myself and be much more productive, but that if I'm in the office, I talk to people and we generate ideas, we come up with new projects, so the creative process is probably better. Measuring that is a much harder thing to do, and their methodology, which I honestly I couldn't even really totally describe to you, but it has to do with the idea of using citations of things and the idea being that if you have something that's truly groundbreaking, that's going to be the thing that everyone cites back to the original at some point, sort of looking for those. I don't know that I totally buy that that is a perfectly valid way to assess whether or not more breakthroughs are occurring. I can't say why though. So it's not like I can poke specific holes in that methodology. There's just something about it that doesn't feel...it's not that I feel like it's wrong, I can't say to you, I look at that and I'm immediately convinced, and so I'm left sort of feeling like, well, this fits with what I believe, but that doesn't make it right. So that's where I am. Jessica Leibler (31:51): I think that was my experience too, that I felt like this seems to make sense, but the methods were beyond really what I was going to grasp. Allegra? Allegra Gordon (31:59): Yeah, as I was reading it, I kept thinking, this is confirmation bias. Jessica Leibler (32:02): Right, right. Matt Fox (32:02): That's what I think. Allegra Gordon (32:02): Like, "I agree with this. This must be sound because it is exactly what I believe," but... Matt Fox (32:10): Is that how you evaluate? Allegra Gordon (32:12): That's how I approach all science. Jessica Leibler (32:13): Also how I view every aspect of the literature. Matt Fox (32:13): Must be sound, I believe it, it must be sound. Allegra Gordon (32:16): I just want my own views to be reinforced like all humans. Matt Fox (32:19): I totally understand. Allegra Gordon (32:21): But I was trying to keep that in check and it was a little bit challenging to evaluate because I had never encountered a summary of disruption or new thinking like this, understandably, because I think it's really hard for us to think about. Maybe one challenge is that it felt like this is unidimensional, right? They're hanging everything on what they call their "D score" or their "disruption score," which was a little hard to wrap my mind around, but it's an interesting proposition. Right, okay. I think the way they described it is when a key paper is considered disruptive, the papers after it will cite that paper alone and not its predecessor papers, which generally I don't like becuase I like it when we cite the whole history, but that makes sense. That becomes such a novel idea that people only start citing that paper, not the earlier one, and that's how they kind of created this disruption score. I would like people to think creatively. I mean, I'd be curious if you have other ideas on how else do we measure really interesting creative thinking and new ideas? I was left realizing, well, we have the D score. I don't know if I have other ideas. Jessica Leibler (33:31): Right, right. Matt Fox (33:33): I don't know that I do either, but can I just ask you, so if you think about your field of research and you just sort of walk back through the past 10 years, how many ideas immediately come to your mind that you would say, "these are paradigm shifting ideas?" I'm... Jessica Leibler (33:49): Ideas that we've had or ideas that you have encountered? Like ideas that Allegra has had herself or... Matt Fox (33:55): No, no, no. The larger field. I've had zero. So for me that is completely useless. Jessica Leibler (34:01): But maybe that's because you work alone too much. Matt Fox (34:04): That's probably it, but no sort in the field of HIV, we've gone through several paradigm... Jessica Leibler (34:08): How many ideas are we really talking about? Matt Fox (34:09): ...shifting changes around treatment as prevention? Allegra Gordon (34:12): Absolutely. Matt Fox (34:12): We've had introduction of new drugs that have completely changed the field, but I don't think of those as breakthrough ideas like paradigm shifting ideas. But I do think that the shift from HIV treatment can prevent people from getting infected in the future, and therefore we should treat as many people as possible as early as possible under the idea that that could, in the future, end the epidemic. I can think of three or four of those kind of ideas within the field. I don't know, if I were to go back, if you were to go back longer, you would say, oh, there've been a ton more. But even if there are, how much of that is just because the field was younger then and there are a lot more new ideas, and over time... Jessica Leibler (34:50): Right. Matt Fox (34:50): ...there are just going to be fewer breakthrough ideas within a field? So it is hard for me to say what it means to say there have been fewer over time. Allegra Gordon (34:59): Yeah. I think one piece that is maybe the most useful take home to me, if we buy their analysis and their use of this score, is that they did an analysis related to training and mentorship where, based on their comparing people based on their kind of impact score and sort of differences, so assuming that more junior folks like me are going to have a lower impact score, and if I'm publishing with somebody with higher impact scores, there's some assumed mentorship happening, and if we're code generating ideas together, that to them is value added. That's us generating new ideas. I thought that was interesting. And the idea...their theory is that, which they say they have evidence for, is that that happens more effectively in-person rather than remote. Matt Fox (35:45): Yep, yep. Jessica Leibler (35:46): Mhmm. Allegra Gordon (35:46): And I thought that had some of the most concrete implications to my mind, even if we're not sure if this is the only way that we could measure creativity, I really liked that point and it really underscored for me the value of thinking about training lineages and how much I've benefited from in-person environments where I've been invited to engage more deeply. Matt Fox (36:05): Agreed. Jessica Leibler (36:06): It led me to think about how we can, in the reality that many of us in our field work remotely for at least some of the week, and we enjoy these collaborations with people who don't live near us, that would not have been so easy or at all possible in decades past because of technology. How can we most effectively use time we have together in person, even if it's limited. Like I just got back from a week long meeting in Guatemala about kidney disease in Central America, for example, and we had all the leading researchers in this field were all at this meeting. And I was thinking about it in context of reading this article. And what we did is we kind of summarized, we got in groups and we summarized research to date trying to come up with these almost like summary documents. (36:58): And this is a disease that's been very intractable to understanding causality over many, many years. And then I was wondering if we, in light of this paper, if we could have structured those in-person interactions differently to try to generate new ideas or to try and we work very productively in our remote teams, but it was just a very kind of concrete example. How can you structure those in-person if they're brief, if they're limited, if they're once a year or once every three or four years, how can you best structure those to generate new ideas and to kind of push away some of the other stuff that you can do fine working over Zoom and Slack? Matt Fox (37:34): Yeah, it's a really interesting point because prior to the pandemic, I work a lot in South Africa, we went a lot to South Africa. Pandemic hit. We had to stop going for a year and a half, two years, whatever it was. And what we learned over that time was they don't actually need us there as much as we thought they needed us. They're really good at what they do and they don't need us there. The downside of that though is therefore I have less contact and communication, and I have absolutely noticed fewer new ideas because we're just not having those conversations where you say, "oh, I saw this interesting thing, or they're thinking about changing this policy and what do you think it's going to mean?" And then you think what the implications are and that leads to new ideas and there's just been fewer of them because we're not engaged in the same way. And so to your point, I think, yeah, I think we need to think about ways to structure the time when we are together to be focused on the stuff that I think is going to be groundbreaking and say we can do the rest remotely. Jessica Leibler (38:32): Right. And maybe that even relates to kind of the symposium-style meeting where people are reporting on research findings that maybe those are activities that can be done very effectively over Zoom, someone just reporting on their research. But then how do you create those maybe smaller conversations that then can lead to new ideas or new collaborations? Allegra Gordon (38:52): Yeah. Or you use those as the launchpad, the fodder for more structured breakout groups. And on the flip side, I'm thinking about...there's been a lot of work in thinking around digital learning, but less around digital research collaborations and what's the equivalent of pedagogy for our digital research collaborations. So your point on the flip side made me think, how can we try...are there ways we can create digital spaces like meetings that are dedicated to a freewheeling conversation? Are there ways that we can foster that? Does it have to be in-person? Is it possible? We tend to not block out that kind of time, right? In our digital meetings, we tend to have these highly structured meetings that are very task oriented, which is aligned with these findings. Matt Fox (39:39): Yep. Allegra Gordon (39:39): But maybe we can also think creatively as we have with remote learning and classroom spaces about how to have more flexible, fun, free-thinking conversations too. Jessica Leibler (39:51): It's true that, I mean, the implication of the digital learning and remote learning is really interesting. For the first time I taught in our online program this past year, and it's a really different format where it's almost a different kind of teaching where the students have very limited live interaction, and certainly no in-person interaction unless they happen to do it on their own because they're in the same place or whatnot, which they certainly could do. But it's interesting to think about the strengths and limits of that approach where you lose that real, real life and you lose often the unscripted conversations also. Matt Fox (40:28): I would not thrive in that environment. I'd be very studious about doing the work, but what I wouldn't get is the conversations... Jessica Leibler (40:36): Critical thinking. Matt Fox (40:37): ...that you have when you...that make you realize you didn't understand it as well as you thought you did. I think for some people that digital learning is great and it works perfect. I just think for me it wouldn't for the same reasons. Jessica Leibler (40:48): It's challenging. Matt Fox (40:48): That's just that what you're doing with that space is very different from what you're doing from in-person communication. Allegra Gordon (40:55): And it's harder to push into challenging and uncomfortable conversations remotely that can happen in a different way when we're gathered. Matt Fox (41:01): Good point. Jessica Leibler (41:02): No, I think that's really true. I think that's really true. Well, thank you for this really interesting conversation. I can see Matt's getting a little anxious again that we move on to the next segment. Matt Fox (41:11): The timer is facing me. And so if you hadn't done that, I wouldn't have gone into my... Jessica Leibler (41:17): We, we started like eight minutes late. So we're okay, but we're going to... Matt Fox (41:20): I didn't know that. Jessica Leibler (41:21): We did. Yes, we did. We're going to shift to segment three, amazing and amusing. Our last segment, which needs no introduction for our dedicated listeners. So let's just get into it. Allegra, you want to go first and share with us what you have? Allegra Gordon (41:33): Yes. This is public health adjacent. Jessica Leibler (41:35): Public health adjacent. Matt Fox (41:36): Alright. It does not have to be. Even better if it isn't. Allegra Gordon (41:39): It's not direct. Alright, so you know those bristly spikes that humans put on top of buildings and ledges to deter pigeons and other birds from roosting in places that we humans don't want them? This article I came across this week is about the ability of living creatures to make lemonade out of lemons, if you will. It's a piece by Hiemstra and colleagues. It was published last summer in the Journal of the Natural History Museum of Rotterdam. Not a journal I traditionally look at, but... Matt Fox (42:06): Oh, I read very episode, every issue. Allegra Gordon (42:09): ...I was delighted to find this. It's titled Bird Nests Made from Anti-Bird Spikes. Jessica Leibler (42:15): Ooh. Matt Fox (42:16): Oh, smart birds. Allegra Gordon (42:17): Yes. They're taking 'em back. So in this very charming piece, the authors document...first they document, which I found interesting, the history of research on birds use of human-made, sometimes very sharp, materials for nest building. So apparently the first report of a crow's nest made of barbed wire was in 1933. Jessica Leibler (42:34): Oh, wow. Allegra Gordon (42:34): So this goes back. Then they describe what they see as a really new development, which is evidence of two species of birds, carrion crows and magpies, tearing off anti-bird spikes and using them to build their nests. And they were particularly thrilled about one case where the magpies, they weren't only using the anti-bird spikes to make the sort of nest base, they were putting them on top to deter avian predators from above in a way that was sort of analogous to what they would do in less urban environments with thorn branches to protect their nests. So basically they are putting them to use as anti-bird spikes as birds. So although this isn't exactly... Matt Fox (43:17): Birds are... Allegra Gordon (43:19): And they're ripping them off. Matt Fox (43:20): ...creating anti-bird spikes. Jessica Leibler (43:22): Wow. Allegra Gordon (43:23): Yes, they're taking them from the local hospital, building it... Jessica Leibler (43:26): It's kind of like a big middle finger to humans who are trying to keep the birds out, right? They're like... Allegra Gordon (43:31): It's beautiful. Jessica Leibler (43:32): ...we're going to take these and we're going to put them in our homes for decoration. I dunno, they don't have really a middle finger, but... Matt Fox (43:40): Birds' middle claw? Allegra Gordon (43:42): This is the equivalent. It's a spiked finger. Matt Fox (43:42): Talon? Talon? Jessica Leibler (43:43): Talon, middle talon. Right! Allegra Gordon (43:46): So I do feel it's public health adjacent because it really speaks to resilience, creativity, and a rebellion against the dominant power structure. Jessica Leibler (43:54): And they probably could not figure, could not have figured out how to do that over Zoom. That's like an... Matt Fox (43:59): No, no. Have you been to a bird meeting on Zoom? It's not good. Jessica Leibler (44:04): They're pretty dry. They're pretty dry. Oh, that is so cool. Matt Fox (44:07): Nothing gets done. Jessica Leibler (44:07): Thank you for sharing that. Matt, what about you? Matt Fox (44:09): Okay, so I have talked about this before, but there was a new paper, so I just wanted to bring it up. So this was a Nature piece for end of last year, but they were talking about a paper that was published on a surge in the number of supercharged researchers. You know what these are, supercharged researchers? People publishing more than 60 publications per year kind of thing, right? So immediately... Jessica Leibler (44:34): We have some of those in our midst. Matt Fox (44:36): ...immediately you hear that and you feel bad about your own productivity, but they are sort of talking about this in the context of...effectively if you are publishing 60, more than 60 papers a year, are you really reading these papers? Are you potentially...not all these are real, you're paying for authorship kind of thing? Or even potentially fraud involved kind of thing? So this group of researchers noted, they sort of went and looked at this and noted that there's been this sort of large increase in up to four times more researchers putting out more than 60 papers a year than there were a decade ago. And then they looked sort of by country and noted that places like Saudi Arabia and Thailand are seeing the sharpest increases in the number of such scientists. And they postulated various reasons why this might be happening. They specifically looked, by the way, actually that was interesting. They looked over lots of different fields. They took out physics where I guess in physics there must be some, maybe they publish, they publish really, really large... Jessica Leibler (45:41): They publish very infrequently large... Matt Fox (45:42): Oh no, I think they were saying the opposite. Jessica Leibler (45:46): Oh. Matt Fox (45:46): My understanding was they were saying the publication culture is different, but they had, I'm guessing you have like... Jessica Leibler (45:51): Many authors. Matt Fox (45:52): Yeah, and so you'd have your name on lots of things, but I could be wrong about that. It says they tend to publish large numbers of papers because authorship practices in the field differ. So I'm guessing that just means putting a ton of people on lots of papers, but we do that too, but I'm guessing not quite as large except in our cases where you have these big... Jessica Leibler (46:08): Consortium. Matt Fox (46:08): Consortium or genomic collaboration kind of things. And so they saw...agriculture, fisheries, and forestry saw the speedies growth increasing 14 fold between 2016 and 22. And then they looked at and they speculated that some of these changes may be due to, well, sorry, but the biggest one of course was clinical medicine where you see the most of them. Jessica Leibler (46:28): Of course. Matt Fox (46:28): As you might not be surprised to find out. And then they looked at some of the practices that might be associated with this, that they were talking about Thailand and saying there's a focus on increasing university rankings. (46:41): And that may have led to it. That of course doesn't say anything nefarious is going on that just says if you put more emphasis on things, people may go out and respond in ways of publishing more, but it could also be going about it in ways that we probably wouldn't support. Okay. So all that is interesting. What I find the most interesting about the whole thing is this article interviewed a professor from Stanford who was a co-author on this paper and talked to him about it. And if you go and look up said professor, guess how many papers said professor published in 2023. Jessica Leibler (47:13): 60. Matt Fox (47:13): More than 60. Jessica Leibler (47:17): I was going to say 61. Matt Fox (47:18): I went through the Google citations and I tried to exclude anything that I thought could be an abstract, a letter or a pre-print, and I still got over 60. So I just find that fascinating to say there's somebody out there who is writing papers and then speculating on the reasons for this. Jessica Leibler (47:34): I see. Matt Fox (47:35): That could be reasons we wouldn't accept as appropriate. Basically saying, "I can do it and obviously I do it with integrity, but other people who are doing it might not..." Jessica Leibler (47:47): That's interesting. Matt Fox (47:48): That worries me a little. So I thought that was an interesting take on a really interesting trend. Jessica Leibler (47:54): It's really interesting. I mean, I feel like, not in this exact example, but I feel like this is something I find myself thinking about a lot in terms of in a very small sense where you have a data set or you have a finding and you could say this could be one paper or this could be five papers depending on how you splice and dice. It could be one heavy duty paper or it could be a number of smaller, less consequential papers on the same findings, the same data. And I feel like I see that a lot where there is this disaggregation of data across multiple papers, but that doesn't answer this question of people who are publishing level, at this quantity. Matt Fox (48:33): That is referred to as salami slicing. Jessica Leibler (48:37): No one likes a big rubbery piece of salami, but if it's too skinny, then you don't get the flavor. Right? Matt Fox (48:43): Well, so they refer to salami slicing with the implication that it's bad. I don't know that it is necessarily bad, but even in that case that you're describing, those five papers, you're going to put a lot of effort into those five papers. It's going to take you a while. You're not going to be putting out 60 papers. And I think their point here is, are these people putting out 60 papers? Do they really know what's going on in all of these papers? Are they all legit? Are they all...and I suspect most of them are. And so just the question becomes, "what do we make of this trend?" Jessica Leibler (49:15): Right? I mean, it's interesting too. I feel like the element that I think about a lot is our research in our domain now is very collaborative. And so to have a lot of papers, you have to be engaged across many, not even just one collaborative research group, often multiple collaborative research groups where you are...some things you lead and some things you're the grant writer and other things, you are just a co-author and other people are moving things forward. And what are the characteristics for success in that environment, which are different than if you're a solo operator in terms of what are the characteristics of success for having this diversified engagement across multiple research teams? But again, that doesn't necessarily lead to 60 or more papers a year. Matt Fox (49:55): We have highly productive people and they're not publishing...I wouldn't expect anyone to be publishing 60 papers a year. There are some people who do it, but I'm just saying I wouldn't use that as a metric for what we are looking for. Jessica Leibler (50:11): Although it's not surprising in an environment where we are often "graded" for quantity over quality of manuscripts. Matt Fox (50:18): For sure. Jessica Leibler (50:18): And so it's kind of the natural consequence of that sort of thing. Anyway, I have something that is very silly but yet was written up in major news sources. This was a study published in the journal Scientific Reports about the association between lifespan in dogs and the shape of their snouts, the shape of their noses. And with the ultimate conclusion, large breeds and breeds, they were focused here on nose size, breeds with flattened faces, had shorter average lifespans than smaller dogs and dogs with elongated snouts, the researchers found. Matt Fox (50:52): Do we know why? Jessica Leibler (50:53): And then they ranked...so then they hypothesized. So this is the part why I really liked this study. So they were saying dogs with flat faces had shorter lifespans. And then they had a finding about size, which I think we all knew that kind of larger dogs had shorter lifespans than smaller dogs. (51:09): That's something dog people kind of know, but the flatness of the nose. And so they didn't know why, but they had a few hypotheses that to really flatten the dog's nose, it required genetic inbreeding. Matt Fox (51:20): Mhmm. Jessica Leibler (51:20): That that was maybe not a natural state of affairs. And so the dogs with flat noses...and then they went into kind of life experience and behavioral characteristics that maybe dogs with flattened faces were more likely to kind of be coddled and have cozier life experiences because they're perceived to be "cuter." Matt Fox (51:39): So they don't toughen up. Jessica Leibler (51:42): They're not tough. And so they have... Allegra Gordon (51:44): No grit. Matt Fox (51:44): No grit! Jessica Leibler (51:46): No grit associated with the breeds and that the long nose snout, that maybe there was a connotation between the long nose snouts and hard work. And so those dogs were more likely to be more asked to, I dunno what kind of pets are asked to do hard stuff. But apparently this was a study of 600,000 British dogs, so maybe some of them were working dogs and these dogs were more likely to have the longer snouts. So it was a combination of humans creating dogs to suit our perception of cuteness, which then led to genetic inbreeding and the flat snouts and shorter lifespans. And then also maybe the way they look reflects tasks or jobs or something, expectations in some kind of strange way. Allegra Gordon (52:30): I'm highly skeptical of the sort of Protestant work ethic element of this hypothesis, but the inbreeding makes...when I think about the really cute French bulldog who I used to know and how many breathing problem...no, she had so many breathing problems. Matt Fox (52:45): Every flat nosed dog I have known couldn't breathe. And you hear them...but I'm going strongly disagree that they're cute. Strongly. I did not find...if they were breeding those for cuteness... Jessica Leibler (52:58): They were very slobbery. Matt Fox (53:00): ...they failed in my view. They failed miserably. I do not find those cute. Allegra Gordon (53:04): In the eye of the beholder. Matt Fox (53:06): Absolutely. Jessica Leibler (53:06): The eye of beholder. Matt Fox (53:07): Absolutely. Jessica Leibler (53:07): This was just a weird but true. Matt Fox (53:08): The eye does not behold cuteness. Jessica Leibler (53:10): Oh my goodness. And then they ranked, obviously, the dogs by their sout size and lifespan. Matt Fox (53:17): So the take home message here is longer nose, live longer? Jessica Leibler (53:20): Yes! Matt Fox (53:20): Okay. Long nose. Jessica Leibler (53:21): In dogs only. This was, no, that's the question. I don't if we want to extend this to humans, but yes, maybe that was the...they didn't make any commentary, but that was kind of left to your imagination for those of us with long noses. Matt Fox (53:33): Ok. Generalizability. Jessica Leibler (53:33): There we go. That is the end of our program. If you have any feedback on this or any other episode or you want to suggest a study or topic for us to take on, you can reach us at Boston University's Population Health Exchange website: pophealthex.org. As always, we want to thank Nick Gooler at BUSPH for sound, producing, and editing. Thank you for joining us. We hope you enjoyed it, and we hope you join us again for our next episode.