Correcting the Record Regarding My Essay in The Free Press

Last week, I wrote an opinion piece in The Free Press on perverse incentives in scientific publishing.

I described the strong incentives researchers face to publish high-profile papers and how those incentives naturally push researchers to mold their research questions and the presentation of work so that they are palatable to ‘high-impact’ journals.

I also described how I believe high-profile papers put an inordinate focus on the negative impacts of climate change and underemphasize other relevant areas, especially the study of societal resilience to climate.

I think that this is a problem because I believe that a hyper-focus on seeking out and highlighting negative impacts of climate change represents a lost opportunity to study practical on-the-ground solutions to climate-related problems. Hence, I argued that the incentives facing researchers are not aligned with the production of the most useful knowledge for society.

I have written about these problems in the scientific literature many times before, but this time, I used my own paper on wildfires as an example. My aim was to simultaneously criticize myself and criticize a broader system.

In response, I received a great deal of support for my editorial privately from other Ph.D. researchers who agree with me, are exasperated, and would like to see change, and a lot of very negative public reaction from high-profile climate researchers and journalists, some of whom have characterized my public airing of decisions that I made to increase the likelihood of the publication of my Nature paper as a “scandal.”

Much of the public criticism revolves around highly misleading (and in some cases patently false) claims about the research approach that I took in designing the study and what then transpired during the peer review process. One outlet falsely stated that I manipulated data. The editor-in-chief of Nature incorrectly suggested that peer reviewers instructed me to include changes in non-climate factors in our wildfire projections.

There is much more to be said about why I chose to write my essay and why the issues I raise are important for both the integrity of the climate science endeavor and the efficacy of societal efforts to mitigate and adapt to climate change. But here I address specifically the false and misleading claims that have been made about my Nature paper and the peer-review process.

Was the publication of my Nature paper, followed by my essay in The Free Press, a hoax or a sting operation?

Absolutely not. When I started working on what would become my Nature paper, I had no intention of writing anything about the publication process. I started doing wildfire research in 2019, and I realized there was an opportunity for a high-profile paper in the fall of 2021. I followed this pathway, shaping the research question and the presentation of the research for a high-profile paper, and submitted it to Nature in the summer of 2022.

I had been bothered by what I saw as strong biases in the scientific literature for a long time, and I started thinking and writing more about this problem as the paper was going through peer review. I gradually realized that I was applying a major double standard, criticizing other researchers’ papers but not my own. Over the course of this past year, and as the Nature paper was getting over the finish line, I decided to write a piece critiquing it. I did not think that my experience was anything exceptional, so I discussed the general incentives facing climate impact researchers and how these incentives make research less useful than it could be.

Did you manipulate data in order to ensure the publication of your paper?

As I state in the essay, I chose to frame the research question in my paper narrowly, to focus only on the contribution that climate change was making to wildfire behavior. In doing so, my methodology left out (held constant) the myriad of causal factors that affect wildfire behavior (e.g., non-climate factors like human ignition patterns and fuel loads) and could be altered in the future to mitigate wildfire danger. The paper is honest about leaving those factors out, so there is nothing explicitly wrong with the paper itself. However, at the end of the day, what gets communicated to the public is just part of the story and not the full truth.

This has been standard practice in published peer-reviewed literature that attempts to quantify the impact that climate change is having or will have on a wide range of weather-related phenomena and the resulting impacts that those phenomena have upon society. My Free Press essay simply acknowledged that I, like many others, make these choices, sometimes believing that doing so increases the likelihood that high-impact journals would be interested in the research.

Do you stand by the research findings in your Nature paper? Should it be retracted?

I do stand by the research findings, and there is no basis for retracting the paper on its methods or merits. As I have consistently said, I am proud of the research overall, and I think it significantly advances our understanding of climate change’s role in day-to-day wildfire behavior. There is nothing explicitly wrong with the paper, and it follows the regular conventions of many other papers. My point is simply that molding the presentation of the research for a high-impact journal made it less useful than it could have been.

Did editors or reviewers pressure you into highlighting climate change in your paper?

My paper was conceived from the outset to focus exclusively on climate change's impact on wildfire behavior partially because I thought that would increase its chance of publication in a high-profile journal. I did not claim that editors or reviewers pressured me into highlighting climate change after the submission of the paper. My editorial is about how high-profile venues display preferences for certain research results, and that indirectly solicits research that is customized to those preferences.

Did reviewers ask you to include other non-climate factors in your wildfire projections?

Nowhere in the peer review process did reviewers challenge the usefulness of focusing solely on the impact of climate change when projecting long-term changes in wildfire behavior. The main non-climate factors that I said were important (but held constant) in my essay were ignition patterns and fuel loads. Reviewers brought up ignitions a single time, and fuel loads came up six times. In all cases, reviewers raised these factors in relation to questions about whether my methodology was sufficiently robust to accurately quantify the contribution of climate factors to wildfire growth, not to tell us that we should expand our methodology to include changes in these other non-climate factors in our long-term projections.

Let’s go over them all:

From page 7 of the peer review document,

“The second aspect that is a concern is the use of wildfire growth as the key variable. As the authors acknowledge, there are numerous factors that play a confounding role in wildfire growth that are not directly accounted for in this study (L37-51). Vegetation type (fuel), ignitions ( lightning and people), fire management activities ( direct and indirect suppression, prescribed fire, policies such as fire bans and forest closures) and fire load.”

The reviewer is expressing concern that the “confounding” factors will make it difficult for us to quantify the influence of temperature on wildfire growth in our historical dataset (a relationship we can then use later for our climate change projections). They are implying that it might be easier to quantify the influence of temperature on some other wildfire characteristic other than growth. They are not telling us that we should expand our methodology to include changes in these other non-climate factors in our projections.

In my reply, I pointed out that our models were able to predict wildfire growth well in our historical dataset, even without information on the confounding factors.

From page 8 of the peer review document,

“Did you consider using other fuel moisture variables such as 1 hour and 10 hour fuel that can be important is some fuel types ( grass and shrub etc.)”

“Fuel moisture variables” in this context are climate variables and do not refer to non-climate factors like fuel loads. The reviewer is suggesting ways to better quantify the impact of climate change on wildfires.

From page 9 in the peer review document,

“My main concern is on the robustness of the empirical models due to the extremely unbalanced samples for the binary response variables (extreme vs non-extreme fire days: 380 vs. 18,000) and the very small size for the occurrence samples, especially considering the diverse landscape in California in terms of fuel types and topography.”

The reviewer is concerned that there may not be enough samples of the extreme wildfire growth days that we are studying to get a good quantification of the influence of temperature in our historical dataset (a relationship we can then use later for projections). We demonstrate that the models have good predictive skill on out-of-sample data in our historical dataset.

From page 12 in the peer review document,

“1) The climate change scenario only includes temperature as input for the modified climate. However, changes in atmospheric humidity would also be highly relevant for predicting changes in VPD or fuel moisture.”

The reviewer is asking that we include another climate factor (absolute humidity) in our projections, not asking us to consider any non-climate factors in our projections.

In my response, I acknowledge that all climate variables other than temperature (and temperature’s influence on aridity) are held constant in the projections, and I bring up the fact that other non-climate factors are also held constant.

“We agree that climatic variables other than temperature are important for projecting changes in wildfire risk. In addition to absolute atmospheric humidity, other important variables include changes in precipitation, wind patterns, vegetation, snowpack, ignitions, antecedent fire activity, etc. Not to mention factors like changes in human population distribution, fuel breaks, land use, ignition patterns, firefighting tactics, forest management strategies, and long-term buildup of fuels.”

I further articulated my stance that the temperature signal dominates other climate variables in its influence on wildfire behavior, and that is why focusing on warming alone (and not other climate-related changes) was valid for representing the entirety of the influence of climate change.

“Accounting for changes in all of these variables and their potential interactions simultaneously is very difficult. This is precisely why we chose to use a methodology that addresses the much cleaner but more narrow question of what the influence of warming alone is on the risk of extreme daily wildfire growth. We believe that studying the influence of warming in isolation is valuable because temperature is the variable in the wildfire behavior triangle (Fig 1A) that is by far the most directly related to increasing greenhouse gas concentrations and, thus, the most well-constrained in future projections. There is no consensus on even the expected direction of the change of many of the other relevant variables. We do not believe that absolute humidity is an exception to this, and thus it is just as justifiable to hold it constant as any of the other variables mentioned above. Support for this can be summarized with the following two points…”

This is all to say that if we can hold everything else constant in the projections, then there is no reason we should not also be able to hold absolute humidity constant. Further, it is an argument that temperature is sufficient to represent the influence of climate change. It is not an argument making the case that focusing solely on the impact of climate change is more useful than an expansive assessment that includes other non-climate factors.

Page 23 of the peer review document,

“Line 73 - It is worth mentioning that fuels including both fuel types, structure, and amount were held constant in addition to fuel moisture.”

The reviewer asks for us to state that fuel characteristics are held constant at a particular place in the manuscript (even though that is already stated in other places).

Page 26 of the peer review document,

“Lastly, area burned is influenced by many variables and I think that fire management effort is one that deserves some mention. Fire management effectiveness varies due to a number of factors, but fire load/resource availability is a key factor that could confound your results.”

The reviewer is expressing concern that fire management will make it more difficult for us to quantify the influence of temperature on wildfire growth in our historical dataset (a relationship we can then use later for our climate change projections). We address this by inserting a caveat.

Reviewers did not challenge the usefulness of focusing solely on the impact of climate change when calculating long-term changes in wildfire behavior in any of these discussions. Rather, they are focused on making sure the methodology is able to get an accurate quantification of the impact of climate change.

Wouldn't accounting for these other non-climate factors in your wildfire projections make your paper better and help it get published in Nature?

A paper that appropriately accounted for projections in other relevant non-climate factors would probably be inconclusive in even the direction of future wildfire change with uncertainty ranges that overlap with zero. That research - which is more useful research - would tell a much less clean story and would be less likely to be a high-profile paper. This is the sense in which ‘leaving out the full truth’ (other relevant causal factors) made the paper more compelling, which then makes it more worthy of a high-profile venue. This “positive results bias,” where researchers are more likely to submit, and editors are more likely to accept, results that demonstrate a clear significant relationship over an inconclusive one, is a well-known phenomenon.

To conclude

My aim is to highlight a problem and push for reform so that the incentives facing researchers are better aligned with what produces the most useful knowledge for society.

There are many reasonable reasons that climate researchers and journalists might disagree with my essay. Nature has published papers that push back against the narrow focus on climate impacts that I describe. I can’t say for certain that a paper focused more broadly on climate and non-climate factors would not have been published in Nature. And any criticism of climate science practice will assuredly be amplified by right-wing media eager to promote skepticism about climate change.

But the specific claims above, which have been widely repeated and have constituted much of the effort to ignore the issues I have raised, are false and misleading.

More broadly, I’d like to push for changes in attitudes and norms from researchers to institutions to journals and the media. My main thrust is simply that research on climate and society should not have such an inordinate focus on identifying and highlighting negative climate impacts at the expense of studying the effectiveness of solutions that can help people today. I am interested in hearing ideas and forming networks of people who would also like to see reform in this direction, and I hope, perhaps unrealistically, that those determined to loudly discredit the concerns I have raised will consider whether doing so, over the long term, really serves the credibility of the climate science enterprise.