The Largest-Ever Opinion Poll on Climate Change Is Also One of the Worst

Is climate change a global emergency? Yes or no?

Surveys of the public's attitudes about climate change can be of markedly poor quality. But their faults aren’t random — they generally serve to create the appearance of a public clamoring for action to solve a problem that it takes ever more seriously.

A new survey by Cassie Flynn and Eri Yamasumi at the United Nations Development Program (UNDP), in partnership with researchers from Oxford University, provides a rather egregious case in point.

It’s the largest climate attitude survey ever conducted, with over 1.2 million respondents from 50 countries. Its centerpiece is a yes/no item that is, in the parlance of survey research, double-barrelled, simultaneously asking respondents about the existence and severity of climate change in a way that one can’t discern the intention of their answers. Sixty-four percent of respondents answered yes to the question, "Do you think climate change is a global emergency?”

But what does this mean?

Respondents can’t answer that yes, climate change is happening, but no, it’s not an emergency. Nor can they answer “don’t know,” or “unsure,” which, based on decades of survey research, many respondents surely would.

The following item suggests that the headline finding is indeed highly misleading. Asked “what should the world do about it," over 40% of respondents who answered “Yes” to the climate emergency question said that they thought the world should “act slowly while we learn more about what to do,” that “the world is already doing enough,” or should “do nothing.”

If the results are taken at face value, the belief that climate change is an emergency does not well predict support for acting accordingly. Faced with this counterintuitive finding, Flynn and Yamasumi call for more education, “even for those who already think climate change is a global emergency.” But the more likely explanation is that this surprising finding is simply a residue of, generously, a poorly designed survey or, less generously, an intentional strategy to inflate the appearance of public concern about climate change.

Similarly, the rest of the survey creates the misleading appearance of, in the author’s words, “a broad-based appetite for policy action.” Participants were invited to endorse anywhere from zero to all 18 of the climate policies provided, and they selected eight on average.

If anything, the policy results show — as abundant research already has — that survey participants prefer doing positive things rather than stopping negative things, even when they have virtually identical meanings. Consider, for example, that “use solar, wind and renewable power” was one of the most popular of the climate policies, and “stop burning fuels that pollute” was one of the least.

Flynn and Yamasumi emphasize that virtually all participants “fully supported climate action by supporting at least one policy.” One out of 18 is a ludicrously low bar for “fully supporting climate action,” and the small number of participants who supported none is still roughly 60 times larger than random guessing would predict.

This survey, unfortunately, is emblematic of climate polling. The well known “Climate Change in the American Mind” survey from Yale and George Mason University, for example, has proclaimed rising climate concern over the past few years but fails to note that it hit a similar peak a decade ago but is now more politically polarized. Researchers have also framed their findings as showing that informing people of the scientific consensus changes their beliefs about climate change, but it only changed their beliefs about what scientists think. Mainstream polling organizations aren’t fully immune either. Gallup offers a chart that seems to show consistently rising climate concern, but the trend line is anchored in an average of 2001–2014 values that hides how much climate concern has vacillated over this period.

Climate polling can be, and has been, conducted in a way that moves the conversation forward. Besides following the best practices of survey research, the most useful climate polling considers how important people think the issue is compared to others, how much they would be willing to pay for climate action, and how climate attitudes intersect with partisanship, political ideology, and electoral politics.

Good climate polling tends to suggest a view of public opinion that doesn't align nearly as well with the narratives of activists, who are consumed by climate change and wish to enlist the public on their side.

Climate change is a serious problem, and people should be calling for action, but climate surveys are too often designed to be used the way a drunk uses a lamppost — for support, not illumination.