The Climate 'Consensus' Trick
Does the latest 'climate consensus' study show a genuine agreement among scientists, or just a statistical construct?
How can a study that claims ‘greater than 99% consensus’ among scientists, simultaneously report that 69% of scientists gave ‘no position’?!
This article reports on a recently published study which declares an overwhelming consensus among experts - more than 99% - as to the human-cause of climate change.
I don’t intend to argue whether there is indeed a consensus among scientists, but rather, I wish to highlight how easy it is to be mislead with ‘scientific consensus’ headlines, and ultimately, why it matters.
Overview
A recent Cornell University publication announced that:
‘99.9% of studies agree; humans caused climate change’
This was based on an analysis undertaken by Mark Lynas et al of more than 80,000 scientific papers in which their titles and abstracts were carefully reviewed, filtered and categorized.
The work was an update of the widely cited ‘97% of experts agree’ study published by the Australian scientist Cook et al in 2013.
Approach
The purpose of the study was to determine the degree of scientific consensus against the following proposition;
‘..agreement in the peer reviewed scientific literature on the principle role of greenhouse gas (GHG) emissions from human activities in driving modern climate change (i.e. since the Industrial Revolution)’. Mark Lynas et al 2021 Environ. Res. Lett. 16 114005
Below is a summary of how the study was undertaken:
A data set of 88,125 climate-related papers published since 2012 on the ‘Web of Science’ (an extensive database of scientific papers) was accessed.
3,000 papers were randomly sub-sampled from these 88,125 papers upon which the analysis was conducted.
From these 3,000 papers, 282 were removed as being ‘not relevant’, leaving a total of 2,718 papers subject to analysis.
The titles and abstracts from the selected papers were reviewed and classified according to their level of endorsement of human-caused climate change, ranging from ‘explicit endorsement with quantification’ through to ‘explicit rejection with quantification’.
The number of papers that fall into these classifications were counted and used to calculate a level of ‘consensus’ against the proposition noted above.
(*For the sake of brevity, I have not described parts of the study that are not relevant to the points I wish to make in this article. I do recommend reading the entire study, references to which I have linked at the bottom of this article.)
Below is a summary of the results showing the numbers of papers that fell into each particular category:
Looking beyond the headlines
Sounds pretty reasonable so far?
When scratching beneath the surface, however, things start to get a bit questionable.
Bias?
The first issue relates to how a paper can be accurately categorized just by looking at it’s title and abstract. This would require, one imagines, a fair amount of judgement from the reviewers side. How would someone become qualified in making such important judgments, and how would one ensure that bias is not driving the decision making?
There are numerous statements from the paper that I believe indicate a level of bias on behalf of the authors, the following being just one example:
‘..the prevalence of mis/disinformation about the role of GHG emissions in modern climate change is unlikely to be driven purely by genuine scientific illiteracy or lack of understanding.’ Mark Lynas et al 2021 Environ. Res. Lett. 16 114005
It appears that, in the authors opinion, those skeptical about mankind’s ability to change the weather - which they call mis/disinformation - are at best scientifically illiterate, but more likely embarking on a campaign of knowingly spreading falsehoods for some other unknown motivation.
From this example I think you can judge for yourself whether there may have been the potential for ‘bias’ to creep into the authors judgments.
Explicit vs Implicit Endorsement
The next issue is the difference between explicit and implicit endorsement. The results show that:
16% of the papers explicitly endorsed the human-caused climate change proposition.
15% of the papers implicitly endorsed the human-caused climate change proposition
But how do you tell the difference between implicit and explicit endorsement?
The definitions of these categories given in the study are broad. But one specific example given raises a glaring contradiction;
‘…we gave rating [*explicit* endorsement] to all papers referencing future emissions scenarios in their abstracts, because emissions scenarios by definition *imply* an evaluation of humanity’s role in GHG emissions and their subsequent impact on climate [sic].’ Mark Lynas et al 2021 Environ. Res. Lett. 16 114005
So if I understand this correctly, a papers’ endorsement can be categorized as explicit, because their topic of study is believed to be implicit in the endorsement of the causes of climate change….?!
Consensus defined as a lack of objection!
If a paper didn’t contain a ‘clearly-stated rejection or disagreement’ it was simply assumed as endorsing the consensus position. Yes you read that correctly. Here it is in the authors own words:
‘..defining consensus in a clearer and more objectively transparent way as simply the absence of clearly-stated rejection or disagreement.’ Mark Lynas et al 2021 Environ. Res. Lett. 16 114005
The study commits what I believe to be a professional foul in assuming that the absence of rejection is the same as endorsement.
Into which specific endorsement category these ‘non-rejection’ papers were placed is not clear, but in the end it didn’t actually make any difference!
Which brings me on to my final point.
The mathematical slight-of-hand
The authors undertook a post-categorization step that allowed them to really boost their consensus paper figures. After categorizing a paper as ‘no position’, they bundled it in with the ‘endorsement’ papers anyway!
One can’t help but wonder, therefore, why they bothered with a ‘no position’ category at all!?
It was this astonishing move that that was behind what appears to be some mathematical trickery, and ultimately, the contradictory headline ‘99% consensus’ figure;
1-(4/2718)) = 99.85%
The papers that either implicitly or explicitly rejected the proposition (4) were removed from the total number of sampled papers (2718), and the result expressed as a percentage of the total sampled papers. This mathematical slight-of-hand allows the ‘no position’ papers (which make up the largest category at 69% of those sampled) to be counted along with the papers categorized as endorsing the proposition.
How can a study that claims ‘greater than 99% consensus’ among scientists, simultaneously report that 69% of scientists gave ‘no position’?!
Why it matters
Cook’s earlier 2013 ‘97% consensus study’, the work that this recent paper published by Lynas et al effectively updated, was extremely powerful in persuading people around the world of the human origin of climate change. From discussions with friends and colleagues, through newspaper headlines and TV panel discussions, to speeches made by the US president in front of the world, the ‘97% of experts agree’ phrase has been used countless times to very powerful effect.
I recently read an excellent summary of the Cook study published by the Daily Sceptic website, which shares many common themes as those highlighted in this article.
Endorsement by ‘experts’ is very influential; that’s why you will often see dentists in white coats pushing toothpaste brands on TV adverts and not random actors - irrespective of how clean their teeth are!
Social psychologists, just like marketeers, understand the persuasive power of endorsement by ‘experts’. Having a consensus among them is even better!
That is why the headlines that this type of study generate are important, something which I believe the authors understand very well.
Conclusion
It would come as no surprise to me if it is the case that the majority of those publishing papers about climate change do, in fact, ‘endorse’ the view that human’s are primarily responsible.
But why bolster the appearance of ‘consensus’ using questionable assumptions and mathematical slights-of-hand?
Notwithstanding the issues of bias and questionable assumptions during the categorization process which I described above, here are the actual results summarized;
15% of papers explicitly endorsed human-caused climate change
16% of papers implicitly endorsed human-caused climate change
69% or papers took ‘no position’, and
less than 1% rejected, either implicitly or explicitly, the human-caused climate change
I could write much on the broader idea of ‘consensus’ in science. But for now, I will just say that science is not an endeavor in which ‘consensus’ should be given any authority over any single scientist’s honest commitment to skepticism, evidence and inquiry.
We can all think of examples of great scientists from the pages of history who were made great precisely because they went against the ‘consensus’ of the day; Galileo, Mendel, Boltzmann & Einstein to name but a few famous examples.
“Whenever you find yourself on the side of the majority, it is time to reform (or pause and reflect).” Mark Twain
The next time you hear someone use the ‘scientific consensus’ argument during a discussion about climate change or any other area of science, I hope that you will pause and consider the possibility that you are being manipulated by clever statistical artistry, and more importantly, whether it even has any relevance at all in a scientific context.
Thanks for reading this far, and I hope you found it interesting!
And don’t forget to subscribe so you don’t miss any future Atlas Reports!
-Tristan
Notes:
The paper that this article discusses was co-written by two representatives from Cornell University, Mark Lynas and Benjamin Z Houlton
There is also a third author from the Bill and Melinda Gates Foundation-funded Alliance for Science (itself affiliated with Cornell University), Simon Perry.
The paper can be accessed and read in full here, and a summary post from Cornell University can be read here.
The study was published in Environmental Research Letters, a part of IOP Science Publishing
The version of the study that informed this article was accessed and downloaded on 20/10/2021
Hello Tristan,
Absolutely fabulously broken down! Great work . I would love to use a portion of this with your consent, and of course give you credit and a link to your substack if I could? I own a local paper called Westward Independent wwind.ca