This essay has appeared, or reappeared; it looks like it was first published some years ago. It concludes, paradoxically:

“In our view, the fact that so many scientists agree so closely about the earth’s warming is, itself, evidence of a lack of evidence for global warming.”

The argument that leads to this conclusion is not clearly laid out and verges on complete incoherence. Nevertheless, some people seem to find it meaningful, so I thought I would look at it in more detail.

It starts by loosely (very loosely) defining some terms like “knowability”. “knowability” seems to have a range of meanings, and is never very carefully defined. Combing through the essay there are the following clues to its meaning:

  1. Knowability is “A broad term loosely encapsulating how possible it is to reduce uncertainty about an idea’s correctness.”
  2. Something with a low knowability is “impossible to verify (K ~ 0)”.
  3. The more easily testable and verifiable a theory, the less debate we would expect. There is little disagreement, for example, about the sum of one plus one or the average distance of the earth from the sun” which implies that high knowability is synonymous with a “more easily testable and verifiable theory
  4. But, as a question becomes more complex and less testable, we would expect an increasing level of disagreement and a lessening of the consensus-think: the existence of god, the best band since the Beatles, or the grand unified theory of physics” which implies that low knowability is synonymous with “more complex and less testable”.

The examples in the last point highlight the confusion because it mixes three qualitatively different types of question together. In total the range covered by the concept jumbles together things that fall either side of Popper’s demarcation line: some are falsifiable, others are not. It also mixes together things that are verifiable in principle but hard or, currently not possible to verify in practice with things that are verifiable both in principle and in current practice.

Clearly (or unclearly, it isn’t clear) “knowability” is a confused concept.

“consensus” is defined as

  1. a measure of the idea’s popularity and general acceptance.”
  2. support for a concept

Furthermore, we learn that “At the upper reaches of consensus, there is less updating of views to account for new information-so much so that supporters of the status quo tend to suppress new facts and hypothesis. Government agencies deny funding to ‘sham’ scientists, tenure boards dissuade young researchers from pursuing ‘the wrong’ track, and the establishment quashes heretical ideas”. There’s clearly more to their understanding of consensus than simply “a measure of the idea’s popularity and general acceptance”. But even given that limited definition, one might ask in what circles the idea is popular and generally accepted?

Anyway…

The basic shape of their argument is that things with low “knowability” should have a low consensus and things with a high “knowability” should have a high consensus. If there is a deviation from this relationship – they focus particularly on low “knowability”/high consensus – then something is up.

There are a couple of graphs that show how they think that the probability of a theory relates to different levels of consensus and knowability regarding that theory. Intriguingly it suggests that there is an optimum level of consensus for theories with low “knowability” which maximises the probability of the truth of that theory. That’s just odd.

Armed with this already wide range of concepts for “knowability” and the troublesomely-loaded meaning of “consensus”, the authors plough into an example.

“Consider the belief that the sun, moon, and stars circle the earth-a reasonable initial proposition. Yet, as additional facts become available (Copernicus, Brahe, Galileo), the dogmatic believers of the consensus condemned these observations as heresy”

Is the proposition that the “sun, moon, and starts circle the earth” of high or low “knowability”? It isn’t clearly stated. It is asserted that there was a consensus, so if the proposition is highly knowable then a strong consensus makes sense, but that is not the way that the argument goes. That implies the authors think the proposition has a low “knowability”, but then it’s not clear in what sense. If there were “additional facts” that bore on the matter then it was possible to “reduce the uncertainty about an idea’s correctness” and it was not “impossible to verify” both of which are associated with high “knowability”.

The authors draw a conclusion from this nonetheless: “Given that we know not the evolutionary stage for any current theory, we arrive thus at the unexpected conclusion that when knowability is low, as the level of consensus increases (without a commensurate increase in knowability) there should be a decrease in the probability assigned to the truth of the matter

Which sounds like a restatement of the usual state of philosophic despair: If we don’t know the evolutionary stage of any current theory then we are effectively in a constant state of low “knowability” by some definition and hence strong consensus viewpoints are always in danger of being wrong. It’s always too soon for a strong consensus. This is a key part of their argument – it’s where the paradox comes from – but as the problem is set up the probability of any viewpoint is low when knowability is low, independent of consensus. Consensus has nothing to do with it, other than that a consensus is a particular view point.

It’s worth noting here that there is no mention of the fact that “incorrect” theories can still provide useful and accurate predictions or that for many important problems we actually want to quantify the relative probability of a range of “theories”. To use one of their examples – the average distance between the sun and the earth – this is a quantity to which we append different probabilities to different values. That is to say the true value is uncertain.

I can try to rebuild their argument in better-defined terms I can understand, though not necessarily agree with. It’s scarcely worth it. If I do, I get something like this:

  • Where evidence about the truth of a proposition is weak, one can expect a range of opinions regarding the truth of that proposition.
  • Any particular position has a reasonable probability of being incorrect
  • If evidence is weak, but there is nonetheless a consensus regarding the truth or otherwise of a proposition, then there is a reasonable probability that the consensus is incorrect.

We don’t really get a lot out of it though. If you accept that the evidence regarding the truth of a proposition is weak then there doesn’t need to be a consensus one way or the other to state that any particular position has a reasonable probability of being incorrect. There is also an interesting “sociological” aspect to consider and that is to what extent and how broad a diversity of views is needed to effectively advance the general understanding of a subject.

The latter half of the essay purportedly uses the example of “climate change” to show how their ideas work (or not) in practice. They don’t clearly state what exactly they mean by “climate change”, seeming to refer to the whole of climate science in one go, eliding as it suits their argument between big questions about changes in the earth’s average temperature and specific impacts such as the number land falling hurricanes hitting the US.

What they do present is a mish-mash of ideas which they believe shows that there are severe limits on empirical verifiability in climate science. We have, amongst others:

  • Models have their problems
  • The earth’s climate is a dynamic, chaotic system.
  • When evaluating the causes of past climate shifts, scientists cannot simply rerun history to test the impact of changing different variables.
  • Although climate scientists can make testable hypotheses about the future, their short-term predictions have an embarrassing record.
  • The debate will be moot by the time we can test their long-term forecasts in the year 2100.

Note that – stripped of the more loaded language – none of these is especially objectionable as a statement. It’s merely a description of the problems faced by climate science – and many other sciences too – and doesn’t at all grapple with the important question of whether the tools and methods developed in the field or brought to it from outside are effective in dealing with these problems.

They also mix in some things that are plain wrong. For example this self defeating couplet:

  • To reach their conclusions, climate scientists have to …[do various things] … without being able to test their hypotheses against reality.
  • Failed attempts to forecast temperature changes for the 2000s.

Which is it guys?

Having presented their mish-mash, they say “We would, therefore, expect this limit on empirical verifiability to birth widely divergent views on the path, causes, and consequences of earth’s future climate.” There’s some more waffle, then “In our view, the fact that so many scientists agree so closely about the earth’s warming is, itself, evidence of lack of evidence for global warming”. Their conclusion is effectively one of their premises – we think the evidence is weak, therefore the evidence is weak. It really has nothing to do with the consensus at all.

One could also take issue with their assertion that there ought to be widely divergent views on the “path, causes and consequences”. The divergence of those views is necessarily constrained by the extant evidence and more well-established theories.

Overall the essay is just bad. The new term they introduce, “knowability”, is practically meaningless. The logic of their argument which leads to their “paradox of consensus” is broken and the connection between their broken argument and their worked example of climate change is weak and relies entirely on their own assessment of the evidentiary support for an ill-defined climate thing.

Advertisements