I’m no historiana. Which self-deprecating preface I would preface further by saying, the list of things I’m not is far longer than the list of things that, tentatively, I would claim I am. That list, while longer, probably, than Descartes’ is still a short one. For the past 19 years, I was paid to do climate science. For most of that time, if you’d asked me whether I was a climate scientist, I would probably have said nob. No one ever asked me though, so I got away with it. I would also say, that not being a climate scientist was no bar to doing climate science. Being a climate scientist certainly would have helped, but it’s not necessary. My lack of credentials thus established, we can continue.
I’m no historian1… so the past constantly surprises mec. I was delightfully surprised to read this quote from the reading of Guy Callendar’s paper – The Artificial Production of Carbon Dioxide and its Influence on Temperature – at the Royal Meteorological Society:
“Sir GEORGE SIMPSON expressed his admiration of the amount of work which Mr. Callendar had put into this paper. It was excellent work. It was difficult to criticise it, but he would like to mention a few points which Mr. Callendar might wish to reconsider. In the first place he thought it was not sufficiently realised by non-meteorologists who came for the first time to help the Society in its study, that it was impossible to solve the problem of the temperature distribution in the atmosphere by working out the radiation.“
The phrase that stuck with me when I first read it was “non-meteorologists who came for the first time to help the Society in its study“. Callendar’s Wikipedia page says he was a steam engineer and inventor as well as being an “amateur climatologist”. Amateur or otherwise, his study is now seen as one of the landmark papers in climate science. He got a lot right in his paper, even if, as the responses to it suggest, the full story was a bit more complicated than that.
Since that time, many non-meteorologists (and latterly, non-climatologists) have come to help the Society and the wider community with their studies. For the most part, this is a wildly positive thing: climate science draws on a very broad range of people with a fascinating diversity of expertise, knowledge and backgroundse. However, it does not always go well. Most climate scientists know the feeling of horror attending the receipt of an email that starts “I am a retired engineer…d“. What follows is rarely edifying, at least not in the sense that the correspondent probably hopes. The instinct is usually to respond along the lines of Sir George, to offer some polite if slightly back-handed compliments, followed by mention of a few things “not sufficiently realised by non-meteorologists who came for the first time etc…” (In the era of social media though, it’s more common and more forgivable to mute and blockm).
Usually, responding to such an email is as simple as providing a stock answer to a stock objection. It requires time, but little thoughtf. It’s rare to come across a novel or meaningfully innovative objection, but very common for the objection to be presented as if it wereg. The responses to Callendar’s paper cover a wide range of the arguments one typically encounters. It’s clear that lots of the “standard” climate grumbler rambling points were already being made in 1938. And by meteorologists no less. A quick scan reveals the following arguments:
- It’s more complex than that.
- climate change has always happened
- it’s natural cycles
- yeah, but is the warming significant?
- the observations are dodgy
Callendar answered these objections the best he could. Since then, these questions have been revisited over and over again and well hashed out in the literature… over and over again. This is the normal process of science. An objection or problem is encountered and, then after a while, it ceases to be a problemk.
If we assume that our retired-engineer (or, whatever) correspondent is sincere, and possessed of the expertise they claim, then how can it be that they are unaware that the arguments they are making have been decisively answered, not just once, but over and over again? How can someone with deep expertise in a technical subject, make trivial mistakes in another subject which they have studied to the extent that they feel confident to question an expert on the topic?
I suspect there are many answers to this question.
Maybe one is that training in a scientific discipline consists of learning a bunch of things that everybody in the area knows but no one writes down except perhaps in passing, in a textbook or training manual. This kind of things is easy to miss if you are new to a field, but working outside of it. Each field has its own ways of doing things, usually adapted to the types of problems it’s dealing withh and usually passed on from one practitioner to another – teacher to student, mentor to mentee, guru to disciple or from one stressed out grad student to another. Sometimes it isn’t taught at all, it’s just something everyone picks up from spending hundreds of tedious hours tickling eels (or whatever the equivalent is). Without this groundwork, without the possibility to ask someone who has hit all the same snags as you (but maybe a year or several earlier (though in the case of the grad student, it might be days)), it’s possible to go quickly awry, get stuck, or spend months (even years) slowly reinventing an item that turns out to be disastrously ill-suited as a wheel substitute. Worse, the people you do have around you might reinforce you poor practices because their background is so similar to yours.
Working in other disciplines can also be subtly disorienting. People within a discipline, can use words in a way that can lead outsiders astray. I once completely misunderstood a paper on global temperature by a statistician because we had completely different understandings of what the phrase “natural variability” meant. To me it means changes in the climate that arise spontaneously within the climate system, things like El Nino. To them it meant the errors peculiar to a single data sets. Working effectively across disciplines takes time and effort. It also takes good will, of which there is usually no shortage in actual research, but which can be lacking in the sometimes fractious online environment.
Some people try to apply methods that “work” in one discipline to problems in another. While this isn’t necessarily a bad thing, it isn’t necessarily a good thing either. Results of this approach are mixed and hard to interpret. What works in a discipline that allows for tightly controlled, physical experiments, wouldn’t work in a discipline where experiments are far harder to control, or impossible to perform at all. What works in a discipline with well developed quantitative theories would not transfer to one without them. Expecting all problems to yield to a single approach is almost comical if you give it a moment’s thoughtL.
Long training or long-immersion in a particular culture can be inimical to thought. It sounds terrible put like that, but it isn’t all bad. If you had to think carefully about every single thing you did all the time, you’d get nothing done. Experience and training provide quick answers in many complex situations, distilling, in some cases, life-times of thought and practical experience into a rule of thumb or shortcut. Without this, it would be hard to make advances in any direction. The elements that are questioned least are generally the most fundamental (there’s a lot resting on them, often teeteringly balanced, so you really don’t want to muck around unnecessarily unless you are that way inclinedo) and these are often the things we learn earlier on and are often not clearly stated (these different answers tread on each other’s toes as these things tend to do).
If your experience of many years as a successful scientist tells you that they way to solve a problem is to apply a Gooblehopper, then you are apt to apply a Gooblehopper to solve all problems. You are also apt to find questionable anyone who claims to have solved a problem without one. If they claim never to have heard of a Gooblehopper, you might even mock their ignorance and wonder how they ever solve problems at all. None of these Gooblehopper-related responses require thought. You don’t, for example, consider what makes a Gooblehopper good at solving your problems, or why it might suck for solving other kinds of problems. You might not ask what this Swarglethumper thing is they are using to solve their problems. You know how science is done: it’s Gooblehoppers all the way.
It might seem a little harsh to suggest that learned, erudite and experienced researchers are not thinking flat-out all of the time. That’s why they exist after all, no? However, I believe it is the case. Thinking is hard work even for people who think for a living and are well-practiced in doing so. People will do almost anything other than think really hard, particularly if they feel they have a good excuse not to (particularly when there’s a good enough answer just sitting there Gooblehopperishly). What might constitute a good excuse? It doesn’t need to be much: a suspicion that the Swarglethumper isn’t quite as smart as you arei, or that Swarglethumpers are politically dubious, or because they once got something else wrong, or because you’re busy or a bit angry, or hungry, or both. Any number of things can distract you from deep thought. The brain is also good at retrofitting reasons to behaviours. You might justify the decision to use your Gooblehopper because you want to “independently check” the claim of a Swarglethumper. This sounds appealing, of course, but it might well ignore why a Swarglethumper was used in the first place as well as the real reason you did it being that they were out of Mars Bars at the snack machine.
This is, I am sure, a very partial list of reasons why smart people make dumb mistakes when stepping outside their area of expertise (it might also serve to explain, again partly, why smart people make dumb mistakes within their nominal area of expertise). I’ve tried to think of more generous reasons than one would usually encounter. I’ve not mentioned the more extreme forms of motivated reasoning, which clearly play a role in some (if not many) situations. No one is quick to question a conclusion they believe in already. Anyone can think up a hundred reasons to question one they find disagreeable.
It’s worth bearing in mind too that people aren’t necessarily as smart as they think they are (or as smart as you think they arei). The obvious caveat to all of this is that this is not my area of expertise either. As mentioned above, it is one of the many things I am not.
-fin-
a Usually a good sign that what follows should be taken with a pinch of salt. I recommend packing what follows in salt for a period of no less than three weeks. Soak in water before cooking.
b The reasons: I had no training in climate science, climate science is a very broad and diverse subject of which I’ve studied only the tiniest fraction, most of the time I wasn’t really doing climate science anywayj, general imposterish feelings, the feeling that I might, if answering positively, be asked to do something.
1 Actually… I have, for years, called myself a probabilistic historian in my twitter bio. This was a silly reference to the uncertainty in historical data sets. Fortunately, no historical knowledge or expertise was required in the writing of this article.
c The present too; apparently my ignorance means I am doomed to experience portions of the past as if for the first time, when I could, with a little effort, have had fair warning. The same people who point this out will often shriek when you reveal minor plot details of a TV show they are watching. Even if I wasn’t so terribly ignorant, lots of other people are, which would mean that I would be well informed, but still unable to stop it. I’m not 100% sure that this is better.
d I’m not picking on engineers here specifically, though they do seem to constitute a large fraction of the people who disagree with the general underpinnings of climate science. I chose engineers because of Callendar’s particular counter-example (as well as the Society’s reaction to it). I might as well have chosen geologists or physicists (also seemingly over represented) though as for that, the dissenters come from all branches of science and, indeed, all walks of life.
e Though, of course, it could always be usefully wider. Much wider.
f There is a large group of people who are experts in this strange world of rehearsed call and response. Some have elevated it to an art form and area of study all its own.
g When actual meaningful objections arise, they become part of the science, get analysed, addressed and the science moves on. It’s interesting to watch how people respond to such a situation. If your response to this process is, “Excellent, we’ve spotted a problem in the science, we can now fix it, woohoo it’s fixed” then congratulations, you are a normal person. If, on the other hand, your response is “We’ve spotted a problem in the science, scientists must be idiots” and you stop there, then you’ve clearly put yourself outside the scientific process.
h Though not always, of course. Poor practices can affect large groups, which can, even once it’s pointed out, be slow to change their ways. I kept wanting to mention the Dunning Kruger effect when writing this but it turns out that the experimental method used to demonstrate its existence in various situations was flawed.
i The conviction that other people are idiots is a common one. I mean, the evidence is everywhere. Everyone says so…
j They say you need to practice something in a dedicated and directed way for 10,000 hours to truly become an expert. I think this is nonsense, but it’s interesting to ponder exactly how many hours a typical researcher racks up for different tasks in the process of their careers. I once did this and concluded that after thirty years, I might have become an expert at poorly organised meetings, going to the coffee machine and screaming at Microsoft products.
k Imagine the problem is a mountain: you can go over it, round it, through it or under it, picking up all manner of new and thrilling skills in the process. You can also pretend you were never really in the mountain-crossing business and develop a very close and careful study of valleys and their contents.
L Like those people who talk about the scientific method, as if repeated application of a fixed procedure will somehow ratchet you towards the truthn.
m Apparently Arthur C. Clarke responded to screeds in this vein like so: “You may be right.” This seems to me nicely disarming, but it would, I fear, lead to some kind of
n Among the more scientifically minded climate grumblers, Popper is considered the purveyor of the finest scientific method. It’s usually clear that their understanding of Popper’s general philosophy is less than passing and the things said in his name would have made Sir Karl howl at the injustice of being so thoroughly misunderstood. In strict Popperian fashion, I should note the observation that howling at injustice was what he used footnotes for and he felt misunderstood to a very high level.
o I don’t know what FAFI means in general parlance, but when I heard it, I thought, that’s science in a nutshell with a BWIDCA(IYASA) hanging on the end – But Write It Down Carefully Afterwards (If You Are Still Alive). If nothing else this footnote serves to illustrate the point (way above) that one’s habits of thought dictate one’s understanding of new information and curtail the natural curiosity and conceptual elasticity that would allow a less hidebound mind to adapt.
Diagram Monkey 
My placeholder comment for now, which I’m gonna try to append if I can, is that this is very good stuff and deserves a lot more discussion than it has received to date, which has been mostly polarized or worse.
So this is a great essay, and there should be a long set of long responses to it, because it touches on many important points. Among many possible responses, it’s hard to know which to make.
There’s no doubt that what you’re describing here is a real thing and I would say that it in fact involves the utterly widespread, and arguably inherent, need for human beings to question things, especially complex (or apparently complex) topics. When your whole career choice specifically involves doing that (scientists, engineers, Pirsig’s motorcycle mechanic, etc.), then you can’t expect otherwise. It would be like asking Miles Davis not to evaluate another trumpet player, when in fact he already did so, internally and automatically, after hearing just the first few bars.
You have exactly the same thing going on when a mathematician or statistician starts looking at your equations and data (or even just your description thereof). He or she doesn’t know jack sh** about climate science, or myriad other disciplines, but as soon as he or she sees something amiss or questionable in terms of the strictly quantitative analysis aspects, the the red flag will go up, and they read the rest of thing with that red flag up, which may then go down, or get even more red, depending on what else gets said. Einstein is a great example on this front, as he was aware that he was not really a ‘true mathematician’ and felt that he was limited by this reality and also that he could potentially be exposed by it–he was potentially in over his head, relative to the understanding of expert mathematicians. It didn’t turn out that way, but if Einstein can feel that he might be open to serious questioning by non-physicists, then that’s good enough for me.
One obvious point is that Callendar is one of the big success stories, so he’s not really the guy to use if one wants to argue against outsiders wading into a field (which I realize you are not arguing for). Callendar (or Simpson?) calling himself an amateur–well in the sense of not getting paid for doing climate science, OK. But in terms of the common connotation of that term as being not fully skilled, or aware, or practiced–no, not the case. More likely a self deprecation so as not to offend people such as Simpson. Another point is that it’s very often smart “outsiders” who are most accutely aware when things aren’t being communicated well to non-professionals, which is all too common.
If somebody like Callendar, or any other smart and informed person who works with complex topics, and especially quantitativly, starts to detect something amiss, or incompletely addressed or communicated, or responded to with misdirections, or any other type of a lack of a complete answer to their question(s)…then there’s now blood in the water, from their perspective. And as soon as there’s blood in the water, that shark’s not going away. In fact, other sharks will now be there soon. This is not a non-phenomenon in climate science, or any other discipline for that matter. I can give several examples just from my own experience, and more from others’. I have seen many in online climate science discussions (blogs, twitter, etc.) get into exactly this predicament, which is pretty much inexcusable, given that you will, for sure, be engaging with non-experts in those forums, which is in fact the whole point.
This was one of the problems with RealClimate, and one of the reasons I left after being a formal member of that group for four years. Those guys frequently had no real clue about how to engage with the public, and they also had an army of commenters who were even worse, whom they gave free reign to make all kinds of outrageous comments to the questions and comments of others. I can assure you that when I joined that group, they had absolutely no sense on this topic–it was like they were specifically trying to make enemies. Instead of just leaving, I decided to take this issue on, and basically this led to an eventual split, because after making a bunch of enemies in so doing, I decided I’d had enough. Took me way too long though, four years in fact.
Tons more to say on this, but this is all I have time for.
As a quasi-side note, but one that relates exactly to the point of the post, is that I kind of don’t understand Simpson’s reply here–I would thought he would have said that “it was impossible to solve the problem of the temperature distribution in the atmosphere WITHOUT working out the radiation”. And my recollection of Callendar’s paper, which admittedly has been a while since I read, was that he wasn’t just focused on radiation–he actually paid attention to the carbon cycle and tried to quantify it, although maybe that was in a separate paper, I don’t remember now. I know he had a definite interest in that. Not at the level of Arrhenius, who was way out ahead of everybody on it, but it was there.
Thanks for the kind words and the thoughtful comments.
Yeah, it’s complex. To cautiously extend the musical analogy, new genres of music can grow out of people breaking the old rules… Not sure where that leaves us though. I’m no jazz aficionado (everything I know I got from Clive James writing about Philip Larkin writing about Jazz) but I suspect that there were probably trumpeters who thought that what Davis was doing wasn’t trumpeting as they understood it, or even music for that matter.
I’ve got draft posts about the absurdities perpetrated in the name of correcting errors or setting the record straight. My favourite sub-genre is when someone makes the exact same error as the person they are correcting or refuting. It’s generally easier to spot flaws of reasoning, fallacies and cognitive biases in other people, but it would be most useful if we could identify them in our own. That’s hard and the more arguments you enter into, the harder it is to maintain strict self-scrutiny and keep calm and patient.
There are so many points where it can all go wrong e.g.
1. Person A asks Person B a question.
2. Person B interprets the question as they understand it and responds
3. Person A, who thought the key point of their question was different to person B’s interpretation.
Then it can go one of two (or more) ways:
4. Person A accuses person B of being evasive and it all unravels…
or
4. Person A says, that’s not quite what my objection is, and restates the question.
5. Person B accuses Person A of moving the goalposts and it all unravels…
The opportunities for misunderstanding explode if you have Person C chipping in with their own half-thought-through responses and responses to responses. And Person D, and E… all the way through to Person Z each with their own understanding or lack of it. And A thinks B is responding to them when B is responding to G who was satirising K who had misunderstood W on X’s point about something entirely different which just happens to have veered, quite by coincidence, towards the original topic.
You pointed out on twitter that twitter is fundamentally a bad place to have sensible discussions (limited and dumbass, I think you said), but I think that comments sections are also a terrible place to have them. It’s impossible to have a sensible discussion in a forum where literally anyone can drop in and contribute. There was a Dutch website (I think, I can’t remember the name right now) that tried to arrange structured discussions between two (or a limited number of) people, but it still had a comments section and the topics and contributors were chosen to be contentious.
I did consider when starting this blog, having a rule that you could only comment if first you could (reasonably) accurately describe the intention and reasoning behind homogenization of climate station data as understood by climate scientists. No one really comments here though, and when they do, they’ve mostly been sensible, so I never implemented it.
But I’m rambling…
More good points, and you especially described the dynamic of open discussion very well–that’s just about how it usually goes: misunderstandings raised to a variable exponent that usually scales with how contentious the public version of the debate on topic x is.
The Dutch forum and blog discussion you’re thinking of is Climate Dialogue (https://mwenb.nl/climate-dialogue-guest-blog/) which was headed by Marcel Crok. There was another American one along somewhat similar lines about the same time called Climate Change National Forum, whose guidance board included Bart Verheggen and Scott Denning. However, both ended very abruptly, even without any warning or notice in the latter case. Both had a lot of potential IMO, but there were also some issues.
I think the issue of moderation is highly critical, and that this has in many cases, been a primary source of problems. Whatever RealClimate, Rabbett Run, Steve McIntyre and Judith Curry had to say, was partly to largely torpedoed by by this issue, because the discussion environment ranged from outright hostile, to highly biased, to just bizarre, often all three at once, and the reason for this situation in each case was because the moderators either wanted it that way, or else weren’t willing to invest the moderation time necessary to keep discussions on the straight and narrow. Nobody who wanders into such environments, wanting only to actually better understand the science issues at hand, wants that kind of garbage discussion and they quickly turn away.
I did bring this up to Judith Curry once and she agreed that it was indeed a serious problem that she wished to correct, but it was a total Pandora’s Box problem at that point and I don’t know what efforts were made on it. And the blog posts themselves promoted this outcome in many cases. They seemed to be primarily after supporters, not education or insight. Contrast this with say, Isaac Held’s blog, where he studiously avoided that crap and focused on just explaining the science of atmospheric radiation physics to non-experts. I tried to do the same, but I also took on topics in which I thought that the standard or common scientific representation thereof was wrong in some important way.
The fact that you get very few comments on posts is frankly pretty disturbing and depressing to me, because it indicates, potentially a bunch of things about the current state of actual interest in climate-related discussions, and topics ancillary to them. A large chunk of people only seem interested if there’s controversy or argument involved.
More to say but out of time again.
Thanks again Jim.
I think the idea of discussing things in public puts a lot of people off. I’ve always had interesting discussions with other scientists of whom only a very small number are active on twitter, blogs, or what have you. Partly, that’s because what they’re doing is already working for them. Chats at the coffee machine or bus stop, emails, meetings, seminars, conferences, peer review, project wikis, instant messaging apps, internal reviews, journal clubs etc. all provide an opportunity to bounce ideas off people, think things through, get criticism and feedback, indulge in friendly mockery and so on. I also think it’s a shame because while there are all sorts of interesting things being discussed, it’s all happening in private. The broader public never gets to see that, let alone participate.
What does exist of scientific discussion in public is a set of interface or edge cases, usually with an explicit purpose beyond just exploring the ideas. So you get people who want to share their already published work, respond to misunderstandings, correct the record, publish their works of misunderstood genius, argue, educate, communicate science (in various ways), practice writing, and so on. A lot of traffic seems to go where the controversy is, which provides a skewed overall impression of what’s available, which is already a skewed view into how science works. For a lot of people, all they see of how scientists interact is either in broadcast mode (we did a thing), or else acting defensively (the thing does not make you grow horns).
When we talk about “open” science, it’s usually in terms of data, code, papers, which are all (in one sense) finished products and while that’s important, it’s not the whole kaboodle. For good or bad in the heyday of blogs, some of the more interesting stuff was on the “skeptical” side as they were doing and showing their work in public. Most scientists would be reluctant to do that, for lots of legitimate reasons – fear of getting scooped, or drawn into protracted arguments, organisational rules about sharing unpublished work, nervousness about making mistakes in public… the list goes on.