Biases in the Interpretation of Weather and Climate Observations

Placing this week's heat wave into context raises all sorts of questions concerning the veracity of weather observations, including extremes and long-term trends.  The relatively high maximum temperatures at the Salt Lake City International Airport have really cast a spotlight on this.  In fact, yesterday the National Weather Service did some measurements in the surrounding area.  They posted a complete summary thread on twitter that you can access beginning with the tweet below.

We remain committed to ensuring the best quality temperature observations at our official Salt Lake City Airport climate site. Today during peak heat we visited a natural landscape location just ¼ mile south of the official site with 5 temperature sensors. A thread🧵 pic.twitter.com/Zco0vNbYS0

— NWS Salt Lake City (@NWSSaltLakeCity) September 8, 2022

The long and short of it is that the area south of the airport is a very hot place.  They found good correspondence between multiple measurements in a dry grass field to the south and the observing site.  Kudos to them for doing this.  There may be few if any meteorologists at the forecast office there who are responsible for the current site and they are doing the best they can to ensure that it records reliable observations.  

Now let's move to a related point concerning biases in the interpretation of weather and climate observations.  Such observations are used for all sorts of applications, from severe weather warnings to long-term climate trends.  I will posit here that there are three kinds of biases in the interpretation of such observations:

1. Instrumentation bias.
2. Representativeness bias.
3. Human bias.

Sometimes #3 is a bigger problem than the first two.  

Instrumentation bias is due to the instrument itself, or the recording and processing processes.  This is especially important for extremes.  The sampling frequency and averaging intervals, for example, can affect the recorded maximum.  I believe NWS maxima are based on a 5-minute average, although I'm not sure what the sampling interval is.  Other instruments may do things differently.  There can be other challenges related to the ventilation of the instrument (radiation shielding and ventilation are really critical for measuring daytime temperatures).  For example, some weather stations in use in the Wasatch Mountains by ski areas are aspirated by the wind and are poorly ventilated when the winds are light, leading to erroneously high temperatures.  

Even if you have an instrument that measures temperature (or some other variable like wind) perfectly, it is essentially a point measurement.  We live in a complicated world, and there can be a lot of spatial variability.  Representativeness bias concerns how representative the observation is of the surrounding area.  The airport, for example, is a hot spot.  During this heat wave, it has reported maximum temperatures 2-4˚F higher than the surrounding stations.  This could be partly related to instrumentation bias, but for this discussion, lets just assume that's a representativeness bias.  On the opposite end are observations collected near the golf courses, which have been considerably cooler than surrounding stations.  The area over which a station might be representative can vary a lot depending on factors like the land-surface or terrain variability.  A wind observation at the mouth of Parley's Canyon, for example, is representative of a very small area.  You move a kilometer to the north or south and you are in an entirely different wind regime.  

Finally we have human bias.  As I said, this can be a bigger problem than the first two.  For weather forecasting, meteorologists must learn to sip from a firehose of information.  This involves making a lot of mental shortcuts.  This can lead to heuristic biases that can sometimes lead to forecast errors.  Similarly, interpretation of climate trends and contexts can affected by such biases.

Human bias in this instance is a compound bias comprised of many biases.  I will steal liberally from Ian McCammon's FACETS acronym for heuristic traps in recreational avalanche accidents.  FACETS summarizes six key heuristic traps that affect backcountry decision making: Familiarity, Acceptance, Consistency, Expert Halo, Tracks/Scarcity, and Social Facilitation.  

Familiarity: In backcountry skiing, parties tend to make riskier choices in familiar terrain. Meteorologists, however, may develop familiarity with observing sites, and this affects their interpretation of data from those sites.  It might, for example, cause them to dismiss an unusual observation because it doesn't fit with their prior experience.  I have done this many times.

Acceptance: Many meteorologists want to be accepted by colleagues as much as backcountry skiers want to be accepted as members of their parties!  If everyone else thinks that an observation or interpretation is correct, you may think so too.  There can also be an anti-acceptance bias in which the meteorologist likes to be a contrarian.  

Consistency: In backcountry travel, this is commitment to an objective.  In meteorology, it might be commitment to a prior forecast or a hypothesis.  When you have been pushing a forecast or arguing for a hypothesis for a long time, it is really hard to change course.

Expert Halo: In backcountry skiing, novices sometimes follow leaders into dangerous situations as they acquiesce to the decision making of a more experienced leader (even guides are not infallible).  Similarly, there is a tendency to give greater credence to more established scientists or forecasters. 

Tracks/Scarcity: This is a tough one to translate to meteorology.  In the backcountry, people may take on more risk if they are competing with others for tracks.  Tracks might not be the right word, but in this latest heat wave, I have seen examples of disinformation due to people rushing to announce records that had not been validated.  

Social Facilitation: It's hard to go against the group, in the backcountry or in meteorology.  On the other hand, sometimes the group is right. 

Scientists are humans too and our biases are real.  On the other hand, the scientific method and processes are designed to minimize these biases and slowly but surely approach a better result or theory.  Feel free to add some additional perspectives.