Friday, April 17, 2020
Lies, Damned Lies, Statistics, and the Importance of Domain Experts
As a teacher, I like pithy quotes. A short, memorable, and forceful quote can help students avoid pitfalls or take advantage of opportunities throughout their lives and careers.
Mark Twain's quote above is one of my favorites.
Along a similar vein is the book How to Lie with Statistics, which was published in 1954, curiously written by a journalist rather than a statistician, but, despite being dated, provides a number of good cautionary tales about the use and interpretation of statistics.
I share the information above not because I am anti-statistics. Like all scientists, statistics are integral to the work that my group does and are often critical for improving forecasts.
Instead, I share it because we are seeing a lot of statistics being used in media reports and social media blurbs about COVID-19. Numbers based on statistical analysis are quite compelling, but it often takes domain experts, people who are authorities in the subject area, to identify underlying issues in sampling or analysis and provide proper context and interpretation.
Additionally, there are people using statistics in nefarious ways to push their agendas and it is very difficult for the lay reader, including trained scientists outside of the subject area, to smell a rat.
I have seen this in the weather and climate business for many years. Time series of temperature, solar output, CO2, and other variables have been cherry picked and manipulated in all sorts of ways to justify every hypothesis "under the sun" (pun intended).
At this critical juncture it is important that we cautiously examine media and social media reports, move forward with the best-available information from domain experts (not just in health, but other disciplines critical for economic recovery and societal resiliency), and encourage our leaders to do the same.