Friday, January 14, 2022

Can You Really Do Your Own Research?

 


The Internet is a blessing and a curse.  So much information at your fingertips.  But alas, interpreting studies is fraught with perils and there's also a lot of misinformation out there.  In fact, there's plenty of snake-oil salespeople, some of them highly trained, making good money on uncertainty and misinformation.  Even amongst ethical scientists, the value and applicability of papers published in the peer reviewed literature varies.  If you want to get to the bottom of critical issues, from COVID to climate change, can you do it?  

I personally am skeptical of the ability of most people to ignore expert guidance, sift through data and publications on complex matters, and figure it out for themselves.  This is not to put experts on a pedestal, but instead to recognize the realities and complexities of interpreting data and scientific studies, as well as our vulnerability to cognitive biases. 

The reality is that it takes time, practice, and experience to build the skills to identify, evaluate, and synthesize research results.  After 30+ years in research, I'm pretty good at finding, evaluating, and applying scientific knowledge within my specialty area, but I still spend hours sometimes digesting the findings and implications of research articles in order to prepare a paragraph for an introduction for one of my papers or to better interpret my research results.  

Some of my knowledge is transferable to other fields or applications.  Knowledge of math, statistics, and physics, for example, is somewhat transferable.  However, such knowledge can also be a trap.  Lies, damned lies, and statistics right?  Well, subject-matter expertise is very important for understanding the strengths and weaknesses of the datasets used in statistical analyses and the interpretation of results.  When my research requires me to cross into other fields, I have found it very useful to consult with people who work in those fields.

One example is trends in snowpack measures.  I am not a snow hydrologist, but I often work with snowpack observations (including for this blog!).  Trends in snowpack water equivalent are pretty easy to calculate, but if you find that snowpack has declined substantially more rapidly on mountain A compared to mountain B, is that a sign of some sort of climate-driven shift in microclimate?  Perhaps, but if you talk with a snow hydrologist, especially one who collects snowpack observations, you learn that there are other possible causes, including changes in vegetation and tree canopy, human disturbance (e.g., snowmobiles), etc.  

Experts of course make plenty of mistakes and like everyone else are vulnerable to cognitive biases.  They can also be unethical, manufacturing data or falsifying results. The scientific method, peer review, and efforts to reproduce results are approaches for minimizing these problems.  Additionally, panels, boards, and other group approaches can be used to reduce the influence of individual bias.  There's also less formal checks and balances.  I've benefited throughout my career from good friends who have "called BS" or challenged my analysis methods or interpretations when my personal cognitive biases were causing me to cut corners or misinterpret results.  

One thing that I notice quite a bit in social media is that we are suckers for a good story.  I see this frequently with climate change.  Someone passionately claims on social media or at one of my talks that "it's the sun" that has caused global warming. All scientists have ever done is talk about CO2.  We've overlooked the most important climate driver of all!

YES!  Decades of climate research by thousands of scientists and nobody ever thought of that!  Lol.  Such a good story, but of course untrue (see https://skepticalscience.com/solar-activity-sunspots-global-warming.htm for one reason why).  I can provide more and more examples.  Cosmic rays are causing climate change.  Global warming has stoped.  Antarctica is gaining ice.  It goes on and on and on.  

You may accept climate change and think you wouldn't fall for those arguments, but we all suffer from cognitive bias.  I get fooled often.  Did you see this one?


Pretty good story.  I fell for it hook, line, and sinker, except it was apparently 20 eagles tracked for a year rather than one tracked for one year.  I'm not even sure the latter is real at this point.  Perhaps your BS filter is stronger than mine and if so, good for you!  

As you do your own research, be cognizant that there may be more to the story.  Sometimes what appears logical is really illogical.

2 comments:

  1. So what should the temperature of the earth be? Has any point in history of weather the temperature of the earth remained at that temperature? How long did the temperature hold at that temperature?

    ReplyDelete
    Replies
    1. What should the temperature of the Earth be? Physics says that it should be whatever temperature it needs to be for the outgoing long-wave from the planet to balance absorbed solar radiation.

      If you consider "history" to be the last several thousand years, the Earth's temperature has been relatively stable, having varied by only about a degree celsius. Over the past 2000 years, sea level has also been pretty stable.

      Most of our society and infrastructure has been built based on the temperatures and related climate system characteristics of that period. Where we have built many cities, for example, is based on sea levels that have prevailed over the last few thousand years.

      Thus, for the society and infracstructure that exist today, I argue that the best temperature of the Earth was the one that prevailed for the emergence of human civilization.


      Jim

      Delete