Tuesday, March 19, 2013

Does Precision = Accuracy?

Public forecasts issued by the National Weather Service and most news broadcasts are usually fairly vague with regards to snow accumulations.  They usually provide a range (e.g., 4-8 inches) for a broad period of time (e.g., tomorrow).

There are, of course, many users who need far greater precision than that.  Determining how much salt to apply to highway for the morning rush hour, or whether or not to close a canyon to traffic to perform avalanche control work requires knowing very precisely how much snow is going to fall and when.  The temperature and the strength of the wind might also be important for these applications too.  

With those examples in mind, let's think about the ramifications of the forecasts being produced for tomorrow (Wednesday) afternoon through Thursday morning by the 12-km NAM and its 4-km high resolution nest, which are remarkably different in terms of the timing and amount of precipitation.

For the three-hour period ending at 3 PM tomorrow afternoon, the 12-km NAM sags precipitation into Salt Lake County and the central Wasatch.  In contrast, the 4-km nest keeps the precipitation to the north.  For Little Cottonwood Canyon, this affects how early tomorrow road crews need to start treating and plowing the highway as skiers are preparing to leave the resorts (Note: it is going to be warm tomorrow, with snow levels near 7500 ft, which will help some, but I'm using this as an example, so go with it).     

Once the storm sags into northern Utah, the 4-km NAM generates a stronger storm, that lasts longer, as illustrated by the 3-h accumulated precipitation for the period ending at 9 PM tomorrow night.  

Here's another way to look at it.  Below is a time series of 3-h accumulated precipitation from the 12-km NAM (green) and the 4-km nest (white) from 1200 UTC (6 AM MDT) this morning through 0000 UTC 22 March (6 PM MDT Friday).  There is a tendency for the 12-km NAM to smooth out the precipitation, generating less abrupt peaks and valleys, whereas the 4-km NAM has greater variability, with a more pronounced peaks and valleys, but also more overall precipitation.  

What you are seeing in the 4-km NAM is the direct influence of resolution, which allows it to produce precipitation features with greater structure and detail.  That's great, but precision does not necessarily equal accuracy.  A high resolution model can produce intense, narrow precipitation bands that may look more physically realistic, but if they are in the wrong place, the forecast is arguably worse.  A time series like the one above can be very misleading in such an instance.

These are the growing pains that meteorology is presently experiencing.  Eventually, we will have ensembles of high resolution models, which will allow us to deal with some of the predictability challenges posed by precision.  Until that happens, we have to carefully interpret high resolution forecasts produced by a small number of models.  


  1. As model resolution increases, what is the NMC doing about increasing the resolution of the observational data used to initialize the models? I imagine they're using the same code to "fit" initial conditions to the 4 km resolution NAM that they are using for the 12 km NAM. Would they also be using the same set of observations. Does anyone know?

    It's fascinating to think about how a model's dynamics and various physical parameterizations are affected by increased resolution. The initial data is just as important, right?

  2. Why can't you just continue to use the 12 km models in the ensemble? More models (now the added 4 km ones) should simply improve the overall performance.

  3. I could have said it better, but yes, use all the models at your disposal. I think a key point is that higher resolution isn't a forecast panacea and that the higher resolution NAM is merely something to add to the full ensemble of forecasts.

    With regards to Lisa's comments, over 99% of the data ingested into the models today comes from satellites. The next big leap will probably be in the area of radar data assimilation, which is an area of very active research, especially for the High Resolution Rapid Refresh (HRRR), which will provide conus forecasts at 3-km grid spacing. In situ (e.g., balloon) upper-air observations at high resolution will never happen. Surface observation assimilation, however, has tremendous possibilities...especially the assimilation of high frequency and high density pressure observations, which seem to offer a lot of bang for the buck.