Ceiling and Visibility Products for Alaska

Background

Poor weather conditions, particularly restricted visibility and low cloud tops, were the leading cause of fatal general aviation (GA) accidents in Alaska from 2001-2012.  Traditional weather observations from Alaska’s widely dispersed airfields inadequately forewarn of weather likely to be encountered along routes between stations or, in particular, through hazardous mountain passes with localized conditions.  In 2014, the National Transportation Safety Board (NTSB) included “GENERAL AVIATION: IDENTIFY and COMMUNICATE HAZARDOUS WEATHER” on its Most Wanted List to improve transportation safety. 

In January 2016, scientists and engineers at the National Center for Atmospheric Research’s Research Applications Laboratory NCAR/)RAL received funding from the Federal Aviation Administration’s Aviation Weather Research Program  (FAA/AWRP) to provide enhanced ceiling, visibility, and flight category products to the Alaska Aviation Weather Unit (AAWU). This effort, which includes collaborations with MIT Lincoln Laboratory and NOAA’s Global Systems Division (NOAA/GSD), is aimed at producing rapidly-updated, high resolution, gridded products of ceiling heights and visibility (C&V) conditions across Alaska.  The Ceiling and Visibility Analysis – Alaska (CVA-AK) products serve as a “first guess” estimates of C&V conditions across Alaska at or near instrumented and non-instrumented airfields and along data-sparse routes between airfields including treacherous and heavily-traveled mountain passes.

The first version (Version 1.0) of CVA-AK product combined ceiling and visibility information from the latest EMC RAP model with METAR observations of C&V using data fusion techniques to produce Flight Category, Ceiling Heights and Visibility gridded fields. These fields were updated every 20 min and hourly analysis products have been viewable by AAWU forecasters on the IC4D display system that they use to produce their aviation forecasts.  New versions of the CVA-AK products provided to the AAWU in 2017 include updates and improvements to the RAP/METAR products (Version 1.5) and the inclusion of geostationary satellite observations (Version 2.0) and are discussed in the sections below.

2017 Accomplishments

CVA-AK Version 1.0 Product Performance

Figure 1. NCAR/RAL CVA-AK Flight Category product displayed on the AAWU forecasters IC4D system with similarly coded METAR observations overlaid.  Maroon, red,  blue, and green regions represent the following flight categories for pilots: LIFR, IFR, MVFR, and VFR.
Figure 1. NCAR/RAL CVA-AK Flight Category product displayed on the AAWU forecasters IC4D system with similarly coded METAR observations overlaid.  Maroon, red,  blue, and green regions represent the following flight categories for pilots: LIFR, IFR, MVFR, and VFR.

Version 1.0 of the CVA-AK product ran at the AAWU forecast office for over a year.  Forecasters noted problems with this first version of the product, specifically that it tended to over-forecast low ceiling heights.  This over-forecasting can be seen in Figure 1, which shows a comparison of the CVA-AK Flight Category field compared to METAR-based Flight Category calculated from the surface observations along western coastal region of Alaska.

A thorough evaluation of Version 1.0 was conducted using a cross-validation approach where a set of 25 METAR stations were withheld from the product and used as an independent truth dataset to evaluation the accuracy of the CVA-AK real-time product.  This cross-validation was done in post-analysis mode, not real-time. Data from the nine-month evaluation period were analyzed and summarized in a technical report. Information presented included seasonal and regional assessments of performance of ceiling and visibility individually, as well as flight category.  An example of performance broken down by regional statistics is shown in Figure 2, for the CVA-AK flight category product.  The evaluation showed that the CVA-AK has skill at diagnosing aviation hazards. Regional differences in performance are being investigated (e.g., Anchorage; Region 2).  Detailed analyses showed that the complex terrain of Alaska affects the C&V performance in the model.  Use of a higher resolution model such as the HRRR could help improved performance in this regard.  Seasonal differences in performance are pronounced for ceiling.  Visibility has consistent performance across the seasons but overall is less accurate than ceiling performance.

Figure 2: Boxplots of CVA-AK Ceiling by METAR observed ceiling category for each region.
Figure 2: Boxplots of CVA-AK Ceiling by METAR observed ceiling category for each region.

Changes were made to address the forecasters’ concerns and to correct other problems identified with the products.  These changes included making using of the NDFD-based terrain data, using the RAP model terrain grid for ceiling AGL computations and allowing the METAR station observations to override the RAP model values when METAR observations show worse weather conditions.

METAR-RAP Comparative Analysis and Real-Time Calibration Technique

Understanding the level of consistency between surface-based observations and RAP forecasts of C&V is critical for developing a useful analysis product. The model provides much greater detail in terms of resolving coastal and terrain driven gradients in the frequency of low ceilings, but notable biases in the model predictions are also evident. These differences become more clear when looking at frequency difference (modeled minus observed frequency) plots. These comparisons can be used to remove first order bias in the model and to better understand the representativeness of the surface observation measurements and how this information should be spread in the gridded C&V analysis product.

flow chart
Data flow diagram

Two new applications were developed for producing model calibration files in real-time. The data flow diagram that shows how these two new applications interface with CVmodelCal is shown below.   The application called ObsClimoFreq reads in the METAR station data for a configurable number of days and calculates the frequency of occurrence for the requested fields (ceiling, visibility) using a configurable list of thresholds. The output is a comma-separated ascii file of frequencies for each METAR station along with latitude and longitude.  The application called FrequencyMatch reads in this ascii file along with the gridded RAP model fields (ceiling and visibility) for a configurable number of days. From the model data, it calculates the frequency of occurrence using the same set of thresholds that were calculated for the METAR stations at each of the METAR station locations. The RAP model and METAR frequencies are then compared to determine if calibration is necessary. If deemed necessary, the threshold used to compute the RAP model frequency is adjusted until the model frequency matches the METAR frequency as closely as possible. Several simple data quality checks are performed along with checks to ensure that an adequate amount of data was available to perform these comparisons. FrequencyMatch then uses a circular filter to place these optimized thresholds onto a RAP grid.  For grid points where multiple METAR stations have influence, we perform a distance-weighted average to determine the optimized threshold value.

An example of the calibration data for a single level (2.5 km) is shown in Figure 3 below. In this example, adjustments are evident as the non-green areas. For example, the RAP model had a consistent over-forecasting bias in the vicinity of Anchorage. The resulting adjustment is to increase the threshold value used to identify ceilings of 2.5 km or less. This is done in order to reduce the frequency of occurrence at this ceiling height threshold. Note that values are converted to the requisite units of kFT for display and user evaluation.

Figure 3. Example of the ceiling calibration values for a single level (2.5 km) as obtained on 12 October 2017.
Figure 3. Example of the ceiling calibration values for a single level (2.5 km) as obtained on 12 October 2017.

These two applications take the place of a time intensive manual offline process that had previously only been done intermittently.  The new procedure runs once per day, thus providing the capability of the system to reliably respond to changes in model bias. As shown in the diagram above, the output of FrequencyMatch feeds into the previously developed application called CVmodelCal which reads in the model data along with the ceiling and visibility calibration files and adjusts the model ceiling and visibility values in areas that are consistently out of line with the observations.  The real-time calibration process has been included in the CVA-AK Version 2.0 products.

VA-AK Version 2.0 Product

Figure 4: Ceiling difference between CVA-AK version 2 (with satellite cloud mask) and version 1.
Figure 4: Ceiling difference between CVA-AK version 2 (with satellite cloud mask) and version 1.

CVA-AK Version 2.0 was installed at the AAWU in October of this year.  This version of the product includes the real-time calibration technique discussed above, and the integration of a GOES satellite cloud mask (Jedlovec et al) into the product.  The Jedlovec algorithm uses GOES 3.9 and 11 micron channels to identify clear and cloud areas at every pixel and to create the cloud mask. This information is then applied over the RAP/METAR analysis and is used to “clear” the ceiling only.  This mask is not applied to the visibility field. A preliminary comparison was done for a 30-day initial period (April 18-May 17, 2017), and used a grid-to-grid match of the ceiling values. These results show how the incorporation of the satellite cloud mask increases ceilings in version 2 over version 1, particularly over the oceans, as can be seen in Fig. 4.  A few terrestrial locations also show some large differences, such as between METAR stations on the north slope of Alaska. When METAR observations of ceiling are available, these override the other data sources in the product. Thus, the small differences seen within the METAR ‘circles’ occur only when METAR ceiling reports are not available (e.g. missing). The most significant effect of inclusion of the mask into the product is over the maritime regions and off-shore coastal areas where there are no METAR observations.  The mask helps to constrain errors in the model over these area where there are no surface observations.

Plans for 2018

Version 2.0 of the CVA-AK product will be statistically evaluated during the coming year and compared to Version 1.5 performance.  Collaborations will also begin with MIT/LL to develop the methodology to blend web camera retrieved visibilities from the Alaska web cameras into the CVA-AK.  Collaboration will also begin with EMC’s RTMA team to establish methodologies within RTMA similar to the real-time model calibration technique in the CVA-AK system.