Global Modeling

BACKGROUND

A research to operations (R2O) initiative was established in 2014 by NOAA to upgrade the current operational Global Forecast System (GFS) to run as a unified and fully coupled Next Generation Global Prediction System (NGGPS). NOAA’s long-term plan seeks to integrate the capabilities of its short-term (GFS), ensemble (GEFS), and sub-seasonal (CFS) NWP applications under the infrastructure of NGGPS. A key challenge during this process is to develop a common physics infrastructure that works across all temporal and spatial scales as well as to accommodate an efficient R2O pipeline that effectively uses the expertise in both the research and operational communities. As part of this effort, the Global Model Testbed (GMTB) team was established within the Developmental Testbed Center (DTC) to facilitate community involvement in the development of NGGPS through several avenues: contributing to select aspects of code management and infrastructure for the community to interact with the system, supporting a hierarchical testing framework to NGGPS developers, and facilitating and performing testing and evaluation of innovations for the operational system. The GMTB consists of scientists and software engineers within RAL’s Joint Numerical Testbed(JNT) who take active roles in supporting R2O for global numerical weather prediction (NWP) by closely collaborating with NOAA’s Environmental Modeling Center (EMC) and the research community to develop an Interoperable Physics Driver (IPD), a Common Community Physics Package (CCPP), and a physics testbed.  In addition to supporting NOAA’s global applications, the DTC is providing testing and evaluation support to the Air Force for upgrades to their global forecast system GALWEM (Global Air-Land Weather Exploitation Model), which is based on the UKMet Office Unified Model.

FY2017 ACCOMPLISHMENTS

Figure 1: Schematic of collaborative environment that IPD/CCPP framework will provide for the research and operational communities to work together towards advancing atmospheric physics representation in numerical weather prediction models.
Figure 1: Schematic of collaborative environment that IPD/CCPP framework will provide for the research and operational communities to work together towards advancing atmospheric physics representation in numerical weather prediction models.

Interoperable Physics Driver (IPD) and Common Community Physics Package (CCPP)

A modular physics suite accessible both in-line as part of a prediction model, and off-line for isolated testing, will enable physics innovation and contribution from the broader community.  In support of this goal, the DTC and NGGPS put forth the concept of a Common Community Physics Package (CCPP) coupled with an Interoperable Physics Driver (IPD) that would be setup and maintained by GMTB to facilitate efficient and effective collaborative development of next generation physics suites. Although a one-way IPD was developed by EMC and used for the NGGPS Dynamical core Test Group (DTG), an IPD with two-way interoperability that can be easily used with all schemes and suites of the CCPP needed to be developed for the vision of the CCPP to be realized. Significant progress was made during FY2017 toward this goal.  Over the past year, the requirements for the IPD and CCPP underwent extensive review and refinement, and informed a software design. GMTB’s contribution to the IPD effort will enable constructing physics suites at runtime by parsing a user-friendly configuration file, allowing for running the parameterizations within the CCPP in a very flexible manner.  Work is currently underway to populate the CCPP with the operational GFS physics suite, which will serve as the baseline for assessing the performance of more advanced physics suites.  Another important component of this work is advancing the documentation efforts undertaken in FY16.  Comprehensive technical web documents describing the IPD and CCPP are now available from the DTC website.  Figure 1 shows the ecosystem that is envisioned for enabling various levels of engagement from the community in the effort to advance atmospheric physics.

Hierarchical Testing

Figure 2: Diagram illustrating the testing hierarchy plan.  LR indicates low resolution, MR medium resolution, and HR high resolution. Shading indicates where groups are anticipated to focus their efforts.
Figure 2: Diagram illustrating the testing hierarchy plan.  LR indicates low resolution, MR medium resolution, and HR high resolution. Shading indicates where groups are anticipated to focus their efforts.

To facilitate the development of an advanced physics suite for NGGPS, the JNT, working through the DTC, is developing a uniform ‘test harness’ to enable in-depth investigation of various physical parameterizations. The principal purpose of this physics testbed is to assist the research and operational communities in streamlining the testing process to accelerate the transfer of worthy improvements into operations. The testbed should see use as both a tool for physics developers to display merit and further improve upon their schemes and as an addition to EMC’s physics development decision-making arsenal. The test harness represents the logical progression for testing newly developed parameterizations that typically takes place within the scientific community. Components and complexity are gradually added and iterated upon as one moves through the hierarchy until the full forecast model complexity is reached. The hierarchy is designed to complement both the existing testing protocol at operational centers and independent testing typically performed by parameterization developers. The natural sequence of testing new physics schemes tends to follow tiers of progressively difficult and computationally intensive model runs as merit warrants, and the GMTB mimics this progression.  Figure 2 shows a diagram illustrating the sequence of physics testing. 

Single Column Model (SCM)

As part of the GMTB Physics Test Harness, a SCM that makes use of the IPD was developed in FY16.  The “catalog” of cases to use with the SCM is a work-in-progress, with one shallow convective case based on the transition from stratocumulus-to-cumulus as observed during the Atlantic Stratocumulus to cumulus Transition EXperiment (ASTEX) field campaign and two deep convective cases based on observations collected during the Tropical Warm Pool – International Cloud Experiment (TWP-ICE) field campaign and the Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) intensive observation period from the summer of 1997.  All cases are initialized and forced based on observations made during their respective field campaigns.  Depending on individual case requirements, it is possible to easily configure the SCM to apply forcing in the three main ways described in Randall and Cripe (1999): using horizontal advective tendencies together with prescribed vertical motion, using total tendencies that prescribe forcing due to both horizontal and vertical motion together, and using the relaxation method. Going forward, the GMTB will add more cases to the catalog, including cases that will require changes to the underlying GFS physics code (e.g., ability to turn off specified physics schemes within the suite) and cases of interest to the general community, such as those from the recent LES ARM Symbiotic Simulation and Observation (LASSO) project. The cases added by the GMTB can also serve as an example for community members to add cases of interest. In addition, the SCM is set up to easily run using forcing ensembles that can be used to understand a physics suite’s response to uncertainty in the forcing.

Workflow for Low-Resolution/Medium-Resolution Global Forecast Mode

To facilitate three-dimensional testing that provides information about the interaction between the physics packages and feedback on the large-scale flow, the GMTB assembled an end-to-end workflow for the atmospheric component of the GFS [Global Spectral Model (GSM)] in FY16 that includes post-processing, comprehensive verification, and production of graphics. Building on previous progress, the GMTB successfully established an end-to-end workflow system for running NEMS/Global Spectral Model (GSM) and UPP (strongly leveraging EMC capabilities), as well as running DTC-contributed components. The end-to-end-workflow system reached a mature state that allowed for running a test of the Grell-Freitas (GF) convective parameterization.  The DTC-contributed workflow components for creating Python-based forecast plots (e.g. temperature, moisture, convective vs. non-convective precipitation) and verification results (e.g., near-surface, upper-air, and precipitation verification) continue to be upgraded to include additional features and flexibility. This work included adding a ‘scorecard’ capability to the verification arsenal; the ‘scorecard’ is a way to summarize cohesive signals in the performance differences between two configurations, including level of significance, for specified metrics, variables, levels, regions, and times.

Figure 3: root mean squared difference between the temperature forecasts of the two models for the whole month of June 2017 for a specific lead time (12), issue time (12) and model level (8).
Figure 3: root mean squared difference between the temperature forecasts of the two models for the whole month of June 2017 for a specific lead time (12), issue time (12) and model level (8).

Work is also underway to expand the testbed capabilities to equip physics developers with a wide range of tools to assess strengths and deficiencies of physics. The capability to produce bias information from GSI diagnostic files, which provide O-B (observation – background) information will soon be available. In addition, the GMTB is collaborating with NGGPS PI Jason Otkin to include synthetic satellite output from UPP to help with evaluating the model’s ability to accurately simulate clouds and moisture. The capability to perform 6-h global precipitation verification has also been added to the testbed.

Testing and Evaluation

Figure 4: Same as Fig. 3 except for the highest model level (64) with a long lead time (54).
Figure 4: Same as Fig. 3 except for the highest model level (64) with a long lead time (54).

During FY17, the JNT participated in two extensive testing and evaluation activities utilizing its physics testbed to assess the potential impacts of including the Grell-Freitas convection parameterization in the advanced physics suite for future GFS implementations.  The GFS physics suite with the Simplified Arakawa Schubert (SAS) convective scheme served as the baseline for the first test, whereas the scale aware SAS served as the baseline for the second test.  The second test also included both cold start and cycled global forecasts.  Both tests produced mixed results where it depended on the field and/or forecast lead as to which configuration produced the best verification statistics.

In addition to the physics testing, the JNT performed a validation exercise to assist the Air Force with assessing the readiness of its GALWEM 10.4 implementation.  This evaluation focused on comparing forecasts provided by the UKMet Office for its 10.4 implementation and forecasts provided by the Air Force for a comparable implementation.  The evaluation concluded that the differences between the two forecasts were consistent with that expected for parallel systems where the only difference is the computing platforms, contributing to the Air Force’s decision to proceed with the operational implementation.  Figure 3 provides one example, showing the root mean squared difference between the temperature forecasts of the two models for the whole month of June 2017 for a specific lead time (12), issue time (12) and model level (8). Differences are quite small and fairly random. Figure 4 shows the highest model level (64) with a long lead time (54), where somewhat larger, more organized differences appear. These differences are still within expected tolerances, but the circular structure of the differences over North America may be of interest. These results were determined via inspection. Future work may include automated detection of areas with organized, larger model differences with the MODE software. Then model differences of interest can be filtered by size, magnitude, and approximate location in a faster, more automated manner. The JNT also provided preliminary input for the Air Force’s Post Processing Products upgrade, work that will continue into FY18. 

Model Evaluation for Research Innovation Transition (MERIT)

The Mesoscale Model Evaluation Tools (MMET) kit was developed for use with regional models.  In 2017, work was begun to expand the MMET concept to include the global modeling framework; the new framework has been named Model Evaluation for Research Innovation Transition or MERIT.  MERIT focuses on a testing framework for selected meteorological cases, which will be studied in depth to help expose shortcoming with the ultimate goal of helping improve operational NWP.  MERIT fits well within the hierarchical testing framework established within the GMTB 

FY2018 PLANS

During FY18, work to transition the current operational GFS physics suite into the CCPP framework will be completed, followed by the addition of schemes that are candidates for the advanced physics suite slated for NCEP’s next GFS implementation.  Work will continue toward expanding the capabilities in the physics testbed in order to equip physics developers with a wide spectrum of tools to assess strengths and deficiencies of physics parameterizations.  Global model testing activities supporting the AF implementation cycle will include completing the post-processing products upgrade evaluation and conducting an evaluation of an upgrade to the Land Information System (LIS) that provides land surface states and fluxes to GALWEM.  A validation exercise directed at assessing the AF implementation of the Unified Model’s data assimilation system will also be a likely activity during FY18.