Regional Modeling Systems


Regional modeling activities in the Joint Numerical Testbed program (JNT; are focused primarily within the Developmental Testbed Center (DTC; The DTC is a distributed facility with components in the JNT at NCAR's Research Applications Laboratory (RAL), and the Global Systems Division (GSD) of NOAA's Earth System Research Laboratory (ESRL). It facilitates the transfer of research results into operations and provides the research community with an easily accessible state-of-the-art Numerical Weather Prediction (NWP) system for research. One of the DTC's focal points is regional forecasting systems, with a goal of accelerating the rate at which new technology is infused into operational weather forecasting. The DTC meets its goals by maintaining and supporting community codes that represent the latest NWP technology, performing extensive testing and evaluation of new NWP technology, maintaining a state-of-the-art verification package, and connecting the NWP research and operational communities through its visitor program. In addition to regional modeling work within the DTC, JNT staff have been working to transfer technologies in support of mesoscale weather prediction for the Colombian Civil Aviation Authority, Saudi Arabia and sparsely observations regions of the world.


Community Codes

Community code is a free and shared resource with distributed development and centralized support. The DTC's community code efforts are collaborative activities with developers at NCEP's Environmental Model Center (EMC), NCAR's Mesoscale and Microscale Meteorology (MMM) Division, NOAA/ESRL/GSD, NOAA's Geophysical Fluid Dynamics Laboratory (GFDL), National Aeronautics and Space Administration’s (NASA) Global Modeling and Assimilation Office (GMAO), National Environmental Satellite, Data and Information Service (NESDIS), the University of Rhode Island (URI), and NOAA’s Atlantic Oceanographic and Meteorological Laboratory (AOML) Hurricane Research Division (HRD). During 2017, the DTC worked with the following software packages:

The DTC contributes to the software management and user support for publically released versions of these systems, which include the latest developments of new capabilities and techniques. Prior to each official release to the user community, the DTC ensures the integrity of all community code software components through a broad range of testing. The DTC also strives for system evolution, in particular through increased interoperability of existing system components, as well as adding new capabilities or techniques. In addition, the DTC provides user support for these packages in the form of Users' Guides, webpages, email helpdesks, and online and on–site tutorials.

Testing and Evaluation (T&E)

The DTC provides a trusted facility that developers and the operational community can rely on for unbiased assessments of the operational prediction systems and potential new additions to those systems. Testing and evaluation undertaken by the developers of new NWP techniques from the research community are generally focused on case studies. However, in order to adequately assess these new technologies, extensive testing and evaluation must be performed to ensure they are indeed ready for operational consideration. Testing and evaluation by the DTC focuses on either extended retrospective time periods or real–time forecast experiments. These forecasts can be generated by the DTC or provided by external modeling groups. The DTC's evaluations include the use of standard verification techniques, as well as new verification techniques. All verification statistics include a statistical significance (SS) and practical significance (PS) assessment when appropriate.

During 2017, the focus within the regional modeling group was on ensemble systems. In most existing regional ensemble systems, model-related uncertainty is addressed by using multiple dynamic cores, multiple physics suites, or a combination of these two approaches. While these approaches have demonstrated potential, it is time-consuming and costly to maintain such systems, especially in operations. In order to move toward a more sustainable and unified system, stochastic parameter perturbations within the High-resolution Rapid Refresh (HRRR) physics suite are being investigated with an initial focus on planetary boundary layer (PBL) and land surface model (LSM) processes, and more recently microphysical processes.  In addition to investigating the stochastic perturbation approach, the DTC is partnering with the Hazardous Weather Testbed to address the merits of different approaches to representing model-related uncertainty by conducting an evaluation of a subset of the members of the Community Leveraged Unified Ensemble (CLUE) from the 2016 and 2017 Spring Experiments.

Using the test harness established in 2016, the DTC conducted functionally similar end-to-end testing of the HRRR model in both a deterministic and ensemble mode. Model Evaluation Tools (MET) verification tasks were used to evaluate the deterministic and probabilistic forecast output. The inclusion of MET provides the opportunity to not only verify the final products, but to also iteratively adjust the ensemble design while examining how probabilistic statistics change when different approaches are utilized.  This work required extensive high performance computing (HPC) resources, which were provided through the NCAR Strategic Capability (NSC) project. The convective allowing ensemble using three stochastic approaches where the stochastic parameter perturbations (SPP) were included in the PBL and LSM schemes performed comparably to the multi-physics control configuration.  Current work is looking at the impact of including SPP in the HRRR microphysics scheme.

The CLUE evaluation activity is focused on addressing the question of whether there is an advantage to using an ensemble composed of multiple microphysics and PBL schemes over a single physics suite ensemble.  Prior studies of multi-physics convective allowing model ensemble systems have focused on composite reflectivity, whereas this study is considering multiple verification metrics and methods.  This approach has demonstrate the need to consider a broader perspective when evaluating the merits of various ensemble approaches before reaching any conclusions as to which approach provides the best forecast skill.

Mesoscale Model Evaluation Testbed (MMET)

The Mesoscale Model Evaluation Testbed (MMET; provides the opportunity for the research community to conduct their own T&E of a new technique.  Datasets for a number of cases deemed to be of high interest by EMC are distributed via RAMADDA, a Repository for Archiving, Managing and Accessing Diverse DAta ( The MMET datasets of opportunity include a variety of initialization and observation datasets, as well as baselines for select operational configurations that were established by the DTC. Operational model output for several NWP systems (both deterministic and probabilistic) were evaluated and the objective verification scores are provided to the research community through MMET as baseline results for comparison with the forecast performance of their innovation. The operational systems include North American Mesoscale (NAM), Rapid Refresh (RAP), High-Resolution Rapid Refresh (HRRR), and Hurricane WRF (HWRF) models for deterministic forecasts and the Storm Scale Ensemble of Opportunity (SSEO) for probabilistic forecasts. For each deterministic model, MMET cases of high relevance were determined (select MMET cases were aged off and new, relevant cases were identified) and operational data were obtained. For the probabilistic forecasts, a multi-week period of SSEO data collected during the 2016 Hazardous Weather Testbed (HWT) Spring Experiment was gathered, and staged for distribution through RAMADDA.


Many times the biggest hurdle when running a new software system is getting it set up and compiled on the intended computer platform. Building complex systems that require a number of external libraries can be a prohibitive hurdle for users to overcome.  In order to reduce some of this difficulty, software containers are being exploited to ship complete software systems to users. The containers have everything that is needed to run a software application, including the necessary operating system components (tools and libraries) and compiled executable (or code and compiler), thus, allowing for the user to quickly produce output without being delayed by technical issues. DTC staff members created UPP and MET containers to supplement those containers that had already been established by others in the community (including, WPS, WRF, and NCL) so that an end-to-end NWP system can be fully employed through containers. Along with the software containers, datasets that make up two Mesoscale Model Evaluation Testbed (MMET) cases were also bundled in a container. By establishing these additional containers, the DTC is assisting the user community (especially students) with efficiently running NWP components and making connections with future collaborators.  To further assist the community, the DTC is offering an AMS short course in January 2018 geared toward raising awareness of these tools for testing and evaluation of NWP innovations.

Real-time modeling systems

JNT staff have participated in technology transfer activities in support of the Colombian Civil Aviation Authority’s weather prediction needs. A prediction system based on the WRF model and GSI was developed and deployed in collaboration with Sutron/Meteostar, which is responsible for operational support and visualization. The work has leveraged JNT capability developed under DTC funding. Workflows based on Rocoto provide a stable and modular deployment environment.  The high-elevation tropical weather of Colombia provides unique prediction challenges associated with deep convection and fog, and allows for evaluation in an environment not commonly studied. The focus in FY2017 has been on improving the model through the implementation of the GSI data assimilation system for data assimilation of surface and satellite observations.  The project has also developed a 10-member multi-physics ensemble system in an effort to improve forecast capabilities.  Retrospective testing and evaluation provide the basis for optimizing system configuration, and for measuring improvements from upgrades.

JNT staff also participated in a research project with partners at King Abdullah University of Science and Technology (KAUST) in Jeddah, Saudi Arabia. The goal of the project is to better understand the effects of dust and anthropogenic emissions on air quality, clouds and climate in the coastal and central regions of the Arabian Peninsula and to document the main source regions and mobilization processes of airborne dust over the Arabian Peninsula using existing observations.  The focus of research in FY2017 has been on understanding the ability of WRF to predict dust storm conditions.  WRF simulations have been compared to aircraft, radar, and surface observations of dust-generating mesoscale convective storms that were observed in the central region of Saudi Arabia.

In addition, JNT staff supported the evaluation of observation data quality from a USAID-funded project to develop and deploy low-cost weather instrumentation in sparsely observed regions of the world.  The project uses innovative new technologies such as 3D printers, Raspberry Pi computing systems, and wireless communications to develop a sustainable system that can be built locally in under-developed countries.  The study evaluated data quality of temperature, pressure, humidity, wind, and precipitation observations collected for the NCAR and NOAA testbed sites in Boulder, CO and Sterling, VA. Results indicate the low-cost sensors provide high quality data that could be used for applications for agriculture, water resource monitoring, health, and monitoring of hazard weather conditions.


In the coming year, the JNT--through the DTC--will continue to support various community codes, including NWP systems, GSI and MET. The DTC will also help organize and support tutorials on the community codes that it supports, as well as on mesoscale models, data assimilation, hurricanes, and forecast verification.  Relevant workshops will be offered to stimulate discussion among the research and operational modeling communities on future directions of development. In addition, efforts will continue related to evaluating deterministic and ensemble-based probabilistic model output.