Four-Dimensional Weather System (4DWX)

BACKGROUND

Since the middle 1990s, the U.S. Department of Defense’s (DOD’s) Army Test and Evaluation Command (ATEC), then known as TECOM, has sponsored RAL to conduct research, development, and technology-transfer of the Four-Dimensional Weather (4DWX) system.  4DWX is an advanced numerical weather prediction (NWP) system that analyzes current weather and makes detailed predictions of weather over the next several days across many scales of phenomena.  4DWX’s NWP core is the Weather Research and Forecasting (WRF) Model.  4DWX ingests observations into the NWP core through RAL’s Real-Time Four-Dimensional Data Assimilation (RTFDDA) scheme.  RAL upgrades 4DWX software several times per year.

4DWX is used by ATEC meteorologists and other DOD staff at eight test ranges across five major climate zones: White Sands Missile Range, New Mexico; Electronic Proving Ground, Arizona; Dugway Proving Ground, Utah; Aberdeen Test Center, Maryland; Redstone Test Center, Alabama; Airborne and Special Operations Test Directorate, Fort Bragg, North Carolina; Yuma Proving Ground, Arizona; and Cold Regions Test Center, Alaska.  4DWX is also used at other locations when ATEC meteorologists are required to support temporary exercises in places such as San Nicholas Island, California; Spaceport America, New Mexico; Fort Wingate, New Mexico; Isle of Benbecula, Scotland; Woomera, Australia; Pacific Missile Range Facility, Hawai’i; and Kwajalein Atoll, Marshall Islands.

Thanks to 4DWX, ATEC meteorologists have greater access than ever to technology and expertise that help them produce weather forecasts and analyses at the scales, and with the accuracy and utility, required to support safe and cost-effective testing by the DOD.  For RAL and its collaborators in the university community, one of the most attractive elements of the 4DWX project is that the ATEC test ranges serve as natural laboratories for atmospheric research, complete with dense observing networks and specialized data that permit study of mesoscale and microscale phenomena in complex terrain.  Continual improvements to 4DWX and to community numerical weather prediction models, such as the WRF Model, are made possible through this collaboration with DOD.

PRIMARY 4DWX TECHNOLOGY

Weather Research and Forecasting (WRF) Model

The predictive core of 4DWX is the Advanced Research version of the WRF Model (sometimes abbreviated ARW), a long-established industry standard for NWP in operations and research.   The model code is open source.  It was developed by a group of partners including NCAR, the National Oceanic and Atmospheric Administration, the Air Force Weather Agency, the Federal Aviation Administration, and the university community.  The model is used across many scales, from global to microscale.

Real-Time Four-Dimensional Data Assimilation (RTFDDA)

The project continues to rely on Real-Time Four-Dimensional Data Assimilation (RTFDDA) as one way to ingest observations and define the atmosphere’s current state for 4DWX’s NWP core, the WRF Model.  RTFDDA involves modifications to an NWP model’s predictive equations so the model can be gently adjusted, or nudged, toward observed conditions during the model’s analysis stage, before the forecast stage begins.  The scheme is computationally efficient and preserves the precise timing of observations, which gives 4DWX a particularly accurate depiction of the weather at any instant.  RTFDDA continues to show itself superior to, or the equal of, many alternative methods of data assimilation in operational systems.  RTFDDA assigns quality flags to observations during the analysis and forecast cycling, rather than as a pre-processing step, providing more accurate and stable assessments of each observation’s usefulness in data assimilation.  RTFDDA also has an improved means of dealing with cases when a ground-based observing site’s actual elevation differs significantly from the simulated terrain height in the model, which is a mundane but under-appreciated problem in applied NWP.

Ensemble 4DWX (E-4DWX) 

Figure 1.  Probability density functions (PDFs) of an uncalibrated (solid black line) and a calibrated (solid blue line) 24-h ensemble forecast of 2-m air temperature (°C) from E-4DWX for station 01 at Dugway Proving Ground.  Calibration reduces bias, broadens spread, and increases sharpness in the ensemble forecast when compared with observations, such as 25.9°C (thick red bar) in this case, and when compared to the baseline prediction one could get from the climatological PDF for summer (dashed black line).  From article by Knievel et al. (Weather and Forecasting, 2017).
Figure 1.  Probability density functions (PDFs) of an uncalibrated (solid black line) and a calibrated (solid blue line) 24-h ensemble forecast of 2-m air temperature (°C) from E-4DWX for station 01 at Dugway Proving Ground.  Calibration reduces bias, broadens spread, and increases sharpness in the ensemble forecast when compared with observations, such as 25.9°C (thick red bar) in this case, and when compared to the baseline prediction one could get from the climatological PDF for summer (dashed black line).  From article by Knievel et al. (Weather and Forecasting, 2017). 

Since 2007, Dugway Proving Ground has used an ensemble version of 4DWX (called E-4DWX) developed by RAL.  E-4DWX provides a suite of 30 forecasts valid at the same place and time, each producing slightly different but similarly realistic forecasts.  Differences among ensemble members are induced by varying initial conditions, boundary conditions, and model physics.  All members are based on the WRF Model.  The ensemble captures the forecasts’ probability information that varies with changes in weather regime.  In 2014, E-4DWX was expanded to include three additional ranges in the intermountain West: White Sands Missile Range, Yuma Proving Ground, and Electronic Proving Ground.  E-4DWX products include maps and time series of means, standard deviations, or fractions of the ensemble exceeding thresholds.

A subset of output from E-4DWX is calibrated so that the probability of E-4DWX’s forecasts being realized matches the observed probability (Figure 1).  Benefits of calibration include: 1) reducing forecast error of the ensemble mean, partly by reducing bias; 2) increasing reliability, resolution, and sharpness, including for predicting extreme and potentially devastating weather; and 3) providing a measure of forecast uncertainty through the spread among ensemble members.  Calibration is performed on moments of the overall probability density function, no matter the size of the ensemble membership, as opposed to calibrating output from specific ensemble members.  This makes E-4DWX particularly robust, even if individual members of the ensemble fail at some point during the forecast.  E-4DWX’s calibration algorithms combine logistic regression with quantile regression.  To ensure the ensemble’s reliability, it is pre-processed, then the calibration is explicitly conditioned on the ensemble dispersion.  Regressions are always performed with cross-validation to minimize the likelihood of overfitting.  

Forecasts of severe weather

The 4DWX system has components that predict severe weather on two scales: the next few hours, based on both observations and model predictions blended via the AutoNowcaster (at Redstone Test Center and White Sands Missile Range) and the AutoNowcaster-Lite; and the next few days, based on model predictions alone.  The AutoNowcaster and AutoNowcaster-Lite systems employ the dual polarization data available from the nation’s WSR-88D (i.e., NEXRAD) network as well as Terminal Doppler Weather Radar (TDWR).  An algorithm called Trident alerts forecasters to conditions that could lead to flash flooding.  Trident predictions are at 10-min intervals to a lead time of 1-hour.  The algorithm currently uses a standard Z-R relationship to relate radar reflectivity to precipitation rate. 

Predictions of lightning

At all ranges, 4DWX now includes a tool for tactical prediction of lightning (lead times of minutes to tens of minutes) and a tool for strategic prediction of lightning (lead times of hours to days).  The former is based on WSR-88D radar data, used to monitor reflectivity above the melting level.  The latter is based on numerical output from 4DWX’s predictive core.  Algorithms are calibrated at each range independently, based on summer and winter cases from previous years.

Predictions of flood-inducing rainfall

To warn test ranges about the potential for flood-inducing rainfalls, 4DWX relies on Trident.  Trident uses two methods of calculating warning criteria, one based on maximum rainfall in a drainage basin, another based on the percentage of a basin covered by rainfall of various thresholds.  Those algorithms now include radar-based estimates of rainfall rate calculated from dual-polarization moments.

Analog ensemble 

Figure 2.  Example of the 4DWX display of AnEn predictions of near-surface conditions at station 1 of Dugway Proving Ground.  The top panel shows the mean prediction of 2-m air temperature (°C in dark blue) as a function of valid time (hour, month, and day) within an envelope of ± 1 standard deviation (cyan) about the mean.  The bottom panel shows the same but for 10-m wind speed (m s-1).  Observations at station 1 are in gray on both panels.
Figure 2.  Example of the 4DWX display of AnEn predictions of near-surface conditions at station 1 of Dugway Proving Ground.  The top panel shows the mean prediction of 2-m air temperature (°C in dark blue) as a function of valid time (hour, month, and day) within an envelope of ±1 standard deviation (cyan) about the mean.  The bottom panel shows the same but for 10-m wind speed (m s-1).  Observations at station 1 are in gray on both panels.

4DWX’s Analog Ensemble (AnEn) uses a set of algorithms to calculate probabilistic predictions that rely on archives of observations and model output to collect an ensemble of prior forecasts made under analogous weather patterns (Figure 2).  Predictions from analog-based methods are inherently calibrated, so an extra calibration step is not required.

Coupled applications

Direct weather analyses and predictions from 4DWX and E-4DWX are the core of the weather information used by forecasters at the ATEC ranges, but that information can be made even more valuable when it is supplied to decision support systems (DSSs) that simulate how the weather affects other processes and conditions, such as sound propagation and the transport and dispersion of airborne hazards.  Examples of DSSs that have been linked to 4DWX and/or E-4DWX include:

  • Noise Assessment and Prediction System (NAPS)
  • Second-order Closure Integrated Puff (SCIPUFF) model
  • Lewis Rocket Trajectory Model
  • Open Burn / Open Detonation Model (OBODM)

4DWX Web Portal

The primary interface to the 4DWX system at all ATEC ranges is the 4DWX Portal.  The Portal’s flexibility, accessibility, modularity, and extensibility are ideally suited to the customized weather support that RAL provides to forecasters, who have eagerly welcomed how the Portal improves their workflow.  Weather maps and related graphics from 4DWX include optional color palettes that can be accurately seen by the color-blind.  The Portal’s dashboard has a flexible, configurable layout, with streamlined access to portlets for coupled applications.  The list of output formats that the Portal supports includes the third-party BUFKIT and RAOB applications.

Integrated Data Viewer (IDV)

RAL collaborates with UCAR’s Unidata program to include among 4DWX’s display options the Integrated Data Viewer (IDV), which is a sophisticated, flexible, Java-based application for analyzing and displaying geophysical data.  IDV is the primary means by which range forecasters explore in greater depth the weather analyses and forecasts from 4DWX.  This more complex and flexible exploration complements the “virtual map wall” that is available through the 4DWX Web Portal, whose purpose is to provide the easiest and quickest interface to a standard suite of pre-generated weather maps.  IDV is also a research tool and is employed by scientists and engineers during their testing, development, and refinement of 4DWX.

OUTREACH AND TRAINING

Each year, RAL provides to ATEC meteorologists at each test range several days of on-site training on 4DWX technology.  Not only does 4DWX improve every year, but the test support required of ATEC meteorologists also changes frequently.  Moreover, turnover among ATEC forecasters also points to the need for a regular training cycle, independent of how rapidly 4DWX technology changes.  Close interaction between ATEC and RAL is critical for maintaining the project’s success.

SELECTED ACCOMPLISHMENTS IN FY2019

4DWX build and install on two clusters at DOD Supercomputing Resource Centers (DSRCs)

After redesigning key elements of 4DWX to be more platform-independent, modifying data feeds, and improving the scope and sophistication of 4DWX’s system monitoring, in FY2018 RAL began running 4DWX operationally at the DSRCs.  In FY2019, this effort was expanded to include fully automated builds and turn-key installations of the latest version of 4DWX at the DSRCs.

Moved 4DWX code repository to Git

Modernization of the 4DWX base code continued in FY2019, with the move to Git.  This allows for greater coordination of software updates and easier methods of deploying new software to the ATEC ranges and DSRCs.

Analog Ensemble (AnEn)

A full analysis of Cold Air Damming (CAD) events was conducted for Aberdeen Test Center.  Classic CAD cases have been identified and are being used to develop a prognostic algorithm to use 4DWX output to forecast the likelihood of CAD.

FastEddy

FastEddy is a new hybrid CPU/GPU-accelerated, large-eddy simulation (LES) model developed by RAL.  The model comprises resident-GPU code, so all prognostic calculations are carried out in an accelerated manner on GPUs, with CPUs used only for model configuration and input/output.  In FY2019, RAL began testing FastEddy for predicting the statistical properties of turbulence and other microscale phenomena at specially chosen test sites at the ATEC ranges.

Integration of new GOES satellite data into 4DWX

New datasets, including addition infrared channels, the Geostationary Lightning Mapper (GLM) and high temporal and special resolution products, are now available with the new GOES satellites.  These products have been integrated into 4DWX.

Display of 4DWX-predicted weather impacts

RAL has developed a new tool that displays the impacts of predicted weather conditions on tests at the ATEC ranges.  This tool allows the ATEC meteorologists to quickly assess weather threats and when safety and test operations may be impacted by unfavorable weather conditions. 

Testing predictions of wet-bulb globe temperature

Wet-bulb globe temperature (WBGT) is a key index of heat stress commonly used in the military and in the sports community to assess instances when heat, humidity, lack of wind, and intense sunshine combine to make working outside unsafe.  WBGT is an empirical quantity, but RAL has developed test algorithms to diagnose an approximate WBGT from 4DWX forecasts.  RAL evaluated several formulae for the algorithm, which will soon be applied in the form of a predictive ensemble for improving the safety of DOD’s outdoor test exercises.

SELECTED KEY PLANS FOR FY2020

Port E-4DWX to DSRCs

Following the success of 4DWX’s transition to the supercomputers at DSRC’s, RAL will begin to port E-4DWX to the same systems.  Because the available nodes at DSRC’s far exceed what has been available on ATEC’s dedicated clusters, porting the ensemble offers the promise of extending ensemble forecasting to every supported ATC range, beyond just the four that currently use the system.

FastEddy

In FY 2020, RAL will expand FastEddy to include moist atmospheric processes.  More accurate representations of turbulence and fine-scale meteorological phenomena are expected with the addition of these processes.  In FY2020 and beyond, RAL will design and implement post-processed products and images from FastEddy for operational use by ATEC meteorologists.

Improved displays of 4DWX model output

RAL will continue to improve the WRF output products available from 4DWX.  Engineers will develop a new animation tool that utilizes layering to overlay multiple variables and allows the user to easily query model output for each grid point.

Data assimilation

In FY2020 and beyond, RAL will continue to develop and optimize a scheme for using RTFDDA to assimilate lightning data.

Radar-based icing algorithm (RadIA)

RAL will adapt the RadIA algorithm for use in 4DWX.  This algorithm uses moments from dual polarization radar data to represent when and where in-flight icing hazards are occurring. 

Prediction of turbulence for unmanned aerial systems (UAS)

RAL will continue adapt the Laboratory’s Graphical Turbulence Guidance (GTG) for use at the test ranges to predict turbulence at altitudes and over areas relevant to UAS, which are the focus of substantial DOD testing.  Data sets from UAS tests over Dugway Proving Ground are now being used to adapt GTG.

Prediction of cloud cover and cloud ceiling

Accurate cloud cover and cloud ceiling predictions are needed for aviation-related testing that occurs at ATEC ranges, including UAS missions and air drops of cargo and personnel.  RAL will integrate relative humidity and microphysics predictions from WRF to develop cloud prediction products for 4DWX.

Wildfires

RAL continues to explore the potential of adding to 4DWX the physics module WRF-Fire for use at selected test ranges, such as White Sands Missile Range.  WRF-Fire simulates two-way coupling between wildfires and their environment, so the model can predict a fire’s spread, severity, smoke, and other characteristics. 

Observing and modeling the Chesapeake Bay breeze

A tool will be added to the 4DWX Portal that displays observations and model predictions of the Chesapeake Bay breeze.  ATC’s weather is often influenced by the breeze, so testing there will likely be improved by having explicit predictions of the breeze’s onset, duration, and extension inland.

Range climatographies

RAL will resume our previously paused effort to develop a full-grid climatography over each test range, based on 4DWX final analyses spanning a length of time to be determined. To accompany the climatographies, RAL is developing a method to extract and display data for particular locations, seasons, and times of day, and to display the data as 2- dimensional (2-D) isosurfaces.

Incorporating the second-generation Advanced Weather Interactive Processing System (AWIPS-II) into 4DWX

RAL will continue to work with ATEC meteorologists to determine how best to incorporate AWIPS-II into their workflow, and to define the modifications that must be made to 4DWX so it can be used most effectively in combination with AWIPS-II.