Four-Dimensional Weather System (4DWX)

BACKGROUND

Since the middle 1990s, the U.S. Department of Defense’s (DOD’s) Army Test and Evaluation Command (ATEC), then known as TECOM, has sponsored RAL to conduct research, development, and technology-transfer of the Four-Dimensional Weather (4DWX) system.  4DWX is an advanced numerical weather prediction (NWP) system that analyzes current weather and makes detailed predictions of weather over the next several days across many scales of phenomena.  4DWX’s NWP core is the Weather Research and Forecasting (WRF) Model.  4DWX ingests observations into the NWP core through RAL’s Real-Time Four-Dimensional Data Assimilation (RTFDDA) scheme.  RAL upgrades 4DWX software several times per year.

4DWX is used by ATEC meteorologists and other DOD staff at eight test ranges across five major climate zones: White Sands Missile Range, New Mexico; Electronic Proving Ground, Arizona; Dugway Proving Ground, Utah; Aberdeen Test Center, Maryland; Redstone Test Center, Alabama; Airborne and Special Operations Test Directorate (ASOTD), Fort Bragg, North Carolina; Yuma Proving Ground, Arizona; and Cold Regions Test Center, Alaska.  4DWX is also used at other locations when ATEC meteorologists are required to support temporary exercises in places such as San Nicholas Island, California; Spaceport America, New Mexico; Isle of Benbecula, Scotland; Woomera, Australia; Pacific Missile Range Facility, Hawai’i; and Kwajalein Atoll, Marshall Islands.

Thanks to 4DWX, ATEC meteorologists have greater access than ever to technology and expertise that help them produce weather forecasts and analyses at the scales, and with the accuracy and utility, required to support safe and cost-effective testing by the DOD.  For RAL and its collaborators in the university community, one of the most attractive elements of the 4DWX project is that the ATEC test ranges serve as natural laboratories for atmospheric research, complete with dense observing networks and specialized data that permit study of mesoscale and microscale phenomena in complex terrain.  Continual improvements to 4DWX and to community numerical weather prediction models, such as the WRF Model, are made possible through this collaboration with DOD.

PRIMARY 4DWX TECHNOLOGY

Weather Research and Forecasting (WRF) Model

The predictive core of 4DWX is the WRF Model, a long-established industry standard for NWP in operations and research.   The model code is open source.  It was developed by a group of partners including NCAR, the National Oceanic and Atmospheric Administration, the Air Force Weather Agency, the Federal Aviation Administration, and the university community.  The model is used across many scales, from global to microscale.

Real-Time Four-Dimensional Data Assimilation (RTFDDA)

The project continues to rely on Real-Time Four-Dimensional Data Assimilation (RTFDDA) as one way to ingest observations and define the atmosphere’s current state for 4DWX’s NWP core, the WRF Model.  RTFDDA involves modifications to an NWP model’s predictive equations so the model can be gently adjusted, or nudged, toward observed conditions during the model’s analysis stage, before the forecast stage begins.  The scheme is computationally efficient and preserves the precise timing of observations, which gives 4DWX a particularly accurate depiction of the weather at any instant.  RTFDDA continues to show itself superior to, or the equal of, many alternative methods of data assimilation in operational systems.  RTFDDA assigns quality flags to observations during the analysis and forecast cycling, rather than as a pre-processing step, providing more accurate and stable assessments of each observation’s usefulness in data assimilation.  RTFDDA also has an improved means of dealing with cases when a ground-based observing site’s actual elevation differs significantly from the simulated terrain height in the model, which is a mundane but under-appreciated problem in applied NWP.

Ensemble 4DWX (E-4DWX)

Figure 1.  Probability density functions (PDFs) of an uncalibrated (solid black line) and a calibrated (solid blue line) 24-h ensemble forecast of 2-m air temperature (°C) from E-4DWX for station 01 at Dugway Proving Ground.  Calibration reduces bias, broadens spread, and increases sharpness in the ensemble forecast when compared with observations, such as 25.9°C (thick red bar) in this case, and when compared to the baseline prediction one could get from the climatological PDF for summer (dashed black line).  From article by Knievel et al. (Weather and Forecasting, 2017).
Figure 1.  Probability density functions (PDFs) of an uncalibrated (solid black line) and a calibrated (solid blue line) 24-h ensemble forecast of 2-m air temperature (°C) from E-4DWX for station 01 at Dugway Proving Ground.  Calibration reduces bias, broadens spread, and increases sharpness in the ensemble forecast when compared with observations, such as 25.9°C (thick red bar) in this case, and when compared to the baseline prediction one could get from the climatological PDF for summer (dashed black line).  From article by Knievel et al. (Weather and Forecasting, 2017).

Since 2007, Dugway Proving Ground has used an ensemble version of 4DWX (called E-4DWX) developed by RAL.  E-4DWX provides a suite of 30 forecasts valid at the same place and time, each producing slightly different but similarly realistic forecasts.  Differences among ensemble members are induced by varying initial conditions, boundary conditions, and model physics.  All members are based on the WRF Model.  The ensemble captures the forecasts’ probability information that varies with changes in weather regime.  In 2014, E-4DWX was expanded to include three additional ranges in the intermountain West: White Sands Missile Range, Yuma Proving Ground, and Electronic Proving Ground.  E-4DWX products include maps and time series of means, standard deviations, or fractions of the ensemble exceeding thresholds.

A subset of output from E-4DWX is calibrated so that the probability of E-4DWX’s forecasts being realized matches the observed probability (Figure 1).  Benefits of calibration include: 1) reducing forecast error of the ensemble mean, partly by reducing bias; 2) increasing reliability, resolution, and sharpness, including for predicting extreme and potentially devastating weather; and 3) providing a measure of forecast uncertainty through the spread among ensemble members.  Calibration is performed on moments of the overall probability density function, no matter the size of the ensemble membership, as opposed to calibrating output from specific ensemble members.  This makes E-4DWX particularly robust, even if individual members of the ensemble fail at some point during the forecast.  E-4DWX’s calibration algorithms combine logistic regression with quantile regression.  To ensure the ensemble’s reliability, it is pre-processed, then the calibration is explicitly conditioned on the ensemble dispersion.  Regressions are always performed with cross-validation to minimize the likelihood of overfitting.  

Forecasts of severe weather

The 4DWX system has components that predict severe weather on two scales: the next few hours, based on both observations and model predictions blended via the AutoNowcaster; and the next few days, based on model predictions alone.  The AutoNowcaster employs the dual polarization data available from the nation’s NEXRAD network as well as Terminal Doppler Weather Radar (TDWR).  An algorithm called Trident helps to alert forecasters to conditions that could lead to flash flooding.  Trident predictions are at 10-min intervals to a lead time of 1-hour.  The algorithm currently uses a standard Z-R relationship to relate radar reflectivity to precipitation rate. 

Predictions of lightning

At all ranges, 4DWX now includes a tool for tactical prediction of lightning (lead times of minutes to tens of minutes) and a tool for strategic prediction of lightning (lead times of hours to days).  The former is based on WSR-88D radar data, are used to monitor reflectivity above the melting level.  The latter is based on numerical output from 4DWX’s predictive core.  Algorithms are calibrated at each range independently, based on summer and winter cases from previous years.

Predictions of flood-inducing rainfall

To warn test ranges about the potential for flood-inducing rainfalls, 4DWX relies on Trident.  Trident uses two methods of calculating warning criteria, one based on maximum rainfall in a drainage basin, another based on the percentage of a basin covered by rainfall of various thresholds.  Those algorithms now include radar-based estimates of rainfall rate calculated from dual-polarization moments.

Analog ensemble 

Figure 2.  Example of the 4DWX Portal’s display of 4DWX AnEn predictions of near-surface conditions at station 2 of White Sands Missile Range.  The top panel shows the mean prediction of 2-m air temperature (°C in dark blue) as a function of valid time (hour, month, and day) within an envelope of ± 1 standard deviation (cyan) about the mean.  The bottom panel shows the same but for 10-m wind speed (m s-1).  Observations at station 2 are in gray on both panels.
Figure 2.  Example of the 4DWX Portal’s display of 4DWX AnEn predictions of near-surface conditions at station 2 of White Sands Missile Range.  The top panel shows the mean prediction of 2-m air temperature (°C in dark blue) as a function of valid time (hour, month, and day) within an envelope of ± 1 standard deviation (cyan) about the mean.  The bottom panel shows the same but for 10-m wind speed (m s-1).  Observations at station 2 are in gray on both panels.

4DWX’s Analog Ensemble (AnEn) uses a set of algorithms to calculate probabilistic predictions that rely on archives of observations and model output to collect an ensemble of prior forecasts made under analogous weather patterns (Figure 2).  Predictions from analog-based methods are inherently calibrated, so an extra calibration step is not required.

Coupled applications

Direct weather analyses and predictions from 4DWX and E-4DWX are the core of the weather information used by forecasters at the ATEC ranges, but that information can be made even more valuable when it is supplied to decision support systems (DSSs) that simulate how the weather affects other processes and conditions, such as sound propagation and the transport and dispersion of airborne hazards.  Examples of DSSs that have been linked to 4DWX and/or E-4DWX include:

  • Noise Assessment and Prediction System (NAPS)

  • Second-order Closure Integrated Puff (SCIPUFF) model

  • Lewis Rocket Trajectory Model

  • Open Burn / Open Detonation Model (OBODM)

4DWX Web Portal

The primary interface to the 4DWX system at all ATEC ranges is the 4DWX Portal.  The Portal’s flexibility, accessibility, modularity, and extensibility are ideally suited to the customized weather support that RAL provides to forecasters, who have eagerly welcomed how the Portal improves their workflow.  Weather maps and related graphics from 4DWX include optional color palettes that can be accurately seen by the color-blind.  The Portal’s dashboard has a flexible, configurable layout, with streamlined access to portlets for coupled applications.  The list of output formats that the Portal supports includes the third-party BUFKIT and RAOB applications.

Integrated Data Viewer (IDV)

RAL collaborates with UCAR’s Unidata program to include among 4DWX’s display options the Integrated Data Viewer (IDV), which is a sophisticated, flexible, Java-based application for analyzing and displaying geophysical data.  IDV is the primary means by which range forecasters explore in greater depth the weather analyses and forecasts from 4DWX.  This more complex and flexible exploration complements the “virtual map wall” that is available through the 4DWX Web Portal, whose purpose is to provide the easiest and quickest interface to a standard suite of pre-generated weather maps.  IDV is also a research tool and is employed by scientists and engineers during their testing, development, and refinement of 4DWX.

Outreach and training

Each year, RAL provides to ATEC meteorologists at each test range several days of on-site training on 4DWX technology.  Not only does 4DWX improve every year, but the test support required of ATEC meteorologists also changes frequently.  Moreover, turnover among ATEC forecasters also points to the need for a regular training cycle, independently of how rapidly 4DWX technology changes.  Close interaction between ATEC and RAL is critical for maintaining the project’s success.

SELECTED ACCOMPLISHMENTS IN FY2018

4DWX running operationally at two DOD Supercomputing Resource Centers (DSRCs)

Until recently, ATEC relied solely on dedicated computer clusters to run 4DWX for operational support at the ranges.  Now shared clusters at DOD Supercomputing Resource Centers (DSRCs) permit more cost-effective and powerful platforms for running 4DWX.  After redesigning key elements of 4DWX to be more platform-independent, modifying data feeds, and improving the scope and sophistication of 4DWX’s system monitoring, in FY17 RAL and ATEC first ported 4DWX to two Cray supercomputers, one at the Army Research Laboratory, Maryland, and the other at Stennis Space Center, Mississippi.  Thanks to invaluable assistance and cooperation from the Army and Navy, in FY2018 ATEC is now running 4DWX operationally at the DSRCs.

More streamlined technology transfer

During FY2018, RAL made many improvements to 4DWX’s engineering and to how the code is installed on DOD computing systems.  The improvements have significantly streamlined the project’s technology transfer, improved 4DWX’s reliability and efficiency, and made it more robust in the event of disaster recovery.

Data assimilation

RAL has developed a scheme for assimilating total lightning data into 4DWX.  Assimilating lightning provides several benefits, including a) providing information about storms where radar coverage is poor, b) improving inference of water-vapor mixing ratio from radar reflectivity where lightning and radar data co-exist, and c) improving short-term forecasts because assimilating lightning data improves prediction of model variables directly associated with lightning diagnosis.  The lightning data assimilation scheme has been integrated into the most recent version of RTFDDA.  The next step is to use observations and case studies to optimize the scheme’s performance.

Improved lightning prediction

RAL has developed a new application that creates radar volumes every minute using the latest set of elevation angles available from radars local to each test range.  This improves the radar-based lightning nowcast application that is currently implemented in 4DWX.  Another improvement to lightning prediction was achieved through development of a new single truth field for lightning data, which combines three separate lightning data sources for the lightning indicator circles used in the 4DWX displays at the test ranges. 

Lightning observations

RAL has established with the Alaska Interagency Coordination Center a memorandum of understanding that enables us to obtain real-time observations of lightning from a Bureau of Land Management network for use at Cold Regions Test Center.

Predicting flash floods

In FY2018 we adapted for 4DWX some of the code from the RAL-developed WRF-Hydro Model used by the NOAA National Water Center as the primary component of the U.S.’s National Water Model.  For the 4DWX project, the end goal of the effort is to forecast flash floods in the arroyos of Yuma Proving Ground.  So far, the code has been tested only against individual cases.  More steps toward implementation are scheduled for next year.

Prototype real-time very large-eddy-simulation (VLES) version of 4DWX

Much of ATEC’s testing is sensitive to microscale weather, so RAL continues to work on extending 4DWX into the range of very-large-eddy simulation (VLES) resolutions (grid intervals of 100s of meters).  In FY2018, RAL began running in real-time a prototype VLES configuration for ATC.  The four domains have intervals of 9 km, 3 km, 1 km, and 200 m.  Radar data are being assimilated on domains 1-3 every 10 minutes.  Results from the real-time simulations will inform the full deployment of 4DWX-VLES when ATEC’s computational resources permit.

Predicting wet-bulb globe temperature

Wet-bulb globe temperature (WBGT) is a key index of heat stress commonly used in the military and in the sports community to assess instances when heat, humidity, lack of wind, and intense sunshine combine to make working outside unsafe.  WBGT is an empirical quantity, but RAL has developed test algorithms to diagnose an approximate WBGT from 4DWX forecasts.  RAL is evaluating several formulae for the algorithm, which in the future will be applied in the form of a predictive ensemble for improving the safety of DOD’s outdoor test exercises.

SELECTED KEY PLANS FOR FY2019

Improvements to 4DWX’s NWP core and data assimilation

RAL is improving the WRF Model’s code for modifying wind speed through drag that is a function of sub-grid topography.  The scheme, with the recent addition of stability-dependent sub-grid friction, performs very well in weak wind over flat terrain, but it tends to exacerbate underprediction of strong wind.  RAL is working on a solution to this problem.

Data assimilation

In FY2019 and beyond, RAL will continue to develop and optimize a scheme for using RTFDDA to assimilate lightning data.

Prediction of turbulence for unmanned aerial systems (UAS)

RAL will adapt the lab’s Graphical Turbulence Guidance (GTG) for use at the test ranges to predict turbulence at altitudes and over areas relevant to UAS, which are the focus of substantial DOD testing.

Wildfires

RAL will explore the potential of adding to 4DWX the physics module WRF-Fire for use at selected test ranges, such as White Sands Missile Range.  WRF-Fire simulates two-way coupling between wildfires and their environment, so the model can predict a fire’s spread, severity, smoke, and other characteristics. 

Analog Ensemble (AnEn)

AnEn will be tested as a method for predicting cold-air damming east of the Appalachians, which has a key forecast challenge at Aberdeen Test Center.

FastEddy

FastEddy is a new hybrid CPU/GPU-accelerated, large-eddy simulation (LES) model developed by RAL.  The model comprises resident-GPU code, so all prognostic calculations are carried out in an accelerated manner on GPUs, with CPUs used only for model configuration and input/output.  In FY2019, RAL will begin testing FastEddy for predicting the statistical properties of turbulence at specially chosen test sites at the ATEC ranges.

Predicting flash floods

RAL will implement a prototype configuration of WRF-Hydro in 4DWX to forecast flash floods in the arroyos of Yuma Proving Ground.

Observing and modeling the Chesapeake Bay breeze

A tool will be added to the 4DWX Portal that displays observations and model predictions of the Chesapeake Bay breeze.  ATC’s weather is often influenced by the breeze, so testing there will likely be improved by having explicit predictions of the breeze’s onset, duration, and extension inland.

Predicting wet-bulb globe (WBGT) temperature

RAL will deploy a prototype algorithm to diagnose an approximate WBGT from 4DWX forecasts with the goal of improving the safety of DOD’s outdoor test exercises.

Range climatographies

RAL will resume our previously paused effort to develop a full-grid climatography over each test range, based on 4DWX final analyses spanning a length of time to be determined. To accompany the climatographies, RAL is developing a method to extract and display data for particular locations, seasons, and times of day, and to display the data as 2- dimensional (2-D) isosurfaces.

Port E-4DWX to DSRCs

Following the success of 4DWX’s transition to the supercomputers at DSRC’s, RAL will begin to port E-4DWX to the same systems.  Because the available nodes at DSRC’s far exceed what has been available on ATEC’s dedicated clusters, porting the ensemble offers the promise of extending ensemble forecasting to every supported ATEC range, beyond just the four that currently use the system.

Incorporating the second-generation Advanced Weather Interactive Processing System (AWIPS-II) into 4DWX

In FY2019, RAL will work with ATEC meteorologists to determine how best to incorporate AWIPS-II into their workflow, and to define the modifications that must be made to 4DWX so it can be used most effectively in combination with AWIPS-II.