Four-Dimensional Weather System (4DWX)

BACKGROUND

Since the middle 1990s, the U.S. Army Test and Evaluation Command (ATEC), then known as TECOM, has sponsored RAL to conduct research, development, and technology-transfer of the Four-Dimensional Weather (4DWX) system.  4DWX is an advanced numerical weather prediction (NWP) system that analyzes current weather and makes detailed predictions of weather over the next several days across many scales of phenomena.  4DWX’s NWP core is the Weather Research and Forecasting (WRF) Model.  4DWX ingests observations into the NWP core through NCAR’s Real-Time Four-Dimensional Data Assimilation (RTFDDA) scheme.  RAL upgrades 4DWX software several times per year.

4DWX is used by ATEC meteorologists at seven test ranges across five major climate zones: White Sands Missile Range (WSMR), New Mexico; Electronic Proving Ground (EPG), Arizona; Dugway Proving Ground (DPG), Utah; Aberdeen Test Center (ATC), Maryland; Redstone Test Center (RTC), Alabama; Yuma Proving Ground (YPG), Arizona; and Cold Regions Test Center (CRTC), Alaska.  4DWX is also in other locations when ATEC staff are required to support temporary exercises at locations such as San Nicholas Island, CA; Spaceport America, NM; Isle of Benbecula, Scotland; Woomera, Australia; Pacific Missile Range Facility, HI; and Kwajalein Atoll, Marshall Islands.

Thanks to 4DWX, ATEC forecasters have greater access than ever to technology and expertise that help them produce weather forecasts and analyses at the scales, and with the accuracy and utility, required to support safe and cost-effective testing by the Department of Defense (DOD).  For NCAR and its collaborators in the university community, one of the most attractive elements of the 4DWX project is that the ATEC test ranges serve as natural laboratories for atmospheric research, complete with dense observing networks and specialized data that permit study of mesoscale and microscale phenomena in complex terrain.  Continual improvements to 4DWX and to community numerical weather prediction models, such as the WRF Model, are made possible through this collaboration with DOD.

PRIMARY 4DWX TECHNOLOGY

Weather Research and Forecasting (WRF) Model

The WRF Model is a long-established industry standard for NWP in operations and research.   The model code is open source.  It was developed by a group of partners including NCAR, the National Oceanic and Atmospheric Administration, the Air Force Weather Agency, the Federal Aviation Administration, and the university community.  The model is used across many scales, from global to microscale.  The WRF Model is the predictive core of 4DWX.

Real-Time Four-Dimensional Data Assimilation (RTFDDA)

The project continues to rely on Real-Time Four-Dimensional Data Assimilation (RTFDDA) as one way to ingest observations and define the atmosphere’s current state for 4DWX’s NWP core, the WRF model.  RTFDDA involves modifications to an NWP model’s predictive equations so the model can be gently adjusted, or nudged, toward observed conditions during the model’s analysis stage, before the forecast stage begins.  The scheme is computationally efficient and preserves the precise timing of observations which gives 4DWX a particularly accurate depiction of the weather at any instant.  RTFDDA continues to show itself superior to, or the equal of, many alternative methods of data assimilation in operational systems.  RTFDDA assigns quality flags to observations within the analysis and forecast cycling, rather than as a pre-processing step, providing more accurate and stable assessments of each observation’s usefulness in data assimilation.  RTFDDA also has an improved means of dealing with cases when a ground-based observing site’s actual elevation differs significantly from the simulated terrain height in the model, which is a mundane but under-appreciated problem in applied NWP.

Ensemble 4DWX (E-4DWX)

Figure 1. Time series of E-4DWX’s mean 10-m wind speed (bold black line in m s-1) and ensemble members’ wind directions (wind roses) at an unspecified site.  This 48-h uncalibrated forecast (plus 6 h of data assimilation at the start of the period, to the left of the red semi-circle on the x axis) is on domain 3 (dx = 3.3 km) on an unspecified day in spring.  Each wind rose shows for a given time of day (UTC) the percentage of ensemble members (range rings at intervals of 20%) that predict ranges of wind speed (colors) from specific directions (sectors 22.5° wide).  Each rose’s position along the y axis marks the ensemble’s mean wind speed at that time.  From article by Knievel et al. (Weather and Forecasting, 2017).
Figure 1. Time series of E-4DWX’s mean 10-m wind speed (bold black line in m s-1) and ensemble members’ wind directions (wind roses) at an unspecified site.  This 48-h uncalibrated forecast (plus 6 h of data assimilation at the start of the period, to the left of the red semi-circle on the x axis) is on domain 3 (dx = 3.3 km) on an unspecified day in spring.  Each wind rose shows for a given time of day (UTC) the percentage of ensemble members (range rings at intervals of 20%) that predict ranges of wind speed (colors) from specific directions (sectors 22.5° wide).  Each rose’s position along the y axis marks the ensemble’s mean wind speed at that time.  From article by Knievel et al. (Weather and Forecasting, 2017).

Since 2007, DPG has used an ensemble version of 4DWX (called E-4DWX) developed by RAL.  E-4DWX provides a suite of 30 forecasts valid at the same place and time, each producing slightly different but similarly realistic forecasts.  Differences among ensemble members are induced by varying initial conditions, boundary conditions, and model physics.  All members are based on the WRF model.  The ensemble captures the forecasts’ probability information that varies with changes in weather regime.  In 2014, E-4DWX was expanded to include three additional ranges in the intermountain West: WSMR, YPG, and EPG.  E-4DWX products include maps and time series of means, standard deviations, or fractions of the ensemble exceeding thresholds (Figure 1).

A subset of output from E-4DWX is calibrated so that the probability of E-4DWX’s forecasts being realized matches the observed probability.  Benefits of calibration include: 1) reducing forecast error of the ensemble mean, partly by reducing bias; 2) increasing reliability, resolution, and sharpness, including for predicting extreme and potentially devastating weather; and 3) providing a measure of forecast uncertainty through the spread among ensemble members.  Calibration is performed on moments of the overall probability density function, no matter the size of the ensemble membership, as opposed to calibrating output from specific ensemble members.  This makes E-4DWX particularly robust, even if individual members of the ensemble fail at some point during the forecast.  E-4DWX’s calibration algorithms combine logistic regression with quantile regression.  To ensure the ensemble’s reliability, it is pre-processed and then the calibration is explicitly conditioned on the ensemble dispersion.  Regressions are always performed with cross-validation to minimize the likelihood of overfitting.

Forecasts of severe weather

The 4DWX system has components that predict severe weather on two scales: the next few hours, based on both observations and model predictions blended via the AutoNowcaster, and the next few days, based on model predictions alone.  The AutoNowcaster employs the dual polarization data available from the nation’s NEXRAD network as well as Terminal Doppler Weather Radar (TDWR).  An algorithm called Trident helps to alert forecasters to conditions that could lead to flash flooding.  Trident predictions are at 10-min intervals to a lead time of 1-hour.  The algorithm currently uses a standard Z-R relationship to relate radar reflectivity to precipitation rate. 

Coupled applications

Direct weather analyses and predictions from 4DWX and E-4DWX are the core of the weather information used by forecasters at the ATEC ranges, but that information can be made even more valuable when it is supplied to decision support systems (DSSs) that simulate how the weather affects other processes and conditions, such as sound propagation and the transport and dispersion of airborne hazards.  Examples of DSSs that have been linked to 4DWX and/or E-4DWX include:

•           Noise Assessment and Prediction System (NAPS)

•           Second-order Closure Integrated Puff (SCIPUFF) model

•           Lewis Rocket Trajectory Model

•           Open Burn / Open Detonation Model (OBODM)

4DWX Web Portal

The primary interface to the 4DWX system at all ATEC ranges is the 4DWX Portal.  The Portal’s flexibility, accessibility, modularity, and extensibility are ideally suited to the customized weather support that RAL provides to forecasters who have eagerly welcomed how the Portal improves their workflow.  Weather maps and related graphics from 4DWX include optional color palettes that can be accurately seen by the color-blind.  The Portal’s dashboard has a flexible, configurable layout, with streamlined access to portlets for coupled applications.  The list of output formats that the Portal supports includes the third-party BUFKIT and RAOB applications.

Integrated Data Viewer (IDV)

RAL collaborates with UCAR’s Unidata program to include among 4DWX’s display options the Integrated Data Viewer (IDV), which is a sophisticated, flexible, Java-based application for analyzing and displaying geophysical data.  IDV is the primary means by which range forecasters explore in greater depth the weather analyses and forecasts from 4DWX.  This more complex and flexible exploration complements the “virtual map wall” that is available through the 4DWX Web Portal, whose purpose is to provide the easiest and quickest interface to a standard suite of pre-generated weather maps.  IDV is also a research tool and is employed by scientists and engineers during their testing, development, and refinement of 4DWX.

Outreach and training

Each year, RAL provides to ATEC meteorologists at each range several days of on-site training on 4DWX technology.  Not only does 4DWX improve every year, but the test support required of ATEC meteorologists also changes frequently.  Moreover, turnover among ATEC forecasters also points to the need for a regular training cycle, independently of how rapidly 4DWX technology changes.  Close interaction between ATEC and RAL is critical for maintaining the project’s success.

SELECTED ACCOMPLISHMENTS IN FY17

4DWX ported to clusters at two DOD Supercomputing Resource Centers (DSRCs)

Since the project’s inception, ATEC has relied on dedicated computer clusters to run 4DWX for operational support at the ranges.  Now shared clusters at DOD Supercomputing Resource Centers (DSRCs) offer the promise of more cost-effective and powerful platforms for running 4DWX.  After redesigning key elements of 4DWX to be more platform-independent, modifying data feeds, and improving the scope and sophistication of 4DWX’s system monitoring, RAL and ATEC ported 4DWX to two Cray supercomputers, one at the Army Research Laboratory, MD, and the other at Stennis Space Center, MS.  Thanks to invaluable assistance and cooperation from the Army and Navy, prototype configurations of 4DWX are now running in real-time and are being evaluated by each of the seven test ranges.

Predictions of lightning

At all ranges, 4DWX now includes a tool for tactical prediction of lightning (lead times of minutes to tens of minutes) and a tool for strategic prediction of lightning (lead times of hours to days).  The former is based on WSR-88D radar data that are used to monitor reflectivity above the melting level.  The latter is based on numerical output from 4DWX’s predictive core.  Algorithms are calibrated at each range independently, based on summer and winter cases from previous years.

Predictions of flood-inducing rainfall

In FY17, Trident was upgraded at all the ranges in the conterminous western US.  Trident uses two methods of calculating warning criteria, one based on maximum rainfall in a drainage basin, another based on the percentage of a basin covered by rainfall of various thresholds.  Those algorithms now include radar-based estimates of rainfall rate calculated from dual-polarization moments.

Analog ensemble

Figure 2.  Example of the 4DWX Portal’s display of 4DWX AnEn predictions of near-surface conditions at WSMR’s station 2.  The top panel shows the mean prediction of 2-m air temperature (C in dark blue) as a function of valid time (hour, month, and day) within an envelope of  1 standard deviation (cyan) about the mean.  The bottom panel shows the same but for 10-m wind speed (m s-1).  Observations at station 2 are in gray on both panels.
Figure 2.  Example of the 4DWX Portal’s display of 4DWX AnEn predictions of near-surface conditions at WSMR’s station 2.  The top panel shows the mean prediction of 2-m air temperature (°C in dark blue) as a function of valid time (hour, month, and day) within an envelope of ± 1 standard deviation (cyan) about the mean.  The bottom panel shows the same but for 10-m wind speed (m s-1).  Observations at station 2 are in gray on both panels.

4DWX’s Analog Ensemble (AnEn) uses a set of algorithms to calculate probabilistic predictions that rely on archives of observations and model output to collect an ensemble of prior forecasts made under analogous weather patterns.  Predictions from analog-based methods are inherently calibrated, so an extra calibration step is not required.  The initial operational version of 4DWX AnEn generated forecasts to just 24 hours and was deployed at only three test ranges: ATC, RTC, and CRTC.  In FY17, 4DWX AnEn’s code and databases were upgraded to generate 48-hour forecasts, and analog forecasts are now being used by all seven test ranges.

Improvements to the 4DWX Portal

 

The 4DWX Portal was upgraded to display output from 4DWX run on the two DSRC supercomputers described above.  Many other smaller upgrades were made as well, including the ability to display 4DWX AnEn predictions (Figure 2).

SELECTED KEY PLANS FOR FY2018

Complete migration away from dedicated hardware

RAL will continue to optimize 4DWX’s configurations to take advantage of the increased computing power at the DSRCs described above.  Much of the dedicated computing hardware will gradually be phased out of the operational chain starting in FY18.

Improvements to 4DWX’s NWP core and data assimilation

RAL is improving the WRF model’s current code for modifying wind speed through drag that is a function of sub-grid topography.  The scheme with the recent addition of stability-dependent sub-grid friction performs very well in weak wind over flat terrain, but it tends to exacerbate underprediction of strong wind.  RAL is working on a solution to this problem.

Data assimilation

RAL has developed a scheme for assimilating total lightning data into 4DWX.  Assimilating lightning provides several benefits, including a) providing information about storms where radar coverage is poor, b) improving inference of water-vapor mixing ratio from radar reflectivity where lightning and radar data co-exist, and c) improving short-term forecasts because assimilating lightning data improves prediction of model variables directly associated with lightning diagnosis.  RAL plans to adopt lightning DA into 4DWX in FY18.

Analog Ensemble (AnEn)

AnEn will be tested as a method for predicting three specific phenomena that are problematic for the dynamical 4DWX and critical for testing at some ranges: chinooks, cold-air damming, and drainage flow.

Prototype real-time very large-eddy-simulation (VLES) version of 4DWX

Much of ATEC’s testing is sensitive to microscale weather, so RAL continues to work on extending 4DWX into the range of very-large-eddy simulation (VLES) resolutions (grid intervals of 100s of meters).  In 2018, RAL will begin running in real-time a prototype VLES configuration for ATC.  The four domains will have grid intervals of 9 km, 3 km, 1 km, and 200 m.  Radar data will be assimilated on domains 1-3.  Output products will be available every 15-minutes.

Lightning observations

RAL is working with the Alaska Interagency Coordination Center to obtain real-time observations of lightning from a Bureau of Land Management network for use at CRTC.

Stream-flow prediction

RAL will begin developing a prototype implementation of WRF-Hydro in 4DWX to forecast flash floods in the arroyos of YPG.

Observing and modeling the Chesapeake Bay breeze

A tool will be added to the 4DWX Portal that displays observations and model predictions of the Chesapeake Bay breeze.  ATC’s weather is often influenced by the breeze, so testing there will likely be improved by having explicit predictions of the breeze’s onset, duration, and extension inland.

Predicting wet-bulb globe temperature

Wet-bulb globe temperature (WBGT) is a key index of heat stress commonly used in the military to assess those instances when heat, humidity, lack of wind, and the sun’s intensity combine to make working outside unsafe.  It is an empirical quantity, but RAL is developing algorithms to diagnose an approximate WBGT from 4DWX forecasts.  A combination of several formulae will be used to produce an ensemble of diagnoses that the test ranges can apply to improve the safety of outdoor test exercises.