Globally Accessible Data Environment

The GLobally Accessible Data Environment (GLADE) provides centralized high-performance file systems spanning supercomputing, data post-processing, data analysis, visualization, and HPC-based data transfer services. GLADE provides computation, analysis, and visualization work spaces common to all CISL HPC resources. Project space is allocated through NCAR’s allocation panels, while scratch and user space is available to all users of NCAR HPC resources. GLADE also hosts data from NCAR’s Research Data Archive (RDA), NCAR’s Community Data Portal, and the Earth System Grid which curates CMIP5/AR5 data.

GLADE usage
GLADE usage for FY2015 showing 236 PB of data written and 142 PB of data read.

GLADE’s architecture shifts user workflows from a design that centers on serving the supercomputer to a more scientifically efficient design that facilitates the flow of data. Through a globally accessible storage infrastructure, users now arrange their workflows to use stored data directly without first needing to move or copy it. Additional services like high-performance data transfer protocols enhance CISL’s ability to bring computational data from other sites to NCAR for post-processing, analysis, and visualization.

In FY2015 GLADE resources continued to provide stable centralized file services for all CISL HPC resources. GLADE resources were fully allocated in FY2015, and bandwidth usage has begun to show regular spikes at maximum throughput. The new Globus data sharing service was moved from a proof-of-concept environment to the production GLADE environment enabling collaborative data sharing for all projects.

This work supports CISL’s computing imperative for hardware cyberinfrastructure by provisioning storage and networking systems customized to support efficient workflows for the atmospheric and related sciences. Specifically, GLADE facilitates typical user workflows plus special efforts like supporting data flows for the IPCC AR5. GLADE also advanced CISL’s computing imperative for facilities by demonstrating high-performance data services that were critical for the next-generation resources that now operate at NWSC and will continue to be critical as we move toward next-generation resources in FY2016.

GLADE equipment was purchased with NSF Special funds, and it is supported by NSF Core funds including CSL funding.