Provide a capacious and reliable data archive

HPSS Growth
This chart shows the growth since early 2011 of the HPSS archive located at the NCAR-Wyoming Supercomputing Center. The vertical blue line marks the date that the 1,504-teraflops computer Yellowstone entered production alongside the 77-teraflops computer Bluefire.

The High Performance Storage System (HPSS) is an advanced, highly scalable and flexible mass storage and archival resource that supports both NCAR’s supercomputing environment as well as divisional servers run by other NCAR laboratories and UCAR programs. CISL also uses the HPSS to support disaster recovery for irreplaceable data by maintaining HPSS infrastructure at the NCAR Wyoming Supercomputing Center (NWSC) in Cheyenne, Wyoming and at the NCAR Mesa Lab in Boulder, Colorado. The tape libraries in Boulder are used to store critical data specified in the CISL Business Continuity plan. This disaster recovery service currently supports the RDA, NCAR’s EOL, and UCAR’s COSMIC program.

Tailored to the needs of the Earth System science community, this capacious and reliable data archive is a strategic component in the cyberinfrastructure that CISL provides and maintains to expand the productivity of the research community. CISL’s ongoing leadership in providing discipline-focused computing and data services is a critical role for NCAR as a national center.

The HPSS system is maintaining a steady growth rate: Data holdings in FY2016 grew by nearly 15 PB of new data and an additional 25 million files. Holdings as of October 2016 stand at around 66 PB and 241 million files, with growth since Yellowstone began production averaging around 1.25 PB per month. After recent acquisition of two additional tape libraries at NWSC, the HPSS system has doubled its maximum data capacity from 160 to 320 PB. Further augmentation of the archival system has included purchases of a newer, faster metadata server and additional data movers to support the increase in load from the new Cheyenne system and the overlap period with Yellowstone. In addition to these improvements a major HPSS upgrade is underway, scheduled to be completed in the second quarter of FY2017.

To meet the increased data load from the new Cheyenne system, analysis and projection exercises were conducted to size an augmentation of the current HPSS equipment. Based on this analysis and consideration of projected technology advances, CISL is planning to extend the current archival system subcontract, procuring next-generation tapes and drives that will significantly increase total HPSS capacity and throughput via both augmentation of existing resources as well as re-use of existing tapes at higher capacities and bandwidth, thus preserving existing capital investment and substantially reducing overall cost.

The NCAR HPSS is managed by CISL under the UCAR/NSF Cooperative Agreement and is supported by NSF Core funds and CSL funding.