Provide the HPFL test and exploration laboratory

High Performance Futures Laboratory
Two racks of hardware in the High Performance Futures Laboratory. The HPFL provides hardware and software infrastructure for testing and evaluating emerging technologies.

Since 2014, CISL has operated its High Performance Futures Laboratory (HPFL) which is comprised of small, experimental systems located both at the Mesa Lab facility and at the NWSC. The HPFL is a flexible, multi-system testbed where NCAR staff and collaborators can evaluate new hardware and software that represent experimental or emerging technologies and/or that may be part of future production systems. Initially focused on computational technologies, the HPFL was expanded during FY2016 to include storage, I/O, and data archive test resources.

The High Performance Futures Lab is intended to provide strategic opportunities for NCAR staff and collaborators to gain valuable experience with emerging hardware and software technologies, and CISL also uses it to study and mitigate risks arising from new technologies. New HPC and storage technologies such as high-performance stacked and non-volatile memory, and hybrid solid-state and non-volatile disk technologies are rapidly transforming the HPC and storage landscape. Such technologies are poised to transform system architectures by introducing additional layers of memory, a hierarchy of I/O devices, as well as tighter integration of those technologies with computing elements. CISL is using the HPFL and vendor partnerships, such as the SGI-NCAR Joint Center of Excellence and the Intel Parallel Computing Center, to learn how to apply these technologies effectively to atmospheric and geoscience applications and to meet the data requirements of future systems and applications.

The HPFL provides a ready-to-use environment where cutting-edge technology can be deployed and tested by system administrators, consulting staff, and computational scientists. Current research is examining areas such as heterogeneous architectures, GPGPUs, coprocessors, resource managers, job schedulers, Message Passing Interface (MPI) software, benchmarks, performance tuning, file systems, and a variety of computation- and I/O-intensive applications.

HPFL status

CISL continued operating and enhancing the HPFL during FY2016. By year end, the HPFL contained test platforms that included Intel’s new Xeon E5 v4 (Broadwell) and Phi (Knights Landing) processors, Mellanox EDR InfiniBand and Intel OmniPath high-speed interconnect technologies, 40-gigabit Ethernet, and SGI’s UV300 large-shared-memory system. Additionally, CISL added an I/O innovation component to the Lab that is focused on storage futures, I/O optimization, and better data-hosting services. This I/O component included SSD technology, Data Direct Networks (DDN) storage systems, and a test environment for new HPSS releases with HPSS on Linux and rolling upgrades to the NCAR HPSS archive.

HPFL software evaluation activities in FY2016 included testing of operating systems and their management (Salt, CentOS), resource managers (SLURM, PBS Professional), MPI implementations (Intel MPI, OpenMPI, MPICH-3, and MVAPICH), containers (Docker), the Spectrum Scale (formerly GPFS) and Lustre parallel file systems, DDN’s Infinite Memory Engine (IME), and an HPSS test environment. The HPFL also served as an initial NWSC-2 software test environment prior to the delivery of the NWSC-2 test system named Laramie.

HPFL and NWSC-2a/NWSC-2b procurements

New non-volatile memory and many-core technologies, some examples of which were tested in the HPFL during FY2016, are key components of the NWSC-2a and NWSC-2b procurements planned for the upcoming year. While NWSC-2a will be focused primarily on acquiring a production-quality data analysis and visualization platform to replace the existing Geyser and Caldera systems, it will also include GPUs, large shared-memory nodes for in-memory computation, and non-volatile memory, SSD, and/or burst-buffer hardware and software for accelerating the I/O operations critical to analyzing large datasets. The NWSC-2b procurement will look more toward an experimental computational-accelerator-based system utilizing many-core technologies. NWSC-2b will help NCAR evaluate the efficacy of alternate processor technologies for use by geoscience codes and to advance our understanding of how such technologies perform and can be managed in a semi-production computational environment.

Funding

CISL's HPFL is made possible by NSF Core funds and through partnerships with and equipment donations from leading HPC and storage vendors.