Foster research and technical collaborations

CISL’s research efforts are enhanced by a robust set of ongoing partnerships, workshops, and training activities. These partnerships take the form of joint appointments with other NCAR laboratories and long-term research and development projects that include resource investments by vendor, industry, university, and research laboratory partners. Recurring workshops, symposia, hackathons and other training events foster community understanding and use of emerging tools, technologies and techniques developed through CISL research activities.

Joint appointments

NCAR joint appointments are positions that are funded by two or more NCAR laboratories on a semi-permanent basis under a formal agreement that is reviewed and approved by the NCAR Directorate. CISL has a long-standing project scientist joint appointment with the Mesoscale and Microscale Meteorology Laboratory. The position combines computer models and observations to understand the behavior and predictability of severe thunderstorms and other high-impact weather phenomena.

In FY2018 CISL initiated several new joint appointments in response to CISL Advisory Panel recommendations. These include two project scientist positions, shared with the Research Applications Laboratory (RAL), devoted to machine learning. The positions will not only serve to strengthen ties between RAL and CISL, but also help create an NCAR-wide network of machine learning interest and expertise that spans applied research and stakeholder-driven applications. Similarly, two new joint appointments between CISL and the Climate and Global Dynamics (CGD) Laboratory are being planned. The first will enhance collaboration between the labs in the area of Earth system prediction by sharing project scientist expertise in this area. The second will do the same for data workflow expertise between CISL and CGD by formalizing a long-standing, co-funded position in that area.

HPC R&D partnerships

CISL maintains a wide spectrum of strategic research and development partnerships in high-performance computing. These activities enable CISL to gain early access to and evaluate emerging technologies and to do advance software development and optimization so critical software components – like NCAR’s community models – will be able exploit them. This work is done on an open-source basis that furthers progress on the scientific goals of the NCAR and university scientific communities.

Following are examples of these activities.

CISL and IPCC project

CISL participated in Intel Corporation’s Parallel Computing Center (IPCC) for Weather and Climate Simulations project to develop tools and techniques for getting more performance from Intel Xeon and Xeon Phi processors. This enabled CISL to develop the Kernel Generator (KGEN), an automated unit test extraction tool used for optimization and verification by a steadily growing community of scientists and engineers. The IPCC project wrapped up its final year of funding in mid-FY2018.

Weather and Climate Alliance

The Weather and Climate Alliance (WACA) is a CISL partnership with NVIDIA Corporation, the University of Wyoming, and the Korean Institute for Science and Technology Information. The WACA partnership, which focused initially on GPU application acceleration, pivoted in mid-FY2018 to a new focus on machine learning. To this end, NVIDIA supplied $75,000 to fund CISL activities in the new Analytics and Integrative Machine Learning (AIML) group. This funding from the private sector has fostered rapid progress on three machine learning auto-encoding projects discussed elsewhere in this report.

Joint development agreement with IBM/TWC

An ongoing joint development agreement with IBM Corporation and its subsidiary, the Weather Company (TWC), seeks to create an operational, GPU-enabled weather prediction system based on the MPAS atmospheric model. In FY2018 NCAR continued this collaboration to complete the optimization, integration, and verification of a GPU-enabled version of the model for use in global weather predictions by TWC on the IBM Power Server architecture. The joint development activity provides $200,000 of funding to CISL and is also supported with cosponsored staff time from NSF Core funds.

Joint Center of Excellence with HPE

A Joint Center of Excellence (JCoE) with Hewlett Packard Enterprise (HPE) – formed in 2017 to advance atmospheric and climate-related applications on the Cheyenne supercomputer and to prepare for future HPC architectures – continued in FY2018 to explore ways to improve CISL’s Cheyenne computing environment. CISL and HPE undertook a number of collaborative projects through a multi-faceted effort spanning systems administration, user services, and computing research. CISL HPC system engineers and HPE architects and system engineers collaborated on the design and architecture of HPE Performance Cluster Manager (HPCM) so that it includes features and functions most relevant to the operation of Cheyenne and future HPC systems. Significant work was done to provide working examples of features and functions that improve the stability and usability of HPC systems. HPE has already incorporated many such features and functions to optimize the integration of system components, enhance system usability and monitoring capabilities, and allow integration of new functions for improving power utilization. CISL will continue to solicit improvements of future HPCM releases. An additional benefit of the JCoE is HPE’s commitment to perform the migration of Cheyenne to HPCM on site at no cost to NCAR.

Workshops

Recurring workshops and training activities give NCAR a way to track emerging research fields and technologies, monitor the state of the art, and promote their use in the community. Three such activities are highlighted below, focusing on climate informatics, the use of many-core processors in modeling applications, and hands-on training in machine learning.

Climate informatics and hackathon

Climate informatics broadly refers to any research combining climate science with approaches from statistics, machine learning, and data mining. The Climate Informatics Workshop series, in its eighth year, seeks to build collaborative relationships between researchers from all of those disciplines. The workshop September 19-21, 2018, at the NCAR Mesa Laboratory attracted 125 attendees.

Multicore 8

The eighth annual Multicore Workshop (MC-8) was held September 18-19, 2018, and was the largest yet with 82 participants. The workshop is a forum for open discussion and learning to better understand the application of new high-performance computing technologies for the next generation of weather, climate, and Earth system models. The new generation of HPC platforms has diverse, heterogeneous architectures that present significant challenges to the community working on these models. In addition to providing a forum for participants to share their experience and lessons learned from developing models on these platforms, the workshop goals include creating community of developers to collect and enunciate requirements for next-generation HPC programming models, tools, systems, and hardware. Topics discussed at MC-8 included techniques for programming, performance analysis and optimization, and I/O strategies; scalability of codes on future platforms; advancements in machine learning and deep learning techniques for modeling, data analysis and data assimilation; and involving students and early career computational scientists in HPC for Earth system applications.

Multicore Workshop participants group photo
The two-day Multicore 8 workshop drew 82 participants in FY2018. NCAR uses this strategic workshop to share knowledge and expertise about emerging computing technologies with peer centers. In its eighth year, the workshop featured student posters and travel support for the first time.

Beyond P-Values: Machine Learning

This annual three-day course was held at the NCAR Mesa Laboratory April 17-19, 2018. It gave 12 attendees a hands-on introduction to fundamental methods used in machine learning. The course included dimension reduction methods, followed by an overview of unsupervised vs. supervised learning. Instructors covered various cluster analysis methods such as k-means and also introduced data-driven approaches such as regression trees and modeling-based approaches with a special focus on artificial neural networks and deep learning.