Explore many-core and accelerator-based architectures

HOMME execution time
This chart shows the execution times for the High Order Method Modeling Environment (HOMME), which is a dynamical core that is used by the Community Atmosphere Model (CAM) using the Intel compiler on several different Intel processors (SNB2 = 2-socket Sandy Bridge node, BDW2 = 2-socket Broadwell node; KNL 1-socket Knights Landing node). Note that while optimizations have reduced execution time on all three architectures, a factor of four reduction in execution time is achieved on KNL. Depending on the particular scientific configuration, the HOMME dynamical core consumes 23% of CAM and 81% of the Whole Atmosphere Chemistry (WACCM) model. Reducing the cost of the HOMME dynamical core will increase the amount of science that can be performed on our existing and future supercomputers.

CISL collaborates with NCAR’s science laboratories to provide new tools for exploiting many-core architectures such as general-purpose graphics processing units (GPGPUs). This allows us to increase model performance on advanced many-core architectures, such as those in NCAR’s next supercomputer. CISL also plans to give our users access to advanced systems by acquiring a many-core cluster.

Using codes developed under this initiative, our scientific users will gain experience with production systems composed of these emerging technologies. These collaborations have enabled advances in application performance and opportunities to help train the next generation of scientists and engineers who will apply these new technologies to challenges of societal importance.

In FY2016, CISL’s Application Scalability and Performance (ASAP) group has been involved in several collaborations that focus on preparing NCAR applications for future generations of microprocessor architectures. These collaborations include: An Intel Parallel Computing Center (IPCC) focused on Weather and Climate Simulation (IPCC-WACS) funded by Intel in collaboration with the University of Colorado at Boulder (CU Boulder); A National Energy Research Scientific Computing Center (NERSC) Exascale Science Application Program (NESAP) in collaboration with NERSC and Cray Inc, the Indian Institute of Science in Bangalore, India, and the University of Wyoming.

This effort has focused on weather and climate applications, including the Community Earth System Model (CESM), the Weather Research and Forecasting model (WRF), and the Model for Prediction Across Scales (MPAS), three of the most widely used applications in the field. All three are large Fortran-based simulation codes – for instance, CESM is estimated to have about 1.5 million lines of code.

CISL has made significant progress optimizing several sections of CESM and MPAS that reduced their computational costs. Also, the execution time of multiple physics modules including those within CAM was shortened, and this reduced the total cost of CAM by 15%. Moreover, the HOMME dynamical core used within CAM received additional optimizations that reduced the total cost of HOMME from 23 to 75%, depending on the scientific configuration. Work was also done to address the problematic data structures and computationally intensive subroutines within MPAS that were inhibiting compiler optimizations. Addressing these data structure issues enabled a speedup on the entire MPAS dynamical core by a factor of two on Yellowstone.

Work will continue with the science and model development teams at NCAR to both optimize existing application codes and provide guidance for future code development.

The IPCC-WACS project is funded by a donation from Intel Corporation. Additional optimization efforts within ASAP are supported by NSF Core funds.