Lead regional cyberinfrastructure engagements

CISL’s regional cyberinfrastructure (CI) leadership strategy is to engage the community in numerous activities to advance high performance computing (HPC), data storage, and networking technologies, and to improve regional, national, and international CI. NCAR’s CI partnership strategy is to engage research CI communities in activities that complement NCAR’s goals and mission, that broadly advance HPC, data storage, and networking technologies, and that improve regional, national, and international CI. All of these activities are aligned with NCAR’s leadership role as a Federally Funded Research and Development Center. Further, this work also addresses CISL’s imperative to broaden participation in technology, education, and applied research.

Information technology is a fast-paced business, driven by exponential growth in the capabilities of the underlying technologies, so it requires an agile approach to the partnerships surrounding it. Consequently, partnerships in computational science and technology at NCAR have typically grown up from the challenges created by emerging, disruptive technologies. Two key components of NCAR’s partnership development strategy are technology tracking and applied research. The former activity alerts the lab to emerging challenges, while the latter enables the lab to exploit them as opportunities. Knowledge gained from both serves as a guide to help direct and prioritize CISL's research and development partnerships and to realize its goals in operational systems.

High Performance Computing

CISL is actively engaged in regional HPC partnerships that facilitate exchange of cyberinfrastructure (CI) experience and ideas among software and storage engineers and that help to inform CISL’s processes for CI acquisition, deployment, and maintenance.

In FY2017, HSS staff participated in panels and sessions at the annual Rocky Mountain Advanced Computing Consortium (RMACC) symposium and presented an NWSC Site Update, including lessons learned and challenges of deploying a multi-petaflops supercomputer. The presentation focused on the lessons learned in the NWSC-2 procurement and the deployment of NCAR’s new supercomputer, Cheyenne, while reporting the current status of all NWSC supercomputing environments. The presentation was attended by approximately 30 participants ranging from peer centers to students. Multiple questions were fielded in the areas of user environment setup, tool usage, and why specific CI elements were selected and how they perform together.

Also in FY2017, CISL collaborated with Intel to host a free, one-day training session on Intel’s performance and threading analysis tools named Intel Parallel Studio. Detailed sessions covering Intel MPI, cluster tools, Python distribution, and Intel libraries were also provided. This training afforded software engineers from NCAR and other institutions the option to attend either in person or via webcast.

In FY2017, CISL hosted three interns who conducted research and testing in the area of Supercomputer Infiniband Fabric Analysis, where the students developed and published Tulip plug-in code to analyze the high-performance interconnect of our supercomputers by isolating bottlenecks and congestions as well as highlighting the efficiencies in the network. A second project focused on improving HPC scheduling throughput by applying machine learning and statistical techniques to analyze the historical job data produced on the supercomputers. This work provided information that will be used to improve the efficiency of our job schedulers.

In collaboration with the University of Wyoming, CISL hosted the 2017 LCI HPC Workshop on 22-26 May 2017 in Laramie, Wyoming. HSS leveraged Amazon Cloud Services to build the infrastructure for the training program. This on-demand, reusable infrastructure is now available for future needs. The workshop was targeted toward system administrators who were either new to or had not had professional exposure to High-Performance Computing (HPC), and it provided advanced technical training in the design, provisioning, and maintenance of HPC clusters. During the five days, workshop attendees learned about HPC system administration concepts and technologies and how to apply them in a production HPC environment. Users also received individual hands-on skill development by building a small test cluster and setting up the hardware and software environment to run MPI jobs. The workshop received favorable reviews from the attendees as well as the LCI representative who participated in the workshop. The workshop concluded with a tour of the NCAR-Wyoming Supercomputer Center, where participants viewed NCAR’s Yellowstone and Cheyenne supercomputers and the GLADE high-performance file system storage environment.

LCI HPC workshop
2017 LCI HPC workshop participants at the NWSC.


CISL actively initiates and leads networking partnerships and collaborations and engages in regional HPC activities such as the Wyoming-NCAR Alliance, the Rocky Mountain Advanced Computing Consortium (RMACC), and the Pacific Research Platform. In networking technology, CISL will provide the technical leadership for operating several regional networks that are tightly integrated with national networks. Examples include the Front Range GigaPop (FRGP) and the Bi-State Optical Network (BiSON).

CISL’s Network Engineering and Telecommunications Section (NETS) is a recognized leader and participant in many regional networking projects that support national networks. NETS provides a vital service to NCAR’s research communities by linking scientists to supercomputing resources and each other. These activities are essential for the effective use of UCAR/NCAR scientific resources, and they foster the overall advancement of scientific inquiry.

CISL’s involvement with regional networking consortia includes the Front Range GigaPoP (FRGP), the Bi-State Optical Network (BiSON), Western Regional Network (WRN), Boulder Point-Of-Presence (BPOP), Boulder Research and Administration Network (BRAN), the City of Boulder CG4 inter-building cabling, the Colorado Department of Transportation (CDOT), and the National Research and Educational Networks (RENs) consortium named The Quilt. These collaborations and networks are all designed to provide NCAR/UCAR and other regional institutions with robust regional and wide-area data pathways.

On behalf of UCAR, CISL continues to lead and participate in these important metropolitan, regional, and national networking initiatives. The tangible benefits of such participation include economical, diverse, high-performance networking for UCAR and its member universities. However, the intangible benefits are at least as important: participating in these initiatives reinforces UCAR’s public mission of providing services to its members while simultaneously fostering cooperative ventures, collaborations, and relationships among these institutions. As a leading participant in such community alliances, NETS contributes to strengthening UCAR’s value as an institution and helps UCAR fulfill its leadership and outreach obligations for NSF funding.

Women In Networking at SC (WINS)

WINS is a three-year National Science Foundation and DOE/ESnet funded program for early-to-mid-career women from diverse regions of the U.S. research and education community IT field to participate in the construction of SCinet from the ground up. SCinet is one of the fastest and most advanced computer networks in the world. WINS is a joint effort between the Energy Sciences Network (ESnet), the Keystone Initiative for Network Based Education and Research (KINBER), and University Corporation for Atmospheric Research (UCAR). Aimed at increasing hands-on training opportunities for women in the Information Technology, the program integrates them in the research and education (R&E) network community to participate in the setup, build, and live operation of SCinet, SC’s ultra-high-performance network that supports large-scale computing demonstrations. SC is considered the premier U.S. conference on HPC, networking, data storage, and data analysis and is attended by over 10,000 of the leading minds in these areas of research.

Internet2 Gender Diversity Initiative

Internet2 began the Gender Diversity Initiative (GDI) in 2013 as a community-driven effort to increase diversity in advanced networking community at all stages of professional development because I2 believes that a more diverse and inclusive community is a stronger one and will generate better solutions, tools, and results for the many people we serve around the world. Marla Meehl from NETS has co-chaired the GDI Steering Committee since its inception with Laurie Burns McRobbie from Indiana University. At each Internet2 Global Summit and Technology Exchange, the Diversity Steering group of the Gender Diversity Initiative chooses one or more awardees to attend the meeting. The Goals of Gender Diversity Initiative are to:

  • Provide a national forum for discussion and sharing of best practices in improving gender diversity in technical fields, particularly advanced networking.

  • Explore and support specific projects that have application to the membership community at large.

  • Partner with other national organizations engaged in similar work, such as the National Center for Women in Technology (NCWIT), Society of Women Engineers (SWE), and other organizations with a national scope.

  • Communicate regularly to the membership and the public on effective approaches and progress toward improved diversity.

Commitment to HPC partnerships

CISL is actively engaged with regional HPC partnerships. These activities allow CISL to gain hands-on experience in collaborating with campus IT staff through the processes of CI acquisition, deployment, and resource federation.

Wyoming-NCAR Alliance and regional campus partnerships

Perhaps CISL’s most important regional partnership is the Wyoming-NCAR Alliance (WNA), which governs the joint activities of NCAR and the State and University of Wyoming related to the NWSC facility, supercomputing environment, and its education, outreach, and training activities. Through this partnership, CISL collaborates with Wyoming to enhance their campus HPC capabilities and extend Wyoming’s research partnerships with other EPSCoR states. CISL has also supported several University of Wyoming-focused STEM education activities and proposals.

Intel Parallel Computing Center for Weather And Climate Simulation

CISL and the University of Colorado at Boulder continued their collaboration on an Intel award for the Intel Parallel Computing Center for Weather And Climate Simulation (IPCC-WACS). This collaborative center promotes the discovery of new methods for optimizing the performance of weather and climate models on Intel Xeon and Xeon Phi hardware and accelerates the adoption of these optimizations in key weather and climate community models.

Rocky Mountain Advanced Computing Consortium

The Rocky Mountain Advanced Computing Consortium (RMACC) is a collaboration among academic and research institutions located throughout the intermountain states. Its mission is to facilitate widespread effective use of regional high performance computing. CISL continued its participation in RMACC, collaborating in areas such as education, outreach, training, and HPC research. Each year RMACC holds an HPC Symposium that provides opportunities for professional development via scientific and technical presentations and training lectures. CISL helps organize and plan the Symposium. RMACC also showcases student research through a student poster competition at the symposium. Many of CISL’s SIParCS students participate in this Symposium. The RMACC community also considers other collaborative measures to benefit the regional development of HPC, such as writing joint infrastructure proposals and having a Consortium booth at the IEEE Supercomputing Conference.

Rocky Mountain Cyberinfrastructure Mentoring and Outreach Alliance

CISL was awarded an NSF Collaborative Research CC*IIE Region Proposal titled the “Rocky Mountain Cyberinfrastructure Mentoring and Outreach Alliance (RMCMOA).” Colorado State University (CSU), the Idaho Regional Optical Network (IRON), the University Corporation for Atmospheric Research (UCAR), the University of Colorado at Boulder (UCB), and the University of Utah (UU) have a long and fruitful history of collaboration, leadership, and innovation in regional and state networking, CI, and HPC technology and infrastructure operations. These partners are leveraging their expertise and organizational structures to lead and manage an outreach effort to better inform, educate, and drive adoption and expansion of advanced networking and CI technologies to small colleges and universities in the western region of the United States, specifically those in Colorado, New Mexico, Idaho, Utah, and Wyoming. The RMCMOA grant received a No Cost Extension to 2/28/17, which allowed tRMCMOA participants to return to the January 2017 Westnet CIO and Westnet meetings.


CISL’s work in this area is supported by NSF Core funding, NSF MRI grant CNS-0821794, NSF ACI 1440642, NSF ACI 1640987, and UCAR non-federal funds.