NCL data analysis and visualization software

The NCAR Command Language (NCL) is a data analysis and visualization environment developed primarily at CISL in deep collaboration with a number of core scientific groups. NCL enables scientists to easily and effectively read, analyze, and visualize their geoscientific data on platforms ranging from personal laptops to high performance computers. PyNIO and PyNGL are Python modules based on NCL’s file input/output and visualization capabilities; these modules offer NCL’s software components to a wider and more mainstream user base.

The NCL and Python tools have been embraced broadly across the international Earth System sciences community spanning research, education, operational, military, government, and commercial organizations. They are used for data analysis and display to generate publication-quality visualizations for journals, for post-processing and comparing data from a wide range of Earth System models, for real-time data access and display at operational centers, and by hundreds of graduate students as the primary tool for their research. Critical to the importance of these tools is that they are free, well supported, and run across a wide variety of UNIX systems, allowing students and scientists to continue their research uninterrupted as they change organizations.

This work advances one of NCAR’s goals to support, enhance, and extend the capabilities of the university community and the broader scientific community both nationally and internationally. It also addresses CISL’s strategic imperative for software cyberinfrastructure through developing and supporting software specific to the simulation, analysis, and forecasting needs of the atmospheric and related sciences. Specifically, this effort fulfills CISL’s strategic action item to continue the support and development of community data analysis and visualization software.

In FY2015 we released a major version of NCL with over 20 new computational and visualization routines and major improvements to internal OpenMP code for drawing raster graphics. We laid some of the research groundwork for developing a web-based GUI for NCL. We released two versions of PyNIO for use in parallelized workflows for climate model diagnostic packages, and for eventual incorporation into “xray,” an extension of a popular high-performance data analysis library for scientific Python (“pandas”).

Also in FY2015 we added Python-based task parallelism to a large suite of NCL scripts in the Climate Variability Diagnostics Package (CVDP) – an analysis tool that documents the major modes of climate variability in models and observations – resulting in a 4x to 6x speed-up. We released a beta version of PyNGL that incorporates the latest graphical features of NCL with contouring support for complex grids and the new color model. We put PyNIO under “conda,” a package manager that allows for very fast and easy installation of Python packages that require complicated dependencies. We created an NCL / Python Power Users Group called “nclpy-group.”

There were 21,769 downloads of NCL, PyNGL, and PyNIO software in FY2015. Our email lists had 3,156 total postings on 1,210 unique subjects.

Our core development and maintenance of NCL and Python tools is primarily supported by NSF Core funding.