Data

Data has been and will continue to be the ultimate product from field campaigns EOL supports. As such, the data we provide must be of high quality and well managed and preserved. Furthermore, in view of the President’s 2013 Open Data Executive Order for public access to data and NSF’s increasing emphasis on multi-disciplinary science, EOL must ensure that its current and historical data is well documented, discoverable, and feed seamlessly into scientific analysis workflows. Imperative 3 describes our activities to meet these challenges, which are divided into three key areas: 1) acquisition, quality control, and data management; 2) standardization of data formats and distribution; and 3) data citation and metrics.

Enhanced Reporting Tools for Field Campaigns 

EOL has been working with field researchers for many years to help document project operations and highlight important events that occur during a scientific field campaign with the EOL Field Catalog. This web-based tool provides information for mission planning and real-time decision-making in the field as well as providing tools for investigators to use for summarizing daily operations, research flights and other important data collection activities. One key component of the Field Catalog that researchers often consult after a field program ends is a “Mission Table” that provides a summary of major sampling events during a field campaign and provides links to reports and catalog products related to that event. 

During FY 2019, EOL has developed and implemented an Interactive Mission Table that allows project participants to create and annotate events in the Mission Table. This new tool allows for more complete documentation of events by field investigators, including the option to identify particular phenomena that were sampled during a specific project observing period. 

One of the goals of this work is to link the mission table information with the field project data archive to improve the ability for researchers to search for measurements of specific phenomena or weather events. Making that linkage between the Field Catalog and the Data Archive will be the next step in this project to improve data discovery across the EOL Field Project Data Archives. We look forward to project investigators contributing improved information on phenomena sampling from upcoming field campaigns in 2020 including Targeted Observations by Radars and UAS of Supercells (TORUS) and the Sundowner Winds Experiment (SWEX).

Legacy Data Rescue

NCAR has been involved in field campaigns for over 50 years and since 1967, EOL (previously the Atmospheric Technology Division (ATD)) has been involved in over 450 field campaigns. The data/metadata and publications resulting from a field campaign are the final legacy products and EOL has a responsibility to steward the data for future reference and research. 

Field Project webpage for the TOGA-COARE project (1992-1993), for which legacy data rescue is being performed. The page contains information on the project and its objectives along with links to important datasets and other documentation.

EOL has unique campaign archives, most of which are not available elsewhere. However, mMuch of the earlier campaign archives - or legacy data - are in poor condition (e.g. lacking complete documentation, undigitized data, unsupported formats, etc.). That means that a considerable amount of the data/documentation is unusable or currently unavailable in their present form.  Time is therefore critical to ‘rescue’ these data, as it is becoming more and more difficult to read old vintage data media and formats and campaign participants are now passing away, leading to the danger is that the data, institutional knowledge, and campaign legacy may be lost forever.

As time passes, there have also been increased requests and inquiries in EOL’s field campaign archive data, mainly pertaining to climate related studies and data “reuse”, and many by graduate students seeking new research topics. In addition, past campaign Principal Investigators (PI)s and previously unknown field data archives at various institutions, are now submitting their data and documentation to EOL.  Interest in these data is also driven by scientists returning to previous field campaign locations who are trying to combine similar measurements taken there in the past to enhance their current research.

Rescuing the legacy data includes the development of an inventory of all legacy publications, photographs and digital media on site, development of project web pages for legacy campaigns not previously part of our collection, the transfer of many datasets, particularly radar and aircraft, from obsolete tape media to online digital files, and the conversion of some legacy radar data from unsupported file formats to more modern standards. We have also received a great deal of help from external investigators who have contributed photos, information, documentation and publications from legacy field campaigns.