Advance and support enterprise IT at NCAR and UCAR

Supporting NCAR’s computational and data services is an information technology (IT) infrastructure challenge with many issues that must be managed across organizational boundaries. Many of the foundational technologies such as networking, enterprise services, and cybersecurity are the domain of CISL’s Enterprise Systems and Services Division (ESSD). ESSD focuses on this area to meet demands driven by rapid change, complexity, and the fact that nearly all science is delivered through some form of information technology.

Networking

CISL’s Network Engineering & Telecommunications Section (NETS) has nearly finished the Mesa Lab Network Access Completion (MLNAC-2) project. Twelve zones were completed by the end of FY2019. The project replaces outdated network cabling to 300+ telecommunications outlets in the Mesa Lab.

During FY2019, NETS also was involved in the network design for a new building project at NCAR’s Research Aviation Facility at the Rocky Mountain Metropolitan Airport. NETS was also responsible for designing, acquiring, and installing transition networking and telecommunications services for temporary offices and continued operation of the hangers in support of the construction project.

Mesa Lab Network Access Completion Project-2
NETS staff removing old cable, and pulling in and terminating new ethernet cable as part of the Mesa Lab Network Access Completion Project-2 (MLNAC-2). This project rewires the A tower, common areas and portions of the 1B level to keep pace with the demands of modern network capabilities.

 Other noteworthy FY2019 accomplishments:

  • NETS worked with UCAR Contracts and the 31 members of the Front Range GigaPop (FRGP) to successfully create new five-year agreements when previous agreements expired at the end of June. This effort also included updating budget estimates and creating new account keys, fully executing the agreements, entering them in UCAR's awards management platform, and updating all related web pages and documentation. The FRGP is the Research and Education Network for Colorado and Wyoming and is managed by NETS.

  • NETS provided two staff members to support SC18 in November 2018 and six staff have supported SC19 planning efforts this summer as part of SCinet teams. The FRGP also facilitated the donation of two pairs of fiber from CenturyLink to support WAN connectivity to the Colorado Convention Center in Denver.

  • Upgrading of the busiest FRGP router to a Juniper MX10003. This project involved replacing an MX480 router with a more efficient and capable device that supports current and future demands for 100Gbps connections. The router provides critical access to Internet2 and other regional and national research and education networks.

  • Deployment of Palo Alto firewalls to provide enhanced security and segmentation for guest and business-critical networks. The firewalls were deployed in a redundant cluster at the NCAR Foothills campus. The cluster provides high availability in a configuration that helps simplify maintenance and changes. This required extensive coordination with UCAR Operations.

  • Deployment of a Clearpass captive portal for the UCAR Visitor wireless network. This portal enables UCAR to positively identify visitors and network users. With the elimination of pre-shared keys, the user experience is improved. Self-enrollment significantly reduces the administrative burden of providing wireless networks.

  • UCAR requested that the organization change the telephone dial-out digit from 9 to 7 to reduce the number of accidental 911 calls. The President’s Council approved the change and NETS created a detailed plan and timeline to test, implement, label phones, and communicate the change. NETS completed the change by the September 23 deadline and placed stickers on all UCAR phones to remind employees to use 7 for outside calls and to dial 911 for emergencies.

  • NETS submitted and was awarded an NSF CC* planning grant: Integrating the Colorado Western Slope Research and Education (R&E) Community into the National R&E Infrastructure (BiSON-West).

  • Aging Cisco 6509 data center switches were replaced with Juniper QFX5100 series switches. The new switches enable a next-generation EVPN fabric and high-density 10Gbps and 40Gbps connectivity for enterprise applications and divisional computing.

Enterprise services

CISL enhanced the identity and access management environment by retiring two-factor CRYPTOCard authentication tokens and upgrading the back-end authentication infrastructure. Retiring CRYPTOCards and moving to DUO enables a simplified workflow that no longer requires CISL to mail tokens, saving operating costs and simplifying the end-user experience for CISL HPC users. The FY2019 deployment of a self-enrolling device portal enables Duo two-factor authentication users to enroll additional devices. Installment of load balancers to distribute CIT authentication load improved the availability of authentication when maintenance tasks need to be performed.

Other noteworthy accomplishments:

  • Increased communication to .gov and .mil agencies via email by implementing DMARC policies due to recent mandates by the Department of Homeland Security requiring those federal agencies to reduce exposure to attacks carried out via email.

  • Registered a UCAR DNS domain within Amazon Web Services’ Route 53 DNS service to easily map to services running on Amazon’s cloud infrastructure.

  • Configured several services organization-wide for authentication to Active Directory.

  • Simplified institutional DNS structure by reconfiguring external DNS presence.

  • Expanded the virtual hosting environment for staff to self-provision virtual machines from a catalog of choices. Additionally started hosting a Windows VM environment for the Climate and Global Dynamics Laboratory.

  • Consolidated to a local, single-system inventory product.

  • Consolidated the virtual environment down to a single hypervisor vendor, VMware.

  • Provisioned storage for the Digital Asset Services Hub small-scale data collection repository.

  • Implemented a consolidated patch management system for Linux RedHat systems.

  • Enhanced security by working with configuration management teams to implement disk encryption across staff systems.

Cybersecurity

The Cybersecurity Program Office (CPO) worked with UCAR Operations to ensure that Workday, the new HR and payroll system, meets an appropriate level of cybersecurity compliance. CPO conducted a security assessment of Workday and other vendors based on documentation provided by UCAR Operations and began working with operations staff to establish a continuous monitoring program.

CPO and CISL’s director have been working with UCAR Operations representing UCAR|NCAR|UCP Cybersecurity in the Enterprise System Security Assessment (ESSA). The ESSA had completed about 50% of the scope by the end of FY2019. This initial phase included external scanning, penetration testing, an evaluation of awareness, and training. In conjunction with IT leadership, CPO worked to remediate many of the discovered vulnerabilities and significantly reduce the UCAR threat exposure.

Other noteworthy accomplishments in FY2019:

  • In coordination with other CISL and NCAR groups, CPO took on the effort to upgrade the UCAR VPN solution for the organization. The new Palo Alto VPN solution uses firewall rules and zones to provide a higher level of protection for the UCAR network than the previous solution provided.

  • Establishment of a process for assessing potential cloud vendors considered for services at UCAR. CPO coordinated with UCAR Contracts and Office of the General Council (OGC) to route cloud vendor requests through CPO and then to OGC for review of security and privacy.