NCAR Technical Notes NCAR/TN-567+PROC
Total Page:16
File Type:pdf, Size:1020Kb
Proceedings of the 2021 Improving Scientific Software Conference Editors Weiming Hu Davide Del Vento Shiquan Su NCAR Technical Notes BY SPONSORED IS NCAR NCAR/TN-567+PROC National Center for Atmospheric Research THE NSF P. O. Box 3000 Boulder, Colorado 80307-3000 www.ucar.edu NCAR TECHNICAL NOTES http://library.ucar.edu/research/publish-technote The Technical Notes series provides an outlet for a variety of NCAR Manuscripts that contribute in specialized ways to the body of scientific knowledge but that are not yet at a point of a formal journal, monograph or book publication. Reports in this series are issued by the NCAR scientific divisions, serviced by OpenSky and operated through the NCAR Library. Designation symbols for the series include: EDD – Engineering, Design, or Development Reports Equipment descriptions, test results, instrumentation, and operating and maintenance manuals. IA – Instructional Aids Instruction manuals, bibliographies, film supplements, and other research or instructional aids. PPR – Program Progress Reports Field program reports, interim and working reports, survey reports, and plans for experiments. PROC – Proceedings Documentation or symposia, colloquia, conferences, workshops, and lectures. (Distribution maybe limited to attendees). STR – Scientific and Technical Reports Data compilations, theoretical and numerical investigations, and experimental results. The National Center for Atmospheric Research (NCAR) is operated by the nonprofit University Corporation for Atmospheric Research (UCAR) under the sponsorship of the National Science Foundation. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. National Center for Atmospheric Research P. O. Box 3000 Boulder, Colorado 80307-3000 NCAR/TN-567+PROC NCAR Technical Note _____________________________________________ 2021-08 Proceedings of the 2021 Improving Scientific Software Conference Editors Weiming Hu Davide Del Vento Shiquan Su NCAR Laboratory NCAR Division ______________________________________________________ NATIONAL CENTER FOR ATMOSPHERIC RESEARCH P. O. Box 3000 BOULDER, COLORADO 80307-3000 ISSN Print Edition 2153-2397 ISSN Electronic Edition 2153-2400 How to Cite this Document: Hu, Weiming, Davide Del Vento, Shiquan Su, (Eds.). (2021). Proceedings of the 2021 Improving Scientific Software Conference (No. NCAR/TN-567 +PROC). doi:10.26024/p6mv-en77 Information about future workshops and other SEA news can be found on our website, -- > https://sea.ucar.edu/sea The website of the 2021 workshop is https://sea.ucar.edu/conference/2021 To be added to the workshop mailing list, please send an email to [email protected]. i Proceedings of the 2021 Improving Scientific Software Conference Table of Contents Organizing Committee .................................................................................................... iii SEA 2021 Peer-Reviewed Papers ...............................................................................1-49 PyELM-MME: A Python Platform For Extreme Learning Machine Based Multi-Model Ensemble.........................................................................................1-4 Nachiketa Acharya and Kyle Joseph Chen Hall Anomaly Detection in Particle Accelerators using Autoencoders................5-11 Jonathan P. Edelen and Nathan M. Cook Empirical Inverse Transform Function for Ensemble Forecast Calibration........................................................................................................12-22 Weiming Hu, Laura Clemente, George S. Young, and Guido Cervone Expanding Impact Metrics Contexts With Software Citation*......................23-30 Keith E. Maull and Matt Mayernik A Portable Framework for Multudimensional Spectral-like Transforms At Scale.................................................................................................................31-39 Dmitry Pekurovsky ii Organizing Committee Conference Chairs Davide Del Vento, National Center for Atmospheric Research (NCAR) Shiquan Su, National Center for Atmospheric Research (NCAR) Program Committee Chairs Davide Del Vento, National Center for Atmospheric Research (NCAR) Shiquan Su, National Center for Atmospheric Research (NCAR) Weiming Hu, The Pennsylvania State University Proceedings Committee Weiming Hu, The Pennsylvania State University Davide Del Vento, National Center for Atmospheric Research Shiquan Su, National Center for Atmospheric Research Michael Flanagan, National Center for Atmospheric Research Taysia Peterson, National Center for Atmospheric Research Steering Committee Andrew Younge, Sandia National Laboratories Brian Vanderwende, National Center for Atmospheric Research (NCAR) Davide Del Vento, National Center for Atmospheric Research (NCAR) Edward Hartnett, National Oceanic and Atmospheric Administration (NOAA) Guido Cervone, The Pennsylvania State University Joseph Schoonover, Fluid Numerics Julia Collins, University of Colorado Keith Maull, National Center for Atmospheric Research (NCAR) Maggie Sleziak, National Center for Atmospheric Research (NCAR) Mick Coady, National Center for Atmospheric Research (NCAR) Sheri Mickelson,National Center for Atmospheric Research (NCAR) Shiquan Su, National Center for Atmospheric Research (NCAR) Srinath Vadlamani, Arm Weiming Hu, The Pennsylvania State University Workshop Administrator Taysia Peterson, National Center for Atmospheric Research (NCAR) iii This page is intentionally left blank. iv PyELM-MME: A Python Platform For Extreme Learning Machine Based Multi-Model Ensemble Nachiketa Acharya Kyle Joseph Chen Hall Center for Earth System Modeling, Analysis, & Data (ESMAD) International Research Institute for Climate & Society Department of Meteorology and Atmospheric Science The Earth Institute at Columbia University The Pennsylvania State University Palisades, NY USA University Park, PA [email protected] [email protected] Abstract— The generation of a multi-model ensemble (MME) PyELM-MME implements ELM, as well as traditional is a well-accepted way to improve the skill of the climate forecasts MME methods like the ensemble mean (EM) and multiple produced by individual general circulation models. Recently, linear regression (MLR) as benchmarks. One can compare and there has been significant interest in exploring the potential to contrast the prediction skill of the different methods using improve climate prediction using Machine Learning based MME. PyELM-MME’s forecast verification module. In this study, we One such Machine Learning method is the Extreme learning describe the co-design, co-development, and skill assessment Machine (ELM), which is a state-of-the-art non-linear regression of the PyELM-MME. method based on single-hidden-layer feed-forward neural networks. We developed PyELM-MME, a Python platform for II. BACKGROUND AND GOAL producing ELM-based MME climate predictions and comparing them with those produced by other traditional MME methods like The SLFN, a simple form of ML, has been extensively ensemble mean and multiple linear regression. PyELM-MME also studied from both theoretical and practical perspectives for its includes a forecast verification module, which allows one to assess learning capacity and fault tolerance. However, the efficacy of the relative prediction skill of the different methodologies. In this SLFN-based methods is highly dependent on appropriate tuning study, we describe the co-design, co-development, and skill of their adjustable hyperparameters, e.g., transfer function, assessment of PyELM-MME. learning rate, and the number of nodes in the hidden layer. Additionally, there are several disadvantages to traditional Keywords—Multi-Model Ensemble, Extreme Learning Machine, SLFN-based methods, including long computation time, over- Python Platform fitting, and vanishing gradient. I. INTRODUCTION To overcome such shortcomings, a novel learning algorithm The generation of a multi-model ensemble (MME) is a well- for SLFN called Extreme Learning Machine (ELM) has been accepted way to improve the skill of forecasts generated by proposed [6]. In the proposed algorithm, the network’s input individual general circulation models (GCMs). There are two weights and hidden biases are randomly chosen, and its output common approaches to making an MME: one either combines weights are determined analytically using the Moore-Penrose the individual forecasts with equal weights, or weights them generalized inverse. The basic principle distinguishing ELM according to their prior performance [1,2]. Numerous studies from the traditional neural network methodology is that the have shown that multi-model ensembles, regardless of which parameters of the feedforward network are not trained through methods have been used, exhibit increased prediction skill when backpropagation during ELM model fitting. Implementing an compared to single-model forecasting [3,4]. After previous MME using ELM requires the following steps: pioneering work, there is strong interest in exploring the • Selecting input and output neurons (training dataset) potential of Machine Learning (ML) based MME for improving seasonal forecasts [5]. This previous work proposed generating • Scaling the input dataset MME forecasts using a state-of-the-art non-linear regression • Selecting the activation function method based on single-hidden-layer feed-forward neural networks (SLFN) called Extreme Learning Machine (ELM) • Training and testing the model [5].