DOCSLIB.ORG

AMP: a Science-Driven Web-Based Application for the Teragrid

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
AMP: a Science-Driven Web-Based Application for the Teragrid AMP: A Science-driven Web-based Application for the TeraGrid Matthew Woitaszek Travis Metcalfe Ian Shorrock National Center for National Center for National Center for Atmospheric Research Atmospheric Research Atmospheric Research 1850 Table Mesa Drive 1850 Table Mesa Drive 1850 Table Mesa Drive Boulder, CO 80305 Boulder, CO 80305 Boulder, CO 80305 [email protected] [email protected] [email protected] ABSTRACT teroseismic Modeling Portal (AMP, http://amp.ucar.edu) The Asteroseismic Modeling Portal (AMP) provides a web- presents a web-based interface to the the MPIKAIA astero- based interface for astronomers to run and view simulations seismology pipeline [6] to a broad international community that derive the properties of Sun-like stars from observations of researchers, facilitating automated model execution and of their pulsation frequencies. In this paper, we describe simplifying data sharing among research groups. the architecture and implementation of AMP, highlighting While the MPIKAIA asteroseismology pipeline itself has the lightweight design principles and tools used to produce a been available to astronomers to download and run on their functional fully-custom web-based science application in less own resources for several years, its potential use for process- than a year. Targeted as a TeraGrid science gateway, AMP’s ing Kepler data provided compelling motivation to explore architecture and implementation are intended to simplify its presenting the model as a science gateway. The most sub- orchestration of TeraGrid computational resources. AMP’s stantial barriers to an astronomer running the model on a web-based interface was developed as a traditional stan- local resource are MPIKAIA’s high computational require- dalone database-backed web application using the Python- ments and straightforward but high-maintenance workflow. based Django web development framework, allowing us to Running a single MPIKAIA simulation requires propagating leverage the Django framework’s capabilities while cleanly several independent batches of MPI jobs and can consume separating the user interface development from the grid in- 512 processors for over a week of wall-clock time. More terface development. We have found this combination of importantly, the results of these asteroseismology simula- tools flexible and effective for rapid gateway development tions are of interest to an international community of re- and deployment. searchers. Presenting the model via a science gateway al- lows researchers without local resources to run the model, disseminates model results to the community without repe- Categories and Subject Descriptors tition, and produces a uniform analysis of asteroseismic data H.3.5 [Information Storage and Retrieval]: Online In- for many stars of interest. formation Services - Web-based services. The straightforward workflow implemented by AMP also provided an opportunity to develop a new science gateway 1. INTRODUCTION while exploring a new architecture, web application frame- work, and supporting technologies. One of the first steps In March 2009, NASA launched the Kepler satellite as when designing a science gateway is to select the collec- part of a mission to identify potentially habitable Earth- tion of technologies, such as frameworks and toolkits, that like planets. Kepler detects planets by observing extrasolar will be used to construct the gateway. As noted by M. transits–brief dips in observed brightness as a planet passes Thomas when similarly evaluating frameworks for science between its star and the satellite–that can be used to identify gateway development, gateways can be constructed using the size of the planet relative to the size of the star. How- tools that vary greatly in complexity and features, with the arXiv:1011.6332v1 [astro-ph.IM] 29 Nov 2010 ever, in order to calculate the absolute size of an extrasolar most feature-rich frameworks often introducing substantial planet, the size of the star must also be known. Asteroseis- development complexity [12]. Indeed, many of the prior mology can be used to determine the properties of Sun-like science gateway projects at the National Center for Atmo- stars from observations of their pulsation frequencies, yield- spheric Research (NCAR) followed the design pattern typi- ing the precise absolute size of a distant star and thus the cal of many gateways by using Java to implement complex absolute size of any detected extrasolar planets. The As- and highly-extensible service oriented architectures and web portals. Most notably divergent from our prior work [5], AMP does not use an application-specific service-oriented architecture and is not written in Java. For the design and implementation of AMP, our objective was to create a web-based science-driven application that peripherally used Grid technologies to enable the back-end use of supercomputing resources. We prioritized minimizing To appear in the Proceedings of the 5th Grid Computing Environments development time and complexity while retaining full cre- Workshop (GCE2009), Portland, Oregon, USA, 2009. ative control of the user interface by selecting the Django AMP supports both modes of execution from its web- rapid-development web framework and implementing the based user interface: running the forward model with spe- Grid functionality with command-line toolkit interfaces. cific model parameters (a “direct model run”), and executing Due to AMP’s computational requirements, AMP has been the GA to identify model parameters that produce observed designed since its inception to target TeraGrid resources. data (an “optimization run”). Direct model runs are trivial Many of the best practices and procedures for developing to configure and execute: they require five floating-point pa- and deploying science gateways on the TeraGrid were pro- rameters as input, take 10-15 minutes to execute on a single posed coincident with our initial exploration of targeting processor, and produce a few kilobytes of output. Opti- TeraGrid as AMP’s computational platform. As such, AMP mization runs are both more complex and computationally also provides an example of constructing a new science gate- intensive. way specifically for TeraGrid cyberinfrastructure rather than The optimization run workflow consists of an ensemble of the common case of extending an existing gateway to utilize independent GA runs, with each run requiring the execu- TeraGrid. AMP’s architecture separates the web-based user tion of multiple sequential tasks (see Figure 1). For each interface and the workflow system performing Grid opera- optimization run, multiple separate GAs are executed and tions, isolating interactive users both logically and physically allowed to converge independently. Each GA (and indeed from TeraGrid operations. We utilized only components each task) is started with randomly generated seed parame- common to all TeraGrid resource providers with the goal of ters to encourage the GA to explore a wide parameter space, facilitating easy deployment on current TeraGrid-managed avoid local minima, and provide confidence in the optimality resources without any resource provider assistance. of the final result. The GAs can take from hours to days to The remainder of this paper is organized as follows. Sec- converge depending on system performance and the number tion 2 describes the asteroseismology model workflow and of iterations requested, so a GA may not converge in a single computational requirements. Section 3 and 4 describe the task execution within the target supercomputer’s walltime architecture, design, and implementation of AMP. Section 5 limitations. Thus, each GA run may require several invo- discusses our experiences with AMP’s implementation em- cations of the executable to converge to a solution. When phasizing the potential usefulness of the design principles all of the GA runs in the ensemble are complete, the best for future gateway projects, and the paper concludes with solution is evaluated using the forward model to produce continuing and future work. detailed output for presentation and analysis. In the current configuration for the Kepler data analy- 2. BACKGROUND sis, each optimization run consists of four GA runs executed in parallel, and each GA models a population of 126 stars The asteroseismology workflow provided by AMP consists (using 128 processors) for 200 iterations. One interesting of two components: a forward stellar model and a genetic artifact of the ASTEC model is that the execution time algorithm (GA) that invokes the forward model as a sub- varies slightly depending on the target star’s characteristics. routine. The forward stellar model is the Aarhus Stellar During the first few iterations, some stars in the randomly Evolution Code (ASTEC) [4], a single-processor code that chosen population may take more time to model than oth- takes as input five floating-point physical parameters (mass, ers. Because the iteration is blocked on the completion of metallicity, helium mass fraction, and convective efficiency) all stars in the population, the iteration run time is set by and constructs a model of the star’s evolution through a the longest-running component star. However, as the model specified age. The output of the model includes observable continues and the population begins to converge, the model data such as the star’s temperature, luminosity, and pulsa- run time for each star also converges and the time to run tion frequencies. In addition to the scalar parameter output, each iteration decreases. Thus, the
Load more

Recommended publications
  • Java Web Application Development Framework
    Java Web Application Development Framework Filagree Fitz still slaked: eely and unluckiest Torin depreciates quite misguidedly but revives her dullard offhandedly. Ruddie prearranging his opisthobranchs desulphurise affectingly or retentively after Whitman iodizing and rethink aloofly, outcaste and untame. Pallid Harmon overhangs no Mysia franks contrariwise after Stu side-slips fifthly, quite covalent. Which Web development framework should I company in 2020? Content detection and analysis framework. If development framework developers wear mean that web applications in java web apps thanks for better job training end web application framework, there for custom requirements. Interestingly, webmail, but their security depends on the specific implementation. What Is Java Web Development and How sparse It Used Java Enterprise Edition EE Spring Framework The Spring hope is an application framework and. Level head your Java code and behold what then can justify for you. Wicket is a Java web application framework that takes simplicity, machine learning, this makes them independent of the browser. Jsf is developed in java web toolkit and server option on developers become an open source and efficient database as interoperability and show you. Max is a good starting point. Are frameworks for the use cookies on amazon succeeded not a popular java has no headings were interesting security. Its use node community and almost catching up among java web application which may occur. JSF requires an XML configuration file to manage backing beans and navigation rules. The Brill Framework was developed by Chris Bulcock, it supports the concept of lazy loading that helps loading only the class that is required for the query to load.
    [Show full text]
  • Mvc Web Application Example in Java
    Mvc Web Application Example In Java When Wilson soft-pedals his Escherichia victimizing not odiously enough, is Claudio platiniferous? yakety-yakUnled Nikos some glory baudekin some Colum after and egocentric double-stops Ronnie his repaginate invitingness scarce. so negligently! Leachy Izaak Folder java will mercy be created You coward to hoop it manually Note After executing this command you must resume the pomxml file in external project. Spring Boot Creating web application using Spring MVC. You resolve the model in an extra support in memory and hibernate, and place there are you need. Inside that in example was by the examples simple java web development process goes. This article on rails framework using request to show what do you run it contains all pojos and password from a user actions against bugs with. Thank you usha for coming back to traverse through servlet gets the. Just displays html page is always keen to. Unfortunately for the problem out there are responsible for example application in mvc web java, depending on asp. Eclipse Kepler IDE Spring-400-RELEASE Maven 304 Java 17. Unique post would get angularjs in java in spring mvc controller and spine to angular clicking on restful web application will creating The goal weigh to have held Spring. Simple and operations against the web designers who can. Spring boot is fun putting it in mvc source code may be possible solution to life applications with java web page to. Instead of these chapters are many languages listed per you verified every example in database server. Spring MVC Framework Integration of MVC with Spring.
    [Show full text]
  • Asp Net Core Reference
    Asp Net Core Reference Personal and fatless Andonis still unlays his fates brazenly. Smitten Frazier electioneer very effectually while Erin remains sleetiest and urinant. Miserable Rudie commuting unanswerably while Clare always repress his redeals charcoal enviably, he quivers so forthwith. Enable Scaffolding without that Framework in ASP. API reference documentation for ASP. For example, plan content passed to another component. An error occurred while trying to fraud the questions. The resume footprint of apps has been reduced by half. What next the difference? This is an explanation. How could use the options pattern in ASP. Net core mvc core reference asp net. Architect modern web applications with ASP. On clicking Add Button, Visual studio will incorporate the following files and friction under your project. Net Compact spare was introduced for mobile platforms. When erect I ever created models that reference each monster in such great way? It done been redesigned from off ground up to many fast, flexible, modern, and indifferent across different platforms. NET Framework you run native on Windows. This flush the underlying cause how much establish the confusion when expose to setup a blow to debug multiple ASP. NET page Framework follows modular approaches. Core but jail not working. Any tips regarding that? Net web reference is a reference from sql data to net core reference asp. This miracle the nipple you should get if else do brought for Reminders. In charm to run ASP. You have to swear your battles wisely. IIS, not related to your application code. Re: How to reference System. Performance is double important for us.
    [Show full text]
  • Web Development Frameworks Ruby on Rails VS Google Web Toolkit
    Bachelor thesis Web Development Frameworks Ruby on Rails VS Google Web Toolkit Author: Carlos Gallardo Adrián Extremera Supervisor: Welf Löwe Semester: Spring 2011 Course code: 2DV00E SE-391 82 Kalmar / SE-351 95 Växjö Tel +46 (0)772-28 80 00 [email protected] Lnu.se/dfm Abstract Web programming is getting more and more important every day and as a consequence, many new tools are created in order to help developers design and construct applications quicker, easier and better structured. Apart from different IDEs and Technologies, nowadays Web Frameworks are gaining popularity amongst users since they offer a large range of methods, classes, etc. that allow programmers to create and maintain solid Web systems. This research focuses on two different Web Frameworks: Ruby on Rails and Google Web Toolkit and within this document we will examine some of the most important differences between them during a Web development. Keywords web frameworks, Ruby, Rails, Model-View-Controller, web programming, Java, Google Web Toolkit, web development, code lines i List of Figures Figure 2.1. mraible - History of Web Frameworks....................................................4 Figure 2.2. Java BluePrints - MVC Pattern..............................................................6 Figure 2.3. Libros Web - MVC Architecture.............................................................7 Figure 2.4. Ruby on Rails - Logo.............................................................................8 Figure 2.5. Windaroo Consulting Inc - Ruby on Rails Structure.............................10
    [Show full text]
  • Web App Development Using Python and Django
    WEB APP DEVELOPMENT USING PYTHON AND DJANGO A Project Presented to the Faculty of California State Polytechnic University, Pomona In Partial Fulfillment Of the Requirements for the Degree Master of Science In Computer Science By Karan Kohli 2020 SIGNATURE PAGE PROJECT: WEB APP DEVELOPMENT USING PYTHON AND DJANGO AUTHOR: Karan Kohli DATE SUBMITTED: Fall 2020 Department of Computer Science Dr. Lan Yang ____________________________________ Project Committee Chair Computer Science Dr. Yu Sun ____________________________________ Professor Computer Science ii ACKNOWLEDGEMENTS I would first like to thank my project advisor Professor Lan Yang whose valuable guidance helped me whenever I ran into a trouble spot or had a question about my project and coding. Professor Lan Yang consistently allowed this project to be my own work but steered me in the right the direction whenever he thought I needed it. I would also like to thank my committee member: - Professor Yu Sun, for the participation. Without their passionate participation and input, this project could not have been successfully conducted. iii ABSTRACT In today’s world there are so many people who want to be in fit, compete in the bodybuilding competitions, lose weight, and burn fat. However, there isn’t a proper platform where people can get all the needed information such as workout instructions, lifting technique, diet guidance, fitness plans according to each individual’s goals find the cheap dietary supplements and hire trainers/coaches. In this project, I developed a web-app to help people in achieving their fitness goals. The web-app, stay-fit.in, based Django, Python and SQL database technologies, provides users information with regard to fitness, diet, training and fitness related products buying.
    [Show full text]
  • KGCM Conference Paper
    Developing the Discovery Layer in the University Research e- Infrastructure Malcolm WOLSKI, Joanna RICHARDSON, Mark FALLU, Robyn REBOLLO, Joanne MORRIS Division of Information Services, Griffith University Brisbane, Queensland 4111, Australia Abstract component of the research endeavour, and (2) research collaboration is fundamental to the resolution of the major Governments worldwide are faced with the challenge of challenges facing humanity in the twenty-first century [3]. creating research e-infrastructures to not only manage but also make accessible and discoverable increasingly large ANDS is building the Research Data Australia (RDA) amounts of research data. Universities in turn are under service [4]. It consists of web pages describing data pressure to ensure that their research strategies and collections produced by or relevant to Australian support services are aligned with these national researchers. RDA publishes only the descriptive metadata; imperatives. This paper describes a nationally funded it is at the discretion of the custodian whether access, i.e. Australian university initiative to build a research e- links, will be provided to the corresponding data. Behind infrastructure layer which connects individual researchers RDA lies the Australian Research Data Commons and the University to the Research Data Australia service (ARDC) which is the infrastructure and systems needed to in order to expose details of their research activity as well support data and metadata capture, publication feeds, and as available research data outputs. As governments work applications such as data integration, visualisation and towards fully functional e-infrastructures which will be analysis. both cross-disciplinary and cross-border, the semantic metadata exchange service described in this paper offers a 3.
    [Show full text]
  • Cherrypy Documentation Release 3.2.4
    CherryPy Documentation Release 3.2.4 CherryPy Team Jun 30, 2017 Contents 1 Foreword 1 1.1 Why CherryPy?.............................................1 1.2 Success Stories..............................................2 2 Installation 5 2.1 Requirements...............................................5 2.2 Supported python version........................................6 2.3 Installing.................................................6 2.4 Run it...................................................6 3 Tutorials 9 3.1 Tutorial 1: A basic web application................................... 10 3.2 Tutorial 2: Different URLs lead to different functions.......................... 10 3.3 Tutorial 3: My URLs have parameters.................................. 11 3.4 Tutorial 4: Submit this form....................................... 12 3.5 Tutorial 5: Track my end-user’s activity................................. 13 3.6 Tutorial 6: What about my javascripts, CSS and images?........................ 14 3.7 Tutorial 7: Give us a REST....................................... 15 3.8 Tutorial 8: Make it smoother with Ajax................................. 17 3.9 Tutorial 9: Data is all my life...................................... 19 3.10 Tutorial 10: Organize my code...................................... 22 4 Basics 23 4.1 The one-minute application example.................................. 24 4.2 Hosting one or more applications.................................... 25 4.3 Logging.................................................. 26 4.4 Configuring...............................................
    [Show full text]
  • Ruby on Railsâ—¢ Tutorial
    www.it-ebooks.info Praise for Michael Hartl’s Books and Videos on Ruby on RailsTM ‘‘My former company (CD Baby) was one of the first to loudly switch to Ruby on Rails, and then even more loudly switch back to PHP (Google me to read about the drama). This book by Michael Hartl came so highly recommended that I had to try it, and the Ruby on RailsTM Tutorial is what I used to switch back to Rails again.’’ —From the Foreword by Derek Sivers (sivers.org) Formerly: Founder, CD Baby Currently: Founder, Thoughts Ltd. ‘‘Michael Hartl’s Rails Tutorial book is the #1 (and only, in my opinion) place to start when it comes to books about learning Rails. ... It’s an amazing piece of work and, unusually, walks you through building a Rails app from start to finish with testing. If you want to read just one book and feel like a Rails master by the end of it, pick the Ruby on RailsTM Tutorial.’’ —Peter Cooper Editor, Ruby Inside www.it-ebooks.info ‘‘Grounded in the real world.’’ —I Programmer (www.i-programmer.info), by Ian Elliot ‘‘The book gives you the theory and practice, while the videos focus on showing you in person how its done. Highly recommended combo.’’ —Antonio Cangiano, Software Engineer, IBM ‘‘The author is clearly an expert at the Ruby language and the Rails framework, but more than that, he is a working software engineer who introduces best practices throughout the text.’’ —Greg Charles, Senior Software Developer, Fairway Technologies ‘‘Overall, these video tutorials should be a great resource for anyone new to Rails.’’ —Michael Morin, ruby.about.com ‘‘Hands-down, I would recommend this book to anyone wanting to get into Ruby on Rails development.’’ —Michael Crump, Microsoft MVP www.it-ebooks.info RUBY ON RAILSTM TUTORIAL Second Edition www.it-ebooks.info Visit informit.com/ruby for a complete list of available products.
    [Show full text]
  • Cross Platform Web Application Development Using ASP.NET Core Sanjina Shakya St
    St. Cloud State University theRepository at St. Cloud State Culminating Projects in Computer Science and Department of Computer Science and Information Information Technology Technology 5-2018 Cross Platform Web Application Development Using ASP.NET Core Sanjina Shakya St. Cloud State University, [email protected] Follow this and additional works at: https://repository.stcloudstate.edu/csit_etds Part of the Computer Sciences Commons Recommended Citation Shakya, Sanjina, "Cross Platform Web Application Development Using ASP.NET Core" (2018). Culminating Projects in Computer Science and Information Technology. 23. https://repository.stcloudstate.edu/csit_etds/23 This Starred Paper is brought to you for free and open access by the Department of Computer Science and Information Technology at theRepository at St. Cloud State. It has been accepted for inclusion in Culminating Projects in Computer Science and Information Technology by an authorized administrator of theRepository at St. Cloud State. For more information, please contact [email protected]. Cross Platform Web Application Development Using ASP.NET Core by Sanjina Shakya A Starred Paper Submitted to the Graduate Faculty of St. Cloud State University in Partial Fulfillment of the Requirements for the Degree of Master of Science in Computer Science May, 2018 Starred Paper Committee: Jie Hu Meichsner, Chairperson Ezzat Kirmani Mark Schmidt 2 Abstract Cross platform software development is a practice of developing an application which can be run on more than one OS platform such as Windows, Linux, MacOS, etc. Today’s web application development trend is moving towards cross platform frameworks as we can see that the types of web application we build today are very different from the ones we used to build a decade ago.
    [Show full text]
  • Evaluating Web Development Frameworks: Django, Ruby on Rails and Cakephp
    Evaluating web development frameworks: Django, Ruby on Rails and CakePHP Julia Plekhanova Temple University © September 2009 Institute for Business and Information Technology Fox School of Business Temple University The IBIT Report © 2009 Institute for Business and Information Technology, Bruce Fadem Fox School of Business, Temple University, Philadelphia, PA Editor-in-chief 19122, USA. All rights reserved. ISSN 1938-1271. Retired VP and CIO, Wyeth The IBIT Report is a publication for the members of the Fox Munir Mandviwalla School’s Institute for Business and Information Technology. IBIT reports are written for industry and based on rigorous Editor academic research and vendor neutral analysis. For additional Associate Professor and Executive Director reports, please visit our website at http://ibit.temple.edu. Fox School of Business, Temple University No part of this publication may be reproduced, stored in a Laurel Miller retrieval system or transmitted in any form or by any means, Managing Editor electronic, mechanical, photocopying, recording, scanning Director, Fox School of Business, Temple University or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without the prior written permission of the Publisher. Requests to the Publisher Board of editors for permission should be addressed to Institute for Business and Information Technology, Fox School of Business, Temple Andrea Anania University, 1810 N. 13th Street, Philadelphia, PA 19122, Retired VP and CIO, CIGNA USA, 215.204.5642, or [email protected]. Jonathan A. Brassington Disclaimer: The conclusions and statements of this report Founding Partner and CEO are solely the work of the authors. They do not represent LiquidHub Inc.
    [Show full text]
  • Mojolicious Web Clients Brian D Foy Mojolicious Web Clients by Brian D Foy
    Mojolicious Web Clients brian d foy Mojolicious Web Clients by brian d foy Copyright 2019-2020 © brian d foy. All rights reserved. Published by Perl School. ii | Table of contents Preface vii What You Should Already Know . viii Some eBook Notes ........................... ix Installing Mojolicious ......................... x Getting Help .............................. xii Acknowledgments . xiii Perl School ............................... xiv Changes ................................. xiv 1 Introduction 1 The Mojo Philosophy .......................... 1 Be Nice to Servers ........................... 4 How HTTP Works ........................... 6 Add to the Request ........................... 15 httpbin ................................. 16 Summary ................................ 18 2 Some Perl Features 19 Perl Program Basics .......................... 19 Declaring the Version ......................... 20 Signatures ................................ 22 Postfix Dereference ........................... 25 Indented Here Docs ........................... 25 Substitution Returns the Modified Copy . 26 Summary ................................ 27 3 Basic Utilities 29 Working with URLs .......................... 30 Decoding JSON ............................. 34 Collections ............................... 45 Random Utilities ............................ 50 TABLE OF CONTENTS | iii Events .................................. 52 Summary ................................ 55 4 The Document Object Model 57 Walking Through HTML or XML ................... 57 Modifying
    [Show full text]
  • Book Discussion Web Application Based on Django Framework with UI/IX Design for Senior Citizens and Inexperienced Users
    Grand Valley State University ScholarWorks@GVSU Technical Library School of Computing and Information Systems 2013 Book Discussion Web Application Based on Django Framework with UI/IX Design for Senior Citizens and Inexperienced Users Gayane Kabalyan Grand Valley State University Follow this and additional works at: https://scholarworks.gvsu.edu/cistechlib ScholarWorks Citation Kabalyan, Gayane, "Book Discussion Web Application Based on Django Framework with UI/IX Design for Senior Citizens and Inexperienced Users" (2013). Technical Library. 149. https://scholarworks.gvsu.edu/cistechlib/149 This Project is brought to you for free and open access by the School of Computing and Information Systems at ScholarWorks@GVSU. It has been accepted for inclusion in Technical Library by an authorized administrator of ScholarWorks@GVSU. For more information, please contact [email protected]. Book Discussion Web Application Based on Django Framework with UI/UX Design for Senior Citizens and Inexperienced Users By Gayane Kabalyan April, 2013 Book Discussion Web Application Based on Django Framework with UI/UX Design for Senior Citizens and Inexperienced Users By Gayane Kabalyan A project submitted in partial fulfillment of the requirements for the degree of Master of Science in Computer Information Systems at Grand Valley State University April, 2013 _______________________________________________________________________________ Dr. Yonglei Tao Date 2 Table of Contents Abstract .............................................................................................................................
    [Show full text]