DOCSLIB.ORG

Tethys Platform: a Development and Hosting Platform for Water Resources Web Apps

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Tethys Platform: a Development and Hosting Platform for Water Resources Web Apps Brigham Young University BYU ScholarsArchive Theses and Dissertations 2015-06-01 Tethys Platform: A Development and Hosting Platform for Water Resources Web Apps Nathan R. Swain Brigham Young University - Provo Follow this and additional works at: https://scholarsarchive.byu.edu/etd Part of the Civil and Environmental Engineering Commons BYU ScholarsArchive Citation Swain, Nathan R., "Tethys Platform: A Development and Hosting Platform for Water Resources Web Apps" (2015). Theses and Dissertations. 5832. https://scholarsarchive.byu.edu/etd/5832 This Dissertation is brought to you for free and open access by BYU ScholarsArchive. It has been accepted for inclusion in Theses and Dissertations by an authorized administrator of BYU ScholarsArchive. For more information, please contact [email protected], [email protected]. Tethys Platform: A Development and Hosting Platform for Water Resources Web Apps Nathan R. Swain A dissertation submitted to the faculty of Brigham Young University in partial fulfillment of the requirements for the degree of Doctor of Philosophy E. James Nelson, Chair Norman L. Jones Daniel P. Ames Gustavious P. Williams A. Woodruff Miller Department of Civil and Environmental Engineering Brigham Young University June 2015 Copyright © 2015 Nathan R. Swain All Rights Reserved ABSTRACT Tethys Platform: A Development and Hosting Platform for Water Resources Web Apps Nathan R. Swain Department of Civil and Environmental Engineering, BYU Doctor of Philosophy The interactive nature of web applications or “web apps” makes it an excellent medium for conveying complex scientific concepts to lay audiences and creating decision support tools that harness cutting edge modeling techniques. However, the technical expertise required to develop them represents a barrier for would-be developers. The barrier can be characterized by the following hurdles that developers must overcome: (1) identify, select, and install software that meet the spatial and computational capabilities commonly required for water resources modeling; (2) orchestrate the use of multiple FOSS and FOSS4G projects and navigate their differing application programming interfaces (APIs); (3) learn the multi-language programming skills required for modern web development; and (4) develop a web-safe and fully featured web site to host the app. This research has resulted in two primary products that effectively lower the barrier to water resources web app development: (1) a literature review of free and open source software (i.e. software review) and (2) Tethys Platform. The software review included earth science web apps that were published in the peer-reviewed literature in the last decade and it was performed to determine which FOSS4G and FOSS web software has been used to develop such web apps. The review highlights 11 FOSS4G software projects and 9 FOSS projects for web development that were used to develop 45 earth sciences web apps, which constitutes a significantly reduced list of possible software projects that could be used to meet the needs of water resources web app development—greatly lowering the barrier for entry to water resources web development. While the software review addresses the hurdle of identifying FOSS software to provide a web framework and spatial data capabilities for water resources web apps, there are still other hurdles that needed to be overcome to make development more viable. Tethys Platform was developed to address these other hurdles and streamline the development of water resources web apps. It includes (1) a suite of free and open source software that address the unique data and computational needs common to water resources web app development, (2) a Python software development kit for incorporating the functionality of each software element into web apps and streamlining their development, and (3) a customizable web portal that is used to deploy the completed web apps. Tethys Platform has been used to develop a broad array of web apps for water resources modeling and decision support. Keywords: Tethys Platform, water resources, modeling, software development kit, web app ACKNOWLEDGEMENTS At the outset of my doctoral degree and my involvement with CI-WATER I could not have anticipated the experiences that I would have or the ways in which I would grow both as a professional and as an individual. Though I fully expected to develop bleeding-edge technologies, I did not see Tethys Platform on the horizon or the overwhelming interest and response to this work. I am overcome with gratitude for the many people who afforded me this opportunity. I would like to thank my advisors, Jim Nelson and Norm Jones. They provided a compelling vision that inspired my efforts and they provided me the latitude to explore many different avenues that eventually led to Tethys Platform. I would also like to acknowledge Dan Ames for his guidance and especially for his recommendation to port the project from CKAN to Django—a decision that was critical for the longevity of the project. I would also like to thank Gus Williams and Wood Miller for their input and feedback as members of my committee. I am indebted to Scott Christensen for his patience and invaluable advice. I would rarely implement a new feature or move forward with a plan until I had pitched the idea to Scott, because he always helped me root out the flaws and perfect the ideas. I would like to thank Jocelynn Anderson as the first Tethys app developer. Her work on the GSSHA Index Map Editor app was a driver for the development of many features. I am also grateful for Alan Snow and his work on the Streamflow Prediction Tool app, which put wind under the wings of Tethys Platform affording it national attention. I also want to recognize the efforts of the other early Tethys app developers on the CI-WATER and HydroShare teams. Tethys Platform would be nothing without the people who use it. I also need to recognize my parents for instilling in me an attitude of open mindedness and the confidence to reach beyond myself. Lastly, I would be remiss if I didn’t recognize my wife, Sunni. She has been a web developer much longer than I have, which was a large motivation behind my decision to embark on the CI-WATER project. In addition to countless acts of selfless love and support, she provided invaluable advice as I learned the ins and outs of web development. I would not have been able to complete or even participate in this project without her support. This material is based upon work supported by the National Science Foundation under Grant No. 1135483. TABLE OF CONTENTS LIST OF TABLES ..................................................................................................................... viii LIST OF FIGURES ..................................................................................................................... ix 1 Introduction ............................................................................................................................. 1 2 Software Review ...................................................................................................................... 6 2.1 Web GIS Software Review ................................................................................................ 7 2.1.1 Spatial Databases ........................................................................................................ 8 2.1.2 Spatial Data Publishing ............................................................................................. 11 2.1.3 Mapping Libraries ..................................................................................................... 13 2.1.4 Spatial Analysis ........................................................................................................ 16 2.2 Web Development Software Review ............................................................................... 18 2.2.1 Programming Languages .......................................................................................... 20 2.2.2 Web Frameworks ...................................................................................................... 21 2.2.3 Content Management Systems .................................................................................. 23 3 Tethys Platform ..................................................................................................................... 25 3.1 Tethys Software Suite ...................................................................................................... 26 3.1.1 Spatial Database ........................................................................................................ 27 3.1.2 Geoprocessing ........................................................................................................... 27 3.1.3 Map Rendering .......................................................................................................... 28 3.1.4 Visualization ............................................................................................................. 28 3.1.5 Distributed Computing .............................................................................................. 29 3.1.6 File Dataset Storage .................................................................................................. 30 3.1.7 Docker Installation .................................................................................................... 30 3.2 Tethys Software Development Kit .................................................................................
Load more

Recommended publications
  • Build It with Nitrogen the Fast-Off-The-Block Erlang Web Framework
    Build it with Nitrogen The fast-off-the-block Erlang web framework Lloyd R. Prentice & Jesse Gumm dedicated to: Laurie, love of my life— Lloyd Jackie, my best half — Jesse and to: Rusty Klophaus and other giants of Open Source— LRP & JG Contents I. Frying Pan to Fire5 1. You want me to build what?7 2. Enter the lion’s den9 2.1. The big picture........................ 10 2.2. Install Nitrogen........................ 11 2.3. Lay of the land........................ 13 II. Projects 19 3. nitroBoard I 21 3.1. Plan of attack......................... 21 3.2. Create a new project..................... 23 3.3. Prototype welcome page................... 27 3.4. Anatomy of a page...................... 30 3.5. Anatomy of a route...................... 33 3.6. Anatomy of a template.................... 34 3.7. Elements............................ 35 3.8. Actions............................. 38 3.9. Triggers and Targets..................... 39 3.10. Enough theory........................ 40 i 3.11. Visitors............................ 44 3.12. Styling............................. 64 3.13. Debugging........................... 66 3.14. What you’ve learned..................... 66 3.15. Think and do......................... 68 4. nitroBoard II 69 4.1. Plan of attack......................... 69 4.2. Associates........................... 70 4.3. I am in/I am out....................... 78 4.4. Styling............................. 81 4.5. What you’ve learned..................... 82 4.6. Think and do......................... 82 5. A Simple Login System 83 5.1. Getting Started........................ 83 5.2. Dependencies......................... 84 5.2.1. Rebar Dependency: erlpass ............. 84 5.3. The index page........................ 85 5.4. Creating an account..................... 87 5.4.1. db_login module................... 89 5.5. The login form........................ 91 5.5.1.
    [Show full text]
  • Pyqgis Developer Cookbook Release 2.18
    PyQGIS developer cookbook Release 2.18 QGIS Project April 08, 2019 Contents 1 Introduction 1 1.1 Run Python code when QGIS starts.................................1 1.2 Python Console............................................2 1.3 Python Plugins............................................3 1.4 Python Applications.........................................3 2 Loading Projects 7 3 Loading Layers 9 3.1 Vector Layers.............................................9 3.2 Raster Layers............................................. 11 3.3 Map Layer Registry......................................... 11 4 Using Raster Layers 13 4.1 Layer Details............................................. 13 4.2 Renderer............................................... 13 4.3 Refreshing Layers.......................................... 15 4.4 Query Values............................................. 15 5 Using Vector Layers 17 5.1 Retrieving information about attributes............................... 17 5.2 Selecting features........................................... 18 5.3 Iterating over Vector Layer...................................... 18 5.4 Modifying Vector Layers....................................... 20 5.5 Modifying Vector Layers with an Editing Buffer.......................... 22 5.6 Using Spatial Index......................................... 23 5.7 Writing Vector Layers........................................ 23 5.8 Memory Provider........................................... 24 5.9 Appearance (Symbology) of Vector Layers............................. 26 5.10 Further
    [Show full text]
  • Civil Good: a Platform for Sustainable and Inclusive Online Discussion
    Civil Good: A Platform For Sustainable and Inclusive Online Discussion An Interactive Qualifying Project submitted to the faculty of Worcester Polytechnic Institute In partial fulfillment of the requirements for the degree of Bachelor of Science by: Steven Malis (Computer Science), Tushar Narayan (Computer Science), Ian Naval (Computer Science), Thomas O'Connor (Biochemistry), Michael Perrone (Physics and Mathematics), John Pham (Computer Science), David Pounds (Computer Science and Robotics Engineering), December 19, 2013 Submitted to: Professor Craig Shue, WPI Advisor Alan Mandel, Creator of the Civil Good concept Contents 1 Executive Summary1 1.1 Overview of Recommendations......................2 2 Authorship5 3 Introduction 10 3.1 Existing Work - Similar Websites.................... 11 4 Psychology 17 4.1 Online Disinhibition........................... 17 4.2 Format of Discussions.......................... 22 4.3 Reducing Bias with Self-Affirmation................... 28 4.4 Other Psychological Influences...................... 34 5 Legal Issues 38 5.1 Personally Identifiable Information................... 38 5.2 Intellectual Property........................... 42 5.3 Defamation................................ 45 5.4 Information Requests........................... 46 5.5 Use by Minors............................... 49 5.6 General Litigation Avoidance and Defense............... 51 6 Societal Impact 52 6.1 Political Polarization........................... 52 6.2 Minority Opinion Representation.................... 55 6.3 History and Political
    [Show full text]
  • Making Story from System Logs with Elastic Stack
    SANOG36 18 - 21 January, 2021 Making story from system logs with stack [email protected] https://imtiazrahman.com https://github.com/imtiazrahman Logs syslog Audit SNMP NETFLOW http METRIC DNS ids What is Elastic Stack ? Store, Analyze Ingest User Interface a full-text based, distributed NoSQL database. Written in Java, built on Apache Lucene Commonly used for log analytics, full-text search, security intelligence, business analytics, and operational intelligence use cases. Use REST API (GET, PUT, POST, and DELETE ) for storing and searching data Data is stored as documents (rows in relational database) Data is separated into fields (columns in relational database) Relational Database Elasticsearch Database Index Table Type Row/Record Document Column Name Field Terminology Cluster: A cluster consists of one or more nodes which share the same cluster name. Node: A node is a running instance of elasticsearch which belongs to a cluster. Terminology Index: Collection of documents Shard: An index is split into elements known as shards that are distributed across multiple nodes. There are two types of shard, Primary and replica. By default elasticsearch creates 1 primary shard and 1 replica shard for each index. Terminology Shard 1 Replica 1 Replica 2 Shard 2 Node 1 Node 2 cluster Terminology Documents { • Indices hold documents in "_index": "netflow-2020.10.08", "_type": "_doc", serialized JSON objects "_id": "ZwkiB3UBULotwSOX3Bdb", "_version": 1, • 1 document = 1 log entry "_score": null, "_source": { • Contains "field : value" pairs
    [Show full text]
  • Learning Javascript Design Patterns
    Learning JavaScript Design Patterns Addy Osmani Beijing • Cambridge • Farnham • Köln • Sebastopol • Tokyo Learning JavaScript Design Patterns by Addy Osmani Copyright © 2012 Addy Osmani. All rights reserved. Revision History for the : 2012-05-01 Early release revision 1 See http://oreilly.com/catalog/errata.csp?isbn=9781449331818 for release details. ISBN: 978-1-449-33181-8 1335906805 Table of Contents Preface ..................................................................... ix 1. Introduction ........................................................... 1 2. What is a Pattern? ...................................................... 3 We already use patterns everyday 4 3. 'Pattern'-ity Testing, Proto-Patterns & The Rule Of Three ...................... 7 4. The Structure Of A Design Pattern ......................................... 9 5. Writing Design Patterns ................................................. 11 6. Anti-Patterns ......................................................... 13 7. Categories Of Design Pattern ............................................ 15 Creational Design Patterns 15 Structural Design Patterns 16 Behavioral Design Patterns 16 8. Design Pattern Categorization ........................................... 17 A brief note on classes 17 9. JavaScript Design Patterns .............................................. 21 The Creational Pattern 22 The Constructor Pattern 23 Basic Constructors 23 Constructors With Prototypes 24 The Singleton Pattern 24 The Module Pattern 27 iii Modules 27 Object Literals 27 The Module Pattern
    [Show full text]
  • Zope Documentation Release 5.3
    Zope Documentation Release 5.3 The Zope developer community Jul 31, 2021 Contents 1 What’s new in Zope 3 1.1 What’s new in Zope 5..........................................4 1.2 What’s new in Zope 4..........................................4 2 Installing Zope 11 2.1 Prerequisites............................................... 11 2.2 Installing Zope with zc.buildout .................................. 12 2.3 Installing Zope with pip ........................................ 13 2.4 Building the documentation with Sphinx ............................... 14 3 Configuring and Running Zope 15 3.1 Creating a Zope instance......................................... 16 3.2 Filesystem Permissions......................................... 17 3.3 Configuring Zope............................................. 17 3.4 Running Zope.............................................. 18 3.5 Running Zope (plone.recipe.zope2instance install)........................... 20 3.6 Logging In To Zope........................................... 21 3.7 Special access user accounts....................................... 22 3.8 Troubleshooting............................................. 22 3.9 Using alternative WSGI server software................................. 22 3.10 Debugging Zope applications under WSGI............................... 26 3.11 Zope configuration reference....................................... 27 4 Migrating between Zope versions 37 4.1 From Zope 2 to Zope 4 or 5....................................... 37 4.2 Migration from Zope 4 to Zope 5.0..................................
    [Show full text]
  • ELK: a Log Files Management Framework
    ELK: a log files management framework Giovanni Bechis <[email protected]> LinuxCon Europe 2016 About Me I sys admin and developer @SNB I OpenBSD developer I Open Source developer in several other projects searching through log files, the old way $ man 1 pflogsumm $ grep [email protected] /var/log/maillog | awk '{print $1 "-" $2 "-" $3;}' $ grep -e 'from=.*@gmail\.com' /var/log/maillog | grep "550" \ | awk {'print $1 "-" $2 "-" $3 " " $7 " " $10 " " $11 " " $13;}' $ vi logparser.sh $ git clone https://github.com/random/parser_that_should_work $ man 1 perltoc $ man 1 python searching through log files, the old way $ cssh -a 'mylogparser.py' host1 host2 host3 host4 | tee -a /tmp/parsedlogs.txt $ man syslogd(8) searching through log files, the new way ELK open source components I Beats: collect, parse and ship I Logstash: collect, enrich and transport data I Elasticsearch: search and analyze data in real time I Kibana: explore and visualize your data ELK closed source components I Watcher: alerting for Elasticsearch I Shield: security for Elasticsearch I Marvel: monitor Elasticsearch I Graph: analyze relationships Elasticsearch I open source search engine based on lucene library I nosql database (document oriented) I queries are based on http/json I APIs for lot of common languages, (or you can write your own framework, is just plain http and json) Elasticsearch: security I not available in open source version, you need Shield I Elasticsearch should not be exposed on the wild, use firewalling to protect your instances I manage security on your
    [Show full text]
  • Revealing Injection Vulnerabilities by Leveraging Existing Tests
    Revealing Injection Vulnerabilities by Leveraging Existing Tests Katherine Hough1, Gebrehiwet Welearegai2, Christian Hammer2 and Jonathan Bell1 1George Mason University, Fairfax, VA, USA 2University of Potsdam, Potsdam, Germany [email protected],[email protected],[email protected],[email protected] Abstract just one of over 8,200 similar code injection exploits discovered in Code injection attacks, like the one used in the high-prole 2017 recent years in popular software [44]. Code injection vulnerabilities Equifax breach, have become increasingly common, now ranking have been exploited in repeated attacks on US election systems [10, #1 on OWASP’s list of critical web application vulnerabilities. Static 18, 39, 61], in the theft of sensitive nancial data [56], and in the analyses for detecting these vulnerabilities can overwhelm develop- theft of millions of credit card numbers [33]. In the past several ers with false positive reports. Meanwhile, most dynamic analyses years, code injection attacks have persistently ranked at the top rely on detecting vulnerabilities as they occur in the eld, which of the Open Web Application Security Project (OWASP) top ten can introduce a high performance overhead in production code. most dangerous web aws [46]. Injection attacks can be damaging This paper describes a new approach for detecting injection vul- even for applications that are not traditionally considered critical nerabilities in applications by harnessing the combined power of targets, such as personal websites, because attackers can use them human developers’ test suites and automated dynamic analysis. as footholds to launch more complicated attacks. Our new approach, Rivulet, monitors the execution of developer- In a code injection attack, an adversary crafts a malicious in- written functional tests in order to detect information ows that put that gets interpreted by the application as code rather than may be vulnerable to attack.
    [Show full text]
  • Spatialist Documentation
    spatialist Documentation John Truckenbrodt, Felix Cremer, Ismail Baris Jul 17, 2020 Contents 1 Installation 1 1.1 Installation of dependencies.......................................1 1.2 Installation of spatialist..........................................2 2 API Documentation 3 2.1 Raster Class...............................................3 2.2 Raster Tools...............................................8 2.3 Vector Class............................................... 12 2.4 Vector Tools............................................... 16 2.5 General Spatial Tools........................................... 18 2.6 Database Tools.............................................. 21 2.7 Ancillary Functions........................................... 21 2.8 ENVI HDR file manipulation...................................... 24 2.9 Data Exploration............................................. 25 3 Some general examples 27 3.1 in-memory vector object rasterization.................................. 27 4 Changelog 29 4.1 v0.4.................................................... 29 4.2 v0.5.................................................... 29 4.3 v0.6.................................................... 30 5 Indices and tables 31 Python Module Index 33 Index 35 i ii CHAPTER 1 Installation The most convenient way to install spatialist is by using conda: conda install --channel conda-forge spatialist See below for more detailed Linux installation instructions outside of the Anaconda framework. 1.1 Installation of dependencies 1.1.1 GDAL spatialist requires
    [Show full text]
  • SWE 642 Software Engineering for the World Wide Web
    SWE 642 Software Engineering for the World Wide Web Fall Semester, 2017 Location: AB L008 Time: Tuesday 7:20-10:00pm Instructor Overview Textbook and Readings Grading Schedule Academic Integrity Professor: Dr. Vinod Dubey Email: [email protected] Class Hours: Tuesday 7:20-10:00, AB L008 Prerequisite: SWE 619 and SWE Foundation material or (CS 540 and 571) Office Hours: Anytime electronically, or by an appointment Teaching Assistant: Mr. Shravan Hyderabad, [email protected] Overview OBJECTIVE: After completing the course, students will understand the concepts and have the knowledge of how web applications are designed and constructed. Students will be able to engineer high quality building blocks for Web applications. CONTENT: Detailed study of the engineering methods and technologies for building highly interactive web sites for e-commerce and other web-based applications. Engineering principles for building web sites that exhibit high reliability, usability, security, availability, scalability and maintainability are presented. Methods such as client-server programming, component-based software development, middleware, and reusable components are covered. After the course, students should be prepared to create software for large-scale web sites. SWE 642 teaches some of the topics related to the exciting software development models that are used to support web and e-commerce applications. We will be studying the software design and development side of web applications, rather than the policy, business, or networking sides. An introductory level knowledge of HTML and Java is required. SWE 619 is a required prerequisite and SWE 632 is a good background course. The class will be very practical (how to build things) and require several programming assignments.
    [Show full text]
  • Strange History
    A STRANGE HISTORY. A DRAMATIC TALE, IN EIGHT CHAPTERS. BY MESSRS. SLINGSBY LAWRENCE AND CHARLES MATHEWS, Authors of " A Chain of Events," &c. &c. THOMAS HAILES LACY, WELLINGTON STREET, STRAND, LONDON. First Performed at the Royal Lyceum Theatre, on Easter Monday, March 29th, 1853. CHARACTERS. JEROME LEVERD ..................................... Mr. CHARLES MATHEWS. LEGROS Mr. FRANK MATTHEWS. NICOLAS Mr. ROBERT ROXBY. MAURICE BELLISLE ................................ Mr. COOPER. DOMINIQUE Mr. JAMES BLAND. ALFRED DE MIRECOUR ..................... Mr. BELTON. JEAN BRIGARD ............................. Mr. BASIL BAKER. AMEDEE Mr. ROSIERE. HECTOR DE BEAUSIRE ...................... Mr. H. BUTLER. CAPTAIN OF GENDARMES ................. Mr. ABBOTT. PIERRE Mr. HENRY. CHRISTINE Madame VESTRIS. MADAME LEGROS .......................................... Mrs. FRANK MATTHEWS. NICOTTE Miss JULIA ST. GEORGE. ESTELLE Miss M. OLIVER. COUNTESS DE MIRECOUR ................... Mrs. HORN. MANETTE Miss MASON. MARGUERITE Miss WADHAM. Time in Representation, Three Hours. If performed in Five Acts, Two Hours and Twenty Minutes. COSTUMES. In the first three chapters Tyrolean ; and French soldiers. In the rest, Breton peasants and French gentlemen of 1810-15. JEROME.—1st Dress—Peasant's brown jacket, black sleeves, red vest; full brown trunks i grey leggings ; shoes ; felt Swiss hat and feather ; over-coat for the mountains, brown camels hair, trimmed with fur. 2nd Dress—Brown open jacket, without sleeves ; holland shirt. 3rd Dress—Light blue open jacket, slashed sleeves of brown ; red em- broidered vest; light drab full trunks, all trimmed with white and coloured gimp ; white buttons ; striped silk sash. 4th Dress—Long blue dress coat (Paris cut), gilt buttons; white vest, trimmed with white fringe; white silk stockings ; short blue breeches ; Hessian boots, &c.; sugar-loaf hat; long hair.
    [Show full text]
  • Comparing Performance of Plain PHP and Four of Its Popular Frameworks
    Thesis Project Comparing Performance of Plain PHP and Four of Its Popular Frameworks Author: Jone Samra Supervisor: Johan Hagelbäck Examiner: Sabri Pllana Semester: VT 2015 Subject: Computer Science Abstract The objective of this study is to evaluate performance of four popular PHP frameworks Laravel, Symfony, CodeIgniter and Phalcon together with the plain PHP. The decision of making this study was based on the fact that there is a lack of comparison tests between the most popular PHP frameworks. Visiting the official websites of these frameworks, the first thing to notice is the slogans that have been made by the core teams. The majority of these slogans contain quality attributes like speed and high performance. As a developer looking for performance in your next project, choosing the right PHP framework by these slogans is not that easy. Therefor, the performance of these frameworks is put to the test. And to do that, three experiments are conducted in which five functionally equivalent PHP applications are developed and used as targets. One version of these applications represents the plain PHP and the other four represent the four mentioned frameworks. The experiments are conducted in two sessions. The first session deals with the execution time and the stack trace measurements while the second one is covering the measurement of the memory usage consumption. The result outcome of these experiments has been analyzed and interpreted in order to expose the performance of the targeted frameworks. The experiment results prove that the targeted frameworks perform differently compared with each other and the PHP stack. Plain PHP and Phalcon are performing well while the other three frameworks have both mediocre and low performance.
    [Show full text]