DOCSLIB.ORG

Standard Operating Procedure for a Common Operating Platform for Online 3D Modeling

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Standard Operating Procedure for a Common Operating Platform for Online 3D Modeling Standard Operating Procedure for a Common Operating Platform for Online 3D Modeling Prepared for: Ohio UAS Center Prepared by: University of Cincinnati Revision Date: April 20, 2020 Version Number: 2.0 Table of Contents List of Figures ................................................................................................................................. 3 Abstract ........................................................................................................................................... 4 Additional Reference Materials ...................................................................................................... 5 1.0 Purpose and Scope .................................................................................................................... 6 2.0 System Architecture .................................................................................................................. 6 2.1 The Web Interface................................................................................................................. 6 2.1.1. Graphical User Interface ............................................................................................... 7 2.1.2. Application Programming Interface ......................................................................... 7 2.2 Backend Systems .................................................................................................................. 8 3.0 Installation................................................................................................................................. 9 3.1 System Requirements............................................................................................................ 9 3.2 Configuring Software for the Common Operating Platform .............................................. 10 4.0 Usage.................................................................................................................................. 15 4.1 Register and Login .............................................................................................................. 15 4.2 Navigating the dashboard ................................................................................................... 16 4.3 Processing a 3D model on the Common Operating Platform ........................................ 17 4.4 Visualize a point cloud file ................................................................................................... 18 4.5 Change Password ................................................................................................................ 20 4.6 Create API Keys .................................................................................................................. 21 4.7 Sharing Content ................................................................................................................... 21 References ..................................................................................................................................... 23 2 | Page List of Figures Figure 1: Common Operating Platform System Architecture ........................................................ 6 3 | Page Abstract PLEASE UPDATE This Standard Operating Procedure (SOP) outlines the basic tools and procedures for utilizing small Unmanned Aircraft Systems (sUAS) for mapping construction sites and producing their accurate 3D model representation. This document is prepared for the Ohio UAS Center by the University of Cincinnati. This document intends to provide the Ohio Department of Transportation (ODOT) personnel, who are assigned responsibilities associated with the deployment and the use of Unmanned Aircraft System (UAS) information to map out construction sites and produce 3D models, orthomosaics and façades of the mapped-out area. This document will cover what hardware and software was used, the procedures to collect and process these images, and document case studies in areas of bridge and facilities inspection. Please note that this is document is still currently in a draft form and that images and tables have been taken from other locations to meet the needs of this document. 4 | Page Additional Reference Materials 1. Pix4D Mapper User Manual: LINK 5 | Page 1.0 Purpose and Scope This document is intended to inform the reader about the capabilities of the Common Operating Platform(v1.5) and define a standard operating procedure that covers usage, setup, and maintenance. In a nutshell, the common operating platform is a web-based service that is designed to process 3D models. The platform consists of a website, which is a frontend interface that can be used to create 3D modeling projects. A backend system serves this website and executes all of the processing that it entails. The idea is to have a secure platform, which is accessible to a broad audience, take care of all of their processing requirements and eliminates the need for an uninitiated audience to install highly technical software on their personal computers and maintain any sophisticated computing hardware. Sections 2 and 3 are meant for developers and system administrators who will want to understand how the platform works under the hood. Normal users who want to learn how to use the site should skip to section 4. 2.0 System Architecture The common operating platform consists of mainly two components. 1. A web interface which enables users to upload and interact data to the system 2. A backend processing system responsible for processing uploaded data The website was created using a Django backend with a Postgres database and runs Twitter bootstrap on the frontend. The backend processing system was implemented using a combination of Python, and Bash scripts. The common operating platform also facilitates API programming. The platforms HTTP based REST APIs have been used in tandem with the Microsoft HoloLens for visualizing 3D models (processed by the backend). The following sections delve deeper into the design and architecture of the system. Figure 1: Common Operating Platform System Architecture 2.1 The Web Interface There are two kinds of web interfaces available. The first one a graphical user interface (GUI) which is accessible using a web browser and the second one is an application programming 6 | Page interface (API) to be used by the devices or programs. The website interface was written in HTML, CSS, and JavaScript using Twitter’s bootstrap frontend interface library. J-Query and Ajax were used for implementing the functionality. 2.1.1. Graphical User Interface The standard graphical user interface (GUI) has a username/password authentication available on both interfaces. Users may register for an account using a sign-up page. The GUI has several different views which accomplish specific functionality. One such view is the virtual file system interface. It is akin to a PC’s desktop-like view of the different files and folders on a computer. The virtual file system is designed after a Unix based file system [1] and tries to emulate operations to store data in an orderly fashion online. The user has options to create files and folders as well as features to handle them like delete. For 3D model processing projects, users are required to create special project directories and upload their images into it. The project may be of two types based on the 3D model processing engine the user wants. They are Pix4DEngine Server [2] and OpenDroneMap [3]. The prior application is proprietary, and the latter is open sourced. Once the files have been uploaded, 3D model processing may be initiated by the user using the options provided on the website. Once the processing is initiated, the images are processed in the backend server using the chosen processing engine. The status of the processing can be seen in the backend jobs tab. This view consists of a history of all the jobs that have been processed by the system. All projects that the system processes will show up here along with its status. The view contains links to Potree [4], which is an open source online 3D point cloud viewer where the user can view the finished 3D model once it has been processed. There also links to the project directory which houses the images for the 3D model processing job. Apart from these two views, there is a settings view that provides two features. The first is the change password feature. The user is required to type his/her old password, followed by the new password to reset it. The second feature is the API key manager. An API key is a universally unique identifier (UUID) which must be used in the HTTP header of an API request to authenticate access to the backend server. The user may generate API keys for their automated programs or any devices which are required to query the backend server for information. Apart from creating keys, the option to disable API keys is also available. 2.1.2. Application Programming Interface The server also provides representational state transfer (REST)[5] application programming interfaces (API) for facilitating interoperability between different computer systems over a network. Responses to API queries are exclusively in JavaScript object notation (JSON) form and uses HTTP methods GET and POST. The API requests are authenticated using a universally unique identifier (UUID) API key. API keys are device specific and may be generated by the user using the options in the settings view. The API key should be attached to the header (X- API-Key) of the HTTP request. The different APIs are:- 1. Uploads
Load more

Recommended publications
  • Application of Unmanned Aerial Systems (UAS) to Quantify Biomass, Stem Volume, and Basal Area in a Mature Norway Spruce (Picea Abies) Plantation in Central New York
    SUNY College of Environmental Science and Forestry Digital Commons @ ESF Dissertations and Theses Spring 4-6-2018 Application of Unmanned Aerial Systems (UAS) to Quantify Biomass, Stem Volume, and Basal Area in a Mature Norway Spruce (Picea Abies) Plantation in Central New York Daniel Tinklepaugh [email protected] Follow this and additional works at: https://digitalcommons.esf.edu/etds Recommended Citation Tinklepaugh, Daniel, "Application of Unmanned Aerial Systems (UAS) to Quantify Biomass, Stem Volume, and Basal Area in a Mature Norway Spruce (Picea Abies) Plantation in Central New York" (2018). Dissertations and Theses. 44. https://digitalcommons.esf.edu/etds/44 This Open Access Thesis is brought to you for free and open access by Digital Commons @ ESF. It has been accepted for inclusion in Dissertations and Theses by an authorized administrator of Digital Commons @ ESF. For more information, please contact [email protected], [email protected]. APPLICATION OF UNMANNED AERIAL SYSTEMS (UAS) TO QUANTIFY BIOMASS, STEM VOLUME, AND BASAL AREA IN A MATURE NORWAY SPRUCE (PICEA ABIES) PLANTATION IN CENTRAL NEW YORK by Daniel Tinklepaugh A thesis submitted in partial fulfillment of the requirements for the Master of Science Degree State University of New York College of Environmental Science and Forestry Syracuse, New York April 2018 Division of Environmental Science Approved by: Eddie Bevilacqua, Major Professor Avik P. Chatterjee, Chair Examining Committee Russell Briggs, Director, Division of Environmental Science S. Scott Shannon, Dean, The Graduate School © 2018 Copyright D. Tinklepaugh All rights reserved ACKNOWLEDGEMENTS I’d like to thank those who have helped me during my time as a candidate and ultimate recipient of a Master’s Degree in Environmental Sciences during the past two years.
    [Show full text]
  • Enabling High-Performance Onboard Computing with Virtualization for Unmanned Aerial Systems
    2018 International Conference on Unmanned Aircraft Systems (ICUAS) Dallas, TX, USA, June 12-15, 2018 Enabling High-Performance Onboard Computing with Virtualization for Unmanned Aerial Systems Baoqian Wang and Junfei Xie Songwei Li and Yan Wan Shengli Fu Department of Computing Sciences Department of Electrical Engineering Department of Electrical Engineering Texas A&M University Corpus Christi University of Texas at Arlington University of North Texas Corpus Christi, TX, 78412 Arlington, TX, 76010 Denton, TX, 76203 Email:[email protected] Email: [email protected] Email: [email protected] Kejie Lu Department of Computer Science and Engineering University of Puerto Rico at Mayaguez¨ Mayaguez,¨ Puerto Rico, 00681 Email: [email protected] Abstract—In recent years, unmanned aerial systems (UAS) UAS are limited by the computing resources that they have attracted significant attentions because of their broad can carry. It is crucial to optimize the management of civilian and commercial applications. Nevertheless, most exist- constrained UAS computing resources, and offload less ing UAS platforms only have limited computing capabilities to time-critical tasks to other computing devices. This can be perform various delay-sensitive operations. To tackle this issue, achieved by virtualization, which is known for its powerful in this paper, we develop a high-performance onboard UAS resource management capabilities and live migration support computing platform with the virtualization technique. Specif- ically, we first discuss the selection of microcomputers that that handles dynamic workloads [5]. Virtualization has many are suitable for UAS onboard computing. We then investigate other attributes that can further enhance the capability of virtualization schemes that can effectively manage constrained UAS.
    [Show full text]
  • Fonafifo Geographic Information System Needs Assessment
    FONAFIFO GEOGRAPHIC INFORMATION SYSTEM NEEDS ASSESSMENT Submitted by: Richard J. Kotapish, GISP GISCorps Volunteer July 4, 2017 [Type here] TABLE OF CONTENTS Section Description Page Section 1: Introduction 3 Section 2: Existing Conditions Introduction and history 6 Software 7 Hardware 10 Network 13 GIS Data and GAP Analysis 14 Section 3: Department of Monitoring and Control Workflows Introduction 16 FONAFIFO PSA Workflows 16 Key Documents and Processes 17 Section 4: System Architecture / Technology Options Introduction 23 GIS Software Infrastructure: 23 Open Source, Esri and Cloud-based Considerations - GIS Software and Analysis Tools 26 Hardware Options 29 Unmanned Aerial Systems (UAS, UAV, Drones) 31 Remote Sensing 34 Metadata 34 Section 5: Costs, Benefits & Recommendations Introduction 36 Esri based solutions 36 Open Source GIS software 41 Mobile hardware solutions 42 Servers 42 UAS solutions 43 Shifted PSA Parcel Polygons 43 General Observations 45 Section 6: Next Steps Introduction 46 Task Series 46 Appendix “A” - Prioritized GIS Layers 49 Appendix “B” - Hardware Specifications 50 List of Figures Figure 1 - Existing FONAFIFO Operational Environment 14 Figure 2 - Simplified Geospatial Data Diagram 17 Figure 3 – PSA Application Workflow 19 Figure 4 – PSA Approval Process Workflow 19 Figure 5 - FONAFIFO Functional Environment 22 2 SECTION 1 INTRODUCTION 1.1 INTRODUCTION This GIS Needs Assessment Report is respectfully offered to the National Government of Costa Rica. This report provides various recommendations regarding the current and future use of Geographic Information System (GIS) technologies within the National Forestry Financing Fund (FONAFIFO). This GIS Needs Assessment Report is intended for your use in exploring new strategic directions for your GIS program.
    [Show full text]
  • Viswebdrone: a Web Application for UAV Photogrammetry Based on Open-Source Software
    International Journal of Geo-Information Article VisWebDrone: A Web Application for UAV Photogrammetry Based on Open-Source Software Nathalie Guimarães 1,2,* , Luís Pádua 1,3 , Telmo Adão 1,3 , Jonáš Hruška 1, Emanuel Peres 1,3 and Joaquim J. Sousa 1,3 1 Engineering Department, School of Science and Technology, University of Trás-os-Montes e Alto Douro, 5000-801 Vila Real, Portugal; [email protected] (L.P.); [email protected] (T.A.); [email protected] (J.H.); [email protected] (E.P.); [email protected] (J.J.S.) 2 Centre for the Research and Technology of Agro-Environmental and Biological Sciences, CITAB, Universidade de Trás-os-Montes e Alto Douro, UTAD, 5000-801 Vila Real, Portugal 3 Centre for Robotics in Industry and Intelligent Systems (CRIIS), INESC Technology and Science (INESC-TEC), 4200-465 Porto, Portugal * Correspondence: [email protected]; Tel.: +351-259-350-762 (ext. 4762) Received: 24 September 2020; Accepted: 13 November 2020; Published: 15 November 2020 Abstract: Currently, the use of free and open-source software is increasing. The flexibility, availability, and maturity of this software could be a key driver to develop useful and interesting solutions. In general, open-source solutions solve specific tasks that can replace commercial solutions, which are often very expensive. This is even more noticeable in areas requiring analysis and manipulation/visualization of a large volume of data. Considering that there is a major gap in the development of web applications for photogrammetric processing, based on open-source technologies that offer quality results, the application presented in this article is intended to explore this niche.
    [Show full text]
  • ODM) on Cpouta
    Install and Run Open Drone Map (ODM) on cPouta Step-by-step guide 2019 (Volume 1) Open Geospatial Information Infrastructure for Research (oGIIR) Consortium: Compiled and edited by: Augustine-Moses Gbagir (MSc.)1 Alfred Colpaert (Prof.)2 1,2Department of Geographical and Historical Studies, University of Eastern Finland. Contents CHAPTER 1: SETTING UP A VIRTUAL MACHINE ON CPOUTA .............................................. 3 cPouta virtual infrastructure ............................................................................................................. 3 The basic skills you need to use cPouta ........................................................................................... 3 Basic software you need .................................................................................................................. 3 How to set up your virtual machine (vm) on cPouta ....................................................................... 4 Step 1 Create keypair for your virtual machine(vm) ....................................................................... 4 Step 1.2 Modify, add password and authenticate keypair............................................................ 6 Step 1.3 Create security group for your virtual machine (vm) .................................................... 8 Step 1.4 Add security rules for your vm ...................................................................................... 9 Step 1.5 Creating our virtual machine (vm) ..............................................................................
    [Show full text]
  • Nuno Humberto Mendes Gonçalves Paula Controlo Multi-Drones Com
    Departamento de Eletrónica, Universidade de Aveiro Telecomunicações e Informática 2018 Nuno Humberto Controlo Multi-drones com Suporte a Missões Mendes Gonçalves Autónomas Paula Multi-drone Control with Autonomous Mission Support Departamento de Eletrónica, Universidade de Aveiro Telecomunicações e Informática 2018 Nuno Humberto Controlo Multi-drones com Suporte a Missões Mendes Gonçalves Autónomas Paula Multi-drone Control with Autonomous Mission Support “When you are developing a new technique, there are no recipes to copy, textbooks to consult, or manuals to read to pass on those little tips and secrets that guarantee success. You end up having to try any and every permutation of conditions and ingredients. You are never quite sure which of the many factors is really significant, how they act with and against one another, and so on. To sort out all those variables requires carefully designed trials. This is basic experimentation at its toughest and, if you succeed, at its best.” — John Craig Venter Departamento de Eletrónica, Universidade de Aveiro Telecomunicações e Informática 2018 Nuno Humberto Controlo Multi-drones com Suporte a Missões Mendes Gonçalves Autónomas Paula Multi-drone Control with Autonomous Mission Support Dissertação apresentada à Universidade de Aveiro para cumprimento dos re- quisitos necessários à obtenção do grau de Mestre em Engenharia de Com- putadores e Telemática, realizada sob a orientação científica da Doutora Su- sana Sargento, Professora Associada com Agregação do Departamento de Eletrónica, Telecomunicações e Informática da Universidade de Aveiro, e do Doutor André Braga Reis, investigador do Instituto de Telecomunicações. Apoio financeiro do POCTI no Apoio financeiro da FCT e do FSE âmbito do III Quadro Comunitário no âmbito do III Quadro Comuni- de Apoio.
    [Show full text]
  • Open Geospatial Software and Data: a Review of the Current State and a Perspective Into the Future
    International Journal of Geo-Information Review Open Geospatial Software and Data: A Review of the Current State and A Perspective into the Future Serena Coetzee 1,* , Ivana Ivánová 2 , Helena Mitasova 3 and Maria Antonia Brovelli 4 1 Department of Geography, Geoinformatics and Meteorology, University of Pretoria, Pretoria 0083, South Africa 2 Spatial Sciences, School of Earth and Planetary Science, Curtin University, Bentley, Perth 6102, Australia; [email protected] 3 Center for Geospatial Analytics, North Carolina State University, Raleigh, NC 27695, USA; [email protected] 4 Department of Civil and Environmental Engineering, Politecnico di Milano, P.zza Leonardo da Vinci 32, 20133 Milan, Italy; [email protected] * Correspondence: [email protected] Received: 23 October 2019; Accepted: 27 January 2020; Published: 1 February 2020 Abstract: All over the world, organizations are increasingly considering the adoption of open source software and open data. In the geospatial domain, this is no different, and the last few decades have seen significant advances in this regard. We review the current state of open source geospatial software, focusing on the Open Source Geospatial Foundation (OSGeo) software ecosystem and its communities, as well as three kinds of open geospatial data (collaboratively contributed, authoritative and scientific). The current state confirms that openness has changed the way in which geospatial data are collected, processed, analyzed, and visualized. A perspective on future developments, informed by responses from professionals in key organizations in the global geospatial community, suggests that open source geospatial software and open geospatial data are likely to have an even more profound impact in the future.
    [Show full text]
  • A Web-Based Software Platform for Data Processing Workflows and Its
    A Web-Based Software Platform for Data Processing Workflows and its Applications in Aerial Data Analysis A thesis submitted to the graduate school of the University of Cincinnati in partial fulfilment of the requirements for the degree of Master of Science in the department of Electrical Engineering and Computer Science of the College of Engineering and Applied Sciences June, 2019 by Niranjan R Krishnan B. Tech, Cochin University of Science and Technology, Cochin, India 2013 Committe Chair: Dr. Arthur Helmicki, PhD Committee Members: Dr. Victor Hunt, PhD Dr. Nan Niu, PhD Abstract Given the rapid advances in development of unmanned aerial vehicles (UAV), em- ployment of drones in various business functions becomes reasonable and afford- able. Usage of drones as data collection tools will give us access to a new set of geo-referenced images and videos that were not easily accessible in the past. The ultimate objective of this web based platform for data processing workflows, from here on referred to as the common operating platform, is to enable users to archive, process and visualize aerial data without a need for advanced hardware and software locally. This work details the development of the the common operating platform which consist of a web-based frontend and a backend. The frontend is a web app developed based on Django, Twitter Bootstrap and Javascript where the user authenticates, uploads data, submit processing tasks and visualizes the results. The back end, developed using Python 3, is where data is being stored and various processing tasks are being done based on commercial(Pix4D), open source(OpenDroneMap) and custom(Traffic Monitoring) processing engines.
    [Show full text]
  • Low Cost Remote Sensing and Navigational Methods
    LOW COST REMOTE SENSING AND NAVIGATIONAL METHODS FOR NOCTURNAL OPERATION OF COMMERCIAL UNMANNED AIRCRAFT SYSTEMS by JAMIE MICHAEL MOON PRASAD GOGINENI, COMMITTEE CHAIR CHARLES R. O’NEILL, CO-CHAIR PAUL HUBNER JACOB CHAKARESKI A THESIS Submitted in partial fulfillment of the requirements for the degree of Master of Science in the Department of Aerospace Engineering and Mechanics in the Graduate School of the University of Alabama TUSCALOOSA, ALABAMA 2020 Copyright Jamie Michael Moon 2020 ALL RIGHTS RESERVED ABSTRACT This thesis addresses the development of low-cost remote-sensing and navigational techniques for commercial and private small, unmanned aircraft systems (sUAS) to support a multimodal surveying platform capable of navigating in little to no light. The research in this thesis particularly focuses upon systems that would be beneficial for an sUAS equipped with a multimodal sensor suite for visible and thermal remote sensing. The research seeks to develop a low-cost data collection workflow for gathering visible light and thermal orthomosaic maps and digital elevation models (DEM’s) without relying on expensive software packages. A series of commercial and open-source image reconstruction software packages are compared and augmented with custom tools to support a low-cost aerial mapping and data analysis technique. Additionally, this research explores alternative sensors to enable enhanced obstacle avoidance in complex flight environments in little to no light. Two novel sensor prototypes are discussed including a concept for an ultrasonic thermometer as well as a laser-projection system for detecting thin features. This project seeks to present the building blocks for a hypothetical UAV system capable of rapidly collecting a large amount of aerial imagery and thermal data in a dark environment.
    [Show full text]
  • Implementation of Distributed Cloud System Architecture Using Advanced Container Orchestration, Cloud Storage, and Centralized Database for a Web-Based Platform
    Implementation of distributed cloud system architecture using advanced container orchestration, cloud storage, and centralized database for a web-based platform A thesis submitted to the Graduate School of the University of Cincinnati in partial fulfillment of the requirements for the degree of Master of Science In the Department of Electrical Engineering and Computer Science Of the College of Engineering and Applied Science By Sohan Karkera Bachelor of Technology, University of Mumbai June 2018 Thesis Advisor and Committee Chair: Arthur J. Helmicki, Ph.D. Committee Members: Victor J. Hunt, Ph.D. and Nan Niu, Ph.D. University of Cincinnati, December 2020 Abstract In the case of a distributed system based architecture, the cost to maintain an application might be higher, as the components of the system are distributed across various physical locations. Also, there is repeatedly an issue of latency between components that comes into effect. To resolve this issue cloud computing has played a significant role. Due to rapid technological advances being made, it has become more economical and easier to manage these components by substituting them with the various options available with cloud computing vendors which offer lower latency and dashboards for easy maintenance. Moreover, with the emergence of container- based and orchestration-based technologies, it has become possible to easily manage and deploy these applications by packaging them inside containers. This research focuses on using these advanced cloud computing, container and orchestration-based technologies to improve the scalability of an existing web appli- cation from a single-server-based architecture to a multi-server-based architecture. The application being considered in this research work is the Common Operating Platform (COP).
    [Show full text]
  • Produção De Ortomapas Com Vants E Opendronemap 05
    ISSN 1677-8480 CIRCULAR TÉCNICA Produção de ortomapas 05 com VANTs e OpenDroneMap Campinas, SP Thiago Teixeira Santos Dezembro, 2018 Luciano Vieira Koenigkan 2 Circular Técnica 05 Produção de ortomapas com VANTs e OpenDroneMap1 1. Introdução Veículos Aéreos Não-Tripulados (VANTs) têm sido utilizados como alternativa para sensoriamento remoto em agricultura (Jorge; Inamasu, 2014). Ortomapas produzidos com o uso de VANTs podem apresentar resolução superior à de satélites, com Ground Sample Distance (GSD) inferior a 1 pixel/cm2. VANTs podem sensorear áreas com maior frequência temporal, de acordo com a disponibilidade do operador de voo, do equipamento e da necessidade da aplicação, e não sofrem do problema de oclusão por nuvens, que acomete o sensoriamento por satélite. Equipes interessadas em sensoriamento por VANTs geralmente necessitam de produtos digitais como mapas ortorretificados, modelos georreferenciados de superfície e nuvens de pontos 3-D. Tais dados podem ser utilizados em diversas aplicações para agricultura de precisão, conservação ambiental e aplicações de Geographic Information System (GIS) em geral. Atualmente, diversas empresas oferecem esses produtos na forma de serviços. Contudo, o valor das assinaturas cobradas pode ser proibitivo para diversos usuários interessados, incluindo grupos de pesquisa, extensionistas e pequenos produtores rurais. Além do custo, outros requisitos podem ser de importância aos usuários, como a privacidade dos dados ou a necessidade de maior controle e personalização do pipeline de sensoriamento. OpenDroneMap (ODM)3 é um sistema de código aberto para a geração de produtos digitais georreferenciados a partir de imagens aéreas como as obtidas por VANTs. Ele é composto por diversos módulos integrados que realizam as etapas necessárias a um pipeline de fotogrametria.
    [Show full text]
  • Geocomputing Using CSC Resources Intro to CSC Computing Services
    Geocomputing using CSC resources 24-25.10.2018 1 Geocomputing using CSC resources Kylli Ek, Elias Annila, Eduardo Gonzalez 24-25.10.2018, Espoo CSC – Finnish expertise in ICT for research, education, culture and public administration Intro to CSC computing services Geocomputing using CSC resources 24-25.10.2018 2 Reasons for using CSC computing resources • Computing something takes more than 2-4 hours • Need for more memory • Very big datasets • Keep your desktop computer for normal usage, do computation elsewhere • Need for a server computer • Need for a lot of computers with the same set-up (courses) • Convenient to use preinstalled and maintained software • Free for Finnish university and for state research insitute users CSC main computing resourcs for GIS users • Taito supercluster • cPouta cloud oPre-installed software oThe user sets up the environment oSome GIS data available oMore freedom, and more responsibilities • Before using: • Before using: oMove your data and scripts oSetting up the computer oSmall extra adjustments for running your oInstallationof software scripts oMove your data and scripts Geocomputing using CSC resources 24-25.10.2018 3 The keys to geocomputing: Change in working style & Linux Scripts GUI ArcGIS, QGIS, … R, Python, shellscripts, Matlab, … 5 CSC HPC resources Max Max 672 cores / Max 4 cores / Max 48 cores / 2-4 cores / 9600 cores 1536 GB memory 128 GB memory 240 GB memory 8 GB memory Jobs via Jobs via Interactive SLURM SLURM usage For well-scaling Users build parallel jobs GPU GUI their own environment,
    [Show full text]