Supporting Distributed Visualization Services for High Performance Science and Engineering Applications – a Service Provider Perspective
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
THINC: a Virtual and Remote Display Architecture for Desktop Computing and Mobile Devices
THINC: A Virtual and Remote Display Architecture for Desktop Computing and Mobile Devices Ricardo A. Baratto Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Graduate School of Arts and Sciences COLUMBIA UNIVERSITY 2011 c 2011 Ricardo A. Baratto This work may be used in accordance with Creative Commons, Attribution-NonCommercial-NoDerivs License. For more information about that license, see http://creativecommons.org/licenses/by-nc-nd/3.0/. For other uses, please contact the author. ABSTRACT THINC: A Virtual and Remote Display Architecture for Desktop Computing and Mobile Devices Ricardo A. Baratto THINC is a new virtual and remote display architecture for desktop computing. It has been designed to address the limitations and performance shortcomings of existing remote display technology, and to provide a building block around which novel desktop architectures can be built. THINC is architected around the notion of a virtual display device driver, a software-only component that behaves like a traditional device driver, but instead of managing specific hardware, enables desktop input and output to be intercepted, manipulated, and redirected at will. On top of this architecture, THINC introduces a simple, low-level, device-independent representation of display changes, and a number of novel optimizations and techniques to perform efficient interception and redirection of display output. This dissertation presents the design and implementation of THINC. It also intro- duces a number of novel systems which build upon THINC's architecture to provide new and improved desktop computing services. The contributions of this dissertation are as follows: • A high performance remote display system for LAN and WAN environments. -
Virtualgl / Turbovnc Survey Results Version 1, 3/17/2008 -- the Virtualgl Project
VirtualGL / TurboVNC Survey Results Version 1, 3/17/2008 -- The VirtualGL Project This report and all associated illustrations are licensed under the Creative Commons Attribution 3.0 License. Any works which contain material derived from this document must cite The VirtualGL Project as the source of the material and list the current URL for the VirtualGL web site. Between December, 2007 and March, 2008, a survey of the VirtualGL community was conducted to ascertain which features and platforms were of interest to current and future users of VirtualGL and TurboVNC. The larger purpose of this survey was to steer the future development of VirtualGL and TurboVNC based on user input. 1 Statistics 49 users responded to the survey, with 32 complete responses. When listing percentage breakdowns for each response to a question, this report computes the percentages relative to the total number of complete responses for that question. 2 Responses 2.1 Server Platform “Please select the server platform(s) that you currently use or plan to use with VirtualGL/TurboVNC” Platform Number of Respondees (%) Linux/x86 25 / 46 (54%) ● Enterprise Linux 3 (x86) 2 / 46 (4.3%) ● Enterprise Linux 4 (x86) 5 / 46 (11%) ● Enterprise Linux 5 (x86) 6 / 46 (13%) ● Fedora Core 4 (x86) 1 / 46 (2.2%) ● Fedora Core 7 (x86) 1 / 46 (2.2%) ● Fedora Core 8 (x86) 4 / 46 (8.7%) ● SuSE Linux Enterprise 9 (x86) 1 / 46 (2.2%) 1 Platform Number of Respondees (%) ● SuSE Linux Enterprise 10 (x86) 2 / 46 (4.3%) ● Ubuntu (x86) 7 / 46 (15%) ● Debian (x86) 5 / 46 (11%) ● Gentoo (x86) 1 / -
Release 0.11 Todd Gamblin
Spack Documentation Release 0.11 Todd Gamblin Feb 07, 2018 Basics 1 Feature Overview 3 1.1 Simple package installation.......................................3 1.2 Custom versions & configurations....................................3 1.3 Customize dependencies.........................................4 1.4 Non-destructive installs.........................................4 1.5 Packages can peacefully coexist.....................................4 1.6 Creating packages is easy........................................4 2 Getting Started 7 2.1 Prerequisites...............................................7 2.2 Installation................................................7 2.3 Compiler configuration..........................................9 2.4 Vendor-Specific Compiler Configuration................................ 13 2.5 System Packages............................................. 16 2.6 Utilities Configuration.......................................... 18 2.7 GPG Signing............................................... 20 2.8 Spack on Cray.............................................. 21 3 Basic Usage 25 3.1 Listing available packages........................................ 25 3.2 Installing and uninstalling........................................ 42 3.3 Seeing installed packages........................................ 44 3.4 Specs & dependencies.......................................... 46 3.5 Virtual dependencies........................................... 50 3.6 Extensions & Python support...................................... 53 3.7 Filesystem requirements........................................ -
Vsim User Guide Release 10.1.0-R2780
VSim User Guide Release 10.1.0-r2780 Tech-X Corporation Mar 12, 2020 2 CONTENTS 1 Overview 1 1.1 What is VSimComposer?........................................1 1.2 VSim Capabilities............................................1 2 Starting VSimComposer 3 2.1 Running Locally.............................................3 2.2 Running VSimComposer On a Remote Computer System.......................4 2.3 Visualizing Remote Data.........................................5 2.4 Welcome Window............................................5 3 Creating or Opening a Simulation7 3.1 Starting a Simulation...........................................7 4 Menus and Menu Items 15 4.1 File Menu................................................. 15 4.2 Edit Menu................................................ 18 4.3 View Menu................................................ 21 4.4 Help Menu................................................ 21 4.5 Tools/VSimComposer Menu (Settings/Preferences)........................... 21 5 Simulation Concepts 31 5.1 Simulation Concepts Introduction.................................... 31 5.2 Grids................................................... 32 5.3 Geometries................................................ 36 5.4 Electric and Magnetic Fields....................................... 36 5.5 Particles................................................. 41 5.6 Reactions................................................. 43 5.7 Histories................................................. 44 6 Visual Setup 45 6.1 Setup Window for Visual-setup Simulations.............................. -
HOW to VISUALIZE YOUR GPU-ACCELERATED SIMULATION RESULTS Peter Messmer, NVIDIA
HOW TO VISUALIZE YOUR GPU-ACCELERATED SIMULATION RESULTS Peter Messmer, NVIDIA RANGE OF ANALYSIS AND VIZ TASKS . Analysis: Focus quantitative . Visualization: Focus qualitative . Monitoring, Steering TRADITIONAL HPC WORKFLOW Workstation Analysis, Setup Visualization Supercomputer Viz Cluster Dump, Checkpointing Visualization, Analysis File System TRADITIONAL WORKFLOW: CHALLENGES Lack of interactivity prevents “intuition” Workstation High-end viz Analysis, neglected due Setup Visualization to workflow complexity Supercomputer Viz Cluster Viz resources need I/O becomes main to scale with simulation Dump, simulation bottleneck Checkpointing Visualization, Analysis File System OUTLINE . Visualization applications . CUDA/OpenGL interop . Remote viz . Parallel viz . In-Situ viz High-level overview. Some parts platform dependent. Check with your sysadmin. VISUALIZATION APPLICATIONS NON-REPRESENTATIVE VIZ TOOLS SURVEY OF 25 HPC SITES Surveyed sites: LLNL LLNL- ORNL- AFRL- NASA- NERSC -OCF SCF LANL CCS DOD-ORC DSCR AFRL ARL ERDC NAVY MHPCC ORS CCAC NAS NASA-NCCS TACC CHPC RZG HLRN Julich CSCS CSC Hector Curie NON-REPRESENTATIVE VIZ TOOLS SURVEY OF 25 HPC SITES Surveyed sites: LLNL LLNL- ORNL- AFRL- NASA- NERSC -OCF SCF LANL CCS DOD-ORC DSCR AFRL ARL ERDC NAVY MHPCC ORS CCAC NAS NASA-NCCS TACC CHPC RZG HLRN Julich CSCS CSC Hector Curie VISIT . Scalar, vector and tensor field data features — Plots: contour, curve, mesh, pseudo-color, volume,.. — Operators: slice, iso-surface, threshold, binning,.. Quantitative and qualitative analysis/vis — Derived fields, dimension reduction, line-outs — Pick & query . Scalable architecture . Open source http://wci.llnl.gov/codes/visit/ VISIT . Cross-platform — Linux/Unix, OSX, Windows . Wide range of data formats — .vtk, .netcdf, .hdf5,.. Extensible — Plugin architecture . Embeddable . Python scriptable VISIT’S SCALABLE ARCHITECTURE . -
Package Name Software Description Project
A S T 1 Package Name Software Description Project URL 2 Autoconf An extensible package of M4 macros that produce shell scripts to automatically configure software source code packages https://www.gnu.org/software/autoconf/ 3 Automake www.gnu.org/software/automake 4 Libtool www.gnu.org/software/libtool 5 bamtools BamTools: a C++ API for reading/writing BAM files. https://github.com/pezmaster31/bamtools 6 Biopython (Python module) Biopython is a set of freely available tools for biological computation written in Python by an international team of developers www.biopython.org/ 7 blas The BLAS (Basic Linear Algebra Subprograms) are routines that provide standard building blocks for performing basic vector and matrix operations. http://www.netlib.org/blas/ 8 boost Boost provides free peer-reviewed portable C++ source libraries. http://www.boost.org 9 CMake Cross-platform, open-source build system. CMake is a family of tools designed to build, test and package software http://www.cmake.org/ 10 Cython (Python module) The Cython compiler for writing C extensions for the Python language https://www.python.org/ 11 Doxygen http://www.doxygen.org/ FFmpeg is the leading multimedia framework, able to decode, encode, transcode, mux, demux, stream, filter and play pretty much anything that humans and machines have created. It supports the most obscure ancient formats up to the cutting edge. No matter if they were designed by some standards 12 ffmpeg committee, the community or a corporation. https://www.ffmpeg.org FFTW is a C subroutine library for computing the discrete Fourier transform (DFT) in one or more dimensions, of arbitrary input size, and of both real and 13 fftw complex data (as well as of even/odd data, i.e. -
Upgrading and Performance Analysis of Thin Clients in Server Based Scientific Computing
Institutionen för Systemteknik Department of Electrical Engineering Examensarbete Upgrading and Performance Analysis of Thin Clients in Server Based Scientific Computing Master Thesis in ISY Communication System By Rizwan Azhar LiTH-ISY-EX - - 11/4388 - - SE Linköping 2011 Department of Electrical Engineering Linköpings Tekniska Högskola Linköpings universitet Linköpings universitet SE-581 83 Linköping, Sweden 581 83 Linköping, Sweden Upgrading and Performance Analysis of Thin Clients in Server Based Scientific Computing Master Thesis in ISY Communication System at Linköping Institute of Technology By Rizwan Azhar LiTH-ISY-EX - - 11/4388 - - SE Examiner: Dr. Lasse Alfredsson Advisor: Dr. Alexandr Malusek Supervisor: Dr. Peter Lundberg Presentation Date Department and Division 04-02-2011 Department of Electrical Engineering Publishing Date (Electronic version) Language Type of Publication ISBN (Licentiate thesis) X English Licentiate thesis ISRN: Other (specify below) X Degree thesis LiTH-ISY-EX - - 11/4388 - - SE Thesis C-level Thesis D-level Title of series (Licentiate thesis) 55 Report Number of Pages Other (specify below) Series number/ISSN (Licentiate thesis) URL, Electronic Version http://www.ep.liu.se Publication Title Upgrading and Performance Analysis of Thin Clients in Server Based Scientific Computing Author Rizwan Azhar Abstract Server Based Computing (SBC) technology allows applications to be deployed, managed, supported and executed on the server and not on the client; only the screen information is transmitted between the server and client. This architecture solves many fundamental problems with application deployment, technical support, data storage, hardware and software upgrades. This thesis is targeted at upgrading and evaluating performance of thin clients in scientific Server Based Computing (SBC). Performance of Linux based SBC was assessed via methods of both quantitative and qualitative research. -
Remote Visualization
Remote Visualization Ravi Chityala Mike Knox Nancy Rowe © 2011 Regents of the University of Minnesota. All rights reserved. Remote visualization • Motivation: Large data – Limited local storage – Lengthy data transfer time – Slow local graphics system • Data rendered where calculated • Challenges:Bandwidth – 1280 x 1024 pixels of 24 bits at 30 frames a second = 118 MBps • Latency of network and GPU © 2011 Regents of the University of Minnesota. All rights reserved. What is remote visualization at MSI? • Available on Jay and Koronis • Several ways to access • Experimental © 2011 Regents of the University of Minnesota. All rights reserved. Systems Jay • Nvidia Quadro FX5800 • 64G memory Koronis (SGI Altix UV Constellation); NIH graphics1 graphics[2-4] • Nvidia Quadro FX1800 • Nvidia Quadro FX5800 • 256G memory • 48G memory © 2011 Regents of the University of Minnesota. All rights reserved. Access to Systems Jay • Reservation • qsub -I -q jay • www.msi.umn.edu/hardware/itasca/jay.html Koronis NIH www.msi.umn.edu/hardware/koronis © 2011 Regents of the University of Minnesota. All rights reserved. Systems System Location of Single Aggregate temporary Read/Write Read/Write storage speed speed Jay /lustre ~400MB/s ~10GB/s ~400MB/s ~13.5GB/s Koronis /scratch ~1.5GB/s ~4 - 14GB/s ~1.5GB/s ~7- 16GB/s (highly variable) © 2011 Regents of the University of Minnesota. All rights reserved. Ways to access • KVM • Teradici • VirtualGL • Client/Server © 2011 Regents of the University of Minnesota. All rights reserved. KVM • Keyboard, Video and Mouse • Directly runs software • Installed in SDVL 575 © 2011 Regents of the University of Minnesota. All rights reserved. Teradici card • PCoIP • Host rendering • Just sends pixels • Installed in SDVL 575 © 2011 Regents of the University of Minnesota. -
Chromium Renderserver: Scalable and Open Remote Rendering Infrastructure Brian Paul, Member, IEEE, Sean Ahern, Member, IEEE, E
1 Chromium Renderserver: Scalable and Open Remote Rendering Infrastructure Brian Paul, Member, IEEE, Sean Ahern, Member, IEEE, E. Wes Bethel, Member, IEEE, Eric Brug- ger, Rich Cook, Jamison Daniel, Ken Lewis, Jens Owen, and Dale Southard Abstract— Chromium Renderserver (CRRS) is software infrastruc- ture that provides the ability for one or more users to run and view image output from unmodified, interactive OpenGL and X11 applications on a remote, parallel computational platform equipped with graphics hardware accelerators via industry-standard Layer 7 network proto- cols and client viewers. The new contributions of this work include a solution to the problem of synchronizing X11 and OpenGL command streams, remote delivery of parallel hardware-accelerated rendering, and a performance anal- ysis of several different optimizations that are generally applicable to a variety of rendering architectures. CRRS is fully operational, Open Source software. Index Terms— remote visualization, remote rendering, parallel rendering, virtual network computer, collaborative Fig. 1. An unmodified molecular docking application run in parallel visualization, distance visualization on a distributed memory system using CRRS. Here, the cluster is configured for a 3x2 tiled display setup. The monitor for the “remote I. INTRODUCTION user machine” in this image is the one on the right. While this example has “remote user” and “central facility” connected via LAN, The Chromium Renderserver (CRRS) is software in- the typical use model is where the remote user is connected to the frastructure that provides access to the virtual desktop central facility via a low-bandwidth, high-latency link. Here, we see the complete 4800x2400 full-resolution image from the application on a remote computing system. -
Deconvolution Guide
Deconvolution Guide Rumelo Amor QBI Microscopy Facility, Queensland Brain Institute Research Lane, The University of Queensland St Lucia, 4072 QLD, AUSTRALIA <[email protected]> 1 A note on sampling density When we acquire images using a microscope and store these in a computer, we are using the technique of \sampling", that is, we are converting an analog signal (continuous in time or space) into digital form (discrete steps) [1, 2]. Ideally, when acquiring images, one should aim for a sampling density that satisfies the Nyquist criterion: there must be two samples for every structure one wishes to resolve [3]. Scientific Volume Imaging have a Nyquist calculator online [4]. To use the calculator, choose the appropriate microscope type, the numerical aperture of the imaging objective, the excitation and emission wavelengths, the number of excitation photons (1 for wide-field fluorescence, laser-scanning confocal and spinning disk confocal; 2 for two-photon microscopy), and the refractive index of the immersion medium. For example, for images of the Alexa 488 channel acquired on the Yokogawa spinning disk confocal microscope using the 63x/1.4 NA oil-immersion objective, the parameters should be what are shown in Fig. 1a. Click \Calculate". The calculator then shows the results and for this particular example, a sampling rate of 43 nm is required in the lateral (X and Y) direction and 130 nm in the axial (Z) direction (Fig. 1b). The sampling rate in X and Y is a function of the optics in the light path and the relay optics in front of the detectors and therefore is fixed, but the sampling rate in Z is user-defined. -
Remote Visualisation Using Open Source Software & Commodity
Remote visualisation using open source software & commodity hardware Dell/Cambridge HPC Solution Centre Dr Stuart Rankin, Dr Paul Calleja, Dr James Coomer © Dell Abstract It is commonplace today that HPC users produce large scale multi-gigabyte data sets on a daily basis and that these data sets may require interactive post processing with some form of real time 3D or 2D visualisation in order to help gain insight from the data. The traditional HPC workflow process requires that these data sets be transferred back to the user’s workstation, remote from the HPC data centre over the wide area network. This process has several disadvantages, firstly it requires large I/O transfers out of the HPC data centre which is time consuming, also it requires that the user has significant local disk storage and a workstation setup with the appropriate visualisation software and hardware. The remote visualisation procedures described here removes the need to transfer data out of the HPC data centre. The procedure allows the user to logon interactively to the Dell | NVIDIA remote visualisation server within the HPC data centre and access their data sets directly from the HPC file system and then run the visualisation software on the remote visualisation server in the machine room, sending the visual output over the network to the users remote PC. The visualisation server consists of a T5500 Dell Precision Workstation equipped with a NVIDIA Quadro FX 5800 configured with an open source software stack facilitating sending of the visual output to the remote user. The method described in this whitepaper is an OS-neutral extension of familiar remote desktop techniques using open-source software and it imposes only modest demands on the customer machine and network connection. -
Scientific Visualization on Gpu Clusters Peter Messmer, Nvidia
SCIENTIFIC VISUALIZATION ON GPU CLUSTERS PETER MESSMER, NVIDIA Visualization ≠ Rendering Visualization Coordinate Field Operators transformations Isosurfaces, (Gradient, Curl,.. ) Isovolumes Clip, Slice Streamlines Feature extraction Rendering Binning, Surface Resample Line Rendering Rendering Thresholding Volume Compositing Rendering Visualization Filtering Field Operators Coordinate transformations FieldIsosurfaces Operators, (Gradient, Curl,.. ) (Gradient,Isovolumes Curl,.. ) Streamlines Streamlines Clip, Slice Feature Feature extraction Binning, extraction Rendering Resample Surface Line Rendering Rendering Isosurfaces, Thresholding Line Isovolumes Volume CompositingRendering Rendering THE VISUALIZATION PIPELINE Simulation Filtering Rendering Compositing Visualization Simulation: Data as needed in numerical algorithm Filtering: Conversion of simulation data into data ready for rendering Typical operations: binning, down/up-sampling, iso-surface extraction, interpolation, coordinate transformation, sub-selection, .. Sometimes embedded in simulation Rendering: Conversion of shapes to pixels (Fragment processing) Compositing: Combination of independently generated pixels into final frame PARTICULARITIES IN HPC VIZ Parallelism Remoteness Heterogeneity Images may be subject to copyright OUTLINE Tools Paraview, Visit, others Rendering Enable HW rendering on Tesla Remote rendering Compositing Delivery In-situ visualization High-level overview. Some parts platform dependent. Check with your sysadmin. VISUALIZATION APPLICATIONS PARAVIEW Scalar,