Visualisation Using the Grid Jeff Adie Principal Systems Engineer, SAPK July 2008 Agenda
Total Page:16
File Type:pdf, Size:1020Kb
Headline in Arial Bold 30pt Visualisation using the Grid Jeff Adie Principal Systems Engineer, SAPK July 2008 Agenda • Visualisation Today • User Trends • Technology Trends • Grid Viz Nodes • Software Ecosystem • Remote Visualisation • Computational Steering •Q & A 17-Sep-08 Slide 2 The Visualization Landscape Textures Pixel Fill Rate Shading VisualizationVisualization Gaming/Movies Model Size Polygon Count Compute Power Sci Viz Image courtesy 20th Century Fox 17-Sep-08 Slide 3 Market Forces Î Fragmented Market Sci Viz LAN / WAN Clusters Fragmented Market VisualizationVisualization GPU Complex GP/GPU Workflows Gaming/Movies 17-Sep-08 Slide 4 17-Sep-08 Slide 5 Visualization User Trends • Increasingly Complex Data Sets – Scope: Larger Geographic and Physical Models – Precision: More polygons per model – Depth: More complex interactions • Need for Speed – Run multiple iterations of a problem – Time to solution remains critical, even with very large models – Ability to interact in real-time with very large models • Display & Interactivity Requirements – Very large display environments – Computational Steering and Interactivity – Multiple remote displays – Remote group collaboration 17-Sep-08 Slide 6 Example: Manufacturing CAE 17-Sep-08 Slide 7 Example: NASTRAN job surge> ls -l total 163260624 -rw-r--r-- 1 jeffadie users 1558 2008-08-21 17:08 INPUT2_with_ACMS.dat -rwxr-xr-x 1 jeffadie users 249 2008-08-21 17:20 ana64 -rw-r--r-- 1 jeffadie users 484103 2008-08-21 19:39 input2_64core.f04 -rw-r--r-- 1 jeffadie users 107970679 2008-08-21 19:39 input2_64core.f06 -rw-r--r-- 1 jeffadie users 70541 2008-08-21 19:39 input2_64core.log -rw-r--r-- 1 jeffadie users 83535079275 2008-08-21 19:39 input2_64core.pch -rwx------ 1 jeffadie users 0 2008-08-21 17:19 mkl.cfg -rwxr-xr-x 1 jeffadie users 435 2008-08-21 17:14 run64 surge> 17-Sep-08 Slide 8 Example: NASTRAN Job • Input File : 41MB • Output File: 83.5GB • Network (Singapore Ù USA): 1Mb/s measured • Upload model: 410s ~ 7 minutes • Solution (1000 steps, 64 cores): ~ 1 hour • Download results: 835,000s ~ 232 hours ~ 10 days NO WAY TO REASONABLY GET RESULTS 17-Sep-08 Slide 9 Visualization Technology Trends • Increasing COTS GPU Performance – Price/Performance Advantage – Use as Application Accelerators • Cluster Architectures – Distributed Compute & Render – Expandability • High Speed Data I/O – Necessary to handle large data transfers • Resource Optimization – Identify and utilize available compute & render resources – Grid computing and Virtualization • Smart Workflow Management – Increase user and system productivity 17-Sep-08 Slide 10 Implementing Visualization on Clusters • Leverage Cluster Architecture and Advantages – Clusters dominate mindshare and marketshare – Scalability, Costs, Selection, Performance, Open Source • Reduce Cluster Complexity – Integration, Q/A, Time-to-Productivity • Leverage Graphics (Gaming) Technology – Provide Textures, Pixel Fill Rate, Shading – Graphics Cards don’t directly address Visual Supercomputing Demands • The Challenge of Visualization on Clusters – Scaling Visualization Applications – Maintaining Data I/O Throughput – Software Stack Complexity – Resource Management – Maintaining Interactivity in a Batch Environment 17-Sep-08 Slide 11 SGI Virtu™ VN200 Visualization Node 8 FB-DIMMs 8 to 32 GB RAM DDR 4x IB interconnect Two (2.5”) SATA Drives NVIDIA® Quadro® FX High Performance GPU Redundant (2) Power Supplies Two (2) Intel Quad core Xeon CPUs 17-Sep-08 Slide 12 Software Ecosystem 17-Sep-08 SYSTEM MANAGEMENT STORAGE MANAGEMENT FILE SYSTEM LINUX OPERATING SYSTEM LINUX OPERATING JOB SCHEDULER Slide 13 COMPILERS LIBRARIES PROFILERS, DEBUGGERS Software Ecosystem WORKFLOW ORCHESTRATION BATCH INTERACTIVE SCHEDULERJOB SCHEDULERSCHEDULER COMPILERS LIBRARIES FILE SYSTEM GRAPHICS TOOLKITS SYSTEM MANAGEMENT REMOTE VISUALIZATION PROFILERS, DEBUGGERS PROFILERS, STORAGE MANAGEMENT LINUX OPERATING SYSTEM 17-Sep-08 Slide 14 The Market Demand for Grid Visualization • Global workflow is reality in today’s business • Visualization must fit into the global workflow model • Greater demands on security, data sharing, and resource management • Reduce costs for applications and client systems • Grid virtualization • Laptop/desktop limitations • Without Remote Grid Visualization, the only method to share data is to physically copy the data to a remote location, with duplicate applications and processing power. 17-Sep-08 Slide 15 Remote Grid Visualization: Key Concepts Traditional Approach: Data Paradigm –Download DATA to individual graphics workstations –Multiple copies of the model/data across users –Large data transmissions across networks, iteratively Store Compute Manage Render / Interact 2D, 3D, 4D visual model 2D, 3D, 4D non-visual Compute Admin Service Desktop model Nodes Node workstation Batch Interactive 17-Sep-08 Slide 16 Remote Grid Visualization The Silicon Graphics Difference Visual Information Paradigm –Securely share visual information to all users – no data transfer –Real-time iterative analysis and updates –Single data image – secure, simpler management, consistent Store Compute Manage Interact Visualization 2D, 3D, 4D Visualization Display System models Compute & Visualization & Nodes Admin Node or Thin Client Batch Interactive 17-Sep-08 Slide 17 Visualisation across the Grid The Power of a Reality Center The Power of Fusion The Power of Many Applications The Power of Many Devices 17-Sep-08 Slide 18 Benefits of Grid Visualisation • Remove the physical link between users, their data, and the visualization system • Solve much more difficult problems than possible with desktop workstations—but do it from your desktop • Ability to reserve HPC resources to enable interactive applications • Connect experts with users and their data, independent of their locations • Enable computational steering from anywhere in the world 17-Sep-08 Slide 19 SGI High Performance Visualization Computational Steering Traditional Methods Interactive Discovery Prepare Prepare/Analysis Submit Jobs Simulation Interactively change the simulation Analysis Faster Time to Insight 17-Sep-08 Slide 20 Summary Remote Grid Visualisation benefits: • Data size – One copy of the data is maintained in the database. – User has fast access to the data. – Sharing data • Parallel file systems allows multiply users to analysis the database. • Model size – Distributed rendering allows larger models to be visualized. – Frame buffer limit of the GPU is no longer a limiting factor. • Faster computation – Render nodes are integrated in the Grid and can be used to for MPI jobs when not running visualization. • License consolidation – For end user application licenses can be shared on the Grid as opposed to each user having their own license. 17-Sep-08 Slide 21 Thank You! 17-Sep-08 Slide 22.