Mitglied Mitglied der Helmholtz-Gemeinschaft
Remote Scientific Visualization at Jülich Supercomputing Centre
Jülich Supercomputing Centre, Forschungszentrum Jülich GmbH, Germany Algorithms, Tools & Methods Lab Visualization
Page 1 Visualization at JSC Algorithms, Tools & Methods Lab Visualization
. Scientific Visualization – R&D + support for visualization of scientific data
. Virtual/Augmented Reality – VR visualization based on Unreal Engine, with head mounted displays and tablet computers for data analysis and presentation
. Multimedia – multimedia productions for websites, presentations or on TV
Page 2 Visualization at JSC JUWELS: General Hardware Setup
4 x Visualization Login Nodes JUWELS . juwelsvis.fz-juelich.de . (juwelsvis00 to juwelsvis03 in 4x round-robin fashion) vis login nodes . 768 GB RAM each 12x login . 1 GPUs Nvidia Pascal P100 per node nodes . 12 GB RAM on GPU InfiniBand 2567x compute nodes No specific Visualization Batch Nodes Data GPFS
Keep in mind: Visualization is NOT limited to vis. nodes ONLY. (software rendering is possible on any node) Page 3 Visualization at JSC JURECA-DC: General Hardware Setup
12 x Login Nodes with GPU JURECA-DC . jureca.fz-juelich.de . (jureca01 to jureca12 in 12x round-robin fashion) login nodes . 1024 GB RAM each
. 2 x Nvidia Quadro RTX8000 per node . 48 GB RAM on each GPU InfiniBand 768x compute nodes No specific Visualization Batch Nodes Data GPFS
Keep in mind: Visualization is NOT limited to vis. nodes ONLY. (software rendering is possible on any node) Page 4 Visualization at JSC General Software Setup
Special Software Stack on Vis Nodes: Base Software: X-Server, X-Client (Window-Manager) OpenGL (libGL.so, libGLU.so, libglx.so), Nvidia Middleware: Xpra Virtual Network Computing: VNC-Server, VNC-Client VirtualGL Parallel and Remote Rendering Apps, In-Situ Visualization: ParaView VisIt Other Visualization Packages (installation on user demand): VMD, PyMol, Blender, GPicView
Page 5 Mitglied Mitglied der Helmholtz-Gemeinschaft
Remote 3D Visualization
The following examples are given for JUWELS Jülich Supercomputing Centre, Forschungszentrum Jülich GmbH, Germany Algorithms, Tools & MethodsAccess Lab Visualization to JURECA-DC similar
Page 6 Remote 3D Visualization General Setup
User’s Work- JUWELS station
4x vis login nodes
Firewall 12x login nodes vis login node: InfiniBand - direct user access 2567x - no accounting compute nodes - shared with other users - no parallel jobs (no srun) Data GPFS
Page 7 Remote 3D Visualization at Jülich Supercomputing Centre
. X forwarding + Indirect Rendering slow, maybe incompatible bad idea
. “remote aware” visualization apps (ParaView, VisIt) application dependent error-prone setup
. Xpra - stream application content with H.264 + VirtualGL fast, our recommendation good idea
. VNC (Virtual Network Computing) + VirtualGL full remote desktop, but slower than Xpra medium good idea
Page 8 Remote 3D Visualization with X Forwarding + Indirect Rendering
Traditional Approach (X forwarding + Indirect Rendering) ssh –X
Try to AVOID for 3D visualization.
Page 9 Remote 3D Visualization with Xpra (or VNC) + VirtualGL
. X-applications forwarded by Xpra (or VNC) appear on the local desktop as normal windows . allows disconnection and reconnection without disrupting the forwarded application . advantages . No X is required on user´s workstation (X display on server). . No OpenGL is required on user´s workstation (only images are send). . Quality of visualization does not depend on user´s workstation. . Data size send is independent from data of 3d scene. . Disconnection and reconnection possible. . VirtualGL for hardware accelerated rendering: use vglrun
. Good solution for any OpenGL application e.g. ParaView, VisIt, IDL, VMD, PyMol, …
https://xpra.org/ Page 10 How to use Xpra @ JSC
How to start an Xpra session:
. Recommended way: from JupyterLab@JSC https://jupyter-jsc.fz-juelich.de . See next slides
. Alternative: start Xpra session manually . Neither recommended nor necessary . Just in case you need it: documentation provided on later slides
Page 11 Xpra Integration in JupyterLab@JSC
1. Go to https://jupyter-jsc.fz-juelich.de and login
2. Add a new or start an existing JuperLab on a HPC system. Wait for your server to start
Page 12 Xpra Integration in JupyterLab@JSC
3. In the launcher: click on the Xpra icon
4. Wait for the HTML desktop of Xpra. Start apps from the menue or from the Xterm.
Page 13 Xpra Integration in JupyterLab@JSC
5. Example: Start ParaView in the Xpra environment in your browser, load your data from HPC filesystem
Page 14 Mitglied Mitglied der Helmholtz-Gemeinschaft
ParaView for data visualization
Jülich Supercomputing Centre, Forschungszentrum Jülich GmbH, Germany Algorithms, Tools & Methods Lab Visualization
Page 15 Visualization Pipeline
Source Filter 1 Filter2 initial data Data 1 modify the Data 2 modify the Data 3 input or data in data in generated some way some way
Mapper Actor Mapper Actor Mapper Actor Generates Adjust the Generates Adjust the Generates Adjust the Function calls visible Function calls visible Function calls visible to graphic properties to graphic properties to graphic properties system system system
Renderer
Create images of data
Page 16 . .
. .
. .
.
. .
Page 17 •
•
Page 18 Common Filters: Clip
•
•
Page 19 •
•
Page 20 •
Page 21 •
Page 22 •
•
Page 23 •
Page 24 •
Page 25 •
Page 26 •
•
• Page 27 Thank you for your attention
Questions … ?
rendered with Blender from a DNS of a diesel injection spray of ITV, RWTH Aachen University
Page 28 Mitglied Mitglied der Helmholtz-Gemeinschaft
Appendix: How to start Xpra by Hand
Jülich Supercomputing Centre, Forschungszentrum Jülich GmbH, Germany Algorithms, Tools & Methods Lab Visualization
Page 29 Manual Setup of Xpra with Xpra + VirtualGL
JUWELS
4x vis login nodes
Firewall 12x Users Workstation login nodes HowTo start an Xpra session manually InfiniBand 1. SSH to HPC system, authenticate via SSH key pair 2567x 2. Load modules and start an application via Xpra. compute nodes E.g. start an xterm: xpra start --start=xterm Data Look at the output and note the display number, GPFS e.g. „Actual display used: :3“
3. start local Xpra client and connect to remote display 4. Start visualization application in the xterm 5. Stop the Xpra session by xpra stop :3
Page 30 Manual Setup of Xpra
Step 1: login to a (visualization) login node
. Linux: ssh
Page 31 Setup Xpra
Step 2: start xpra on HPC node and notice the display- number in the output
For example, start an xterm: jwvis02> module --force purge jwvis02> module use otherstages jwvis02> ml Stages/Devel-2020 GCCcore/.9.3.0 xpra/4.0.4-Python- 3.8.5 jwvis02> xpra start --start=xterm ... Actual display used: :3 . The display-number is needed to connect to the Xpra session
Page 32 Setup Xpra Step 3: connect to Xpra session Install Xpra on your local machine. Download from www.xpra.org
Linux: use command local_machine> xpra attach ssh://[email protected]/3
Windows: use Xpra GUI:
Page 33 Setup Xpra Step 4: start visualization application After successful connection, an xterm window will show up on your local desktop. Start your application there, e.g. ParaView:
Step 5: When you are done, stop the session by jwvis02> xpra stop :3 Page 34