Parallel computing performance tests
IT/CFD Team meeting 2013-06-13
2013-06-13 E. Da Riva, M. Plagge 1 “Engineering cluster” & STARCCM+ (2 years ago test) 1/2
60 3,000,000 cells test case (HiRadMat) 50 lxbst2000..2020, STARCCM+
40 MEASURED
THEORETICAL MAXIMUM 30
20
10 # Iterations per Wall Clock Time [iter/min] Time Clock # Iterations per Wall 0 0 8 16 24 32 40 # Parallel Processes
2013-06-13 E. Da Riva, M. Plagge 2 “Engineering cluster” & STARCCM+ (2 years ago test) 2/2
5 6,000,000 cells test case (CAST) lxbst2000..2020, STARCCM+ 4 MEASURED
3 THEORETICAL MAXIMUM
2
1 Domain decomposed across different nodes of the cluster
0
# Iterations per Wall Clock Time [iter/min] Time Clock # Iterations per Wall 0 8 16 24 32 40 # Parallel Processes Domain decomposed across one single node Fast interconnection is essential
2013-06-13 E. Da Riva, M. Plagge 3 “Engineering cluster” & ANSYS Fluent 1/2 Test "engineering cluster" parallel (ptile=8) CAST 6 106 cells mesh 0.20
0.18
0.16 Theoretical Experimental 0.14
0.12
0.10
0.08 Iterationsper second 0.06
0.04
0.02
0.00 1 2 3 4 5 6 7 8 # CPUs
2013-06-13 E. Da Riva, M. Plagge 4 “Engineering cluster” & ANSYS Fluent 2/2 Test "engineering cluster" parallel (ptile=8) CAST 6 106 cells mesh 1.40
1.20 Theoretical 1.00 Experimental
0.80
0.60 Iterationsper second 0.40
0.20
0.00 0 8 16 24 32 40 48 56 64 72 80 88 # CPUs
2013-06-13 E. Da Riva, M. Plagge 5 What happens when things don`t work Test Chutiya cluster parallel (single machine) CAST 6 106 cells mesh 0.6
0.5 Theoretical Experimental
0.4
0.3 Iterationsper second 0.2
0.1
0 1 3 5 7 # CPUs
2013-06-13 E. Da Riva, M. Plagge 6 “Engineering cluster” machine vs HPZ800 (2 years ago test)
2013-06-13 E. Da Riva, M. Plagge 7 Windows 7 vs SLC5 with STARCCM+ (2 years ago test)
2013-06-13 E. Da Riva, M. Plagge 8 Hyperthreading test – HPZ800, Ubuntu 10 LTS, OpenFOAM 2.0
8 CPUs machines with Hyperthreading enabled
Hyper-threading is not effective for parallel CFD computing
From: http://www.simutechgroup.com/Technical-Support-Services/ansys-hardware-support.html Hyper-Threading is now called Simultaneous multithreading or SMT. Customers are recommended to leave SMT enabled on their systems but not over-subscribe physical cores for parallel simulations. While some improvement is possible, the extra performance from the virtual threads is not cost-effective and incommensurate with the additional license costs (which are per process)."
2013-06-13 E. Da Riva, M. Plagge 9 Conclusions
Ideally, CFD parallel computing would require:
• Single machine with several CPUs (avoid external interconnection);
• If interconnection is used, this must be fast and RELAIBLE;
• Hyper-threading is not effective;
• Linux is better than Windows.
2013-06-13 E. Da Riva, M. Plagge 10