Performance of Multi-Process and Multi-Thread Processing on Multi-core SMT Processors Hiroshi Inoue and Toshio Nakatani Abstract—Many modern high-performance processors growing gap between the processor and memory speed, SMT support multiple hardware threads in the form of multiple cores is becoming more important. and SMT (Simultaneous Multi-Threading). Hence achieving To exploit thread-level parallelism available in such a good performance scalability of programs with respect to the system, programs have to be parallelized. A programmer can numbers of cores (core scalability) and SMT threads in one core write a parallel program using multiple processes (the (SMT scalability) is critical. To identify a way to achieve higher performance on the multi-core SMT processors, this paper multi-process model) or multiple threads within one process compares the performance scalability with two parallelization (the multi-thread model) on today’s systems. Multiple threads models (using multiple processes and using multiple threads in in one process share the virtual memory space, while multiple one process) on two types of hardware parallelism (core processes do not share the same memory space. This scalability and SMT scalability). We tested standard Java difference typically results in a larger memory footprint and benchmarks and a real-world server program written in PHP better inter-process isolation for the multi-process model. For on two platforms, Sun's UltraSPARC T1 (Niagara) processor and Intel's Xeon (Nehalem) processor. We show that the example, the Apache Web server supports both a multi-thread model achieves better SMT scalability compared multi-process model (prefork model) and a multi-thread to the multi-process model by reducing the number of cache model (worker model). The prefork model generates multiple misses and DTLB misses. However both models achieve roughly Web-server processes to handle many HTTP connections equal core scalability. We show that the multi-thread model while the worker model generates multiple threads. The generates up to 7.4 times more DTLB misses than the prefork model is more reliable but consumes more memory. multi-process model when multiple cores are used. To take advantage of the both models, we implemented a memory In this paper, we study the interactions between the allocator for a PHP runtime to reduce DTLB misses on parallelization model and the processor architecture aspects multi-core SMT processors. The allocator is aware of the core of a multi-core SMT processor. To identify how to achieve that is running each software thread and allocates memory higher performance on the multi-core SMT processor, this blocks from same memory page for each processor core. When paper focuses on performance comparisons between a using all of the hardware threads on a Niagara, the core-aware multi-thread model and a multi-process model on two types allocator reduces the DTLB misses by 46.7% compared to the default allocator, and it improves the performance by 3.0%. of hardware parallelism (core scalability and SMT scalability). We measure the throughput of standard Java I. INTRODUCTION benchmarks, SPECjbb2005 and SPECjvm2008, using the multi-process model and the multi-thread model, and then we Modern high-performance processors provide thread-level analyze the detailed architecture-level statistics such as cache parallelism by using multiple hardware threads within each misses and TLB misses on two processors, Sun’s Niagara and core using Simultaneous Multi-Threading (SMT)1 [1]. For Intel’s Nehalem. The Niagara has eight cores and four SMT example, Sun’s UltraSPARC T2 (Niagara 2) processor [2] threads in each core, whereas the Nehalem has four cores and provides 8-way multi-threading in each core, UltraSPARC two SMT threads in each core. In the measured benchmarks, T1 (Niagara) processor [3] and IBM’s POWER7 processor each thread did not communicate with each other so that we [4] provide 4-way multi-threading, and Intel’s Core i7 and can focus on the architecture-level behavior. Xeon (Nehalem) processors [5] provide 2-way To test the core scalability, we increased the number of multi-threading. Such SMT processors can increase the cores with only one SMT thread in each core. To test the SMT efficiency of CPU resource usage by allowing multiple scalability, we increased the number of SMT threads in each threads to run on a single core, even for workloads with core. Although many previous studies [6-11] measured core limited instruction-level parallelism. Multiple threads using scalability and SMT scalability of programs on multi-core SMT can also hide memory access latencies. Due to a SMT processors, this is the first work to focus on the effects of the parallelization model on multi-core SMT processors. Our results showed that the multi-thread model achieved 1 A Niagara core cannot execute instructions from multiple software much better SMT scalability than the multi-process model threads in the same processor cycle and thus the multi-threading in Niagara because the multi-thread model generated a smaller number should be called fine-grained multi-threading. In this paper, we use the word SMT in a wider sense to include fine-grained multi-threading. of cache misses and DTLB misses due to a smaller memory footprint. In contrast to better SMT scalability of the H. Inoue and T. Nakatani are with IBM Research – Tokyo, Kanagawa-ken 242-0001 Japan (e-mail: {inouehrs, nakatani}@jp.ibm.com). multi-thread model, on average, both models achieved almost comparable core scalability on Niagara and the multi-process the performance scalability of the SAP Java application model achieved better core scalability than the multi-thread server on a Niagara processor and on Intel’s Xeon model on Nehalem. We found that, contrary to our (Clovertown) processors, which do not support SMT, while expectations, the multi-thread model generated up to 7.4 changing the number of JVMs from one to four. The one times more DTLB misses than multi-process model when we JVM configuration in their work corresponds to the used many cores. multi-thread model, and using more JVMs makes the We confirmed that our observation was not specific to Java program more like the multi-process model, which is an workloads by testing a real-world server program written in extreme case of the multiple-JVM configurations. They PHP on Niagara. The multi-thread model achieved better showed that using more JVMs eased the lock contention SMT scalability than the multi-process model by reducing the while it increased the memory footprint and cache misses. As DTLB misses, but it showed comparable core scalability. a result, a 2-JVM configuration achieved the best We also observed that the performance advantage of the performance on the Niagara system and a 4-JVM multi-thread model was reduced as the number of cores configuration peaked on the Xeon system. They concluded increased. To achieve the better performance on the SMT that the amount of cache memory was the key factor to processors using many cores with SMT threads, we determine the best configuration. Our results showed that the implemented a new memory allocator in the multi-threaded 4-way SMT of Niagara might be another important factor that PHP runtime. This allocator avoided sharing a page among reduced the performance of 4-JVM configuration on Niagara. threads running on different cores. This reduced the DTLB Many previous studies identified lock contention as the misses by about half and improved the performance when dominant cause of poor scalability on multi-core SMT using all of the hardware threads on Niagara. The modern processors [9-11]. For example, Ishizaki et al. [11] reported memory allocators are often aware of the remote and local that the performance of Java server workloads running on a memories on NUMA systems and control the allocations to Niagara2 processor was improved by up to 132% by minimize the costly remote memory accesses. Our allocator removing the contended locks. We found that appropriate use provided similar control at a finer granularity even on a of the parallelization model is also an important factor in non-NUMA system and improved the performance. improving the performance of programs on multi-core SMT There are two main contributions in this paper. (1) We processors. The programs we used in this paper did not demonstrate that the smaller footprint of the multi-thread actually suffer from lock contention even when we used all of model does not always result in the higher performance and it the hardware threads of a Niagara processor. This was may degrade the performance compared to the multi-process because we selected programs that did not share data between model due to more frequent DTLB misses on today’s the software threads so as to focus on interactions between multi-core processors. The performance of the multi-thread parallelization model and underlying micro architecture. model and the multi-process model depend on the underlying Server-side programs for Web applications, in which each processor architecture and that the interactions are thread serves a different client request without complicated. (2) We show that a memory allocator that tracks communicating among each other, are important examples of the processor core executing each program can improve the such programs that did not share data among software threads. performance on a multi-core SMT processor by significantly Other important examples include Hadoop (an open-source reducing the DTLB misses. implementation of the MapReduce runtime written in Java) or The rest of the paper is organized as follows. Section 2 scientific workloads using MPI. covers related work. Section 3 summarizes the experimental In this paper, we propose a new memory allocator for results with Java benchmarks.