The Embedded Java Benchmark Suite Jembench
Total Page:16
File Type:pdf, Size:1020Kb
View metadata,Downloaded citation and from similar orbit.dtu.dk papers on:at core.ac.uk Dec 17, 2017 brought to you by CORE provided by Online Research Database In Technology The embedded Java benchmark suite JemBench Schoeberl, Martin; Preusser, Thomas B.; Uhrig, Sascha Published in: Proceedings of the 8th International Workshop on Java Technologies for Real-time and Embedded Systems (JTRES 2010) Link to article, DOI: 10.1145/1850771.1850789 Publication date: 2010 Document Version Early version, also known as pre-print Link back to DTU Orbit Citation (APA): Schoeberl, M., Preusser, T. B., & Uhrig, S. (2010). The embedded Java benchmark suite JemBench. In Proceedings of the 8th International Workshop on Java Technologies for Real-time and Embedded Systems (JTRES 2010) (pp. 120-127). DOI: 10.1145/1850771.1850789 General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim. The Embedded Java Benchmark Suite JemBench Martin Schoeberl Thomas B. Preußer Sascha Uhrig Department of Informatics and Department of Computer Department of Computer Mathematical Modeling Engineering Science Technical University of Technische Universität University of Augsburg, Denmark Dresden, Germany Germany [email protected] thomas.preusser@tu- [email protected] dresden.de augsburg.de ABSTRACT which perform data-parallel and pipelined execution. Requirements to embedded systems increase steadily. In parallel, As the benchmark suite does not target consumer electronics, but also the performance of the processors used in these systems is embedded control systems, the benchmarks are mainly designed in improved leading to multithreaded and/or multicore architectures. a WCET (Worst Case Execution Time) analyzable way. Hence, a Depending on the type of the embedded system, using Java is a target platform can be examined concerning their real-time capabil- more and more popular way for software development. In this pa- ity by performing a static WCET analysis and compare the results per, we present a Java benchmark suite that enables the comparison to the actual execution. Of course, the real-time capability of the of different embedded Java platforms while solely assuming the used platform cannot be proven but an estimation of the WCET availability of a CLDC API, the minimal configuration defined for analysis’ tightness can be obtained. the J2ME. The core of the benchmark suite consists of adapted real- Depending on the platform, the execution time of an application world applications. Furthermore, the suite contains benchmarks to may vary dramatically. To reach a feasible accuracy in combina- explore multi-core/multi-threaded systems. Hence, it is possible tion with a reasonable execution time independent of the system’s to determine the gain of a parallel execution platform compared to performance, the benchmarks adapt themselves to the capability of sequential execution. Additionally, the penalty of a sequential pro- the target platform. gram running on a parallel platform can be measured. Our bench- The provided benchmark suite contains sequential and mul- marks are structured in micro, kernel, application, parallel, and tithreaded benchmarks, organized in five categories that can be streaming benchmarks. started all together or individually. The first category includes simple micro benchmarks testing individual bytecodes of different complexity. Two typical kernel algorithms are packed together into 1. INTRODUCTION the kernel category. Three applications from industrial use form the Benchmarks are important for the development of embedded third category. These application benchmarks are the main bench- systems. While benchmarks for standard Java, server-oriented marks to measure single threaded performance. The multithreaded Java, and Java on mobile phones are available, no Java benchmark benchmarks consist of two groups: data parallel applications and suite is available for classic embedded control systems. The pre- a streaming benchmark. The number of threads to be used by the sented benchmark suite JemBench fills this gap. multithreaded benchmarks is defined in a utility class that is part of Embedded systems are often resource-constrained. A full JDK is the benchmark suite. usually too big for the embedded target. Our benchmark suite is de- The presented benchmark suite is freely available as open source signed to run with minimal JDK support. Furthermore, the bench- under the GNU GPL license. Instructions how to access the source marks are designed to run with a small memory footprint (less than and run the benchmarks can be found in the abstract. 1 MB) and do not require any file I/O. All benchmarks conform to In the remainder of this paper, an overview over embedded Java the CLDC 1.1 [26]1 standard. Additionally, they can be executed systems is given first by Section2 before Section3 existing bench- on RTSJ and Safety-Critical Java (JSR 302) platforms. marks are discussed. Section4 introduces our JemBench bench- Realistic benchmarks include real applications and not only toy mark suite whose characteristics are then be evaluated in Section5. kernels. For JemBench, we have adapted several embedded Java Finally, Section6 concludes this paper. applications that are in industrial use [20]. Besides sequential benchmarks, we also developed several multithreaded benchmarks, 2. EMBEDDED JAVA 1See also: http://java.sun.com/javame/reference/apis/ In this section, we provide an overview of different Java runtime jsr139/ systems for the embedded domain, the domain JemBench was de- veloped for. A Java virtual machine (JVM) can be implemented as a software environment executed on an arbitrary (embedded) pro- cessor or as a hardware solution using a processor that is able to Permission to make digital or hard copies of all or part of this work for execute Java bytecode natively. personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies 2.1 Embedded JVM Implementations bear this notice and the full citation on the first page. To copy otherwise, to A JVM implemented in software contains the Java library republish, to post on servers or to redistribute to lists, requires prior specific classes, several methods to connect the Java application to the un- permission and/or a fee. JTRES’10 August 19–21, 2010 Prague, Czech Republic derlying operating system and a bytecode interpreter and/or a just- Copyright 2010 ACM 978-1-4503-0122-0/10/08 ...$10.00. in-time (JIT) compiler. Obviously, an operating system is required that is responsible for the native I/O, such as file and network ac- 3.1 Java Standard Edition cess. The supported Java standard depends on the available re- DaCapo [4] is an open-source benchmark for Java that is in- sources and the actual JVM implementation. tended to better serve the Java community than SPEC JVM98. Examples of software implemented JVMs that can run in an em- The collection of benchmarks includes more complex code, more bedded setting (among others) are the JamaicaVM [22], IBM J9 object-oriented applications, and puts more stress on the dynamic 2 3 VM, PERC Pico [2], KertasarieVM, KVM [25], Squawk [23], memory management. DaCapo has become the standard bench- CACAO [14], and Fiji VM [18]. mark for the GC research community. DaCapo uses open-source tools and applications such as a parser generator, bytecode opti- 2.2 Java Processors mizer, graph plotting, Eclipse, an SQL database engine, a python In contrast to the software implementations, Java processors do interpreter, text index and search, source code analyzer, and an not require any interpreter or compiler. In general, the operating XSLT processor. The authors also argue for investing more effort system functionalities are also implemented in Java leading to a in sound benchmarking methodologies [5]. The proposed embed- homogeneous control flow from the application to the I/O. Addi- ded benchmark suite is our contribution to this field for a specific tional abstraction layers are not required. Moreover, the exception application domain, which is still underrepresented. concept can be used also at the I/O level that means an explicit er- The SPECjvm20084 focuses on kernel applications for both ror handling in every hierarchy level is not required (try/catch client and server side Java systems. The benchmark results are blocks do not need extra bytecode instructions in the normal pro- obtained by single-threaded as well as multi-threaded applications gram flow). using multiple logical processor cores (physical cores or hardware Regarding the real-time capability, Java processors benefit from threads). Even though the benchmarks require little I/O and no off- the omitted software JVM and operating system. Hence, it is suf- chip network communication they are not well-suited for embedded ficient that the processor and the JVM libraries are designed in a devices because the full J2SE standard API is required. The SPEC real-time capable way. JOP [17], SHAP [29], jamuth [28], aJile [1], website highlights that the SPECjvm2008 uses real-world applica- and picoJava [9] are some examples of Java processors. tions and also domain specific benchmarks (e.g., XML processing, cryptography). Furthermore, the benchmark results are influenced 2.3 Real-Time and Safety-Critical Java by the operating system and the hardware platform where the JVM The Real-Time Specification for Java (RTSJ) [6] is an approach executes.