Towards Performance Measurements for the Java Virtual Machine’s invokedynamic Chanwit Kaewkasi School of Computer Engineering Suranaree University of Technology Nakhon Ratchasima, Thailand 30000 [email protected]
ABSTRACT JVML are strongly typed. Every symbol is linked before This paper presents a study of a Java Virtual Machine prototype executing a referencing instruction [15]. In the JVM, there are from the Da Vinci Machine project, defined by JSR 292. It four instructions for invoking different kinds of Java method describes binary translation techniques to prepare benchmarks to calls. These instructions serve Java programs perfectly, but not run on the invokedynamic mode of the prototype, resulting that well for other languages that use the JVM as their runtime in the invokedynamic version of the SciMark 2.0 suite. systems [4], [5], [8], [11], [12]. Benchmark preparation techniques presented in this paper are The Da Vinci Machine project [3], also known as the Multi- proven to be useful as the invokedynamic version of Language Virtual Machine (MLVM), is an effort to investigate benchmark programs successfully identified strange slowness and develop prototypes that support a new bytecode instruction, behavior of the invokedynamic mode of the server virtual invokedynamic [15], to extend the Java Virtual Machine machine. (JVM) to other languages, beside the Java language itself. Surprisingly, benchmarking results show that the invoke- Recently, several language implementers have adapted the JVM dynamic mode with direct method handles on the server as a runtime system for their languages, which include virtual machine is just 2-5 times slower than native Java JavaScript [11], JRuby [12], Scala [13], Groovy [8], Clojure [4] invocations, except the Monte Carlo benchmark. But this mode and Jython [5]. Dynamically typed languages among them have on the client virtual machine still requires further performance been trying to simulate dynamic invocation using the existing tuning. infrastructure provided by the JVM, and resulting in several magnitudes of performance degradation (see Chapter 7 in [6]). After years of drafting the Java Specification Request (JSR) 292 Categories and Subject Descriptors by its Expert Group, the referential implementation of C.4 [ Measurement Technique s]; D.3.3 [ Programming invokedynamic has now been a part of the early access Languages ]: Language Contructs and Features; D.3.4 builds of Java Development Kit (JDK) and OpenJDK 7, while Optimization, Code generation. the Da Vinci Machine project has continued to implement several advanced features separately. General Terms Reported by members in the development mailing list of Measurement, Performance, Languages MLVM, the performance of dynamic invocation in recent builds of OpenJDK has been around 10 times faster than invocation Keywords through Java’s reflection APIs [1]. Thus, it is worth measuring Bytecode, invokedynamic, method invocation, benchmarks the performance of the invokedynamic mode of the JVM, even in the early stage of its development, to help identifying possible bottlenecks in both JDK and OpenJDK 1. INTRODUCTION implementations. Java Virtual Machine Language (JVML) is an intermediate language defined by the Java Virtual Machine Specification [9]. Following one of the design principles of JSR 292 that focuses JVML has been designed to be statically verifiable and on performance characteristics of the new bytecode [15], this symbolically linkable as its class file format contains type paper aims at finding translation techniques for replacing all information enough to do so. Classes, methods and fields in traditional Java invocations by equivalent invokedynamic instructions in available benchmark programs in order to measure performance of this new execution mode. This paper chose the SciMark 2.0 benchmark suite [14] as input programs Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are for the translation process because of 1) its simplicity, 2) all not made or distributed for profit or commercial advantage and that benchmarks are CPU-bound, and 3) these benchmarks mainly copies bear this notice and the full citation on the first page. To copy contain primitive operations, thus refactoring them as method otherwise, or republish, to post on servers or to redistribute to lists, calls and replacing them by invokedynamic can represent requires prior specific permission and/or a fee. how well MLVM performs execution and optimization for VMIL’10 , 17-OCT-2010, Reno, USA trivial final methods. Copyright © 2010 ACM 978-1-4503-0545-7/10/10…$10.00
One of the problems this paper addresses is that benchmarks for the class initializer [15]. Recently, JSR 292 allows specifying a a new runtime, like the invokedynamic mode, have not been bootstrap method per call site, rather than having only one widely implemented at this early stage, as the technology is bootstrap method per class [16]. relatively new and its adoption is just started. Another problem Before the bootstrap method returning a call site object, the call is that invokedynamic ’s target users are language site’s target must be assigned an instance of a method handle. implementers. Implementing benchmarks in various languages is difficult to directly compare the performance of 2.3 Method Handle invokedynamic with native Java invocations. Other factors A method handle is a lightweight structure that has been in these languages, such as target method selection algorithms, designed to invoke a JVM method [15]. There are several kinds may affect the measurement. of method handles defined by JSR 292. However, this paper The main contribution of this paper is to present the results of uses only the simplest one, which is called the direct method performance measurements for the JVM’s invokedynamic handle, to minimize runtime overheads that may affect the using the refactored version of the SciMark 2.0 benchmark suite experiments. A direct method handle can be obtained by using [14]. The hypothesis is that the performance of invo- an instance of MethodHandles.Lookup , which provides kedynamic should not be too slow compared with native Java finder methods for each kind of JVM calls, for example, invocations. Other contributions are 1) a binary translator for • findStatic for static methods, compiling normal Java programs into equivalent invoke- dynamic programs, 2) an identification of a potential • findSpecial for inherited methods (super calls), bottleneck in the server (C2) virtual machine (VM), and 3) a limitation of the bytecode verifier when taking invoke- • findConstructor for constructors, and dynamic into account. • findVirtual for normal and interface calls. This paper is organized as follows. Section 2 reviews the invokedynamic mode and its components. Section 3 3. TRANSLATION TO invokedynamic describes a mapping from other invocation instructions to From Section 2, finder methods are reviewed to show that invokedynamic . Section 4 discusses benchmarks prepa- different categories of method handles can be obtained from ration, experiments and results. Section 5 continues on different kinds of them. This section discusses semantics when discussion about important implementation issues, namely, the mapping from traditional invocation instructions to equivalent use of findSpecial and bytecode verification when taking invokedynamic instructions using these finder methods. invokedynamic into account. Section 6 discusses related Semi-formal rules in Figure 1 are used to implement a binary work, and then this paper ends with conclusion and possible translator that compiles class files into the invokedynamic future work in Section 7. version.
2. REVIEW invokedynamic This section reviews the invokedynamic mode of the JVM. Static Method Call: It consists of a newly defined bytecode, invokedynamic , and h=findStatic( CmD , , ,type( e )) other key components, such as the bootstrap method, and invokestatic(CmeD , , ):→ invokedynamic( IheD , , ): method handles. However, adapter method handles are not discussed here as they are beyond the scope of this paper. Constructor Call:
cCh:= findConstructor( Ce ,type( )) 2.1 Bytecode An invokedynamic is a 5-byte instruction. It has no scope invokespecial(C ,
00 62 hese hese LU LU
licity, licity, this refactored irst one re- described in is simple, so Sparse Sparse matmul Client/Unmodified Client/Refactored Client/MLVM Server/Unmodified Server/Refactored Server/MLVM Carlo Carlo
13.7% Monte ck bars with numbers) with numbers) bars ck
U 3.9% L Benchmark score (Mflops) (Mflops) score Benchmark FFT FFT SOR 31.1% ars). ars). Table 1. Experimental results. results. 1. Experimental Table site site
5.3% Score
Compo-
matmul
e
s r
4.5% Spa Configuration version of SciMark benchmarks used 2.0, in this paper. there The first one are version of is SciMark the 2.0, two and the second one versions is the compiledf by of a binary translator according t to rules Section 3. The main reason of choosing SciMark 2.0 is that it that it can be manually refactored. Beside its simp Client / Client / Unmodified 547.08 Client / 436.59 Refactored 634.91 557.48 Client / 98.65 471.93 MLVM 627.85 641.89 / Server 937.41 96.79 Unmodified 28.47 637.05 895.33 / Server 939.72 618.17 9.98 Refactored 909.51 893.68 / Server 37.13 342.32 603.63 MLVM 741.64 25.52 906.24 1865. 346.29 33.00 208.52 732.64 36.71 1879. 134.15 407.03 15.28 230.69 255.44
25.9%
te Carlo te n
Mo V 44.8% is an ethods e ion (white bars) and the invokedynamic version (bla version invokedynamic the and (white ion bars) ch is an from from the in these and enchmarks enchmarks . . Auxiliary name-and-
5.8% the original handle must
as a method xecution will SOR m der methods. It means C isa direct method means a sequence h : : ē to represent classes, e
21.7% E , , against the original version of SciMark 2.0 SciMark of (gray version b original the against D , , and , e C as an object,
2.3% invokedynamic FFT . . instruction, must contain name- C this means this instruction returns an , , D c . A dot notation means sequence : : n
23.3% e java.dyn.InvokeDynamic …
returns a sequence of types of expressions.
0
5.2% re , and of course writing o e . . They use
D
is the constructor andname,
Sc
e t
invokedynamic
osi
void mp type() invoke<*>(…)
represents
I Co
concatenation. An arrow means have the similar translation. semantics to those Finder in JSR 292 m [16] function Writing benchmark suite is fit for measuring performance of configurations are insignificantly small. SciMark 2.0 contains invokedynamic because it is CPU-bound, i.e. there is no five benchmark programs. Performance is measured in Mflops benchmark program in this suite that performs input/output unit. The first column, namely Composite Score, is an average operations. More importantly, using this benchmark suite to of remaining five columns. Experimental results are shown in measure invokedynamic reflects how well this new mode Table 1. can execute and optimize trivial final methods that contain only Performance results of invokedynamic on the server VM a simple primitive operation. from FFT, SOR and Sparse matrix multiplication are faster than The original version of SciMark 2.0 cannot be used to correctly expected. They are about 22-44% of native Java invocations. measure invokedynamic because its benchmarks mainly use The invokedynamic mode on the server VM performed primitive operations. Thus, refactoring all primitive operations strangely with the Monte Carlo benchmark as it is clearly slower to method calls is required. In fact, there are two major reasons than the client VM, while this benchmark on the normal server why refactoring is required for existing benchmark programs VM ran faster than that on the normal client VM. Figure 2 before invokedynamic translation can be applied First, illustrates the normalized benchmark results of all six converting all primitive operations to be method calls in order to configurations against the original version of SciMark 2.0. make benchmark applicable for invokedynamic . Secondly, For the invokedynamic mode of the client VM, it is clearly avoiding limitations of the current build of MLVM by manually that its performance is not that good due to the beginning of its changing every unsupported method signature. For every implementation. Note that the MLVM development team has method to be called through invokedynamic , internal firstly implemented invokedynamic for the server VM, and mechanism of MLVM requires reordering its primitive then ported it to the client VM later. parameters to be after reference parameters for performance reasons. For example, void m1(int, String) is invalid to dispatch, while void m2(String, int) is a valid method 5. IMPLEMENTATION NOTES signature. However, the automatic reordering algorithm has not There are two additional issues found during the implementation been implemented in MLVM yet, at the time of writing this of the binary translator for compiling benchmark programs. paper. A manual process is still needed to do so. Firstly, the finder method findSpecial should be able to obtain method handles of
1 The invokedynamic version of SciMark 2.0 is available to download from http://github.com/chanwit/jdyn . 9. REFERENCES [9] Lindholm, T. and Yellin, F. 1999 Java Virtual Machine Specification . 2nd. Addison-Wesley Longman Publishing Co., Inc. [1] Bodden, E. Invokedynamic slower than reflection? (ca. 2010) URL: http://www.mail-archive.com/mlvm- [10] Mathew, J. A., Coddington, P. D., and Hawick, K. A. 1999. dev@openjdk.java.net/msg01720.html. Analysis and development of Java Grande benchmarks. In Proceedings of the ACM 1999 Conference on Java Grande [2] Blackburn, S. M., Garner, R., Hoffmann, C., Khang, A. M., (San Francisco, California, United States, June 12 - 14, McKinley, K. S., Bentzur, R., Diwan, A., Feinberg, D., 1999). JAVA '99. ACM, New York, NY, 72-80. DOI= Frampton, D., Guyer, S. Z., Hirzel, M., Hosking, A., Jump, http://doi.acm.org/10.1145/304065.304101. M., Lee, H., Moss, J. B., Moss, B., Phansalkar, A., Stefanovi ć, D., VanDrunen, T., von Dincklage, D., and [11] Mozilla Corp. Rhino: JavaScript for Java (ca. 2010). URL: Wiedermann, B. 2006. The DaCapo benchmarks: Java http://www.mozilla.org/rhino. benchmarking development and analysis. In Proceedings of [12] Nutter, C. et al. JRuby project (ca. 2010). URL: the 21st Annual ACM SIGPLAN Conference on Object- http://kenai.com/projects/jruby. Oriented Programming Systems, Languages, and [13] Odersky, M. and Zenger, M. 2005. Scalable component Applications (Portland, Oregon, USA, October 22 - 26, abstractions. In Proceedings of the 20th Annual ACM 2006). OOPSLA '06. ACM, New York, NY, 169-190. SIGPLAN Conference on Object-Oriented Programming, DOI= http://doi.acm.org/10.1145/1167473.1167488. Systems, Languages, and Applications (San Diego, CA, [3] Da Vinci Machine project, the. (ca. 2010). URL: USA, October 16 - 20, 2005). OOPSLA '05. ACM, New http://openjdk.java.net/projects/mlvm. York, NY, 41-57. DOI= [4] Hickey, R. et al. Clojure project (ca. 2010). URL: http://doi.acm.org/10.1145/1094811.1094815. http://clojure.org. [14] Pozo, R. and Miller, B. Java SciMark 2.0 (ca. 2010). URL: [5] Hugunin, J. et al. Jython project (ca. 2010). URL: http://math.nist.gov/scimark2. http://www.jython.org. [15] Rose, J. 2009. Bytecodes meet combinators: [6] Kaewkasi, C. An Aspect-Oriented Approach to Productivity invokedynamic on the JVM. In Proceedings of the Third Improvement for A Dynamic Language using Typing Workshop on Virtual Machines and intermediate Concerns . PhD thesis, The University of Manchester, 2009. Languages (Orlando, Florida, October 25 - 29, 2009). VMIL '09. ACM, New York, NY, 1-11. DOI= [7] Kiczales, G., Lamping, J., Mendhekar A., Maeda, C., http://doi.acm.org/10.1145/1711506.1711508. Lopes, C., Loingtier, J. and Irwin, J. Aspect-Oriented Programming, In Proceedings of ECOOP'97 , Lecture [16] Rose, J. 2008. JSR 292: Supporting dynamically typed Notes in Computer Science, Springer, vol.1241, pp.220- languages on the Java platform. 242, 1997. http://jcp.org/en/jsr/detail?id=292. [8] Koenig, D., Glover, A., King, P., Laforge, G., and Skeet, J. [17] Standard Performance Evaluation Corporation. SPEC 2007 Groovy in Action . Manning Publications Co. jvm98 Documentation, release 1.03 edition, March 1999.