Jeannie: Granting Java Native Interface Developers Their Wishes Martin Hirzel Robert Grimm IBM Watson Research Center New York University [email protected] [email protected] Abstract higher-level languages and vice versa. For example, a Java Higher-level languages interface with lower-level languages project can reuse a high-performance C library for binary such as C to access platform functionality, reuse legacy li- decision diagrams (BDDs) through the Java Native Inter- braries, or improve performance. This raises the issue of face [38] (JNI), which is the standard FFI for Java. how to best integrate different languages while also recon- FFI designs aim for productivity, safety, portability, and ciling productivity, safety, portability, and efficiency. This efficiency. Unfortunately, these goals are often at odds. For paper presents Jeannie, a new language design for integrat- instance, Sun’s original FFI for Java, the Native Method In- ing Java with C. In Jeannie, both Java and C code are nested terface [52, 53] (NMI), directly exposed Java objects as C within each other in the same file and compile down to JNI, structs and thus provided simple and fast access to object the Java platform’s standard foreign function interface. By fields. However, this is unsafe, since C code can violate Java combining the two languages’ syntax and semantics, Jean- types, notably by storing an object of incompatible type in a nie eliminates verbose boiler-plate code, enables static error field. Furthermore, it constrains garbage collectors and just- detection across the language boundary, and simplifies dy- in-time compilers, since changes to the data representation namic resource management. We describe the Jeannie lan- may break native code. In fact, NMI required a conserva- guage and its compiler, while also highlighting lessons from tive garbage collector because direct field access prevents composing two mature programming languages. the Java virtual machine (JVM) from tracking references in native code. Finally, native code that depends on a virtual Categories and Subject Descriptors D.3.3 [Programming machine’s object layout is hard to port to other JVMs. Languages]: Language Constructs and Features; D.2.12 In contrast to the Native Method Interface, the Java Na- [Software Engineering]: Interoperability; D.3.4 [Program- tive Interface is well encapsulated. As a result, C code is ming Languages]: Processors easily portable while also permitting efficient and differing General Terms design, languages Java virtual machine implementations. But JNI’s reflection- like API requires verbose boiler-plate code, which reduces Keywords programming language composition, Java, C, productivity when writing and maintaining native code. Fur- JNI, foreign function interface, xtc, modular syntax, Rats! thermore, it is still unsafe—typing errors cannot be checked statically, nor does the JNI specification require dynamic 1. Introduction checks. It is also less efficient than unportable FFIs that ex- Higher-level languages must interface with lower-level lan- pose language implementation details. guages, typically C, to access platform functionality, reuse This paper presents Jeannie, a new FFI for Java that pro- legacy libraries, and improve efficiency. For example, most vides higher productivity and safety than JNI without com- Java programs execute native code, since several methods promising the latter’s portability. Jeannie achieves this by of even class Object at the root of Java’s class hierarchy combining Java and C into one language that nests program are written in C. Foreign-function interfaces (FFIs) accom- fragments written in either language within each other. Con- plish this task, providing access to C code and data from sequently, Jeannie inherits most of the syntax and semantics from the two languages, with extensions designed to seam- lessly bridge between the two. By analyzing both Java and C together, the Jeannie compiler can produce high-quality Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed error messages that prevent many a maintenance nightmare. for profit or commercial advantage and that copies bear this notice and the full citation Our compiler is implemented using Rats! [29], a parser gen- on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. erator supporting modular syntax, and xtc [28], a language OOPSLA’07, October 21–25, 2007, Montreal,´ Quebec,´ Canada. composition framework. It accepts the complete Jeannie lan- Copyright c 2007 ACM 978-1-59593-786-5/07/0010. $5.00 package my.net; `.C { 1 import java.io.IOException; #include <sys/ioctl.h> #include <errno.h> public class Socket { #include <string.h> static { } System.loadLibrary("Network"); package my.net; } import java.io.IOException; protected int native_fd; protected native int available() throws IOException; public class Socket { … protected int native_fd; } protected native int available() throws IOException 2 #include <jni.h> `{ 1 int num; #include <sys/ioctl.h> 3 #include <errno.h> int fd = `this.native_fd ; #include <string.h> if (ioctl(fd, FIONREAD, &num) != 0) jint 4 Java_my_net_Socket_available(JniEnv *env, jobject this) `throw new 5 { 2 IOException( `_newJavaString(strerror(errno)) ); int num, fd; return num; jclass cls; jfieldID fid; 3 } cls = (*env)->GetObjectClass(env, this); … fid = (*env)->GetFieldID(env, cls, "native_fd", "I"); } fd = (*env)->GetIntField(env, this, fid); if (ioctl(fd, FIONREAD, &num) != 0) { Figure 2. Jeannie version of code from Figure 1. jclass exc = 4 (*env)->FindClass(env, "Ljava/io/IOException;"); 5 (*env)->ThrowNew(env, exc, strerror(errno) ); return 0; and for Java field accesses. In the example, C Snippet 3 reads } the Java field this.native_fd. return num; The JNI interface pointer env is the first argument to } all C functions implementing Java native methods. The corresponding type JNIEnv, declared in jni.h, points to Figure 1. JNI example, with Java on top and C on bottom. a struct of function pointers that are implemented by the Java virtual machine. In the struct, the field access func- guage, including Java 1.4 code and most gcc extensions to tion GetIntField used by Snippet 3 of Figure 1 is repli- C, and its backend produces conventional JNI code. Future cated for several other types, including GetBooleanField work will explore alternative backends that, for example, and GetObjectField. JNI also provides similar functions produce code optimized for specific JVMs. for calling methods. Snippet 4 illustrates JNI’s facilities for Jeannie’s contributions are threefold: raising Java exceptions. Since C lacks exception handling fa- cilities, the C code must emulate the abrupt control flow with Improved productivity and safety for JNI; • an explicit return statement. JNI also provides functions to Deep interoperability of full programming languages; • check for pending Java exceptions, entering and leaving syn- Language composition lessons from a large example. • chronization monitors, acquiring and releasing Java arrays We evaluate Jeannie based on a partial port of the JavaBDD for direct access in C, and many more. Finally, JNI functions library [60] and on micro-benchmarks that exercise new lan- use types such as jint, jclass, and jfieldID, which are guage features. Our results demonstrate that Jeannie code is also declared in jni.h. simpler than the corresponding JNI code, while introducing Before first calling a native method, a Java virtual ma- little performance overhead and maintaining JNI’s portabil- chine needs to link the corresponding native library. The ity. System.loadLibrary call in the static initializer of class Socket in Figure 1 identifies the library’s name, which is 2. Point of Departure: JNI mapped to a shared object file (.so or .dll) on the JVM’s This section provides a flavor of JNI, the Java Native Inter- library path. After loading the object file, the JVM locates face, as described in Liang’s book “The Java Native Inter- the C function implementing available by following the face: Programmer’s Guide and Specification” [38]. Figure 1 name mangling rules of the JNI specification. Repeated calls illustrates how a socket abstraction can use JNI to interface to System.loadLibrary for the same library are ignored, with the operating system. The Java class on top declares a so that the C functions for several Java classes can be in- cluded in the same library. native method available with no Java body. The method is implemented by the body (2) of C function Java_my_net_- Socket_available on bottom. A downcall from Java to C 3. Jeannie Overview looks like any other Java method call. On the other hand, C Figure 2 shows the JNI code from Figure 1 rewritten in code uses a reflection-like API for upcalls to Java methods Jeannie. The outer Jeannie code in Figure 2 corresponds to the Java code in Figure 1. Nested code snippets in both figures that correspond to each other are labeled with the Socket.jni same numbers (1 through 5). The Jeannie file starts with a Inject #include <jni.h> `.C block of top-level C declarations included from header files (1). It continues like a normal Java file with a package Socket.jni.pp declaration, imports, and one or more top-level Java class or interface declarations. Unlike in Java, native methods C Preprocessor in Jeannie have a body (2). The backtick ` toggles to the Socket.jni.i opposite language, which is C in the case of Snippet 2. Consequently, Java code can be nested in C (Snippets 3 Jeannie Syntactic Analysis and 4), and C code can be nested in Java (Snippet 5), to Compiler arbitrary depth. The JNI code in Figure 1 illustrates that such Jeannie AST deep nesting occurs naturally. Semantic Analysis Nested Java code in Jeannie does not need to use fully AST and qualified names: where Snippet 4 in Figure 1 refers to SymbolProcess Table java.io.IOException, Snippet 4 in Figure 2 uses the sim- ple name IOException, which Jeannie resolves using the Translation import declaration.