Cloud-Agnostic Serverless Java with the Fn Project and Graalvm

Topics Archives Downloads Subscribe

Cloud-agnostic serverless Java GRAALVM with the Fn project and GraalVM Cloud-agnostic serverless Java Getting started with the cloud with the Fn project and GraalVM Keeping the code portable

Mind your programming Your choice of language has a huge language impact on application portability— The serverless Fn project which can be tremendously important Deployment with Kubernetes for startups building serverless

Oracle Functions applications.

GraalVM native images by Anton Epple

Using GraalVM with Fn September 25, 2020 Conclusion Cloud platforms and especially function-as-a-service (FaaS) Dig deeper services are great for startups. They are easy to get started with, you pay only for what you use, you have guaranteed high availability and low maintenance, and applications scale automatically with growing demand.

When you look at an architecture, think about portability. Although there are efforts to develop standards for portable cloud applications, most platforms offer an integrated system with a set of unique and proprietary tools. That makes it easy to get on board but expensive to leave.

This article is the ﬁrst in a two-part series looking at tools, best practices, and technologies that help to keep applications portable in a multicloud world. In this part, I am going to focus on the beneﬁts of native serverless functions in Java with the Fn project and GraalVM native images.

I’ve created a sample project to demonstrate the tools and techniques shown in this article. If you’d like to deploy the code to Oracle Cloud Infrastructure, as described in the article, I’ve created a tutorial with a helper script.

Getting started with the cloud

In 2018, an old friend asked me for my opinion on his ideas for a startup to manage access rights to electronic locks that would also work in off-grid environments. Mobile apps provide the missing connectivity to the customers’ locks and sensors and connect them to the back end. We did some prototyping and quickly saw that the ideas were feasible. I decided to join the team, and by November 2018, we had founded Smart Access Solutions (SAS).

For startups like ours, IaaS helps focus all efforts on product development without the need to worry about building out data centers. In our research, the best IaaS option to balance scalability and cost control was the FaaS model, also known as a serverless architecture. Since we planned our core product to be a cloud-based system, we decided to go serverless, as you can see in Figure 1.

Figure 1. The Smart Access Solutions secure serverless cloud

Keeping the code portable

The serverless landscape is very different from traditional server-focused Java. It’s harder to choose specific tools and technologies, in part because there are no widely adopted standards, and it’s easy to get locked into a specific platform. Yes, the initial cost of development is attractively low on a well- integrated platform with specialized frameworks, tools, and services—but choosing that platform-specific platform could make it hard to move to a different provider later on.

In a startup, and in many established organizations, it’s a good strategy to stay as platform independent as possible. Design for portability, so you are prepared for a situation such as when a cloud provider stops offering a certain service. You want to be free to move to a different provider with better technology or a more attractive pricing model at any time, with relative ease.

Application portability is inﬂuenced by many factors. A good guiding principle is to look at two cloud providers and think about how your application needs to be designed to run on either platform.

Each provider will have different services for data and file storage, networking, messaging, authentication and authorization, monitoring, and so on. This exercise will help identify the platform-specific services you’re using. You can then minimize any negative impact by replacing platform-specific services with alternative services available on both platforms.

Portability will also guide you in creating abstractions to protect your business code from proprietary APIs. How easy it is to create these abstractions and enforce their API contracts can depend on the programming language. Which programming language to use seems like a minor concern, because most platforms offer runtimes for many languages. But, in fact, your choice of language has a huge impact on portability.

Mind your programming language

Take the Node.js JavaScript runtime, as an example. Node.js worked very well for us in several small customer projects. Serverless functions are typically run in an execution environment like this, where some containerized runtime is launched for each request to execute the function. Compared to Java, JavaScript has a lower cold-start latency and memory footprint.

As a result, JavaScript performs well even if serverless functions are invoked only occasionally. As the utilization cost of serverless functions is billed by execution time and memory usage, using JavaScript can be cheaper than using Java. As an additional beneﬁt, JavaScript development is very fast without the compilation step. Functions are small and isolated units of code, so the missing language concepts for deﬁning and enforcing strict API contracts shouldn’t matter that much. So Node.js seemed to be a good choice. But that’s not what we chose.

After some deliberation, we decided to use Java for our core product, SAS Secure Cloud Core, a multitenant system for managing access rights, sensor data, and predictive analysis.

Why? We need to be able to keep our application as portable as possible, so we can switch cloud providers if desired and also run our application in a multicloud environment or even on- premises.

All vendor-speciﬁc parts need to be kept separate from business logic and hidden behind abstractions, so we can quickly switch to different data storage, another messaging queue, or other services. That’s where Java comes in.

In Java there’s fine-grained control of the visibility of packages, methods, and fields. Interfaces and abstract classes make it easy to define an API or service provider interface (SPI). The compiler naturally enforces the API contract and prevents unintended uses. This is perfect for collaboration in a larger team.

Persisting data is a typical example of such an abstraction. In Java, you can simply create an interface to keep transparent whether the data is stored in a key-value store, a relational database, a graph database, or any incarnation of these. Here’s a simpliﬁed example of what we’re using:

public abstract class Repository { private final Class t; protected Repository (Class t){ this.t = t; }; public abstract Optional create(T enti public abstract Optional get(String id public abstract List getAll(); public abstract boolean update(T entity); public abstract boolean delete(String id)

public final boolean canHandle(Class v) { return t.isAssignableFrom(v); } }

For business code, you can simply provide a ServiceLoader, for example:

public class RepositoryServiceLoader {

public static Repository getReposi

ServiceLoader< Repository> loader = S for (Repository repository : loader) if (repository.canHandle(t)) { return repository; } } throw new UnsupportedOperationExcepti } }

An actual implementation of the service can now use whichever relational database or key-value store your cloud provider supports. Here’s a dummy implementation based on a HashMap you could use for testing:

public class MockRepository extends Repositor private static Map u public MockRepository() { super(MockElement.class); }

@Override public Optional create(MockE MockElement put = users.put(entity.id return Optional.of(entity); }

@Override public Optional get(String i return Optional.ofNullable(users.get( }

@Override public List getAll() { return new ArrayList<>(users.values() }

@Override public boolean update(String id, MockElem MockElement get = users.get(id); return get == null ? false : users.pu }

@Override public boolean delete(String id) { return users.remove(id)==null ? false }

}

Implementations can be registered either as Java modules or via the classpath. If you don’t like the service locator pattern, you can alternatively use an injection framework to provide the services. Either way it’s simple, straightforward, and very well supported in Java.

Dynamic languages such as JavaScript fall short in this aspect. Although we felt conﬁdent using JavaScript for small projects that had one or two developers and a limited lifespan, we decided that for a large multimodule project that forms the core of our business, Java is a much better ﬁt, despite its drawbacks in the serverless space.

Strongly enforced API contracts are also a huge beneﬁt in guiding developers and protecting them from accidentally introducing unwanted dependencies. Users of your API should be able to work with your API even if they are completely clueless about its inner workings.

A strongly typed language such as Java is an invaluable help here, because its IDEs assist developers in writing code that fulﬁlls the contract and provide documentation directly in the editor during coding. In addition to that, the available tooling for larger projects is really mature. Using the Apache Maven project management system, the setup of multimodule builds is easy and the sharing of build artifacts can be managed using Maven repositories. This works great and is integrated in most DevOps platforms, sometimes even for free.

The serverless Fn project

An interesting platform for writing serverless functions in Java (or other languages) is the Fn project, an open source, container native serverless platform based on Docker. It packages your business code as a Docker container using a Function Development Kit (FDK). Fn aims to be cloud agnostic; the landing page says this: “The Fn project is an open-source container-native serverless platform that you can run anywhere—any cloud or on-premise.” Thus, we thought it would be a good addition in our portability toolbox.

The installation process for Fn is simple. To start with, you need Docker installed. On macOS and Linux, you can install the Fn server and command-line interface (CLI) via the shell. On Windows, at least for testing, you should use a virtual machine running Linux, because the Docker setup is difﬁcult.

$ curl -LSs https://raw.githubusercontent.com $ fn start

[ // omitted some lines ] ______/ ____/___ / /_ / __ \ / __/ / / / / /_/ /_/ /_/ v0.3.339

time="2020-05-13T12:23:16Z" level=info msg="F

Once the local installation is up and the development server is running, you can start to develop your functions using the FDK. The following command will create a Maven project in a new directory named function:

$ fn init --runtime java function

In the src folder, you’ll ﬁnd the following demo code:

public class HelloFunction {

public String handleRequest(String input) String name = (input == null || input

System.out.println("Inside Java Hello return "Hello, " + name + "!"; }

}

The generated source code is a method that accepts input as a string and returns the result as a string. There are no dependencies on the framework itself. You don’t need to add any abstractions to protect your business code from leaking dependencies; the code is already perfectly portable. For deployment you need to create an app, which is a unit that groups belonging functions. Then call your function via the Fn CLI, for example: $ fn create app java-app $ fn deploy --app java-app --local $ fn invoke java-app function time="2020-05-17T07:47:58Z" level=info msg="s Hello, world!

The function can also be invoked via an HTTP endpoint. You’ll need to ﬁnd out the address ﬁrst, for example:

$ fn inspect function java-app function { "annotations": { "fnproject.io/fn/invokeEndpoi }, "app_id": "01E88VPC6ANG8G00GZJ0000002 "created_at": "2020-05-14T05:51:18.46 "id": "01E88VSCE1NG8G00GZJ0000003", "idle_timeout": 30, "image": "function:0.0.2", "memory": 128, "name": "function", "timeout": 30, "updated_at": "2020-05-14T05:51:18.46 } $ curl -X "POST" -H "Content-Type: applicatio time="2020-05-18T10:55:10Z" level=info msg="s Hello, world!%

To accept JSON input, convert your method signature to accept a plain old Java object (POJO). Fn will automatically map ﬁelds from incoming JSON messages to your POJO’s ﬁelds and pass them to your handler method as your objects. Again, this doesn’t require any additional dependencies in your code.

Deployment with Kubernetes

Fn projects are not only portable on the source-code level. The platform itself can run in any cloud environment.

To use Fn in production, the project provides a Terraform module for deploying a production-ready environment, including monitoring and metrics, to different clouds.

The only precondition is a Kubernetes cluster, which you can also run on-premises. For our startup, most of our customers will use our hosted solution, but being able to run our functions on- premises was a requirement because a few customers will be disconnected from the internet.

And although setting up a Kubernetes cluster is simple in most cloud environments, and it might be an interesting option later, it’s not the true serverless approach we were aiming for. Fortunately, there’s also a real serverless option: Oracle Functions. Oracle Functions

Oracle Functions is available as part of Oracle Cloud Infrastructure.

With Oracle Functions, you can deploy your Fn functions to a managed environment. This can be done directly via the Fn CLI (see Figure 2). If you’d like to try it yourself, the free Oracle Cloud Infrastructure test account is sufﬁcient. Setup and deployment are described in the tutorial that accompanies this article.

Figure 2. Fn deployment locally using a self-managed Kubernetes cloud and fully managed Oracle Functions

You can run the platform on-premises or in any cloud, or you can use Oracle Functions as a hosted serverless solution. Since the platform doesn’t introduce any dependencies into your function code, reusing your code in a different environment is easy as well. This makes Fn a solid choice to keep your application portable. Yet, to this point, you still have to live with the Java cold-start and memory-usage issues. That’s where another project turns out to be very useful: GraalVM.

GraalVM native images

When we ﬁrst started developing serverless functions in Java, the cold-start issue was very noticeable for us, because for some functions, we had to increase the default timeout our vendor had set for executing a function.

Storage size and memory usage was also a problem. We had to again increase the defaults to avoid out-of-memory issues. We were, therefore, very excited when GraalVM was released.

GraalVM is a JVM and JDK based on OpenJDK and also a whole ecosystem of tools aimed at improving high-performance, JVM-based languages and polyglot applications. I’ve been following this great project by Oracle Labs in Linz since it originated from the “Maxine” project at Sun Microsystems almost 10 years ago. Among many things, GraalVM also offers a solution to the cold-start issue in Java.

Java application performance is usually great, because the HotSpot JVM’s just-in-time (JIT) compiler analyzes the runtime behavior and uses this information to guide compilation of bytecode to native instructions for the most relevant parts of the code. However, this process requires a warm-up period during which your code is ﬁrst executed. Also, class loading and veriﬁcation take some time at application startup. That’s why a measurable delay or even a timeout might happen whenever a new serverless container is started to serve a request.

How can you keep the serverless function warm—that is, avoid any cold-start delay? The most popular solution is to periodically create events that execute your function; this basically converts your stateless serverless function into a stateful server.

There are drawbacks to this approach. Since each container serves only a single request at a given time, additional requests will start a new container, so you’ll need to warm up a pool to serve at least a couple of concurrent requests. You may even have to adjust your code to recognize warm-up events so you can exit early and not block real requests. So instead of a serverless function that is free when not used, you’ll pay for an idle cluster of containerized servers.

Fortunately, with the release of GraalVM, that trick can be avoided. GraalVM can create a native executable from your Java code, called a native image. During this process, the native-image tool performs a static analysis of your code to pack only the required dependencies, thereby reducing the image size to the bare minimum.

Using GraalVM with Fn

You can set up an Fn project with a native image using the Fn CLI, as follows:

$ fn init --init-image fnproject/fn-java-nati

This creates the required resources including a Dockerﬁle. The executable GraalVM produces is platform-speciﬁc, so to run the image on your target platform, it needs to be compiled on the same operating system. For example, an executable created on Apple macOS or on Microsoft Windows wouldn’t work in the BusyBox Linux image that is used for our startup company’s function container. Fn works around this issue by using a multistage Docker build.

The container brings along all required tools and nothing needs to be installed on the developer machine. In addition, I found it extremely helpful that the conﬁguration for reﬂection, Java Native Interface (JNI), dynamic proxies, and resource loading of the runtime itself is already provided in this build.

By the way, we tried native images with other cloud providers’ serverless platforms, and it can take quite some time to set up everything correctly. With Fn, our “Hello World” example worked out of the box without any further conﬁguration. That’s why the seamless integration with Fn can make your life a lot easier— with faster cold-start times and reduced memory consumption.

Let’s compare the call to the earlier “Hello World” function using GraalVM against the native version. In our local measurements, the cold-start time using GraalVM was reduced by around 30% from 720 ms to 476 ms. At the same time, memory usage was reduced by almost a factor of 20, from 18 MB to 1 MB.

If you want to squeeze out the absolute last bit of performance, you can also run the application using GraalVM and collect optimization data to later guide the native image compilation.

Conclusion

For our startup’s SAS Secure Cloud Core multitenant system, portability and stability were the most important concerns when we chose our tools and frameworks. Java turned out to be a great choice for serverless applications.

With mature build tools, testing libraries, and code repositories, Java provides a great environment to keep large applications maintainable. Static typing, ﬁne-grained access modiﬁers, and a module system give API designers better control over their APIs than dynamic languages provide.

At the same time, these APIs provide guidance to API users and prevent accidental misuse. They also provide a way to keep your business code portable through simple layering and abstractions, including a service infrastructure built into the language.

Fn is a good way to keep a serverless project portable and cloud agnostic. The framework doesn’t introduce additional dependencies, so the source code stays portable and testable. Functions can be deployed in a hosted service, in any cloud environment, and also on- premises. The seamless integration with GraalVM native images solves the cold-start issues, while also reducing the memory footprint.

Java, Fn, and GraalVM are very well integrated, yet they can be used independently. You are free to choose in what context you’d like to use them.

In the next part of this series, I’ll complete your toolbox with platform-independent tools for managing your multicloud infrastructure.

Dig deeper The Fn project GraalVM: Native images in containers Java frameworks for the cloud: Establishing the bounds for rapid startups GraalVM native image Improving performance of GraalVM native images with proﬁle-guided optimizations An introduction to the Fn Project Fn project’s Terraform module Oracle Functions concepts Differences between Oracle Functions and Fn project Using the Fn project CLI with Oracle Functions

Anton Epple Based in Munich, Germany, Anton “Toni” Epple is a consultant worldwide for a wide variety of companies, ranging from startups to Fortune 500 companies. In 2013 he joined the Java Champions and received a JavaOne Rockstar Award. In 2014, he received a Duke's Choice Award. Epple is an Apache committer for the NetBeans project. In 2019, he cofounded Smart Access Solutions, a company focused on cloud-based access rights management.

Share this Page FacebookTwitter LinkedInEmail

Contact About Us Downloads and Trials News and Events US Sales: +1.800.633.0738 Careers Java for Developers Acquisitions Global Contacts Communities Java Runtime Download Blogs Support Directory Company Information Software Downloads Events Subscribe to Emails Social Responsibility Emails Try Oracle Cloud Newsroom