Quarkus and Camel K: the Critical Path to Serverless Integration

Quarkus and Camel K: The Critical path to Serverless Integration

Pat Carey / Phil Prosser Domain Solution Architect 15th October 2019 Agenda

Ripping apart the forum demo :

Introduction to Quarkus + demo Introduction to Camel K + demo Quarkus Core Philosophy

Quarkus aims to do as much work as possible at build time, to keep the resulting application as small and fast as possible Quarkus Extensions

... Netty MP JWT RESTEasy Messaging MP Reactive MP OpenAPI Narayana JTA Eclipse Vert.X Apache Camel Hibernate ORM Agroal (conn pool) Hibernate Validator

Quarkus Core Arc (DI) Jandex Gizmo Graal SDK

HotSpot SubstrateVM Quarkus Consists of Two Parts

Core Runtime Undertow Runtime ArC Runtime

Core Deployment Undertow Deployment ArC Deployment Build and Runtime Split

Build Time Runtime All code that determines what should Runtime should only contain classes happen at startup belongs to the that are needed to actually run the deployment module. This will include application. The build will generate things like annotation scanning and bytecode to directly start what is parsing. required. Startup Comparison

Quarkus does most of the Traditional Application Quarkus work that is traditionally ● Scan for annotations ● Start runtime services done at startup and moves ● Parse descriptors it to build time. This results ● Select enabled features in smaller, faster ● Generate metamodel applications that use less ● Start runtime services memory. The end result of a Quarkus build should be just enough customised bytecode to start the services your application requires Not just startup speed

Loading classes is slow, but also takes a lot of memory that is never released. Traditional Java apps hold lots of classes in memory that are used once at startup.

01 Memory usage 140 Mb Traditional Java Application 74 Mb Quarkus JVM

13 Mb Quarkus GraalVM But of course… Quarkus Improves Startup Time

REST

Quarkus + GraalVM 0.014 Seconds

Quarkus + OpenJDK 0.75 Seconds

Traditional Cloud-Native Stack 4.3 Seconds

REST + CRUD

Quarkus + GraalVM 0.055 Seconds

Quarkus + OpenJDK 2.5 Seconds

Traditional Cloud-Native Stack 9.5 Seconds Extensions vs Subsystems

Quarkus extensions are much more lightweight than EAP Subsystems

In many respects they are closer to CDI portable extensions Development Mode

This split only applies to production applications, not quarkus:dev

In developer mode both parts of an extension are present, allowing for fast reload DEMO TIME Extensions for Quarkus Summing Up

Architectural Advantages Integrating Frameworks Quarkus’ unique architecture gives it a Extensions can be used to make home major advantage when memory usage grown and 3rd party frameworks run on and startup time are important Quarkus

Lightweight Extensions Enabling Native Compilation Quarkus extensions are lightweight, and Quarkus provides API’s to help compile can be used in a similar manner to CDI applications to a native image portable extensions Introduction to Camel K What is Apache Camel K?

A lightweight integration platform based on Apache Camel, born on Kubernetes, with serverless superpowers

A community-driven project A sub-project of Apache Camel started on August 31st, 2018 Based on the operator SDK

https://github.com/apache/camel-k What?

Camel DSL, based on EIPs... 1. Create an integration ﬁle (Java, Groovy, Kotlin, JS, XML, …)

// Lookup every second the 'www.google.com' domain 3. It runs on name and log the output Kubernetes / OpenShift from('timer:dns?period=1s') .routeId('dns') .setHeader('dns.domain') .constant('www.google.com') .to('dns:ip') .log('log:dns');

2. Run it

$ kamel run integration.groovy

Camel K CLI (optional) How?

Dev Environment Cluster

Integration Custom Resource Camel K Operator

Live updates!

kamel CLI Running Pod

Fast redeploy! Less than 1 second! Tailored for cloud-native development experience https://github.com/operator-framework/operator-sdk DEMO TIME Camel K + Knative Knative

Knative defines building blocks for “Serverless” applications

Serving Eventing

Scale-to-zero and request-driven Event binding and delivery for compute messaging style integration

<> <> <> <> <>Service <>Channel Service Channel Service Channel

https://knative.dev Camel K Knative Proﬁle

Camel K Operator from("knative:channel/a") .to("http:my-host/api/path");

kind: Deployment kind: Integration Knative no apiVersion: camel.apache.org/v1alpha1 # regular one metadata: profile? ... name: my-integration spec: sources: yes - name: source.groovy content: |- from("knative:channel/a") kind: Service .to("http:my-host/api/path") apiVersion: serving.knative.dev/v1alpha1 ... Knative Serving

rest().post("/path") .to("xx:system1") Kubernetes Namespace .to("xx:system2") 300+ components! Knative Service

System 1

System 2

There’s no container if no one needs it! Knative Serving

rest().post("/path") .to("xx:system1") Kubernetes Namespace .to("xx:system2") 300+ components! Knative Service

Request Pod System 1

System 2

A container is created only when needed! Knative Serving

rest().post("/path") .to("xx:system1") Kubernetes Namespace .to("xx:system2") 300+ components! Knative Service Request

Request Pod System 1

Request Pod System 2 Pod

Multiple containers under high load! Knative Eventing

Mmh? Isn’t that like JMS? https://cloudevents.io/

Subscription

Subscription Producers Channel Consumers

Subscription

Kafka, In-memory, ... Knative Eventing

from("knative:channel/a") .to("xx:system1") Kubernetes Namespace .to("xx:system2") 300+ components! Knative Knative Service Subscription

System 1 Cloud Event Channel a Pod

System 2 Knative Camel Sources Knative Camel Sources

Camel components... “Cloud Edition”

CamelTwitterSource

CamelFacebookSource 300+ components!

CamelTelegramSource

...

CamelServiceNowSource Knative Camel Sources

Camel Sources are an addon to Knative, currently in kind: Integration Knative eventing-contrib repository apiVersion: camel.apache.org/v1alpha1 metadata: It translates Camel-based Knative eventing sources name: my-integration spec: into Camel K integrations source: - content: | - from: uri: timer:tick?period=3s apiVersion: sources.eventing.knative.dev/v1alpha1 steps: kind: CamelSource - set-body: metadata: constant: Hello world! name: camel-timer-flow-source - to: spec: uri: knative://endpoint/sink source: language: yaml flow: | name: flow.yaml from: traits: uri: timer:tick?period=3s knative: steps: configuration: - set-body: configuration: constant: Hello world! Camel Sources '{"services":[{"type":"endpoint","protocol":"http sink: ","name":"sink","host":"camel-test-channel-4dlgs. apiVersion: eventing.knative.dev/v1alpha1 Operator camel-k.svc.cluster.local","port":80,"metadata":{ kind: Channel "service.path":"/"}}]}' name: camel-test

https://github.com/knative/eventing-contrib/tree/master/camel/source Extending Serverless Integration

Service Service Integration Integration Service function KNative function Service

Integration Channel Integration function function Channel

Integration Integration

function Channel External SaaS External SaaS External SaaS Integration Channel function External SaaS

Channel Microservices Integration Microservices function Microservices Channel Microservices Microservices

Integration function Camel K + Quarkus Build time optimization (so far)

● “Service” discovery ● Registry set-up ● Warm up JAXB ● Set-up components ● Set-up Integration with Quarkus

https://github.com/apache/camel-quarkus Build time optimization results

https://github.com/apache/camel-quarkus What is Camel K?

A lightweight integration platform, born on Kubernetes, with serverless superpowers

K K K

1. 2. 3.

Runs on “vanilla” Kubernetes (1), Openshift (2) and gives its best on a Knative-powered cluster (3)!