Quarkus and Camel K: The Critical path to Serverless Integration Pat Carey / Phil Prosser Domain Solution Architect 15th October 2019 Agenda Ripping apart the forum demo : Introduction to Quarkus + demo Introduction to Camel K + demo Quarkus Core Philosophy Quarkus aims to do as much work as possible at build time, to keep the resulting application as small and fast as possible Jandex RESTEasy Netty HotSpot Hibernate ORM Quarkus Core Hibernate Validator Gizmo MP OpenAPI Quarkus Extensions MP JWT Eclipse Vert.X Graal SDK Agroal (conn pool) SubstrateVM Narayana JTA MP Reactive Messaging Arc (DI) Apache Camel ... Quarkus Consists of Two Parts Core Runtime Undertow Runtime ArC Runtime Core Deployment Undertow Deployment ArC Deployment Build and Runtime Split Build Time Runtime All code that determines what should Runtime should only contain classes happen at startup belongs to the that are needed to actually run the deployment module. This will include application. The build will generate things like annotation scanning and bytecode to directly start what is parsing. required. Startup Comparison Quarkus does most of the Traditional Application Quarkus work that is traditionally ● Scan for annotations ● Start runtime services done at startup and moves ● Parse descriptors it to build time. This results ● Select enabled features in smaller, faster ● Generate metamodel applications that use less ● Start runtime services memory. The end result of a Quarkus build should be just enough customised bytecode to start the services your application requires Not just startup speed Loading classes is slow, but also takes a lot of memory that is never released. Traditional Java apps hold lots of classes in memory that are used once at startup. 01 Memory usage 140 Mb Traditional Java Application 74 Mb Quarkus JVM 13 Mb Quarkus GraalVM But of course… Quarkus Improves Startup Time REST Quarkus + GraalVM 0.014 Seconds Quarkus + OpenJDK 0.75 Seconds Traditional Cloud-Native Stack 4.3 Seconds REST + CRUD Quarkus + GraalVM 0.055 Seconds Quarkus + OpenJDK 2.5 Seconds Traditional Cloud-Native Stack 9.5 Seconds Extensions vs Subsystems Quarkus extensions are much more lightweight than EAP Subsystems In many respects they are closer to CDI portable extensions Development Mode This split only applies to production applications, not quarkus:dev In developer mode both parts of an extension are present, allowing for fast reload DEMO TIME Extensions for Quarkus Summing Up Architectural Advantages Integrating Frameworks Quarkus’ unique architecture gives it a Extensions can be used to make home major advantage when memory usage grown and 3rd party frameworks run on and startup time are important Quarkus Lightweight Extensions Enabling Native Compilation Quarkus extensions are lightweight, and Quarkus provides API’s to help compile can be used in a similar manner to CDI applications to a native image portable extensions Introduction to Camel K What is Apache Camel K? A lightweight integration platform based on Apache Camel, born on Kubernetes, with serverless superpowers A community-driven project A sub-project of Apache Camel started on August 31st, 2018 Based on the operator SDK https://github.com/apache/camel-k What? Camel DSL, based on EIPs... 1. Create an integration file (Java, Groovy, Kotlin, JS, XML, …) // Lookup every second the 'www.google.com' domain 3. It runs on name and log the output Kubernetes / OpenShift from('timer:dns?period=1s') .routeId('dns') .setHeader('dns.domain') .constant('www.google.com') .to('dns:ip') .log('log:dns'); 2. Run it $ kamel run integration.groovy Camel K CLI (optional) How? Dev Environment Cluster Integration Custom Resource Camel K Operator Live updates! kamel CLI Running Pod Fast redeploy! Less than 1 second! Tailored for cloud-native development experience https://github.com/operator-framework/operator-sdk DEMO TIME Camel K + Knative Knative Knative defines building blocks for “Serverless” applications Serving Eventing Scale-to-zero and request-driven Event binding and delivery for compute messaging style integration <<custom-resource>> <<custom-resource>> <<custom-resource>> <<custom-resource>> <<custom-resource>>Service <<custom-resource>>Channel Service Channel Service Channel https://knative.dev Camel K Knative Profile Camel K Operator from("knative:channel/a") .to("http:my-host/api/path"); kind: Deployment kind: Integration Knative no apiVersion: camel.apache.org/v1alpha1 # regular one metadata: profile? ... name: my-integration spec: sources: yes - name: source.groovy content: |- from("knative:channel/a") kind: Service .to("http:my-host/api/path") apiVersion: serving.knative.dev/v1alpha1 ... Knative Serving rest().post("/path") .to("xx:system1") Kubernetes Namespace .to("xx:system2") 300+ components! Knative Service System 1 System 2 There’s no container if no one needs it! Knative Serving rest().post("/path") .to("xx:system1") Kubernetes Namespace .to("xx:system2") 300+ components! Knative Service Request Pod System 1 System 2 A container is created only when needed! Knative Serving rest().post("/path") .to("xx:system1") Kubernetes Namespace .to("xx:system2") 300+ components! Knative Service Request Request Pod System 1 Request Pod System 2 Pod Multiple containers under high load! Knative Eventing Mmh? Isn’t that like JMS? https://cloudevents.io/ Subscription Subscription Producers Channel Consumers Subscription Subscription Kafka, In-memory, ... Knative Eventing from("knative:channel/a") .to("xx:system1") Kubernetes Namespace .to("xx:system2") 300+ components! Knative Knative Service Subscription System 1 Cloud Event Channel a Pod System 2 Knative Camel Sources Knative Camel Sources Camel components... “Cloud Edition” CamelTwitterSource CamelFacebookSource 300+ components! CamelTelegramSource ... CamelServiceNowSource Knative Camel Sources Camel Sources are an addon to Knative, currently in kind: Integration Knative eventing-contrib repository apiVersion: camel.apache.org/v1alpha1 metadata: It translates Camel-based Knative eventing sources name: my-integration spec: into Camel K integrations source: - content: | - from: uri: timer:tick?period=3s apiVersion: sources.eventing.knative.dev/v1alpha1 steps: kind: CamelSource - set-body: metadata: constant: Hello world! name: camel-timer-flow-source - to: spec: uri: knative://endpoint/sink source: language: yaml flow: | name: flow.yaml from: traits: uri: timer:tick?period=3s knative: steps: configuration: - set-body: configuration: constant: Hello world! Camel Sources '{"services":[{"type":"endpoint","protocol":"http sink: ","name":"sink","host":"camel-test-channel-4dlgs. apiVersion: eventing.knative.dev/v1alpha1 Operator camel-k.svc.cluster.local","port":80,"metadata":{ kind: Channel "service.path":"/"}}]}' name: camel-test https://github.com/knative/eventing-contrib/tree/master/camel/source Extending Serverless Integration Service Service Integration Integration Service function KNative function Service Integration Channel Integration function function Channel Integration Integration function Channel External SaaS External SaaS External SaaS Integration Channel function External SaaS Channel Microservices Integration Microservices function Microservices Channel Microservices Microservices Integration function Camel K + Quarkus Build time optimization (so far) ● “Service” discovery ● Registry set-up ● Warm up JAXB ● Set-up components ● Set-up Integration with Quarkus https://github.com/apache/camel-quarkus Build time optimization results https://github.com/apache/camel-quarkus What is Camel K? A lightweight integration platform, born on Kubernetes, with serverless superpowers K K K 1. 2. 3. Runs on “vanilla” Kubernetes (1), Openshift (2) and gives its best on a Knative-powered cluster (3)! .