Serverless Integration with Knative & Apache Camel on Kubernetes
Claus Ibsen @davsclaus About me
● Senior Principal Software Engineer at Red Hat ● Java Champion ● 11 years as Apache Camel committer ● Author of Camel in Action books
http://www.davsclaus.com @davsclaus davsclaus ✉️ [email protected] What is Apache Camel?
● The swiss knife of integration
● >10 years of development - still one of the most active Apache projects
● Java-based integration framework that can be installed on multiple runtimes: Spring-Boot, JEE, JBoss, Karaf, Standalone, ...
● Based on Enterprise Integration Patterns
● Supports 250+ components
● Uses a powerful DSL
● Can integrate anything
http://camel.apache.org
+ + + + + + + + = + What is Apache Camel K? https://github.com/apache/camel-k
K
● A platform for directly running integrations on Openshift and Kubernetes
● Based on (ex Core OS) operator-sdk
● A community-driven project
● A subproject of Apache Camel started on August 31st, 2018 Overview of Camel K
A lightweight platform for running Camel integration DSL in the cloud.
● Based on operator-sdk ● Works on Openshift and Kubernetes
File routes.groovy
from(“telegram:bots/bot-id”) .transform()... .to(“kafka:topic”);
from(“kafka:topic”) .to(“http:my-host/api/path”);
$ kamel run routes.groovy Architecture of Camel K
Dev Environment Remote Cloud
“Integration” Custom Resource Camel K
Live updates! Operator
kamel CLI Running Pod
Fast redeploy! Less than 1 second! Tailored for cloud-native development experience Camel K in details
This is what people care from(“telegram:bots/bot-id”) .transform()... about: Camel DSL .to(“kafka:topic”);
from(“kafka:topic”) .to(“http:my-host/api/path”); Camel K Operator kind: Integration apiVersion: camel.apache.org/v1alpha1 metadata: name: my-integration 1. Choose a runtime spec: 2. Create container image kamel CLI sources: - name: source.groovy 3. Create Kubernetes content: |- resources for deployment from(“telegram:...”) .transform()... .to(“kafka:...”) CRDs allow to abstract from all this implementation Camel K Resources Visualization details!
File code.groovy $ kamel run code.groovy from(“telegram:bots/bot-id”) .transform()... .to(“kafka:topic”);
from(“kafka:topic”) <
from(“telegram:..”) .to(“...”) On OpenShift. It uses Kaniko on “vanilla” Kubernetes. Deployment Pod
BuildConfig
ConfigMap ImageStream from(“telegram:..”) .to(“...”) <
File code.groovy $ kamel run code.groovy from(“telegram:bots/bot-id”) .transform()... .to(“kafka:topic”);
from(“kafka:topic”) <
from(“telegram:..”) .to(“...”) On OpenShift. It uses Kaniko on “vanilla” Kubernetes. Deployment Pod
BuildConfig
ConfigMap ImageStream from(“telegram:..”) .to(“...”) <
Time to run a integration using different strategies (in seconds)
Lower is better
F-m-p is the “fabric8-maven-plugin” (http://maven.fabric8.io/) deploying a average spring-boot based integration on Minishift vs. a remote OpenShift cluster (accounting time to upload the fat Jar). Source S2I build has been measured in Red Hat Fuse Online. Can you implement Serverless without functions ? What is serverless again… ?
“Serverless computing refers to the concept of building and running applications that do not require server management. It describes a finer-grained deployment model where applications, bundled as one or more functions, are uploaded to a platform and then executed, scaled, and billed in response to the exact demand needed at the moment”
Source: https://www.cncf.io/blog/2018/02/14/cncf-takes-first-step-towards-serverless-computing/ What is Knative ? Knative Overview - Components "...an extension to Kubernetes exposing building blocks to build modern, source-centric, and container- based applications that can run anywhere".
Build Serving Eventing A pluggable model for An event-driven model Common infrastructure for building artifacts, like jar that serves the container consuming and producing files, zips or containers with your application and events that will stimulate from source code. can "scale to zero". applications. Knative Overview - Builds
● A Build is a list of containers run in-order, with source mounted in (TaskRun) ● BuildTemplates provide reusable, parameterized recipes that can be used to create Builds (Task)
apiVersion: build.knative.dev/v1alpha1 ● Pipelines - The future! kind: Build metadata: ○ Group of Tasks name: example-build spec: serviceAccountName: build-auth-example ● Build Example source: git: url: https://github.com/example/build-example.git ● "Source to URL" revision: master steps: ● S2I for OpenShift users - name: centos-example image: centos args: ["centos-build-example", "SECRETS-example.md"] steps: - image: quay.io/example-builders/build-example args: ['echo', 'hello-example', 'build'] Knative Overview - Serving
● Configurations represent the ‘floating Service HEAD’ of a history of Revisions (my-function)
● Revisions represent immutable snapshot manages of code and configuration Route ● Routes configure ingress over a collection (name) Configuration of Revisions and/or Configurations
10% ● Services (nope, not K8s services) are top- Revision records level controllers that manage a set of 90% history of Routes and Configurations to implement a Revision network service Revision Knative Overview - Eventing
● Goal is to be a generalized eventing framework ● Many influencers also active on CloudEvents specification
Current-ish API: ● Receive Adapter triggers on EventSources that send Events to a sink (channel)
● Channels are named endpoints which accept event delivery (provided by Kafka, AMQP…) ● Subscriptions register to receive traffic from a Channel Knative Eventing - Pipeline
Knative Eventing Docs Knative Eventing - Flow
Knative Eventing Docs Again, beyond functions...
https://12factor.net/
Configuration(configuration.serving.knative.dev) Keep code and configuration separate
Revision(revision.serving.knative.dev) Each configuration change triggers the creation of new revision Route(route.serving.knative.dev) Concurrency (scale out via process) Service(service.serving.knative.dev) Disposability (fast startup/graceful shutdown) Build (buildtemplate.build.knative.dev) Build, release, run (separate build and run stages) What about Camel K & Knative ?
K Camel K and Knative
Knative building blocks relevance to Camel
Knative Knative Build X Knative Serving Eventing <
<
Standardize building container images Auto-scaling and scale-to-zero Messaging for event-based applications Camel K and Knative
Knative building blocks relevance to Camel
Knative Knative Build X Knative Serving Eventing <
<
Standardize building container images Auto-scaling and scale-to-zero Messaging for event-based applications Camel K and Knative
Knative building blocks relevance to Camel
Knative Knative Build X Knative Serving Eventing <
<
Standardize building container images Auto-scaling and scale-to-zero Messaging for event-based applications Camel & Knative in details
Camel K Operator from(“knative:channel/a”) .to(“http:my-host/api/path”);
kind: Deployment kind: Integration Knative no apiVersion: camel.apache.org/v1alpha1 # standard one metadata: profile? name: my-integration spec: sources: yes - name: source.groovy content: |- from(“knative:channel/a”) kind: Service .to(“http:my-host/api/path”) apiVersion: serving.knative.dev/v1alpha1 Camel & Knative in details Same user experience! File code.groovy
from(“knative:channel/a”) .transform()... .to(“slack:#room”); $ kamel run code.groovy
events <
from(“knative:channel/a”) .to(“...”) <
In the knative profile, the operator uses Knative <
Channel Ext System Channel
Channel Integration Function Event EIP Source Camel K & Knative links
● Apache Camel K https://github.com/apache/camel-k ● Camel K Introduction Blog https://www.nicolaferraro.me/2018/10/15/introducing-camel-k ● Camel K and Knative tutorial https://redhat-developer-demos.github.io/knative-tutorial/knative- tutorial-camelk/v1.0.0/index.html ● Camel K and Knative video https://www.youtube.com/watch?v=btf_e2GniXM ● Camel Knative Event Sources https://github.com/knative/eventing- sources/tree/master/contrib/camel/samples
Java Density Problem
Node Node Node NodeJS Go Go Go Traditional Cloud-Native Java Stack NodeJS Go Go Go Traditional Cloud-Native NodeJS Go Go Go Java Stack NodeJS Go Go Go Traditional Cloud-Native NodeJS Go Go Go Java Stack NodeJS Go Go Go Traditional Cloud-Native Java Stack NodeJS Go Go Go
CONTAINER ORCHESTRATION
https://developers.redhat.com/blog/2017/03/14/java-inside-docker/ Java Fast Scaling Problem
SCALE-UP SCALE-DOWN SCALE-TO- ZERO Supersonic Subatomic Java
https://quarkus.io What is Quarkus
A Kubernetes Native Java stack tailored for GraalVM & OpenJDK HotSpot, crafted from the best of breed Java libraries and standards Minimal Footprint - Quarkus
● Minimal footprint Java applications ○ Native vs Quarkus+OpenJDK JVM vs Traditional JVM Container First - Quakrus
● First Class Support for Graal/SubstrateVM ○ Native compiled first-class supported ● Build Time Metadata Processing ○ Pre-Build and "warump" as much as possible during build time ● Reduction in Reflection Usage ○ Reduce startup and memory ● Native Image Pre Boot ○ Most of startup serialized into native image to startup even faster
https://quarkus.io/vision/container-first Apache Camel and Quarkus runs blazingly fast with native compiled binaries. Startup is 7 msec, 28mb binary file, and 15mb RSS memory used. Camel and Quarkus - Links
● Quarkus https://quarkus.io ● Quarkus Camel Extensions https://quarkus.io/extensions/#integration https://github.com/quarkusio/quarkus/tree/master/extensions/camel ● Quarkus Camel Demo https://github.com/quarkusio/quarkus-quickstarts/tree/master/camel-java Where are the slides ?
https://www.slideshare.net/davsclaus Any Questions ?
http://www.davsclaus.com @davsclaus davsclaus ✉️ [email protected]