Serverless Integration with Knative & Apache Camel on Kubernetes Claus Ibsen @davsclaus About me ● Senior Principal Software Engineer at Red Hat ● Java Champion ● 11 years as Apache Camel committer ● Author of Camel in Action books http://www.davsclaus.com @davsclaus davsclaus ✉️ [email protected] What is Apache Camel? ● The swiss knife of integration ● >10 years of development - still one of the most active Apache projects ● Java-based integration framework that can be installed on multiple runtimes: Spring-Boot, JEE, JBoss, Karaf, Standalone, ... ● Based on Enterprise Integration Patterns ● Supports 250+ components ● Uses a powerful DSL ● Can integrate anything http://camel.apache.org + + + + + + + + = + What is Apache Camel K? https://github.com/apache/camel-k K ● A platform for directly running integrations on Openshift and Kubernetes ● Based on (ex Core OS) operator-sdk ● A community-driven project ● A subproject of Apache Camel started on August 31st, 2018 Overview of Camel K A lightweight platform for running Camel integration DSL in the cloud. ● Based on operator-sdk ● Works on Openshift and Kubernetes File routes.groovy from(“telegram:bots/bot-id”) .transform()... .to(“kafka:topic”); from(“kafka:topic”) .to(“http:my-host/api/path”); $ kamel run routes.groovy Architecture of Camel K Dev Environment Remote Cloud “Integration” Custom Resource Camel K Live updates! Operator kamel CLI Running Pod Fast redeploy! Less than 1 second! Tailored for cloud-native development experience Camel K in details This is what people care from(“telegram:bots/bot-id”) .transform()... about: Camel DSL .to(“kafka:topic”); from(“kafka:topic”) .to(“http:my-host/api/path”); Camel K Operator kind: Integration apiVersion: camel.apache.org/v1alpha1 metadata: name: my-integration 1. Choose a runtime spec: 2. Create container image kamel CLI sources: - name: source.groovy 3. Create Kubernetes content: |- resources for deployment from(“telegram:...”) .transform()... .to(“kafka:...”) CRDs allow to abstract from all this implementation Camel K Resources Visualization details! File code.groovy $ kamel run code.groovy from(“telegram:bots/bot-id”) .transform()... .to(“kafka:topic”); from(“kafka:topic”) <<custom-resource>> .to(“http:my-host/api/path”); Integration from(“telegram:..”) .to(“...”) On OpenShift. It uses Kaniko on “vanilla” Kubernetes. Deployment Pod BuildConfig ConfigMap ImageStream from(“telegram:..”) .to(“...”) <<custom-resource>> <<custom-resource>> ImageStreamTag IntegrationContext IntegrationPlatform Camel K Resources Visualization File code.groovy $ kamel run code.groovy from(“telegram:bots/bot-id”) .transform()... .to(“kafka:topic”); from(“kafka:topic”) <<custom-resource>> .to(“http:my-host/api/path”); Integration from(“telegram:..”) .to(“...”) On OpenShift. It uses Kaniko on “vanilla” Kubernetes. Deployment Pod BuildConfig ConfigMap ImageStream from(“telegram:..”) .to(“...”) <<custom-resource>> <<custom-resource>> ImageStreamTag IntegrationContext IntegrationPlatform Fast Deployment of Camel K Time to run a integration using different strategies (in seconds) Lower is better F-m-p is the “fabric8-maven-plugin” (http://maven.fabric8.io/) deploying a average spring-boot based integration on Minishift vs. a remote OpenShift cluster (accounting time to upload the fat Jar). Source S2I build has been measured in Red Hat Fuse Online. Can you implement Serverless without functions ? What is serverless again… ? “Serverless computing refers to the concept of building and running applications that do not require server management. It describes a finer-grained deployment model where applications, bundled as one or more functions, are uploaded to a platform and then executed, scaled, and billed in response to the exact demand needed at the moment” Source: https://www.cncf.io/blog/2018/02/14/cncf-takes-first-step-towards-serverless-computing/ What is Knative ? Knative Overview - Components "...an extension to Kubernetes exposing building blocks to build modern, source-centric, and container- based applications that can run anywhere". Build Serving Eventing A pluggable model for An event-driven model Common infrastructure for building artifacts, like jar that serves the container consuming and producing files, zips or containers with your application and events that will stimulate from source code. can "scale to zero". applications. Knative Overview - Builds ● A Build is a list of containers run in-order, with source mounted in (TaskRun) ● BuildTemplates provide reusable, parameterized recipes that can be used to create Builds (Task) apiVersion: build.knative.dev/v1alpha1 ● Pipelines - The future! kind: Build metadata: ○ Group of Tasks name: example-build spec: serviceAccountName: build-auth-example ● Build Example source: git: url: https://github.com/example/build-example.git ● "Source to URL" revision: master steps: ● S2I for OpenShift users - name: centos-example image: centos args: ["centos-build-example", "SECRETS-example.md"] steps: - image: quay.io/example-builders/build-example args: ['echo', 'hello-example', 'build'] Knative Overview - Serving ● Configurations represent the ‘floating Service HEAD’ of a history of Revisions (my-function) ● Revisions represent immutable snapshot manages of code and configuration Route ● Routes configure ingress over a collection (name) Configuration of Revisions and/or Configurations 10% ● Services (nope, not K8s services) are top- Revision records level controllers that manage a set of 90% history of Routes and Configurations to implement a Revision network service Revision Knative Overview - Eventing ● Goal is to be a generalized eventing framework ● Many influencers also active on CloudEvents specification Current-ish API: ● Receive Adapter triggers on EventSources that send Events to a sink (channel) ● Channels are named endpoints which accept event delivery (provided by Kafka, AMQP…) ● Subscriptions register to receive traffic from a Channel Knative Eventing - Pipeline Knative Eventing Docs Knative Eventing - Flow Knative Eventing Docs Again, beyond functions... https://12factor.net/ Configuration(configuration.serving.knative.dev) Keep code and configuration separate Revision(revision.serving.knative.dev) Each configuration change triggers the creation of new revision Route(route.serving.knative.dev) Concurrency (scale out via process) Service(service.serving.knative.dev) Disposability (fast startup/graceful shutdown) Build (buildtemplate.build.knative.dev) Build, release, run (separate build and run stages) What about Camel K & Knative ? K Camel K and Knative Knative building blocks relevance to Camel Knative Knative Build X Knative Serving Eventing <<custom-resource>> <<custom-resource>> <<custom-resource>> Build Service Channel <<custom-resource>> <<custom-resource>> <<custom-resource>> ... ... ... <<custom-resource>> <<custom-resource>> <<custom-resource>> ... ... ... <<custom-resource>> <<custom-resource>> <<custom-resource>> ... ... ... Standardize building container images Auto-scaling and scale-to-zero Messaging for event-based applications Camel K and Knative Knative building blocks relevance to Camel Knative Knative Build X Knative Serving Eventing <<custom-resource>> <<custom-resource>> <<custom-resource>> Build Service Channel <<custom-resource>> <<custom-resource>> <<custom-resource>> ... ... ... <<custom-resource>> <<custom-resource>> <<custom-resource>> ... ... ... <<custom-resource>> <<custom-resource>> <<custom-resource>> ... ... ... Standardize building container images Auto-scaling and scale-to-zero Messaging for event-based applications Camel K and Knative Knative building blocks relevance to Camel Knative Knative Build X Knative Serving Eventing <<custom-resource>> <<custom-resource>> <<custom-resource>> Build Service Channel <<custom-resource>> <<custom-resource>> <<custom-resource>> ... ... ... <<custom-resource>> <<custom-resource>> <<custom-resource>> ... ... ... <<custom-resource>> <<custom-resource>> <<custom-resource>> ... ... ... Standardize building container images Auto-scaling and scale-to-zero Messaging for event-based applications Camel & Knative in details Camel K Operator from(“knative:channel/a”) .to(“http:my-host/api/path”); kind: Deployment kind: Integration Knative no apiVersion: camel.apache.org/v1alpha1 # standard one metadata: profile? name: my-integration spec: sources: yes - name: source.groovy content: |- from(“knative:channel/a”) kind: Service .to(“http:my-host/api/path”) apiVersion: serving.knative.dev/v1alpha1 Camel & Knative in details Same user experience! File code.groovy from(“knative:channel/a”) .transform()... .to(“slack:#room”); $ kamel run code.groovy events <<custom-resource>> <<custom-resource>> Or even no pods! Integration Channel from(“knative:channel/a”) .to(“...”) <<custom-resource>> Service In the knative profile, the operator uses Knative <<custom-resource>> building blocks instead of a Subscription Pod Pod Pod standard Kubernetes “Deployment” Camel K & Knative Eventing 250+ Camel Camel K: same model for different purposes components! Channel Ext System Channel Channel Integration Function Event EIP Source Camel K & Knative links ● Apache Camel K https://github.com/apache/camel-k ● Camel K Introduction Blog https://www.nicolaferraro.me/2018/10/15/introducing-camel-k ● Camel K and Knative tutorial https://redhat-developer-demos.github.io/knative-tutorial/knative- tutorial-camelk/v1.0.0/index.html ● Camel K and Knative video https://www.youtube.com/watch?v=btf_e2GniXM ● Camel Knative Event Sources https://github.com/knative/eventing-