The Evolution of Distributed Systems on Kubernetes Bilgin Ibryam Product Manager @RedHat @bibryam 1 Bilgin Ibryam ● Product Manager at Red Hat ● Former Architect/Consultant ● Committer at Apache Camel ● Author of “Camel Design Patterns” and @bibryam “Kubernetes Patterns” books ● Latest interest: cloud native data 2 @bibryam What comes after Microservices? 3 Agenda ● Distributed system needs ● Monolithic architectures ● Cloud-native technologies ■ Kubernetes, Istio, Knative, Dapr ● Future architecture trends 4 @bibryam Modern distributed applications ● 100s of components and 1000s of instances ● Polyglot, independent, and automatable components ● Hybrid workloads on hybrid environments ● Open source, open standards, and interoperable ● Based on Kubernetes ecosystem 5 @bibryam What are the needs of distributed applications? 6 Distributed application needs 7 @bibryam Distributed application needs Lifecycle management ● Deployment/rollback ● Placement/scheduling ● Configuration management ● Resource/failure isolation ● Auto/manual scaling ● Hybrid workloads (stateless, stateful, serverless, etc) 8 @bibryam Distributed application needs Advanced networking ● Service discovery and failover ● Dynamic traffic routing ● Retry, timeout, circuit breaking ● Security, rate limiting, encryption ● Observability and tracing 9 @bibryam Distributed application needs Resource bindings ● Connectors for APIs ● Protocol conversion ● Message transformation ● Filtering, light message routing ● Point-to-point, pub/sub interactions 10 @bibryam Distributed application needs Stateful abstractions ● Workflow management ● Temporal scheduling ● Distributed caching ● Idempotency ● Transactionality (SAGA) ● Application state 11 @bibryam Monolithic architectures 12 Traditional middleware capabilities ● Stateful primitives ● Resource bindings ● Networking 13 @bibryam Traditional middleware limitations ● Lifecycle management ○ Single, shared language runtime ○ Manual deployment/rollback ○ Manual placement ○ Manual scaling ○ No resource/failure isolation 14 @bibryam Cloud-native architectures 15 Microservices and Kubernetes 16 @bibryam Microservices and Kubernetes 17 @bibryam Health probes 18 @bibryam Managed start/stop 19 @bibryam Declarative deployment 20 @bibryam Demands & placement Predictable resource demand Automated placement 21 @bibryam Configuration management ● ConfigMaps used in Pods as: ○ environment variables ○ volumes ● Secrets: ○ Minimal Node spread ○ Only stored in memory in a tmpfs ○ Encrypted in the backend store (etcd) ○ Access can be restricted with RBAC 22 @bibryam Foundational kubernetes capabilities More Kubernetes Patterns ● Foundational patterns ● Structural patterns ● Configuration patterns ● Behavioural patterns (For more Kubernetes Patterns, check out the link at the end of the slides) 23 @bibryam Hybrid workloads Batch/Periodic Job Stateless Service 24 @bibryam Stateful Service Global Singleton Lifecycle capabilities ● Deployment/rollback ● Placement/scheduling ● Configuration management ● Resource/failure isolation ● Auto/manual scaling ● Hybrid workloads: stateless, stateful, batch jobs, serverless 25 @bibryam How do we extend Kubernetes? 26 Out-of-process extension mechanism Deployment guarantees Lifecycle guarantees 27 @bibryam Sidecar 28 @bibryam Controller Pattern Default schema Default controllers Managed resources ● ReplicaSet ● replicaset state ● StatefulSet ● statefulset ● Pod ● Job, CronJob ● job, cronjob ● PVC... Custom controller -> Custom behaviour 29 @bibryam Operator Pattern Custom operator Custom application kind: ConfigWatcher apiVersion: k8spatterns.io/v1 ● Go ● AI/ML metadata: name: webapp-config-watcher ● Helm ● Big Data spec: configMap: webapp-config ● Ansible ● Storage podSelector: app: webapp ● Java ● Streaming ● Python ● Monitoring CustomResourceDefinition + Controller = Operator 30 @bibryam Kubernetes based platforms 31 What is Service Mesh? 32 @bibryam What is Service Mesh? 33 @bibryam What is Service Mesh? 34 @bibryam What is Service Mesh? 35 @bibryam Networking capabilities API Gateway Service Mesh Abstract away details and decouple Enhances the reliability and the visibility of the consumers from implementations networking interactions ● Controls what’s allowed in/out ● Telemetry, tracing collection ● Bridging security domains ● Service discovery, load balancing ● Request / response transformation ● TLS termination/origination ● Protocol, data format transformation ● Request routing, traffic splitting ● API composition ● Traffic shadowing ● Rate limiting ● Rate limiting 36 @bibryam What is Knative? Serving Eventing Common infrastructure Common infrastructure for request-driven for consuming and interactions that can producing events "scale to zero". declaratively. Kubernetes-based platform to deploy, and manage serverless workloads. 37 @bibryam Knative Serving concepts apiVersion: serving.knative.dev/v1alpha1 ● Scale-to-zero & activation kind: Service metadata: ● Rapid autoscaling name: lotto ● Traffic splitting spec: replicas: 1 ● Callable by Knative eventing selector: matchLabels: ● Simplified deployment model app: lotto ○ Single Port template: metadata: ○ No PersistentVolumes labels: ○ app: lotto Single Container spec: containers: - image: cds19/lotto 38 @bibryam Knative Eventing concepts ● Sources (Kafka, CronJob, Apache Camel 200+, etc) ● Broker implementations (In-memory, Kafka, etc) ● CloudEvents data format ● Trigger with filters ● Sequence: chaining multiple steps composed of containers 39 @bibryam Lifecycle, networking, binding capabilities ● Knative Serving ○ Simplified deployment for stateless workloads ○ Traffic based autoscaling including Scale-to-Zero ○ Traffic splitting for custom rollout / rollback scenarios ● Knative Eventing ○ External triggers for feeding Knative Services ○ Based on CloudEvents ○ Backed by proven messaging systems ○ Declarative messaging infrastructure 40 @bibryam What is Dapr? Sidecar architecture Building blocks Developer first, standard APIs Make it easy for developers to used from any programming create microservice without language or framework. being an expert in distributed systems. A portable runtime for building distributed applications. 41 @bibryam Dapr building blocks Service Invocation Resource Bindings State Management Act as a reverse proxy with Trigger code through events Provides a key/value-based built-in service discovery, from input and output bindings state API with pluggable state tracing and error handling to external resources. stores for persistence Distributed Tracing Publish & Subscribe Actors See and measure the message Secure, scalable messaging Encapsulate code and data in calls across components and between services reusable actor objects as a networked services common microservices 42 @bibryam Dapr architecture Source: https://github.com/dapr/docs 43 @bibryam Dapr on Kubernetes Source: https://github.com/dapr/docs 44 @bibryam Full circle Connectors Service discovery Eventing Dynamic routing Filtering Resiliency Serverless Observability Bindings State abstraction Deployment Pub/Sub Placement Observability Config mgmt Scaling ● Centralized control plane ● Centralized control plane ● Centralized data plane ● Decentralized, highly-scalable data plane 45 @bibryam Future cloud native trends 46 Lifecycle trends Source: https://operatorhub.io 47 @bibryam Networking trends ● Introduction of Service Mesh Interface specification ● Architecture consolidation of Istio with istiod ● More L7 protocols: MongoDB, DynamoDB, ZooKeeper, MySQL, Redis, Kafka(8188) ○ KIP-559 can enable bridging, validation, encryption, filtering, transformation ● HTTP Cache filter (eCache) ● HTTP tap filter (with matcher) ● WebAssembly (wasm) filters with dynamic loading (C++ -> Rust, Go, etc) 48 @bibryam Binding trends Camel-K Operator: Dev Environment Cloud 1. Choose a runtime - from: 2. Scaffold a project uri: "direct:route" steps: Custom 3. Add boilerplate - split: Resource 4. Add dependencies tokenize: "," 5. Create container image - to: "mock:split" Camel K Operator 6. Create Kubernetes resources for deployment kamel CLI Running Pod Live Fast redeploy! updates! Less than 1 second! Source: https://github.com/apache/camel-k 49 @bibryam State trends Source: https://github.com/cloudstateio/cloudstate 50 @bibryam What does all this mean? 51 Multi-runtime microservices are here 52 @bibryam Smart sidecars and dumb pipes 53 @bibryam What comes after Microservices? 54 @bibryam Thank You @bibryam https://k8spatterns.io 55.