The Evolution of Distributed Systems on

Bilgin Ibryam Product Manager @RedHat @bibryam

1 Bilgin Ibryam

● Product Manager at Red Hat ● Former Architect/Consultant ● Committer at Apache Camel ● Author of “Camel Design Patterns” and @bibryam “Kubernetes Patterns” books ● Latest interest: cloud native data

2 @bibryam What comes after ?

3 Agenda

● Distributed system needs ● Monolithic architectures ● Cloud-native technologies ■ Kubernetes, Istio, Knative, Dapr ● Future architecture trends

4 @bibryam Modern distributed applications

● 100s of components and 1000s of instances ● Polyglot, independent, and automatable components ● Hybrid workloads on hybrid environments ● Open source, open standards, and interoperable ● Based on Kubernetes ecosystem

5 @bibryam What are the needs of distributed applications?

6 Distributed application needs

7 @bibryam Distributed application needs

Lifecycle management ● Deployment/rollback ● Placement/scheduling ● Configuration management ● Resource/failure isolation ● Auto/manual scaling ● Hybrid workloads (stateless, stateful, serverless, etc)

8 @bibryam Distributed application needs

Advanced networking ● Service discovery and failover ● Dynamic traffic routing ● Retry, timeout, circuit breaking ● Security, rate limiting, encryption ● Observability and tracing

9 @bibryam Distributed application needs

Resource bindings ● Connectors for ● Protocol conversion ● Message transformation ● Filtering, light message routing ● Point-to-point, pub/sub interactions

10 @bibryam Distributed application needs

Stateful abstractions ● Workflow management ● Temporal scheduling ● Distributed caching ● Idempotency ● Transactionality (SAGA) ● Application state

11 @bibryam Monolithic architectures

12 Traditional middleware capabilities

● Stateful primitives ● Resource bindings ● Networking

13 @bibryam Traditional middleware limitations

● Lifecycle management ○ Single, shared language runtime ○ Manual deployment/rollback ○ Manual placement ○ Manual scaling ○ No resource/failure isolation

14 @bibryam Cloud-native architectures

15 Microservices and Kubernetes

16 @bibryam Microservices and Kubernetes

17 @bibryam Health probes

18 @bibryam Managed start/stop

19 @bibryam Declarative deployment

20 @bibryam Demands & placement

Predictable resource demand Automated placement

21 @bibryam Configuration management

● ConfigMaps used in Pods as: ○ environment variables ○ volumes ● Secrets: ○ Minimal Node spread ○ Only stored in memory in a tmpfs ○ Encrypted in the backend store (etcd) ○ Access can be restricted with RBAC

22 @bibryam Foundational kubernetes capabilities

More Kubernetes Patterns ● Foundational patterns ● Structural patterns ● Configuration patterns ● Behavioural patterns

(For more Kubernetes Patterns, check out the link at the end of the slides)

23 @bibryam Hybrid workloads

Batch/Periodic Job Stateless Service

24 @bibryam Stateful Service Global Singleton Lifecycle capabilities

● Deployment/rollback ● Placement/scheduling ● Configuration management ● Resource/failure isolation ● Auto/manual scaling ● Hybrid workloads: stateless, stateful, batch jobs, serverless

25 @bibryam How do we extend Kubernetes?

26 Out-of-process extension mechanism

Deployment guarantees Lifecycle guarantees

27 @bibryam Sidecar

28 @bibryam Controller Pattern

Default schema Default controllers Managed resources ● ReplicaSet ● replicaset state ● StatefulSet ● statefulset ● Pod ● Job, CronJob ● job, cronjob ● PVC...

Custom controller -> Custom behaviour

29 @bibryam Operator Pattern

Custom operator Custom application kind: ConfigWatcher apiVersion: k8spatterns.io/v1 ● Go ● AI/ML metadata: name: webapp-config-watcher ● Helm ● Big Data spec: configMap: webapp-config ● Ansible ● Storage podSelector: app: webapp ● Java ● Streaming ● Python ● Monitoring

CustomResourceDefinition + Controller = Operator

30 @bibryam Kubernetes based platforms

31 What is Service Mesh?

32 @bibryam What is Service Mesh?

33 @bibryam What is Service Mesh?

34 @bibryam What is Service Mesh?

35 @bibryam Networking capabilities

API Gateway Service Mesh

Abstract away details and decouple Enhances the reliability and the visibility of the consumers from implementations networking interactions

● Controls what’s allowed in/out ● Telemetry, tracing collection ● Bridging security domains ● Service discovery, load balancing ● Request / response transformation ● TLS termination/origination ● Protocol, data format transformation ● Request routing, traffic splitting ● API composition ● Traffic shadowing ● Rate limiting ● Rate limiting

36 @bibryam What is Knative?

Serving Eventing

Common infrastructure Common infrastructure for request-driven for consuming and interactions that can producing events "scale to zero". declaratively. Kubernetes-based platform to deploy, and manage serverless workloads.

37 @bibryam Knative Serving concepts

apiVersion: serving.knative.dev/v1alpha1 ● Scale-to-zero & activation kind: Service metadata: ● Rapid autoscaling name: lotto ● Traffic splitting spec: replicas: 1 ● Callable by Knative eventing selector: matchLabels: ● Simplified deployment model app: lotto ○ Single Port template: metadata: ○ No PersistentVolumes labels: ○ app: lotto Single Container spec: containers: - image: cds19/lotto 38 @bibryam Knative Eventing concepts

● Sources (Kafka, CronJob, Apache Camel 200+, etc) ● Broker implementations (In-memory, Kafka, etc) ● CloudEvents data format ● Trigger with filters ● Sequence: chaining multiple steps composed of containers

39 @bibryam Lifecycle, networking, binding capabilities

● Knative Serving ○ Simplified deployment for stateless workloads ○ Traffic based autoscaling including Scale-to-Zero ○ Traffic splitting for custom rollout / rollback scenarios ● Knative Eventing ○ External triggers for feeding Knative Services ○ Based on CloudEvents ○ Backed by proven messaging systems ○ Declarative messaging infrastructure 40 @bibryam What is Dapr?

Sidecar architecture Building blocks

Developer first, standard APIs Make it easy for developers to used from any programming create microservice without language or framework. being an expert in distributed systems.

A portable runtime for building distributed applications.

41 @bibryam Dapr building blocks

Service Invocation Resource Bindings State Management

Act as a reverse proxy with Trigger code through events Provides a key/value-based built-in service discovery, from input and output bindings state API with pluggable state tracing and error handling to external resources. stores for persistence

Distributed Tracing Publish & Subscribe Actors

See and measure the message Secure, scalable messaging Encapsulate code and data in calls across components and between services reusable actor objects as a networked services common microservices

42 @bibryam Dapr architecture

Source: https://github.com/dapr/docs

43 @bibryam Dapr on Kubernetes

Source: https://github.com/dapr/docs

44 @bibryam Full circle

Connectors Service discovery Eventing Dynamic routing Filtering Resiliency Serverless Observability

Bindings State abstraction Deployment Pub/Sub Placement Observability Config mgmt Scaling

● Centralized control plane ● Centralized control plane

● Centralized data plane ● Decentralized, highly-scalable data plane

45 @bibryam Future cloud native trends

46 Lifecycle trends

Source: https://operatorhub.io

47 @bibryam Networking trends

● Introduction of Service Mesh Interface specification

● Architecture consolidation of Istio with istiod

● More L7 protocols: MongoDB, DynamoDB, ZooKeeper, MySQL, Redis, Kafka(8188)

○ KIP-559 can enable bridging, validation, encryption, filtering, transformation

● HTTP Cache filter (eCache)

● HTTP tap filter (with matcher)

● WebAssembly (wasm) filters with dynamic loading (C++ -> Rust, Go, etc)

48 @bibryam Binding trends

Camel-K Operator: Dev Environment Cloud 1. Choose a runtime - from: 2. Scaffold a project uri: "direct:route" steps: Custom 3. Add boilerplate - split: Resource 4. Add dependencies tokenize: "," 5. Create container image - to: "mock:split" Camel K Operator 6. Create Kubernetes resources for deployment

kamel CLI Running Pod

Live Fast redeploy! updates! Less than 1 second! Source: https://github.com/apache/camel-k

49 @bibryam State trends

Source: https://github.com/cloudstateio/cloudstate

50 @bibryam What does all this mean?

51 Multi-runtime microservices are here

52 @bibryam Smart sidecars and dumb pipes

53 @bibryam What comes after Microservices?

54 @bibryam Thank You

@bibryam

https://k8spatterns.io

55