Solve the Paradox Less Downtime – More Security Linuxcon Berlin, Germany October 4, 12:10 – 13:00

Solve the paradox Less Downtime – More Security LinuxCon Berlin, Germany October 4, 12:10 – 13:00

Hannes Kühnemund SUSE Product Management Downtime

Considerations for your digital architecture

Take a holistic approach … - End-users (Business) are interested service availability - Application, OS, Cluster, VM, Server, Network, Storage, People, Processes...

... because we understand that components will fail, ... - Failure tolerant architecture, identify weak links

... acceptance of any downtime is decreasing and it is critical to ... - Seek to reduce both planned and unplanned service downtime

... strike a balance. - Cost of IT continuity vs. business impact

2 Downtime Quiz

planned

Regular cadence - monthly - quarterly - yearly On the weekend In alignment with all stakeholders Combination of Taks - software updates / configuration - hardware exchange of defect parts - datacenter maintenance / AC Optimizable with - SUSE Manager 3 Downtime Quiz

planned unplanned

Regular cadence No cadence - monthly - quarterly - yearly On the weekend In alignment with all stakeholders Combination of Taks - software updates / configuration - hardware exchange of defect parts - datacenter maintenance / AC Optimizable with - SUSE Manager 5 Downtime Quiz

planned unplanned

Regular cadence No cadence - monthly - quarterly - yearly On the weekend Usually on Christmas Day In alignment with all stakeholders Combination of Taks - software updates / configuration - hardware exchange of defect parts - datacenter maintenance / AC Optimizable with - SUSE Manager 6 Downtime Quiz

planned unplanned

Regular cadence No cadence - monthly - quarterly - yearly On the weekend Usually on Christmas Day In alignment with all stakeholders No alignment with stakeholders Combination of Taks - software updates / configuration - hardware exchange of defect parts - datacenter maintenance / AC Optimizable with - SUSE Manager 7 Downtime Quiz

planned unplanned

Regular cadence No cadence - monthly - quarterly - yearly On the weekend Usually on Christmas Day In alignment with all stakeholders No alignment with stakeholders Combination of Taks Only one particular problem fixed - software updates / configuration - hardware exchange of defect parts - datacenter maintenance / AC Optimizable with - SUSE Manager 8 Downtime Quiz

planned unplanned

Regular cadence No cadence - monthly - quarterly - yearly On the weekend Usually on Christmas Day In alignment with all stakeholders No alignment with stakeholders Combination of Taks Only one particular problem fixed - software updates / configuration - hardware exchange of defect parts - datacenter maintenance / AC Optimizable with Optimizable with - SUSE Manager - Various technologies available 9 Minimize Unplanned Downtime

RAS System UPS Rollback

High Availability and GEO Virtualization RAID

Load Balancer Live Patching

10 Strike the balance?

11 Strike the balance?

No Downtime Security

12 Since 2005, more than 75 data breaches in which 1,000,000 or more records were compromised have been publicly disclosed.

But what about the non-disclosed ones?

13 Vulnerabilities

Year # vulnerabilities 10000 2010 4258 8000 2011 3532 6000 2012 4347 2013 4794 4000

2014 7038 2000 2015 8822 2010 2011 2012 2013 2014 2015

Vulnerability type 2015 Rank Operating System # vulnerabilities 2015 1 Apple OS X 384 Operating 28% 38% System 2 Microsoft Windows Server 2012 155 Browsers 3 Canonical Ubuntu Linux 152 Mobile Devices 4 Microsoft Windows 8.1 151 ... 18% Applications 16% 11 The Linux Kernel 77

Source: [http://www.cvedetails.com] & [https://nvd.nist.gov/] & [http://www.gfi.com/blog/2015s-mvps-the-most-vulnerable-players/] 14 In a data center, not so long ago …

15 In a data center, not so long ago …

Linux Kernel Nov-11, 2015

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

16 In a data center, not so long ago …

Linux Kernel Nov-11, 2015

 CVE-2015-6937

 CVE-2015-7872

 CVE-2015-7990

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

17 In a data center, not so long ago …

Linux Kernel Nov-11, 2015 CVE: Common Vulnerabilities and Exposures

 CVE-2015-6937 It is a standard naming scheme used by the NVD  CVE-2015-7872

 CVE-2015-7990 NVD: National Vulnerability Database (https://nvd.nist.gov/)

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

18 In a data center, not so long ago …

Linux Kernel Nov-11, 2015

 CVE-2015-6937

 CVE-2015-7872

 CVE-2015-7990

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

19 In a data center, not so long ago …

Linux Kernel Nov-11, 2015 Reboot

 CVE-2015-6937

 CVE-2015-7872 Linux Kernel  CVE-2015-7990 Dec-11, 2015

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

20 In a data center, not so long ago …

Linux Kernel Nov-11, 2015

 CVE-2015-6937

 CVE-2015-7872 Linux Kernel  CVE-2015-7990 Dec-11, 2015  CVE-2016-0728  CVE-2016-0728

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

21 In a data center, not so long ago …

Linux Kernel Nov-11, 2015

 CVE-2015-6937 Linux Kernel  CVE-2015-7872 Reboot  CVE-2015-7990 Dec-11, 2015  CVE-2016-0728  CVE-2016-0728 Linux Kernel Jan-15, 2016

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

22 In a data center, not so long ago …

Linux Kernel Nov-11, 2015

 CVE-2013-7446

 CVE-2015-8539

 CVE-2015-8660

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

23 In a data center, not so long ago …

Linux Kernel Nov-11, 2015

 CVE-2013-7446

 CVE-2015-6937 Linux Kernel  CVE-2015-7872 Dec-11, 2015  CVE-2015-7990  CVE-2013-7446  CVE-2015-8019 Linux Kernel  CVE-2015-8019  CVE-2015-8539 Reboot  CVE-2015-8539 Jan-15, 2016  CVE-2015-8660  CVE-2015-8660  CVE-2016-0728  CVE-2013-7446  CVE-2016-0728 Linux Kernel  CVE-2015-8019

 CVE-2015-8539 Feb-10, 2016

 CVE-2015-8660

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

24 In a data center, not so long ago …

Linux Kernel Nov-11, 2015

 CVE-2013-7446

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

25 In a data center, not so long ago …

Linux Kernel Nov-11, 2015

 CVE-2013-7446

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

26 In a data center, not so long ago …

Linux Kernel Nov-11, 2015

 CVE-2013-7446

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

27 In a data center, not so long ago …

Linux Kernel Nov-11, 2015

 CVE-2013-7446

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

28 In a data center, not so long ago …

Linux Kernel Nov-11, 2015

 CVE-2013-7446

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

29 In a data center, not so long ago …

Linux Kernel Nov-11, 2015

 CVE-2013-7446

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

30 In a data center, not so long ago …

Linux Kernel Nov-11, 2015

 CVE-2013-7446

 CVE-2015-6937 Linux Kernel  CVE-2015-7872 Dec-11, 2015  CVE-2015-7990  CVE-2013-7446  CVE-2015-8019 Linux Kernel  CVE-2015-8019  CVE-2015-8539  CVE-2015-8539 Jan-15, 2016  CVE-2015-8660  CVE-2015-8660  CVE-2015-8709  CVE-2013-7446  CVE-2015-8709  CVE-2015-8812 Linux Kernel  CVE-2015-8019  CVE-2015-8812  CVE-2015-8816  CVE-2015-8539 Feb-10, 2016  CVE-2015-8816  CVE-2016-0728  CVE-2015-8660  CVE-2016-0728   CVE-2016-0758 CVE-2015-8709  CVE-2015-8709  CVE-2016-0758  Linux Kernel  CVE-2016-0774 CVE-2015-8812  CVE-2015-8812  CVE-2016-0774   CVE-2016-1583 CVE-2015-8816 Mar-22, 2016  CVE-2015-8816  CVE-2016-1583   CVE-2016-2053 CVE-2016-0758  CVE-2016-0758  CVE-2016-0758  CVE-2016-2053   CVE-2016-2384 CVE-2016-0774 Linux Kernel  CVE-2016-0774  CVE-2016-1583  CVE-2016-2384   CVE-2016-3134 CVE-2016-1583  CVE-2016-1583  CVE-2016-2053 Jun-09, 2016  CVE-2016-3134   CVE-2016-4470 CVE-2016-2053  CVE-2016-2053  CVE-2016-3134  CVE-2016-4470   CVE-2016-4565 CVE-2016-2384  CVE-2016-0758  CVE-2016-2384  CVE-2016-4470  CVE-2016-4565   CVE-2016-4997 CVE-2016-3134  CVE-2016-2053 Linux Kernel  CVE-2016-3134  CVE-2016-4565  CVE-2016-4997   CVE-2016-5829 CVE-2016-4470  CVE-2016-4470  CVE-2016-4470  CVE-2016-4997 Aug-16, 2016  CVE-2016-5829  CVE-2016-4565  CVE-2016-4565  CVE-2016-4565  CVE-2016-5829  CVE-2016-4997  CVE-2016-4997  CVE-2016-0758  CVE-2016-4997  CVE-2016-5829  CVE-2016-5829  CVE-2016-2053  CVE-2016-5829  CVE-2016-4470

 CVE-2016-4565

 CVE-2016-5829

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

31 In a data center, not so long ago …

Linux Kernel Nov-11, 2015

 CVE-2013-7446

 CVE-2015-6937 Linux Kernel  CVE-2015-7872 Dec-11, 2015  CVE-2015-7990  CVE-2013-7446  CVE-2015-8019 Linux Kernel  CVE-2015-8019  CVE-2015-8539  CVE-2015-8539 Jan-15, 2016  CVE-2015-8660  CVE-2015-8660  CVE-2015-8709  CVE-2013-7446  CVE-2015-8709  CVE-2015-8812 Linux Kernel  CVE-2015-8019  CVE-2015-8812  CVE-2015-8816  CVE-2015-8539 Feb-10, 2016  CVE-2015-8816  CVE-2016-0728  CVE-2015-8660  CVE-2016-0728   CVE-2016-0758 CVE-2015-8709  CVE-2015-8709  CVE-2016-0758  Linux Kernel  CVE-2016-0774 CVE-2015-8812  CVE-2015-8812  CVE-2016-0774   CVE-2016-1583 CVE-2015-8816 Mar-22, 2016  CVE-2015-8816  CVE-2016-1583   CVE-2016-2053 CVE-2016-0758  CVE-2016-0758  CVE-2016-0758  CVE-2016-2053   CVE-2016-2384 CVE-2016-0774 Linux Kernel  CVE-2016-0774  CVE-2016-1583  CVE-2016-2384   CVE-2016-3134 CVE-2016-1583  CVE-2016-1583  CVE-2016-2053 Jun-09, 2016  CVE-2016-3134   CVE-2016-4470 CVE-2016-2053  CVE-2016-2053  CVE-2016-3134  CVE-2016-4470   CVE-2016-4565 CVE-2016-2384  CVE-2016-0758  CVE-2016-2384  CVE-2016-4470  CVE-2016-4565   CVE-2016-4997 CVE-2016-3134  CVE-2016-2053 Linux Kernel  CVE-2016-3134  CVE-2016-4565  CVE-2016-4997   CVE-2016-5829 CVE-2016-4470  CVE-2016-4470 Reboot  CVE-2016-4470  CVE-2016-4997 Aug-16, 2016  CVE-2016-5829  CVE-2016-4565  CVE-2016-4565  CVE-2016-4565  CVE-2016-5829  CVE-2016-4997  CVE-2016-4997  CVE-2016-0758  CVE-2016-4997  Linux Kernel CVE-2016-5829  CVE-2016-5829  CVE-2016-2053  CVE-2016-5829  CVE-2016-4470 Sep-12, 2016  CVE-2016-4565

 CVE-2016-5829

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

32 In a data center, not so long ago …

Linux Kernel Nov-11, 2015 Sample data taken  CVE-2013-7446 Linux Kernel  CVE-2015-6937 on Sept-15, 2016  CVE-2015-7872 Dec-11, 2015  CVE-2015-7990  CVE-2013-7446  CVE-2015-8019 Linux Kernel  CVE-2015-8019  CVE-2015-8539  CVE-2015-8539 Jan-15, 2016  CVE-2015-8660  CVE-2015-8660  CVE-2015-8709  CVE-2013-7446  CVE-2015-8709  CVE-2015-8812 Linux Kernel  CVE-2015-8019  CVE-2015-8812  CVE-2015-8816  CVE-2015-8539 Feb-10, 2016  CVE-2015-8816  CVE-2016-0728  CVE-2015-8660  CVE-2016-0728   CVE-2016-0758 CVE-2015-8709  CVE-2015-8709  CVE-2016-0758  Linux Kernel  CVE-2016-0774 CVE-2015-8812  CVE-2015-8812  CVE-2016-0774   CVE-2016-1583 CVE-2015-8816 Mar-22, 2016  CVE-2015-8816  CVE-2016-1583   CVE-2016-2053 CVE-2016-0758  CVE-2016-0758  CVE-2016-0758  CVE-2016-2053   CVE-2016-2384 CVE-2016-0774 Linux Kernel  CVE-2016-0774  CVE-2016-1583  CVE-2016-2384   CVE-2016-3134 CVE-2016-1583  CVE-2016-1583  CVE-2016-2053 Jun-09, 2016  CVE-2016-3134   CVE-2016-4470 CVE-2016-2053  CVE-2016-2053  CVE-2016-3134  CVE-2016-4470   CVE-2016-4565 CVE-2016-2384  CVE-2016-0758  CVE-2016-2384  CVE-2016-4470  CVE-2016-4565   CVE-2016-4997 CVE-2016-3134  CVE-2016-2053 Linux Kernel  CVE-2016-3134  CVE-2016-4565  CVE-2016-4997   CVE-2016-5829 CVE-2016-4470  CVE-2016-4470  CVE-2016-4470  CVE-2016-4997 Aug-16, 2016  CVE-2016-5829   CVE-2016-6480 CVE-2016-4565  CVE-2016-4565  CVE-2016-4565  CVE-2016-5829  CVE-2016-6480  CVE-2016-4997  CVE-2016-4997  CVE-2016-0758  CVE-2016-4997  CVE-2016-6480  Linux Kernel CVE-2016-5829  CVE-2016-5829  CVE-2016-2053  CVE-2016-5829  CVE-2016-6480  CVE-2016-6480  CVE-2016-4470 Sep-12, 2016  CVE-2016-6480  CVE-2016-4565  CVE-2016-6480  CVE-2016-5829

 CVE-2016-6480

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

33 That reminds me of ...

34 CVEs...? So what...?

• CVE-2016-0728

‒ gain privileges or cause a denial of service

• CVE-2015-8660

‒ local users can bypass intended access restrictions

• CVE-2015-8539

‒ gain privileges or cause a denial of service

• CVE-2015-7990

‒ allows local users to cause a denial of service

• CVE-2015-7872

‒ local users can cause a denial of service (OOPS)

• CVE-2015-6937

‒ local users can cause a denial of service (NULL pointer dereference and system crash)

• CVE-2013-7446

‒ local users to bypass intended AF_UNIX socket permissions or cause a denial of service (panic)

• ...

35 Can’t we patch software while it runs?

Mankind already flew to the moon …

36 Dynamic Software Updates

Trinity Test 1945 (Manhattan Project) • IBM punch card automatic calculators were used to crunch the numbers • A month before the Trinity nuclear device test, the question was: “What will the yield be, how much energy will be released?” • The calculation would normally take three months to complete – recalculating any batches with errors • Multiple colored punch cards introduced to fix errors in calculations while the calculator was running

37 Modern history of kGraft and other DSU technologies

• DSU: Dynamic Software Updates • the goal is to be able to fix bugs and add features either by - changing some functions or - replacing the whole program • kGraft developed as Open Source project by SUSE Labs • Upstream project „klp“ • Takes best of both kGraft (SUSE) and kpatch (Red Hat) • Still in catch up w.r.t. to features required by enterprises

UpStare kpatch klp PoDUS Gupta Erlang Ginseng Ksplice Kitsune kGraft

1990 1995 2000 2005 2010 2015

38 ftrace: return address modification mechanism

39 Common Pitfalls

• Function Inlining → DWARF to the rescue • Static Symbols → kernel keeps list: kallsyms • IPA-SRA (optimization like -O2) → using gcc optimization log • Multiple functions / dependencies → consistency model • Eternal sleepers (getty console 10) → send fake signal SIGKGRAFT / ignore • State transformation (req. for complex fixes) → not in kGraft right now • 3rd party kernel modules → depends on what the module does ...

40 Consistency

Requirement: ensure system consistency when deploying live patches

Freezing the system (kpatch, ksplice) Lazy migration (kGraft)

41 Consistency

Requirement: ensure system consistency when deploying live patches

Freezing the system (kpatch, ksplice) Lazy migration (kGraft)

stop_kernel();

check all stacks, whether any thread is stopped within a patched function

If yes, resume kernel and try again later

If not, flip the switch on all functions and resume the kernel

42 Consistency

Requirement: ensure system consistency when deploying live patches

Freezing the system (kpatch, ksplice) Lazy migration (kGraft)

stop_kernel(); For each thread separately:

Present the old version of functions to check all stacks, whether any thread is the thread until it leaves the kernel then stopped within a patched function give it the updated version

Wake sleeping threads up by a special If yes, resume kernel and try again later signal Prevent the signal from reaching userspace

If not, flip the switch on all functions and Once all threads have exited the kernel resume the kernel at least once we're DONE

43 Consistency

Requirement: ensure system consistency when deploying live patches

Freezing the system (kpatch, ksplice) Lazy migration (kGraft)

stop_kernel(); For each thread separately:

Present the old version of functions to check all stacks, whether any thread is the thread until it leaves the kernel then stopped within a patched function give it the updated version

Wake sleeping threads up by a special If yes, resume kernel and try again later signal. Prevent the signal from reaching userspace

If not, flip the switch on all functions and Once all threads have exited the kernel resume the kernel at least once we're DONE

Do you have better ideas than those two? Join SUSE as Live Patching developer https://jobs.suse.com/job/prague/live-patching-developer/3486/2529381 44 Consistency model for KLP?

The chosen model is a merge of kpatch and kGraft • Combines stack checking and per-thread changes • Non-intrusive, fast finishing • Works well already but requires both:

Reliable stack unwinder (needed by kpatch) Kernel thread model cleanup (needed by kGraft)

• Worked on by Josh Poimboeuf @ Red Hat • Worked on by Petr Mladek @ SUSE • Currently needs FRAME POINTER • Touches both kthreads and workqueues • up 10% slowdown of kernel execution • These parts are the critical core • Could use DWARF • Needs a lot of good planning and review • complex, being developed by SUSE • speed is a concern → Takes time • initial implementation removed from upstream

→ Takes time

45 Live Patching on ppc64le?

[ http://mpe.github.io/posts/2016/05/23/kernel-live-patching-for-ppc64le/ ]

46 In a SUSE data center, today ;-)

47 In a SUSE data center, today ;-)

Linux Kernel Nov-11, 2015

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

48 In a SUSE data center, today ;-)

Linux Kernel Nov-11, 2015

 CVE-2015-6937

 CVE-2015-7872

 CVE-2015-7990

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

49 In a SUSE data center, today ;-)

Linux Kernel Nov-11, 2015

Linux Kernel Dec-11, 2015

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

50 In a SUSE data center, today ;-)

Linux Kernel Nov-11, 2015

 CVE-2016-0728 Linux Kernel Dec-11, 2015

 CVE-2016-0728

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

51 In a SUSE data center, today ;-)

Linux Kernel Nov-11, 2015

Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

52 In a SUSE data center, today ;-)

Linux Kernel Nov-11, 2015

 CVE-2013-7446

 CVE-2015-8019 Linux Kernel  CVE-2015-8539 Dec-11, 2015  CVE-2015-8660  CVE-2013-7446 Linux Kernel  CVE-2015-8019

 CVE-2015-8539 Jan-15, 2016

 CVE-2015-8660  CVE-2013-7446

 CVE-2015-8019

 CVE-2015-8539

 CVE-2015-8660

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

53 In a SUSE data center, today ;-)

Linux Kernel Nov-11, 2015

Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

54 In a SUSE data center, today ;-)

Linux Kernel Nov-11, 2015

 CVE-2015-8709

 CVE-2015-8812 Linux Kernel  CVE-2015-8816 Dec-11, 2015  CVE-2016-0774  CVE-2015-8709  CVE-2016-2384 Linux Kernel  CVE-2015-8812

 CVE-2015-8816 Jan-15, 2016

 CVE-2016-0774  CVE-2015-8709  CVE-2016-2384 Linux Kernel  CVE-2015-8812

 CVE-2015-8816 Feb-10, 2016

 CVE-2016-0774  CVE-2015-8709  CVE-2016-2384  CVE-2015-8812

 CVE-2015-8816

 CVE-2016-0774

 CVE-2016-2384

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

55 In a SUSE data center, today ;-)

Linux Kernel Nov-11, 2015

Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

56 In a SUSE data center, today ;-)

Linux Kernel Nov-11, 2015

 CVE-2016-1583

 CVE-2016-3134 Linux Kernel Dec-11, 2015  CVE-2016-1583 Linux Kernel  CVE-2016-3134 Jan-15, 2016

 CVE-2016-1583 Linux Kernel  CVE-2016-3134 Feb-10, 2016  CVE-2016-1583 Linux Kernel  CVE-2016-3134 Mar-22, 2016

 CVE-2016-1583

 CVE-2016-3134

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

57 In a SUSE data center, today ;-)

Linux Kernel Nov-11, 2015

Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016 Linux Kernel Jun-09, 2016

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

58 In a SUSE data center, today ;-)

Linux Kernel Nov-11, 2015

 CVE-2016-4997 Linux Kernel Dec-11, 2015  CVE-2016-4997 Linux Kernel Jan-15, 2016

 CVE-2016-4997 Linux Kernel Feb-10, 2016  CVE-2016-4997 Linux Kernel Mar-22, 2016  CVE-2016-4997 Linux Kernel Jun-09, 2016

 CVE-2016-4997

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

59 In a SUSE data center, today ;-)

Linux Kernel Nov-11, 2015

Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016 Linux Kernel Jun-09, 2016

Linux Kernel Aug-16, 2016

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

60 In a SUSE data center, today ;-)

Linux Kernel Nov-11, 2015

 CVE-2016-0758

 CVE-2016-2053 Linux Kernel  CVE-2016-4470 Dec-11, 2015  CVE-2016-4565  CVE-2016-0758  CVE-2016-5829 Linux Kernel  CVE-2016-2053

 CVE-2016-4470 Jan-15, 2016

 CVE-2016-4565  CVE-2016-0758  CVE-2016-5829 Linux Kernel  CVE-2016-2053

 CVE-2016-4470 Feb-10, 2016

 CVE-2016-4565  CVE-2016-0758  CVE-2016-5829 Linux Kernel  CVE-2016-2053  CVE-2016-4470 Mar-22, 2016  CVE-2016-4565  CVE-2016-0758  CVE-2016-5829 Linux Kernel  CVE-2016-2053

 CVE-2016-4470 Jun-09, 2016

 CVE-2016-4565  CVE-2016-0758  CVE-2016-5829  CVE-2016-2053 Linux Kernel  CVE-2016-4470 Aug-16, 2016  CVE-2016-4565

 CVE-2016-5829  CVE-2016-0758  CVE-2016-2053

 CVE-2016-4470

 CVE-2016-4565

 CVE-2016-5829

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

61 In a SUSE data center, today ;-)

Linux Kernel Nov-11, 2015

Linux Kernel Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016 Linux Kernel Jun-09, 2016

Linux Kernel Aug-16, 2016

Linux Kernel Sep-12, 2016

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

62 In a SUSE data center, today ;-)

Linux Kernel Nov-11, 2015 Sample data taken  CVE-2016-6480 Linux Kernel on Sept-15, 2016 Dec-11, 2015  CVE-2016-6480 Linux Kernel Jan-15, 2016

 CVE-2016-6480 Linux Kernel Feb-10, 2016  CVE-2016-6480 Linux Kernel Mar-22, 2016  CVE-2016-6480 Linux Kernel Jun-09, 2016

 CVE-2016-6480 Linux Kernel Aug-16, 2016

 CVE-2016-6480 Linux Kernel Sep-12, 2016

 CVE-2016-6480

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

63 In a SUSE data center, today ;-)

Linux Kernel Nov-11, 2015 Sample data taken Linux Kernel on Sept-15, 2016 Dec-11, 2015 Linux Kernel Jan-15, 2016 Linux Kernel Feb-10, 2016 Linux Kernel Mar-22, 2016 Linux Kernel Jun-09, 2016

Linux Kernel Aug-16, 2016

Linux Kernel Sep-12, 2016

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

64 In a SUSE data center, today ;-)

 CVE-2016-6480

December January February March April May June July August September 2015 2016 2016 2016 2016 2016 2016 2016 2016 2016

65 Key Solution Highlights

 Available for SLES 12 onwards (x86-64)

 Provides fixes for Kernel bugs which affect

 Security

 Stability

 Data Integrity

 No runtime performance impact

 No interruption of applications while patching

 Allows full review of patch source code

 Build-in PTF support

 Patches available for most recent maintenance kernels (last 12 months)

 Currently based on kGraft OpenSource project

66 Where does SLE Live Patching make most sense? ... and where not? What‘s your guess?

67 Where does SLE Live Patching make most sense? ... and where not? What‘s your guess?

68 Where does SLE Live Patching make most sense? ... and where not? What‘s your guess?

http://cdn.slashgear.com/wp- (c) creativecommons.org/licenses/by/3.0 content/uploads/2012/10/google-datacenter-tech-21.jpg

69 Where does SLE Live Patching make most sense? ... and where not? What‘s your guess?

http://cdn.slashgear.com/wp- (c) creativecommons.org/licenses/by/3.0 content/uploads/2012/10/google-datacenter-tech-21.jpg

70 Where does SLE Live Patching make most sense? ... and where not? What‘s your guess?

http://cdn.slashgear.com/wp- (c) creativecommons.org/licenses/by/3.0 content/uploads/2012/10/google-datacenter-tech-21.jpg

SAP HANA

71 Outlook

SLE Live Patching for ppc64le SLE Live Patching for IBM z Systems

SLE Live User Space Patching for Live Patching Aarch64

Virtualization Live Patching

72 Further Information

Join SUSE as Live Patching developer https://jobs.suse.com/job/prague/live-patching-developer/3486/2529381

SUSE Linux Enterprise Live Patching – 60 day Eval www.suse.com/products/sles-for-sap/

Forrester – Linux vs. Unix Hot Patching – have we reached the tipping point? http://blogs.forrester.com/richard_fichera/16-05-20- linux_vs_unix_hot_patching_have_we_reached_the_tipping_point

7-11 November, 2016 www.susecon.com 73 Thank you

Hannes Kühnemund SUSE Product Management [email protected] @hakuehnemund www.linkedin.com/in/hanneskuehnemund

74 Backup

75 References

One hour of downtime costs $100k for 95% of all enterprises http://itic-corp.com/blog/2013/07/one-hour-of-downtime-costs-100k-for-95-of- enterprises/

Kernel Live Patching for ppc64le http://mpe.github.io/posts/2016/05/23/kernel-live-patching-for-ppc64le/

Forrester – Linux vs. Unix Hot Patching – have we reached the tipping point? http://blogs.forrester.com/richard_fichera/16-05-20- linux_vs_unix_hot_patching_have_we_reached_the_tipping_point

Using Live Patching to patch a running SAP HANA system with zero interruption https://www.youtube.com/watch?v=E9KwTfWeVLg 76