Dynamically Hacking the Kernel with Containers ContainerCon Japan 2016 Tokyo Quey-Liang Kao National Tsing Hua University, Taiwan About Myself ● Research topics – HPC (nu erical!, Heterogeneous computing – High-end har"ware virtuli#ation ($nfini&an", GPGPU! – Container technology ● Contributions – AUR packager an" maintainer ● runc-git, openscap ● kpatch Outline ● +otivation ● &ac*groun" ● ,e o- *ernel "etouring – FreeB/, on Linux ● Other approaches ● Conclusion +otivation http-44type oon.wi*ia.co 4wi*i4Holy6'rail Higher Perfor ance 3+ Container Lower Isolation Quick Survey Terms ● OS container ● )pp container – Like a 3+ – .or single service – No layered ./ – layered ./ – L7C, 1pen3Z, &/, – ,oc*er, Rocket Jails, /olaris zones https-44(log.risingstack.co 4operating-syste -containers-vs-application-containers4 There Dimensions ● System software – 1/ container's perspective ● Orchestration – )pp container's perspective ● Applications )pplication ,evOps ,oc*er Swar NS, Cgroups Ku(ernetes Security Core1/ Syste 1rchestration How do ' "rchestrate y Container? By Isabel Jimenez )pplication Syste 1rchestration Containers at scale thanks to Kubernetes By Bran"on Philips )pplication Syste 1rchestration A Security State of in&) Container Security in the *nterprise )pplication By Chris Van Tuin Syste 1rchestration How the hell do ' run my containers in production, and will it scale( )pplication By Daniël van 'ils Syste 1rchestration Container Hacks , Fun 'mages By Jessie Frazelle )pplication Syste 1rchestration .ootless Container with Runc By )le*sa /arai )pplication Syste 1rchestration Soft Container Towards 100% Resource 0tilization By )ccela 8hao )pplication Syste 1rchestration Unpri%ileged Containers: What you Always Wanted to Know About 3ame4 spaces But 2ere Too )pplication Afraid To Ask By Ja es Botto ley Syste 1rchestration etc.66 )pplication This wor* shoul" (e here Syste 1rchestration Backgroun" Modules, Live patches, an" Kerenl detouring Kernel Module) 7oading Process- some_ext5*o insmod test.ko syscall- init_ o"ule User Space Kernel Kernel Space Kernel Module) 0sing Process- cat /dev/some_cdev User Space syscall- read Kernel some_ext5*o Kernel Space 7i%e 8atching: Building Current *ernel source Process- patch-fi05*o .ile: build fix.patch fix.patch syscall- init_ o"ule User Space &ug(! Kernel Kernel Space 7i%e 8atching: Applying Process- %05*o insmod fix.ko syscall- init_ o"ule User Space &ug(! Kernel Kernel Space 7i%e 8atching: Applying User Space &ug(! Bug-fixed(! Kernel %05*o ftrace Kernel Space Kernel Detouring Nor al Process Container User Space 2unc(! 2unc(! "etour5*o Kernel (na espace-aware! Kernel Space Demo: Kernel Detouring http-4/kirokueiga.seesaa.net/archives4<=><=?->.ht l -reeBSD binary on 7inux 01000110011100100110010101100101010000100101001101000100 $peci:c Challenges ( FreeBSD ) ● Correspon"ing syste calls – Flag num(ers are not porta(le – "i@erent calling/exiting conventions ● UniAue syste calls – Re-implementation General Challenges ● Insufficient isolation ● Limitation of "evelopment – live patching shoul" only (e a temp. solution Other Binary Compatibility Work ● Wine – /pecial loa"er 2or PDs4,LLs ● Free&/,, Win"ows >= – Kernel (uilt-in compati(ility layer 2or Linux (inary – /ystem call remapping/re-imple entation Possible Applications ● D0peri ental mo"ule4patch test (e" ● $ ages for other 1Ses ● D"ucational purpose – why notE Other approaches ● Hyper-V ● +ulti-Kernel ● Uni*ernel icrosoft Hyper4> ● ) private *ernel per container – strippe" *ernel reduced 2rom Cin"ows server ● Uni0-li*es support – as 3+s ( in a 3+-like container ! ( container-like 3M ! ulti-Kernel ● &arrelfish – Philosophy: separation an" &uplication rather than *eep syncing – 1ne kernel per core – /calability an" heterogeneity ● 3irtuOS, Arra*is, Quest-V, etc. – Per2ormance i provement UniKernel ● Rump Kernel, +irage1S, OSv, etc5 – )pplication oriented ● no ore general-purpose – FCo piler” approach http-44www.pennin*ho2.co 4<=>H4=H4 ini alist-cassan"ra-v -using-osv4 http-44type oon.wi*ia.co 4wi*i4Holy6'rail Higher 3+ Perfor ance Hyper-3 New paradig s Kernel detouring ( Multi-kernel, Uni*ernels ) ( or some kernel-space e0t. ) Lower Container Isolation Conclusion ● The kernel detouring "e o atte pts to in"icate a possi(le ove ent of the "evelopment o2 OS containers – as a proo2-o2-concept ● Future "irection – Make ore 2un – Make it ore complete Q & A .