What's CGManager doing and why is it still relevant Serge Hallyn Canonical, Inc [email protected] October 4, 2014 Serge Hallyn (Canonical) User Namespaces October 4, 2014 1 / 1 Overview Agenda: CGroup intro Containers Requirements from lxc Cgmanager details Demonstration Future and alternatives Serge Hallyn (Canonical) User Namespaces October 4, 2014 2 / 1 About me Started with linux kernel in 1999 Containers since vpid and nsproxy in 2006 Upsteam lxc maintainer Author of cgmanager Serge Hallyn (Canonical) User Namespaces October 4, 2014 3 / 1 Cgroups Introduced in 2007 as "task containers" Group tasks Track and Limit Resource Usage Memory, CPU, BlockIO, etc Administered through lesystem interface Serge Hallyn (Canonical) User Namespaces October 4, 2014 4 / 1 Containers OS level virtualization Uses many kernel features to emulate VM Uses cgroups for tracking and resource control Can be used without privilege Serge Hallyn (Canonical) User Namespaces October 4, 2014 5 / 1 Why cgmanager Support nesting Prevent escaping to parent cgroups even if root Avoid need to grant safe access to cgroupfs Serge Hallyn (Canonical) User Namespaces October 4, 2014 6 / 1 Why cgmanager Simplify cgroup management code Many cgroupfs mounting possibilities No mount at all Bind-mount of container cgroup under /sys/fs/cgroup/subsys Full mount of container cgroup under /sys/fs/cgroup/subsys Fake path up to container cgroup Many cgroupfs mounting possibilities All co-mounted All separately mounted Some co-mounted Cgroup membership may be very dierent per hierarchy Serge Hallyn (Canonical) User Namespaces October 4, 2014 7 / 1 CGManager Full delegation not possible with cgroupfs (devices) Proposed idea in November 2013 Used by lxc, upstart, systemd-shim, and libvirt Serge Hallyn (Canonical) User Namespaces October 4, 2014 8 / 1 Cgmanager Design One daemon per host Requests placed over dbus Cannot send pid across namespaces using dbus Serge Hallyn (Canonical) User Namespaces October 4, 2014 9 / 1 Limits of DBus Request Serge Hallyn (Canonical) User Namespaces October 4, 2014 10 / 1 Cgmanager Design One daemon per host Requests placed over dbus Cannot send pid across namespaces using dbus Send pids and uids as SCM credentials Serge Hallyn (Canonical) User Namespaces October 4, 2014 11 / 1 Enhanced DBus Request Serge Hallyn (Canonical) User Namespaces October 4, 2014 12 / 1 Cgmanager Design One daemon per host Requests placed over dbus Cannot send pid across namespaces using dbus Send pids and uids as SCM credentials One proxy per container Request can be sent as plain dbus to proxy in same ns All proxies connect to the host's cgmanager README details the security guarantees Serge Hallyn (Canonical) User Namespaces October 4, 2014 13 / 1 Proxy architecture Serge Hallyn (Canonical) User Namespaces October 4, 2014 14 / 1 DBus methods Ping, ApiVersion GetPidCgroup Create, Chown, Chmod MovePid, MovePidAbs GetValue, SetValue Limits specied using cgroup lenames Lxc has exported these since 2008 New API would be temporary Minimize churn for lxc GetTasks, GetTasksRecursive, ListChildren Remove, RemoveOnEmpty, Prune ListControllers, ListKeys Serge Hallyn (Canonical) User Namespaces October 4, 2014 15 / 1 Enhance systemd slices Allow users to specify sub-slices Support delegation to user namespaces Continue with CGManager Abstract away limit lenames Keep state (hotplug, etc) Future: Possibilities: Possible Alternatives Enhance cgfs Fake root (cgroup namespaces, Aditya Kali@google) Full delegation Serge Hallyn (Canonical) User Namespaces October 4, 2014 16 / 1 Continue with CGManager Abstract away limit lenames Keep state (hotplug, etc) Future: Possibilities: Possible Alternatives Enhance cgfs Fake root (cgroup namespaces, Aditya Kali@google) Full delegation Enhance systemd slices Allow users to specify sub-slices Support delegation to user namespaces Serge Hallyn (Canonical) User Namespaces October 4, 2014 16 / 1 Future: Possibilities: Possible Alternatives Enhance cgfs Fake root (cgroup namespaces, Aditya Kali@google) Full delegation Enhance systemd slices Allow users to specify sub-slices Support delegation to user namespaces Continue with CGManager Abstract away limit lenames Keep state (hotplug, etc) Serge Hallyn (Canonical) User Namespaces October 4, 2014 16 / 1.