What's CGManager doing and why is it still relevant

Serge Hallyn

Canonical, Inc serge.hallyn@.com

October 4, 2014

Serge Hallyn () User Namespaces October 4, 2014 1 / 1 Overview

Agenda: CGroup intro Containers Requirements from Cgmanager details Demonstration Future and alternatives

Serge Hallyn (Canonical) User Namespaces October 4, 2014 2 / 1 About me

Started with kernel in 1999 Containers since vpid and nsproxy in 2006 Upsteam lxc maintainer Author of cgmanager

Serge Hallyn (Canonical) User Namespaces October 4, 2014 3 / 1

Introduced in 2007 as "task containers" Group tasks Track and Limit Resource Usage Memory, CPU, BlockIO, etc Administered through lesystem interface

Serge Hallyn (Canonical) User Namespaces October 4, 2014 4 / 1 Containers

OS level Uses many kernel features to emulate VM Uses cgroups for tracking and resource control Can be used without privilege

Serge Hallyn (Canonical) User Namespaces October 4, 2014 5 / 1 Why cgmanager

Support nesting Prevent escaping to parent cgroups even if root Avoid need to grant safe to cgroupfs

Serge Hallyn (Canonical) User Namespaces October 4, 2014 6 / 1 Why cgmanager

Simplify cgroup management code Many cgroupfs mounting possibilities No mount at all Bind-mount of container cgroup under /sys/fs/cgroup/subsys Full mount of container cgroup under /sys/fs/cgroup/subsys Fake path up to container cgroup Many cgroupfs mounting possibilities All co-mounted All separately mounted Some co-mounted Cgroup membership may be very dierent per hierarchy

Serge Hallyn (Canonical) User Namespaces October 4, 2014 7 / 1 CGManager

Full delegation not possible with cgroupfs (devices) Proposed idea in November 2013 Used by lxc, upstart, -shim, and libvirt

Serge Hallyn (Canonical) User Namespaces October 4, 2014 8 / 1 Cgmanager Design

One daemon per host Requests placed over dbus Cannot send pid across namespaces using dbus

Serge Hallyn (Canonical) User Namespaces October 4, 2014 9 / 1 Limits of DBus Request

Serge Hallyn (Canonical) User Namespaces October 4, 2014 10 / 1 Cgmanager Design

One daemon per host Requests placed over dbus Cannot send pid across namespaces using dbus Send pids and uids as SCM credentials

Serge Hallyn (Canonical) User Namespaces October 4, 2014 11 / 1 Enhanced DBus Request

Serge Hallyn (Canonical) User Namespaces October 4, 2014 12 / 1 Cgmanager Design

One daemon per host Requests placed over dbus Cannot send pid across namespaces using dbus Send pids and uids as SCM credentials One proxy per container Request can be sent as plain dbus to proxy in same ns All proxies connect to the host's cgmanager README details the security guarantees

Serge Hallyn (Canonical) User Namespaces October 4, 2014 13 / 1 Proxy architecture

Serge Hallyn (Canonical) User Namespaces October 4, 2014 14 / 1 DBus methods

Ping, ApiVersion GetPidCgroup Create, Chown, Chmod MovePid, MovePidAbs GetValue, SetValue Limits specied using cgroup lenames Lxc has exported these since 2008 New API would be temporary Minimize churn for lxc GetTasks, GetTasksRecursive, ListChildren Remove, RemoveOnEmpty, Prune ListControllers, ListKeys

Serge Hallyn (Canonical) User Namespaces October 4, 2014 15 / 1 Enhance systemd slices Allow users to specify sub-slices Support delegation to user namespaces Continue with CGManager Abstract away limit lenames Keep state (hotplug, etc)

Future: Possibilities:

Possible Alternatives Enhance cgfs Fake root (cgroup namespaces, Aditya Kali@) Full delegation

Serge Hallyn (Canonical) User Namespaces October 4, 2014 16 / 1 Continue with CGManager Abstract away limit lenames Keep state (hotplug, etc)

Future: Possibilities:

Possible Alternatives Enhance cgfs Fake root (cgroup namespaces, Aditya Kali@google) Full delegation Enhance systemd slices Allow users to specify sub-slices Support delegation to user namespaces

Serge Hallyn (Canonical) User Namespaces October 4, 2014 16 / 1 Future: Possibilities:

Possible Alternatives Enhance cgfs Fake root (cgroup namespaces, Aditya Kali@google) Full delegation Enhance systemd slices Allow users to specify sub-slices Support delegation to user namespaces Continue with CGManager Abstract away limit lenames Keep state (hotplug, etc)

Serge Hallyn (Canonical) User Namespaces October 4, 2014 16 / 1