Using Linux VServer
Enrico Scholz [email protected]
- p. 1/47 Motivation (1)
Introduction G Motivation (1) Administrator: G Motivation (2) G Requirements G Wishes I lots of services (FTP, HTTP*, LDAP, KRB, DNS, . . . ) G Solutions I vserver updates without side-effects
Security I own hostnames and IPs The Toolsets I access restrictions Base-Operations
Management Prospective Developer:
I tests in different environments (compiler, libraries, programs, distributions) I providing binaries for different distributions
- p. 2/47 Motivation (2)
Introduction G Motivation (1) Businessman: G Motivation (2) G Requirements G Wishes I selling of “root-servers” (IP, CPU power, disk-space, G Solutions root-account) vserver
Security
The Toolsets
Base-Operations Solutions:
Management I Prospective one physical machine per server but: hardware costs, room, cooling, UPS I multiple dedicated servers on the same hardware ⇒ “virtual servers”
- p. 3/47 Requirements
Introduction I behavior like an ordinary server (same binaries, no special G Motivation (1) G Motivation (2) syscalls) G Requirements G Wishes I G Solutions process isolation N vserver kill(2), ptrace(2) N Security /etc/init.d/sshd restart I The Toolsets filesystem isolation N Base-Operations no collisions when using standard-paths N Management keeping of secrets
Prospective I no backdoors N direct hardware-access (/dev/hda) N direct kernel-access (/dev/kmem) ⇒ no influence on the function of other servers or of the host
- p. 4/47 Wishes
Introduction I effective G Motivation (1) G Motivation (2) N performance (CPU, I/O) G Requirements G Wishes N memory (RAM, disk) G Solutions I vserver easy manageable N Security creation N operation The Toolsets ⇒ using of known tools Base-Operations I Management limits/quotas for diskspace, CPU, net
Prospective I migration to other physical machines
- p. 5/47 I vmware/bochs/qemu N usable as usual machine N but: high resource consumption; often for i386 only I UML N middle till high resource consumption I SELinux N fulfills (security related) requirements N no complete virtualization (hostname, ip) N ??? I Linux vserver, BSD Jails, SUN Zones, FreeVPS N grouping of processes N usage of same hardware and kernel N new and already existing access control mechanisms N nearly no overhead
Solutions
Introduction I special hardware (S/390) G Motivation (1) G Motivation (2) G Requirements G Wishes G Solutions
vserver
Security
The Toolsets
Base-Operations
Management
Prospective
- p. 6/47 I UML N middle till high resource consumption I SELinux N fulfills (security related) requirements N no complete virtualization (hostname, ip) N ??? I Linux vserver, BSD Jails, SUN Zones, FreeVPS N grouping of processes N usage of same hardware and kernel N new and already existing access control mechanisms N nearly no overhead
Solutions
Introduction I special hardware (S/390) G Motivation (1) G Motivation (2) I G Requirements vmware/bochs/qemu G Wishes N G Solutions usable as usual machine N vserver but: high resource consumption; often for i386 only
Security
The Toolsets
Base-Operations
Management
Prospective
- p. 6/47 I SELinux N fulfills (security related) requirements N no complete virtualization (hostname, ip) N ??? I Linux vserver, BSD Jails, SUN Zones, FreeVPS N grouping of processes N usage of same hardware and kernel N new and already existing access control mechanisms N nearly no overhead
Solutions
Introduction I special hardware (S/390) G Motivation (1) G Motivation (2) I G Requirements vmware/bochs/qemu G Wishes N G Solutions usable as usual machine N vserver but: high resource consumption; often for i386 only I Security UML N The Toolsets middle till high resource consumption
Base-Operations
Management
Prospective
- p. 6/47 I Linux vserver, BSD Jails, SUN Zones, FreeVPS N grouping of processes N usage of same hardware and kernel N new and already existing access control mechanisms N nearly no overhead
Solutions
Introduction I special hardware (S/390) G Motivation (1) G Motivation (2) I G Requirements vmware/bochs/qemu G Wishes N G Solutions usable as usual machine N vserver but: high resource consumption; often for i386 only I Security UML N The Toolsets middle till high resource consumption
Base-Operations I SELinux Management N fulfills (security related) requirements Prospective N no complete virtualization (hostname, ip) N ???
- p. 6/47 Solutions
Introduction I special hardware (S/390) G Motivation (1) G Motivation (2) I G Requirements vmware/bochs/qemu G Wishes N G Solutions usable as usual machine N vserver but: high resource consumption; often for i386 only I Security UML N The Toolsets middle till high resource consumption
Base-Operations I SELinux Management N fulfills (security related) requirements Prospective N no complete virtualization (hostname, ip) N ??? I Linux vserver, BSD Jails, SUN Zones, FreeVPS N grouping of processes N usage of same hardware and kernel N new and already existing access control mechanisms N nearly no overhead
- p. 6/47 Introduction vserver G Quickstart (1) G Quickstart (2) G Properties (Kernel) (1) G Properties (Kernel) (2) G Userspace vserver G chroot-environments
Security
The Toolsets
Base-Operations
Management
Prospective
- p. 7/47 Quickstart (1)
Introduction 1. download and untaring of the kernel sources vserver G Quickstart (1) $ wget http://ftp.kernel.org/pub/linux/kernel/v2.x/linux-2.x.y.tar.bz2 G Quickstart (2) $ tar xjf linux-2.x.y.tar.bz2 G Properties (Kernel) (1) $ cd linux-2.x.y G Properties (Kernel) (2) G Userspace a G chroot-environments 2. download of the corresponding vserver-patch from Security http://www.13thfloor.at/vserver/project/ and applying of this The Toolsets patch Base-Operations $ bzcat patch-2.x.y-vs1.z.diff.bz2 | patch -p1 Management
Prospective 3. configuration, build and installation of the kernel $ make config $ make dep && make all modules && make install modules_install
a 1.2x – stable, 1.3x and 1.9x – experimental
- p. 8/47 Quickstart (2)
Introduction 4. download of the userspace toolsa (util-vserver) from vserver http://www.nongnu.org/util-vserver G Quickstart (1) G Quickstart (2) G Properties (Kernel) (1) 5. configuration, build and installation G Properties (Kernel) (2) G Userspace $ rpmbuild -ta util-vserver-0.x.y.tar.bz2 \ G chroot-environments [--without xalan] [--without dietlibc] # rpm -Uvh ... Security
The Toolsets oder Base-Operations $ ./configure [--prefix=...]
a 0.x.y → stable if no y, pre if y<90, rc if 90≤y<190 and alpha if 190≤y
- p. 9/47 Properties (Kernel) (1)
Introduction I developer: Herbert Pötzl vserver I G Quickstart (1) one multiswitch syscall for the entire functionality G Quickstart (2) G Properties (Kernel) (1) I attributes for processes: G Properties (Kernel) (2) N xid G Userspace numeric context-ID ( ) G chroot-environments → processes with xid1 invisible for xid2-processes Security I attributes for process-contexts: The Toolsets N hostname resp. a complete utsname entry(2.6) Base-Operations N system- & context-specific(2.6) capabilities Management N flags Prospective N namespace(2.6) N scheduling parameters(2.6)
- p. 10/47 Properties (Kernel) (2)
Introduction I large parts of security based on linux-capabilities vserver (/usr/include/linux/capability.h), e.g. G Quickstart (1) G Quickstart (2) N no new devices without CAP_MKNOD G Properties (Kernel) (1) G Properties (Kernel) (2) N no interface-configuration without CAP_NET_ADMIN G Userspace N G chroot-environments no filesystem-mounting without CAP_SYS_ADMIN N Security . . . The Toolsets I hiding of filesystem-entries Base-Operations some entries in /proc without capability protection, e.g. Management /proc/sysrq-trigger or /proc/scsi/scsi Prospective ⇒ hiding outside of host-context I context-quotas I outbreak-safe chroot(2) environments
- p. 11/47 Userspace
Introduction I two toolsets: “vserver” und “util-vserver” vserver I G Quickstart (1) low-level syscallwrappers G Quickstart (2) G Properties (Kernel) (1) I vserver == chroot-environment + configuration-data G Properties (Kernel) (2) G Userspace I management of the vservers G chroot-environments N creation Security N starting/stopping The Toolsets N optimizations Base-Operations I Management configuration usually under /etc/vservers/ I Prospective starting with “vserver
- p. 12/47 I chroot-environment must be assumed as hostile N execution of arbitrary programs as root N creation, removal, renaming and modification of arbitrary files, symlinks and directories ⇒ special care and kernel-support required
chroot-environments
Introduction I usually at /vservers/
Security
The Toolsets
Base-Operations
Management
Prospective
- p. 13/47 chroot-environments
Introduction I usually at /vservers/
- p. 13/47 I creation of a new “/”: # mount --rbind /vservers/ftp / ⇒ execution in an own namespace N additional kernel features needed for practial application (migrate()) N cleaning up of /proc/mounts possible N not fully implemented currently
Classical chroot-attack
Introduction usr lib ftp vserver
Security vservers G Classical chroot-attack G Excursion: namespaces G Infiltrating foreign chroots / G Symlink-attacks (1) G Symlink-attacks (2) G Other attacks
The Toolsets
Base-Operations
Management
Prospective
- p. 14/47 I creation of a new “/”: # mount --rbind /vservers/ftp / ⇒ execution in an own namespace N additional kernel features needed for practial application (migrate()) N cleaning up of /proc/mounts possible N not fully implemented currently
Classical chroot-attack
Introduction usr lib ftp vserver
Security vservers G Classical chroot-attack G Excursion: namespaces G / Infiltrating foreign chroots chroot("/vservers/ftp"); G Symlink-attacks (1) G Symlink-attacks (2) G Other attacks
The Toolsets
Base-Operations
Management
Prospective
- p. 14/47 I creation of a new “/”: # mount --rbind /vservers/ftp / ⇒ execution in an own namespace N additional kernel features needed for practial application (migrate()) N cleaning up of /proc/mounts possible N not fully implemented currently
Classical chroot-attack
Introduction usr lib ftp vserver
Security vservers G Classical chroot-attack G Excursion: namespaces cwd G / Infiltrating foreign chroots chroot("/vservers/ftp"); G Symlink-attacks (1) G Symlink-attacks (2) chdir("/"); G Other attacks
The Toolsets
Base-Operations
Management
Prospective
- p. 14/47 I creation of a new “/”: # mount --rbind /vservers/ftp / ⇒ execution in an own namespace N additional kernel features needed for practial application (migrate()) N cleaning up of /proc/mounts possible N not fully implemented currently
Classical chroot-attack
Introduction usr lib ftp vserver
Security vservers G Classical chroot-attack G Excursion: namespaces cwd G / Infiltrating foreign chroots chroot("/vservers/ftp"); G Symlink-attacks (1) G Symlink-attacks (2) chdir("/"); G Other attacks chroot("/usr/lib"); The Toolsets
Base-Operations
Management
Prospective
- p. 14/47 I creation of a new “/”: # mount --rbind /vservers/ftp / ⇒ execution in an own namespace N additional kernel features needed for practial application (migrate()) N cleaning up of /proc/mounts possible N not fully implemented currently
Classical chroot-attack
Introduction usr lib ftp vserver
Security vservers G Classical chroot-attack G Excursion: namespaces G / Infiltrating foreign chroots chroot("/vservers/ftp"); G Symlink-attacks (1) G Symlink-attacks (2) chdir("/"); G Other attacks cwd chroot("/usr/lib"); The Toolsets chdir("../.."); Base-Operations
Management
Prospective
- p. 14/47 I creation of a new “/”: # mount --rbind /vservers/ftp / ⇒ execution in an own namespace N additional kernel features needed for practial application (migrate()) N cleaning up of /proc/mounts possible N not fully implemented currently
Classical chroot-attack
Introduction usr lib ftp vserver
Security vservers G Classical chroot-attack G Excursion: namespaces cwd G / Infiltrating foreign chroots chroot("/vservers/ftp"); G Symlink-attacks (1) G Symlink-attacks (2) chdir("/"); G Other attacks chroot("/usr/lib"); The Toolsets chdir("../.."); => −EACCES Base-Operations I static barriers in the filesystem (kernelpatch): not traversable Management by processes outside of the host-context Prospective
- p. 14/47 Classical chroot-attack
Introduction usr lib / vserver ftp
Security vservers G Classical chroot-attack G Excursion: namespaces cwd G / Infiltrating foreign chroots chroot("/vservers/ftp"); G Symlink-attacks (1) G Symlink-attacks (2) chdir("/"); G Other attacks chroot("/usr/lib"); The Toolsets chdir("../.."); Base-Operations I creation of a new “/”: Management # mount --rbind /vservers/ftp / Prospective ⇒ execution in an own namespace N additional kernel features needed for practial application (migrate()) N cleaning up of /proc/mounts possible N not fully implemented currently
- p. 14/47 Excursion: namespaces
Introduction I new namespace with CLONE_NEWNS; documented in vserver clone(2) manpage: Security Every process lives in a namespace. The namespace G Classical chroot-attack G Excursion: namespaces of a process is the data (the set of mounts) describing G Infiltrating foreign chroots G Symlink-attacks (1) the file hierarchy as seen by that process. G Symlink-attacks (2) G Other attacks I relative new (kernel 2.4.19); not for vservers only The Toolsets I conflicts with automounters Base-Operations
Management Example: Prospective
[root@kosh root]# vnamespace --new sh sh-2.05b# mount --bind /bin/rm /bin/ls sh-2.05b# ls /etc/* [root@kosh root]# ls /etc/* ... lieber nicht ... /etc/DIR_COLORS sh-2.05b# exit ... [root@kosh root]# ls /etc/* [root@kosh root]# /etc/DIR_COLORS ... [root@kosh root]#
- p. 15/47 Infiltrating foreign chroots
Introduction ftp home vserver
Security G Classical chroot-attack / vservers foo G Excursion: namespaces mount −−bind ... G Infiltrating foreign chroots G Symlink-attacks (1) G Symlink-attacks (2) G Other attacks home The Toolsets www
Base-Operations
Management
Prospective
I two vservers “www” und “ftp” I commonly used /home directory
- p. 16/47 Infiltrating foreign chroots
Introduction ftp home vserver
Security G Classical chroot-attack / vservers cwd foo G Excursion: namespaces mount −−bind ... G Infiltrating foreign chroots G Symlink-attacks (1) cwd G Symlink-attacks (2) G Other attacks home The Toolsets www
Base-Operations
Management
Prospective
I static barrier at /vservers I root-rights for “red” in “ftp”; “blue” only an ordinary user in “www”
- p. 16/47 Infiltrating foreign chroots
Introduction ftp home vserver
Security G Classical chroot-attack / vservers cwd foo sock G Excursion: namespaces mount −−bind ... G Infiltrating foreign chroots G Symlink-attacks (1) cwd AF_UNIX G Symlink-attacks (2) G Other attacks home The Toolsets www Base-Operations tmp=socket(AF_UNIX,...); tmp=socket(AF_UNIX,...) Management bind(tmp,"/home/foo/sock"); connect(s,"/home/foo/sock");
Prospective listen(tmp) s=accept(tmp);
- p. 16/47 Infiltrating foreign chroots
Introduction ftp home vserver
Security G Classical chroot-attack / vservers cwd foo sock G Excursion: namespaces mount −−bind ... G Infiltrating foreign chroots G Symlink-attacks (1) cwd AF_UNIX G Symlink-attacks (2) fd G Other attacks home The Toolsets www Base-Operations tmp=socket(AF_UNIX,...); tmp=socket(AF_UNIX,...) Management bind(tmp,"/home/foo/sock"); connect(s,"/home/foo/sock");
Prospective listen(tmp) fd=open(".",O_RDONLY); s=accept(tmp);
- p. 16/47 Infiltrating foreign chroots
Introduction ftp home vserver
Security G Classical chroot-attack / vservers cwd foo sock G Excursion: namespaces mount −−bind ... G Infiltrating foreign chroots G Symlink-attacks (1) cwd AF_UNIX G Symlink-attacks (2) fd G Other attacks home The Toolsets www fd Base-Operations tmp=socket(AF_UNIX,...); tmp=socket(AF_UNIX,...) Management bind(tmp,"/home/foo/sock"); connect(s,"/home/foo/sock");
Prospective listen(tmp) fd=open(".",O_RDONLY); s=accept(tmp); recvmsg(s,{&fd,SCM_RIGHTS}); sendmsg(s,{fd,SCM_RIGHTS});
- p. 16/47 Infiltrating foreign chroots
Introduction ftp home vserver
Security G Classical chroot-attack / vservers foo sock G Excursion: namespaces mount −−bind ... G Infiltrating foreign chroots G Symlink-attacks (1) cwd AF_UNIX G Symlink-attacks (2) fd G Other attacks home The Toolsets cwd www fd Base-Operations tmp=socket(AF_UNIX,...); tmp=socket(AF_UNIX,...) Management bind(tmp,"/home/foo/sock"); connect(s,"/home/foo/sock");
Prospective listen(tmp) fd=open(".",O_RDONLY); s=accept(tmp); recvmsg(s,{&fd,SCM_RIGHTS}); sendmsg(s,{fd,SCM_RIGHTS}); fchdir(fd);
- p. 16/47 I unsolved in vserver; perhaps preventable with SELinux or partly with namespaces
Infiltrating foreign chroots
Introduction ftp home vserver
Security G Classical chroot-attack / vservers foo sock G Excursion: namespaces mount −−bind ... G Infiltrating foreign chroots G Symlink-attacks (1) cwd AF_UNIX G Symlink-attacks (2) fd G Other attacks home The Toolsets cwd www fd Base-Operations tmp=socket(AF_UNIX,...); tmp=socket(AF_UNIX,...) Management bind(tmp,"/home/foo/sock"); connect(s,"/home/foo/sock");
Prospective listen(tmp) fd=open(".",O_RDONLY); s=accept(tmp); recvmsg(s,{&fd,SCM_RIGHTS}); sendmsg(s,{fd,SCM_RIGHTS}); fchdir(fd);
⇒ root-rights for “red” in “www”
- p. 16/47 Infiltrating foreign chroots
Introduction ftp home vserver
Security G Classical chroot-attack / vservers foo sock G Excursion: namespaces mount −−bind ... G Infiltrating foreign chroots G Symlink-attacks (1) cwd AF_UNIX G Symlink-attacks (2) fd G Other attacks home The Toolsets cwd www fd Base-Operations tmp=socket(AF_UNIX,...); tmp=socket(AF_UNIX,...) Management bind(tmp,"/home/foo/sock"); connect(s,"/home/foo/sock");
Prospective listen(tmp) fd=open(".",O_RDONLY); s=accept(tmp); recvmsg(s,{&fd,SCM_RIGHTS}); sendmsg(s,{fd,SCM_RIGHTS}); fchdir(fd); open("etc/passwd",...); ⇒ root-rights for “red” in “www” I unsolved in vserver; perhaps preventable with SELinux or partly with namespaces
- p. 16/47 actions executed by the host-administrator at “/”: # rm -f /vserver/ftp/var/lock/subsys/* # touch /vservers/ftp/var/run/utmp # mount /dev/hda1 /vserver/ftp/var/lock/subsys
Symlink-attacks (1)
Introduction
vserver ftp / var Security vservers G Classical chroot-attack G Excursion: namespaces G Infiltrating foreign chroots / G Symlink-attacks (1) G Symlink-attacks (2) G Other attacks
The Toolsets
Base-Operations
Management
Prospective
- p. 17/47 actions executed by the host-administrator at “/”: # rm -f /vserver/ftp/var/lock/subsys/* # touch /vservers/ftp/var/run/utmp # mount /dev/hda1 /vserver/ftp/var/lock/subsys
Symlink-attacks (1)
Introduction run/utmp −> /etc/nologin £ £ £ £ £ £ ¢ ¢ ¢ ¢ ¢ ¢ ¢ vserver ftp / var £ £ £ £ £ £ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¡ ¡ ¡ ¡ ¡ ¡ Security vservers Vserver−Admin: G Classical chroot-attack # ln −s /etc /var/lock/subsys ¡ ¡ ¡ ¡ ¡ ¡ G Excursion: namespaces G Infiltrating foreign chroots / # ln −s /etc/nologin /var/run/utmp ¡ ¡ ¡ ¡ ¡ ¡ G Symlink-attacks (1) etc G Symlink-attacks (2) lock/subsys −> /etc G Other attacks
The Toolsets
Base-Operations
Management
Prospective
- p. 17/47 Symlink-attacks (1)
Introduction run/utmp −> /etc/nologin
¨ ¨ ¨ ¨ ¨ ¨ ¨ vserver ftp / var
© © © © © © Security ¨ ¨ ¨ ¨ ¨ ¨ ¨ vservers Vserver−Admin: G Classical chroot-attack # ln −s /etc /var/lock/subsys © © © © © © G Excursion: namespaces ¨ ¨ ¨ ¨ ¨ ¨ ¨ § § § § § § § § ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ G Infiltrating foreign chroots / # ln −s /etc/nologin /var/run/utmp © © © © © © G ¨ ¨ ¨ ¨ ¨ ¨ ¨ § § § § § § § § ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ Symlink-attacks (1) G Symlink-attacks (2) lock/subsys −> /etc § § § § § § § § G Other attacks ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¤¥ The Toolsets
Base-Operations etc
Management
Prospective actions executed by the host-administrator at “/”: # rm -f /vserver/ftp/var/lock/subsys/* # touch /vservers/ftp/var/run/utmp # mount /dev/hda1 /vserver/ftp/var/lock/subsys
- p. 17/47 Symlink-attacks (2)
Introduction I preventing symlinkattacks by entering the directories in a vserver secure way; often implemented by
Security G Classical chroot-attack chroot ( v s e r v e r _ r o o t d i r ) ; G Excursion: namespaces c h d i r ( d e s t i n a t i o n _ d i r e c t o r y ) ; G Infiltrating foreign chroots a c t i o n ( ) ; G Symlink-attacks (1) G Symlink-attacks (2) G Other attacks I operations only in “.” (exec-cd tool) The Toolsets / / Usage : exec−cd < d i r >
I e.g.: # cd /vservers/ftp && exec-cd /var/lock/subsys mount /dev/hda1 ’.’
- p. 18/47 Other attacks
Introduction I modification of files which are usually writable for root only vserver (/etc/passwd, rpm-database) Security ⇒ overflows G Classical chroot-attack G Excursion: namespaces ⇒ execution of code in host-context G Infiltrating foreign chroots G Symlink-attacks (1) Solution: Important files outside of VServer; G Symlink-attacks (2) G Other attacks helperprograms in VServer-context The Toolsets I dynamic library-loading (/lib/libnss_*) (functional deficiencies Base-Operations also) Management Solution: dietlibc instead of glibc Prospective I races when traversing the filesystem Solution: secure directory-changing; enforcing of stopped VServers I no chroot(2) before entering another context ⇒ hijacking through ptrace(2) Solution: do not do this. . .
- p. 19/47 I lots of open wishes I not applicably in hostile environments because lots of attack-vectors for symlinkattacks and races ⇒ complete redesign necessary I no active development; only bugfixes I no support for new kernel-features
util-vserver, stable (1)
Introduction I “vserver” and stable-branch of “util-vserver”: vserver N nearly the same functionality Security N “util-vserver” forked at “vserver 0.23” The Toolsets I G util-vserver, stable (1) spread widely G util-vserver, stable (2) I G util-vserver, alpha very good documentation G Configuration
Base-Operations
Management
Prospective
- p. 20/47 util-vserver, stable (1)
Introduction I “vserver” and stable-branch of “util-vserver”: vserver N nearly the same functionality Security N “util-vserver” forked at “vserver 0.23” The Toolsets I G util-vserver, stable (1) spread widely G util-vserver, stable (2) I G util-vserver, alpha very good documentation G Configuration I lots of open wishes Base-Operations I Management not applicably in hostile environments because lots of
Prospective attack-vectors for symlinkattacks and races ⇒ complete redesign necessary I no active development; only bugfixes I no support for new kernel-features
- p. 20/47 util-vserver, stable (2)
Introduction configuration in /etc/vservers/
Security IPROOT="192.168.5.32 192.168.5.64"
The Toolsets IPROOTDEV=eth0 G util-vserver, stable (1) G util-vserver, stable (2) S_HOSTNAME=ftp.nowhe.re G util-vserver, alpha G Configuration ONBOOT=yes S_DOMAINNAME= Base-Operations S_NICE=5 Management S_FLAGS="lock nproc fakeinit" Prospective ULIMIT="-HS -u 200" S_CAPS="" ¦ ¥ I bash-scriptlet; applied with source I another script /etc/vservers/
- p. 21/47 I rare documentation http://www.linux-vserver.org/index.php?page=alpha+util-vserver
util-vserver, alpha
Introduction I designgoals: vserver N easy extensible Security N no races and symlinkattacks The Toolsets N embedded solutions for standardtasks G util-vserver, stable (1) G util-vserver, stable (2) N support of new kernelfeatures G util-vserver, alpha G Configuration I retaining of stable’s base-commands, but lots of new Base-Operations program and reimplementation of old ones Management I new configuration scheme Prospective N parseable by C and shell N manageable with cfengine N support of new features
- p. 22/47 util-vserver, alpha
Introduction I designgoals: vserver N easy extensible Security N no races and symlinkattacks The Toolsets N embedded solutions for standardtasks G util-vserver, stable (1) G util-vserver, stable (2) N support of new kernelfeatures G util-vserver, alpha G Configuration I retaining of stable’s base-commands, but lots of new Base-Operations program and reimplementation of old ones Management I new configuration scheme Prospective N parseable by C and shell N manageable with cfengine N support of new features I rare documentation
http://www.linux-vserver.org/index.php?page=alpha+util-vserver
- p. 22/47 Configuration
Introduction I configuration in /etc/vservers/
Base-Operations | | |-- ip | | ‘-- name Management | |-- bcast
Prospective | |-- dev | ‘-- mask |-- run -> /var/run/vservers/ftp |-- run.rev -> ../.defaults/run.rev ‘-- vdir -> /etc/vservers/.defaults/vdirbase/ftp
I files and symlinks; mostly one-entry-per-line/file I path of configuration-directory identifies a vserver; chroot-path freely chooseable I only formal documentation
- p. 23/47 vcontext (1)
Introduction I formerly: chcontext, but no support of new vserver kernel-technologies Security I creation (--create) and entering (--migrate) of The Toolsets process-contexts Base-Operations G vcontext (1) I Take care about security when entering a context! (ptrace(2)) G vcontext (2) G vattribute I usually additional operations between create and migrate G chbind G Misc (1) I G Misc (2) invocation usually as:
Management vcontext --create -- \ vattribute --set -- \ Prospective vlimit ... -- \ vsched ... -- \ vcontext --migrate-self --endsetup -- \
- p. 24/47 vcontext (2)
Introduction vserver Example: Security # vcontext --create ps axh The Toolsets New security context is 49153
Base-Operations 5440 pts/1 R 0:00 ps axh G vcontext (1) G vcontext (2) # vcontext --migrate --xid 43 ps axh G vattribute 5068 ? S 0:00 /sbin/syslogd G chbind 5102 ? S 0:00 /usr/sbin/exim4 -bd -q30m G Misc (1) 5108 ? S 0:00 /usr/sbin/inetd G Misc (2) 5112 ? S 0:00 /usr/sbin/atd Management 5115 ? S 0:00 /usr/sbin/cron 5447 pts/1 R 0:00 ps axh Prospective
- p. 25/47 vattribute
Introduction I setting/removal of attributes and capabilities vserver I syntax for such values: Security N string The Toolsets N number: interpreted as a bitpattern Base-Operations N G vcontext (1) prefix ‘˜’ or ‘!’: unsetting of the pattern G vcontext (2) N G vattribute prefix ‘ˆ’: bitnumber instead of pattern G chbind G Misc (1) G Misc (2) Example: Management # vcontext --create vattribute --set --flag hidemount cat /proc/mounts Prospective # vcontext --create -- \ vattribute --set --secure -- \ vcontext --endsetup --migrate-self -- \ mknod /tmp/test c 1 2 New security context is 49183 mknod: ‘/tmp/test’: Operation not permitted
- p. 26/47 chbind
Introduction I binding of IPs to processes vserver I uncertain future. . . perhaps completely different networking Security or replacing with vnet The Toolsets
Base-Operations G vcontext (1) Example: G vcontext (2) G vattribute # chbind --ip 10.1.0.1 cat /proc/self/status | grep ipv4root G chbind ipv4root is now 10.1.0.1 G Misc (1) G Misc (2) ipv4root: 0100010a/00ffffff ipv4root_bcast: ffffffff Management ipv4root_refcnt: 2
Prospective
- p. 27/47 Misc (1)
Introduction vkill vserver Security I atomic sending of signals to process-contexts The Toolsets
Base-Operations G vcontext (1) G vcontext (2) G vattribute vnamespace G chbind G Misc (1) G Misc (2) I creation and entering of namespaces Management
Prospective vlimit
I setting and showing of resource-limits
- p. 28/47 Misc (2)
Introduction vuname vserver
Security I setting and showing of utsname-entries
The Toolsets
Base-Operations G vcontext (1) G vcontext (2) G vattribute vserver-info G chbind G Misc (1) G Misc (2) I querying of single attributes of contexts and vservers Management I important for bugreports: Prospective vserver−info − SYSINFO
- p. 29/47 Introduction vserver
Security
The Toolsets
Base-Operations Management
Management G vserver creation G vserver . . . build G vserver . . . start G vserver . . . stop G vserver . . . enter|exec G vps G vserver-stat G vrpm (1) G vrpm (2) G vapt-get G setattr, showattr (1) G setattr, showattr (2) G setattr, showattr (3) G vunify
Prospective
- p. 30/47 I Fedora Core: # make -C /usr/src DESTDIR=/vservers/foo install make: Entering directory ‘/usr/src’ make: *** No rule to make target ‘install’. Stop. make: Leaving directory ‘/usr/src’
I lots of distributions with different installation-methods ⇒ implementation of some of them in util-vserver: N “apt-rpm” for Fedora/RH vserver N “debootstrap” for Debian vserver N “skeleton” for base directory-structure and configuration
vserver creation
Introduction I “normal” systeminstallation possible vserver I BSD Jails: Security # make -C /usr/src DESTDIR=/vservers/foo install The Toolsets
Base-Operations
Management G vserver creation G vserver . . . build G vserver . . . start G vserver . . . stop G vserver . . . enter|exec G vps G vserver-stat G vrpm (1) G vrpm (2) G vapt-get G setattr, showattr (1) G setattr, showattr (2) G setattr, showattr (3) G vunify
Prospective
- p. 31/47 ⇒ implementation of some of them in util-vserver: N “apt-rpm” for Fedora/RH vserver N “debootstrap” for Debian vserver N “skeleton” for base directory-structure and configuration
vserver creation
Introduction I “normal” systeminstallation possible vserver I BSD Jails: Security # make -C /usr/src DESTDIR=/vservers/foo install The Toolsets
Base-Operations I Fedora Core: Management # make -C /usr/src DESTDIR=/vservers/foo install G vserver creation make: Entering directory ‘/usr/src’ G vserver . . . build G vserver . . . start make: *** No rule to make target ‘install’. Stop. G vserver . . . stop make: Leaving directory ‘/usr/src’ G vserver . . . enter|exec G vps G vserver-stat I lots of distributions with different installation-methods G vrpm (1) G vrpm (2) G vapt-get G setattr, showattr (1) G setattr, showattr (2) G setattr, showattr (3) G vunify
Prospective
- p. 31/47 vserver creation
Introduction I “normal” systeminstallation possible vserver I BSD Jails: Security # make -C /usr/src DESTDIR=/vservers/foo install The Toolsets
Base-Operations I Fedora Core: Management # make -C /usr/src DESTDIR=/vservers/foo install G vserver creation make: Entering directory ‘/usr/src’ G vserver . . . build G vserver . . . start make: *** No rule to make target ‘install’. Stop. G vserver . . . stop make: Leaving directory ‘/usr/src’ G vserver . . . enter|exec G vps G vserver-stat I lots of distributions with different installation-methods G vrpm (1) G vrpm (2) ⇒ implementation of some of them in util-vserver: G vapt-get G setattr, showattr (1) N “apt-rpm” for Fedora/RH vserver G setattr, showattr (2) N G setattr, showattr (3) “debootstrap” for Debian vserver G vunify N “skeleton” for base directory-structure and configuration Prospective
- p. 31/47 vserver . . . build
Introduction I documented by “vserver - build --help” vserver I Examples: Security N # vserver test0 build -m apt-rpm --hostname test0.nowhe.re \ The Toolsets --interface 10.0.1.0 --netdev eth0 --netprefix 23 \ Base-Operations --context 42 -- -d fc1
Management G vserver creation N # vserver test1 build -m debootstrap --hostname test1.nowhe.re \ G vserver . . . build --interface 10.0.1.1 --netdev eth0 --netprefix 23 \ G vserver . . . start --context 43 -- -d sarge G vserver . . . stop G vserver . . . enter|exec G vps N G vserver-stat # vserver test2 build -m skeleton --hostname test2.nowhe.re \ G vrpm (1) --interface 10.0.1.2 --netdev eth0 --netprefix 23 \ G vrpm (2) --context 44 G vapt-get G setattr, showattr (1) G setattr, showattr (2) I configuration of parameters (mirrors, packet-lists) in G setattr, showattr (3) G vunify /etc/vservers/.defaults/apps/debootstrap/* and Prospective /etc/vservers/.distributions/*
- p. 32/47 vserver . . . start
Introduction 1. creation of namespaces vserver 2. creation of network-interfaces Security 3. directory-mounting The Toolsets
Base-Operations 4. creation of process- and network-contexts
Management 5. activation of limits and capabilities G vserver creation G vserver . . . build 6. execution of the init-process G vserver . . . start G vserver . . . stop I shortcut with “/etc/rc.d/rc 3”, or G vserver . . . enter|exec G vps I regular /sbin/init – often lots of unwanted actions G vserver-stat G vrpm (1) ⇒ G vrpm (2) fakeinit mechanisms needed (getpid()==1) G vapt-get G setattr, showattr (1) G setattr, showattr (2) Attention: at least one process needed in the context G setattr, showattr (3) G vunify Prospective Example: # vserver test0 start # vserver --debug test1 start
- p. 33/47 vserver . . . stop
Introduction I either vserver N sending of SIGINT to the init-process, or Security N execution of “/etc/rc.d/rc 6” The Toolsets I explicit “vkill –xid
Prospective
- p. 34/47 vserver . . . enter|exec
Introduction I execution of commands within the vserver vserver I similar actions as “vserver . . . start”, but entering instead of Security creation of namespace and contexts The Toolsets ⇒ no overriding of parameters and restrictions Base-Operations I Management only for administration-tasks but not for regular service (e.g. G vserver creation G vserver . . . build missing /dev/pts entries) G vserver . . . start G vserver . . . stop G vserver . . . enter|exec G vps Example: G vserver-stat G vrpm (1) # vserver test0 exec ps axh G vrpm (2) 640 ? S 0:00 syslogd -m 0 G vapt-get 1188 pts/1 R 0:00 ps axh G setattr, showattr (1) # vserver test1 enter G setattr, showattr (2) test1:/# G setattr, showattr (3) G vunify # uname -a Prospective Linux delenn 2.6.5ensc-0.3 #1 Thu Apr 15 ... 2004 i686 i686 i386 GNU/Linux # vserver test1 exec uname -a SCO UnixWare test1.nowhe.re 7.1 #1 Sat Feb 29 ... 2003 s390 GNU/Linux
- p. 35/47 vps
Introduction I displays all processes on host + contexts vserver I executed in a special watcher-context (XID 1) Security
The Toolsets Example: Base-Operations # vps ax Management G vserver creation PID CONTEXT TTY STAT TIME COMMAND G vserver . . . build 1 0 MAIN ? S 0:05 /sbin/minit G vserver . . . start 2 0 MAIN ? SWN 0:00 [ksoftirqd/0] G vserver . . . stop ... G vserver . . . enter|exec 5068 43 test1 ? S 0:00 /sbin/syslogd G vps 5102 43 test1 ? S 0:00 /usr/sbin/exim4 -bd -q30m G vserver-stat G vrpm (1) 5108 43 test1 ? S 0:00 /usr/sbin/inetd G vrpm (2) 5112 43 test1 ? S 0:00 /usr/sbin/atd G vapt-get 5115 43 test1 ? S 0:00 /usr/sbin/cron G setattr, showattr (1) ... G setattr, showattr (2) 5256 42 test0 ? S 0:00 syslogd -m 0 G setattr, showattr (3) ... G vunify 5276 1 ALL_PROC pts/1 S 0:00 vps ax Prospective 5277 1 ALL_PROC pts/1 R 0:00 ps ax
- p. 36/47 vserver-stat
Introduction I overview about running vservers resp. process-contexts vserver Security Example: The Toolsets # vserver-stat Base-Operations CTX PROC VSZ RSS userTIME sysTIME UPTIME NAME
Management 0 37 43.8M 4.6K 0m37s86 0m22s52 13h00m44 root server G vserver creation 42 1 1.5M 145 0m00s00 0m00s00 2m56s60 test0 G vserver . . . build 43 5 10.5M 965 0m00s10 0m00s00 8m53s31 test1 G vserver . . . start G vserver . . . stop # vserver-stat G vserver . . . enter|exec CTX PROC VSZ RSS userTIME sysTIME UPTIME NAME G vps 0 48 143.9M 3.5K 19h09m59 8h40m24 56d45h05 root server G vserver-stat G vrpm (1) 2 3 6.7M 172 3h01m34 1h22m00 28d29h14 vpn G vrpm (2) 82 17 90.6M 784 4h44m09 2h13m18 28d26h53 cvs G vapt-get 133 5 1.6M 52 31m03s55 5m46s86 23d33h26 paris G setattr, showattr (1) 146 11 48.9M 2K 37m07s12 8m55s74 28d26h53 mirror G setattr, showattr (2) 147 7 3.8M 180 10h46m17 2h48m54 28d21h46 mirror-master G setattr, showattr (3) 153 9 74.6M 4K 36m31s24 7m08s33 28d25h46 ldap1 G vunify
Prospective
- p. 37/47 vrpm (1)
Introduction I external or internal rpm-database vserver I advantage internal: rpm works within the vserver Security I advantage external: simple bootstrapping (“vserver . . . build”) The Toolsets I Base-Operations switching between both methods with
Management vserver ... pkgmgmt externalize|internalize G vserver creation G vserver . . . build G vserver . . . start Syntax: G vserver . . . stop −− − G vserver . . . enter|exec vrpm
- p. 38/47 vrpm (2)
Introduction External vrpm: vserver Security I LD_PRELOAD wrapper for execv(3), getpwnam(3) et.al. The Toolsets ⇒ execution of %scriplets im the vserver-context Base-Operations ⇒ NSS lookups while unpacking the packages Management I G vserver creation complicated mounting of the database so that access G vserver . . . build G vserver . . . start through vserver-processes or %scriptlets impossible G vserver . . . stop I G vserver . . . enter|exec files at /etc/vservers/
- p. 39/47 vapt-get
Introduction I for rpm-based vservers: both external and internal vserver management possible Security I else: realized with “vserver . . . exec apt-get” The Toolsets Base-Operations Syntax: Management vapt−get
- p. 40/47 manipulations possible, as changes visible on every vserver # echo mycode >/usr/sbin/httpd N no COW oder unionfs in Linux
setattr, showattr (1)
Introduction I often: lots of vservers with the same distribution vserver ⇒ installation and execution of identical packages, binaries Security and data-files The Toolsets I idea: copies with hardlinks (“ln A B”) Base-Operations ⇒ saves diskspace Management ⇒ G vserver creation saves memory (mapping of programs and libraries) G vserver . . . build G vserver . . . start G vserver . . . stop G vserver . . . enter|exec G vps G vserver-stat G vrpm (1) G vrpm (2) G vapt-get G setattr, showattr (1) G setattr, showattr (2) G setattr, showattr (3) G vunify
Prospective
- p. 41/47 setattr, showattr (1)
Introduction I often: lots of vservers with the same distribution vserver ⇒ installation and execution of identical packages, binaries Security and data-files The Toolsets I idea: copies with hardlinks (“ln A B”) Base-Operations ⇒ saves diskspace Management ⇒ G vserver creation saves memory (mapping of programs and libraries) G vserver . . . build G vserver . . . start manipulations possible, as changes visible on every G vserver . . . stop vserver G vserver . . . enter|exec G vps # echo mycode >/usr/sbin/httpd G vserver-stat G vrpm (1) N no COW oder unionfs in Linux G vrpm (2) G vapt-get G setattr, showattr (1) G setattr, showattr (2) G setattr, showattr (3) G vunify
Prospective
- p. 41/47 I additional flag N preventing modifications N allowing to remove files I low-level functionality in setattr und showattr tools N forbidding/allowing of modifications with “–iunlink” N changing of visibility N setting of the chroot-barrier flag ⇒ “setattr --help”
setattr, showattr (2)
Introduction I solution: special immutable-flag; e.g. “chattr +i . . . ” vserver N not settable outside of host-context Security package-management (Updates) not possible anymore
The Toolsets
Base-Operations
Management G vserver creation G vserver . . . build G vserver . . . start G vserver . . . stop G vserver . . . enter|exec G vps G vserver-stat G vrpm (1) G vrpm (2) G vapt-get G setattr, showattr (1) G setattr, showattr (2) G setattr, showattr (3) G vunify
Prospective
- p. 42/47 setattr, showattr (2)
Introduction I solution: special immutable-flag; e.g. “chattr +i . . . ” vserver N not settable outside of host-context Security package-management (Updates) not possible anymore The Toolsets I additional flag Base-Operations N preventing modifications Management N G vserver creation allowing to remove files G vserver . . . build I G vserver . . . start low-level functionality in setattr und showattr tools G vserver . . . stop N G vserver . . . enter|exec forbidding/allowing of modifications with “–iunlink” G vps N G vserver-stat changing of visibility G vrpm (1) N G vrpm (2) setting of the chroot-barrier flag G vapt-get ⇒ “setattr --help” G setattr, showattr (1) G setattr, showattr (2) G setattr, showattr (3) G vunify
Prospective
- p. 42/47 setattr, showattr (3)
Introduction Example: vserver # touch /vservers/test0/{a,b,c} Security # ln /vservers/test0/{a,b,c} /vservers/test1/ # setattr --iunlink /vservers/test0/a The Toolsets # chattr +i /vservers/test0/b
Base-Operations # showattr /vservers/test0/{a,b,c} ---bUI- /vservers/test0/a Management ---buI- /vservers/test0/b G vserver creation ---bui- /vservers/test0/c G vserver . . . build G vserver . . . start # vserver test0 enter G vserver . . . stop G vserver . . . enter|exec [root@test0]# echo a>a G vps bash: a: Permission denied G vserver-stat [root@test0]# echo a>b G vrpm (1) bash: a: Permission denied G vrpm (2) [root@test0]# echo a>c G vapt-get [root@test0]# G setattr, showattr (1) G setattr, showattr (2) G setattr, showattr (3) [root@test0]# rm -f a b c G vunify rm: cannot remove ‘b’: Operation not permitted
Prospective # vserver test1 enter test1:/# cat /c a
- p. 43/47 I complete Fedora Core 1 installation has only approx. 30 MB unsharable files ⇒ 2.6 GB diskspace for 20 vserver á 2 GB
vunify
Introduction I applies the settattr-concept to entire directory-trees vserver I function: Security 1. searches same files The Toolsets 2. sets the iunlink flag Base-Operations
Management 3. creates a hardlink G vserver creation I G vserver . . . build uses static exclude-lists and information of G vserver . . . start G vserver . . . stop package-management about configuration files G vserver . . . enter|exec G vps G vserver-stat G vrpm (1) G vrpm (2) G vapt-get G setattr, showattr (1) G setattr, showattr (2) G setattr, showattr (3) G vunify
Prospective
- p. 44/47 vunify
Introduction I applies the settattr-concept to entire directory-trees vserver I function: Security 1. searches same files The Toolsets 2. sets the iunlink flag Base-Operations
Management 3. creates a hardlink G vserver creation I G vserver . . . build uses static exclude-lists and information of G vserver . . . start G vserver . . . stop package-management about configuration files G vserver . . . enter|exec I G vps complete Fedora Core 1 installation has only approx. 30 MB G vserver-stat G vrpm (1) unsharable files G vrpm (2) ⇒ G vapt-get 2.6 GB diskspace for 20 vserver á 2 GB G setattr, showattr (1) G setattr, showattr (2) G setattr, showattr (3) G vunify
Prospective
- p. 44/47 Prospective
Introduction I new network-concept: tagging of network-packets, iptables, vserver routing-tables Security I documentation The Toolsets I testsuits Base-Operations I → → Management alpha beta stable (before GNU Hurd??)
Prospective G Prospective G References G Questions?
- p. 45/47 References
Introduction I project-homepage http://linux-vserver.org vserver I util-vserver http://www.nongnu.org/util-vserver Security I #vserver at oftc.net The Toolsets
Base-Operations
Management
Prospective G Prospective G References G Questions?
- p. 46/47 Questions?
- p. 47/47