Advanced Linux System Administraon Topic 10. The Linux Kernel Pablo Abad Fidalgo José Ángel Herrero Velasco Departamento de Ingeniería Informáca y Electrónica Este tema se publica bajo Licencia: Creave Commons BY-NC-SA 4.0 The Linux Kernel Index • Introduc2on: – Kernel types. • Stac Reconfiguraon: – Configuraon. – Compilaon. – Install. • Dynamic Reconfiguraon: – /proc. – LKM: LoadaBle Kernel modules. • Device Driver Modules. Introduc2on (Kernel) • Hides HW under an abstract, high level programming interface. • Creates these concepts from low-level HW features: – Processes (?me-sharing, protected address spaces). – Signals and semaphores. – Virtual memory (swapping, paging, mapping). – Filesystem (files, directories namespace). – General input/output (specialty hw, keyboard, mouse, USB). – Communicaon (between processes / network connec?ons). • Linux kernel mostly wriMen in C (+ a few assembly (/linux/arch)). • Source code available (git repository): – hps://github.com/torvalds/linux. Introduc2on (Kernel) Kernel Space User Space • Two basic approaches: •Monolithic Kernel Code CPU Program – Monolithic kernels: Memory Scheduling Manager Loader User Program • All func?onality is compiled together. Error System Security Handling Policies APIs • All code runs in privileged kernel-space. File Device Systems Drivers Kernel Space User Space Kernel Code Third-Party Code • Modular kernel (also monolithic): Memory Manager CPU Device Scheduling Driver – Most func?onality compiled into the kernel, Security Policies Error Device some func?ons loaded dynamically. Handling Driver – File All func?onality runs in kernel-space. System File Systems APIs System Program Loader User Program – Microkernels: • Only essen?al func?onality is compiled. • All other func?onality runs in user space. Introduc2on (Kernel) • Usually, distribu?ons include a kernel generic enough to avoid further reconfiguraon. • However, reconfigura2on is some?mes unavoidable: – Adding a new hardware device. – Performance op?mizaons: • Pre-compiled kernels provide many ============================ unnecessary components (compability). Item Lines % ============================ ./usr 845 0.0042 – Rou?ne updates (security patches). ./init 5,739 0.0283 ./samples 8,758 0.0432 ./ipc 8,926 0.0440 • How can we “adjust” the kernel?: ./virt 10,701 0.0527 … ./tools 232,123 1.1438 – Stacally, re-compiling the whole kernel: ./kernel 246,369 1.2140 ./Docume 569,944 2.8085 • (Source code + compiler + a few more things). ./include 715,349 3.5250 ./sound 886,892 4.3703 – Dynamically, through /proc params or ./net 899,167 4.4307 ./fs 1,179,220 5.8107 ./arch 3,398,176 16.7449 modules. ./drivers 11,488,536 56.6110 ============================= Index • Introduc2on: – Kernel types. • Stac Reconfiguraon: – Configuraon. – Compilaon. – Install. • Dynamic Reconfiguraon: – /proc. – LKM: LoadaBle Kernel modules. • Device Driver Modules. Stac Reconfiguraon Working Directory: /usr/src • Step 1. Obtaining kernel source code: – From www.kernel.org (recommended stable versions): • Complete version: linux-4.X.X.tar.xz (~80MB). • Patches: patch-4.X.X.xz (~50-100k). (Applied to current kernel, with patch command). – From repositories: • apt-get install linux-source-4.X.X. • Step 2. Configuraon: – Kernel configuraon in file /usr/src/linux-4.X.X/.config: • Each line contains a keyword (device/subsystem) and an argument: CONFIG_SCSI=y. • Driver can be not selected (#), built into the kernel (=y) or built as a module (=m). – Extremely complex process, requires deep hw and system understanding. – Two ways to create .config: from scratch or adjus?ng a well known config. Stac Reconfiguraon • Step 2. Configuraon (cont.): – From scratch: make <config/menuconfig/xconfig>: • config: starts a character based ques?on and answer session. • menuconfig: starts a terminal-oriented configuraon tool (requires ncurses package). • xconfig: X based configuraon tool. – From scratch (2): make defconfig: • creates a config file that uses default sengs based on the current system’s architecture. Stac Reconfiguraon • Step 2. Configuraon (cont.): – Adap?ng a pre-built .config: make <oldconfig/silentoldconfig>: • oldconfig: update a config file (copied from another system or from previous kernel) to be compable with the newer kernel source code (ques?ons). • silentoldconfig: do not show ques?ons answered by the config process. – More building op?ons: make help. • Step 3. Compilaon: – Build the kernel + System.map : • [root si ~] make bzImage (aer correct compilaon kernel appears in arch/i386/ boot). – Build modules (see next sec?on): • [root si~] make modules. – Build ramdisk if modules are required to access boo?ng device: • [root si~] mkinitrd –o /boot/initrd-4.X.X.img 4.X.X. (Example: our FS uses LVM/RAID). Stac Reconfiguraon • Step 2. Configuraon (cont.): – Adap?ng a pre-built .config: make <oldconfig/silentoldconfig>: • oldconfig: update a config file (copied from another system or from previous kernel) to be compable with the newer kernel source code (ques?ons). • silentoldconfig: do not show ques?ons answered by the config process. – More building op?ons: make help. Symbol (variable/func?on) to address table • Step 3. Compilaon: Example: ffffffff8104d148 t swap_pages – Build the kernel + System.map : Employed for debugging kernel crashes • [root si ~] make bzImage (aer correct compilaon kernel appears in arch/i386/ boot). – Build modules (see next sec?on): • [root si~] make modules. – Build ramdisk if modules are required to access boo?ng device: • [root si~] mkinitrd –o /boot/initrd-4.X.X.img 4.X.X. (Example: our FS uses LVM/RAID). Stac Reconfiguraon • Step 4. Installaon: – Copy kernel image, System.map and ramdisk to /boot: • [root si~] cp arch/i386/boot/bzImage /boot/bzImage_KERNEL-VERSION. • [root si~] cp System.map /boot/System.map-KERNEL_VERSION. • [root si~] ln –s /boot/System.map-KERNEL_VERSION /boot/System.map. – Install kernel modules (already built): • [root si~] make modules_install (installed in /lib/modules/KERNEL_VERSION). – Configure bootloader (grub2): • [root si~] update-grub. • Do not remove old kernels (new might not boot). Put them in /boot/grub/menu.lst. … title Test Kernel (4.X.X) root (hd0,1) kernel /boot/bzImage-4.X.X ro root=/dev/sda1 ro quiet initrd /boot/initrd-4.X.X.img … Sta2c Reconfigura2on (DEBIAN) • Debian provides tools to compile + build a package for the kernel: – Append compiled kernel informaon to the soqware database. – Ease the management of mul?ple kernels (clean). – All the tools included in kernel-package (apt-get install kernel-package). • Alternave Steps with debian (make-kpkg): – Step 2. Configuraon: make-kpkg --config: • Equivalent to make oldconfig. – Step 3. Compilaon: make-kpkg --initrd kernel_image modules_image: • Generates a .deb file with name: linux-image-[version]_[arch].deb. • Recommended to do a make-kpkg clean previously. – Step 4. Installaon: as easy as dpkg -i linux-image-XXX.deb. Index • Introduc2on: – Kernel types. • Stac Reconfiguraon: – Configuraon. – Compilaon. – Install. • Dynamic Reconfiguraon: – /proc. – LKM: LoadaBle Kernel modules. • Device Driver Modules. Dynamic Reconfiguraon • Kernel recompila?on is not a usual task (very complex and delicate). • Usually, kernel is “fine-tuned” dynamically: – Through /proc directory and/or Loadable Kernel Modules (LKM). • /proc: pseudo File System represen?ng current kernel status: – Details about system hardware (/proc/cpuinfo or /proc/devices). – Informaon about any process currently running: • cmdline: command line arguments. • cwd: link to current working directory. • environ: env variables. [ root si /tmp ] ls /proc/2719 attr environ mem root • exe: executable. auxv exe mountinfo sched • maps: memory maps to executable cgroup fd mounts sessionid & library files. clear_refs fdinfo mountstats smaps cmdline io net stat • mem: memory held by process. coredump_filter limits oom_adj statm • … (see man proc). cpuset loginuid oom_score status cwd maps pagemap task Dynamic Reconfiguraon (/proc) • /proc employed for: – Input (configuraon): echo 32768 > /proc/sys/fs/file-max. – Output (monitoring): /proc/stat. • Command sysctl: configure kernel parameters at run?me: – Syntax: $ sysctl [op?on] <arguments>: • Op?on –a: display all values currently available. • Op?on –w: change a variable value (sysctl –w proc.sys.fs.file-max=32768). • Op?on –p: load sengs from a file. • hps://www.kernel.org/doc/Documentaon/sysctl/kernel.txt. • Permanent modificaons: /etc/sysctl.conf. Dynamic Reconfigura2on (LKM) • LoadaBle Kernel Modules (LKM): – Add code to the kernel while it is running (avoiding recompilaon). • Advantages: – No need to rebuild the kernel (keep using the untouched kernel). – Easier system problem diagnosis: • Kernel -> running; Kernel + LKM -> died; problem located at Module. – Faster development/maintenance (no rebuild/reboot). • But…: – Some pieces MUST be built into the base kernel: • Anything required to boot far enough to load LKMs, for example, the driver of the disk drive that contains root filesystem. Dynamic Reconfigura2on (LKM) • What LKMs are used for: – Device drivers: allow communicaon between kernel and a piece of HW. – Filesystem drivers: interpret the contents of a File System as files and directories. – System calls: make your own syscall or modify an exis?ng one. – Network driver: interprets a network protocol (IPX link -> IPX driver). – TTY line disciplines, executable interpreters. • Where are modules: – Files with extension .o and .ko (since 2.6 version). – /lib/modules/4.X.X. alu@si:/lib/modules/3.16.0-4-amd64/kernel/drivers$ ls arch crypto drivers fs lib mm net sound Dynamic Reconfigura2on