L08: Low‐Level I/O, C++ Preview CSE333, Spring 2018 Low-Level I/O, C++ Preview CSE 333 Spring 2018 Instructor: Justin Hsia Teaching Assistants: Danny Allen Dennis Shao Eddie Huang Kevin Bi Jack Xu Matthew Neldam Michael Poulain Renshu Gu Robby Marver Waylon Huang Wei Lin L08: Low‐Level I/O, C++ Preview CSE333, Spring 2018 Administrivia Exercise 7 posted tomorrow, due Monday Homework 1 due tomorrow night (4/12) . Watch that hashtable.c doesn’t violate the modularity of ll.h . Watch for pointer to local (stack) variables . Use a debugger (e.g. gdb) if you’re getting segfaults . Advice: clean up “to do” comments, but leave “step #” markers for graders . Late days: don’t tag hw1-final until you are really ready . Bonus: if you add unit tests, put them in a new file and adjust the Makefile 2 L08: Low‐Level I/O, C++ Preview CSE333, Spring 2018 Details on x86/Linux Your program A more accurate picture: . Consider a typical Linux process C standard . Its thread of execution can be in POSIX library one of several places: glibc • In your program’s code Linux • In glibc, a shared library system calls containing the C standard library, POSIX, support, and more architecture-independent code • In the Linux architecture- independent code • In Linux x86-64 code architecture-dependent code Linux kernel 3 L08: Low‐Level I/O, C++ Preview CSE333, Spring 2018 Details on x86/Linux Your program Some routines your program invokes may be entirely handled C standard by glibc without involving the POSIX library kernel glibc . e.g. strcmp() from stdio.h . There is some initial overhead when invoking functions in architecture-independent code dynamically linked libraries (during loading) • But after symbols are resolved, architecture-dependent code invoking glibc routines is nearly Linux kernel as fast as a function call within your program itself! 4 L08: Low‐Level I/O, C++ Preview CSE333, Spring 2018 Details on x86/Linux Your program Some routines may be handled by glibc, but they in turn C standard invoke Linux system calls POSIX library . e.g. POSIX wrappers around Linux glibc syscalls • POSIX readdir() invokes the underlying Linux readdir() . e.g. C stdio functions that read architecture-independent code and write from files • , , fopen() fclose() architecture-dependent code fprintf() invoke underlying Linux open(), close(), Linux kernel write(), etc. 5 L08: Low‐Level I/O, C++ Preview CSE333, Spring 2018 Details on x86/Linux Your program Your program can choose to directly invoke Linux system C standard calls as well POSIX library . Nothing is forcing you to link with glibc glibc and use it . But relying on directly-invoked Linux system calls may make your program less portable across UNIX architecture-independent code varieties architecture-dependent code Linux kernel 6 L08: Low‐Level I/O, C++ Preview CSE333, Spring 2018 Details on x86/Linux Your program Let’s walk through how a Linux system call actually works C standard . We’ll assume 32-bit x86 using the POSIX library modern SYSENTER / SYSEXIT x86 instructions glibc • x86-64 code is similar, though details always change over time, so take this as an example – not a architecture-independent code debugging guide architecture-dependent code Linux kernel 7 L08: Low‐Level I/O, C++ Preview CSE333, Spring 2018 Details on x86/Linux 0xFFFFFFFF Your program linux-gate.so Remember our Linux kernel stack process address kernel C standard space picture? POSIX Stack library . Let’s add some glibc details: Shared Libraries architecture-independent code Heap (malloc/free) Read/Write Segment .data, .bss architecture-dependent code Read-Only Segment .text, .rodata Linux kernel CPU 0x00000000 8 L08: Low‐Level I/O, C++ Preview CSE333, Spring 2018 Details on x86/Linux 0xFFFFFFFF Your program linux-gate.so Process is executing Linux kernel stack your program code kernel C standard POSIX Stack library SP glibc Shared Libraries architecture-independent code Heap (malloc/free) Read/Write Segment .data, .bss architecture-dependent code Read-Only Segment IP .text, .rodata Linux kernel unpriv CPU 0x00000000 9 L08: Low‐Level I/O, C++ Preview CSE333, Spring 2018 Details on x86/Linux 0xFFFFFFFF Your program linux-gate.so Process calls into a Linux kernel stack glibc function kernel . e.g. fopen() C standard POSIX . We’ll ignore the Stack library messy details of SP loading/linking glibc shared libraries IP Shared Libraries architecture-independent code Heap (malloc/free) Read/Write Segment .data, .bss architecture-dependent code Read-Only Segment .text, .rodata Linux kernel unpriv CPU 0x00000000 10 L08: Low‐Level I/O, C++ Preview CSE333, Spring 2018 Details on x86/Linux 0xFFFFFFFF Your program IP linux-gate.so glibc begins the Linux kernel stack process of invoking a kernel Linux system call C standard POSIX . glibc’s Stack library fopen() likely SP invokes Linux’s glibc open() system call Shared Libraries . Puts the system call # and arguments into architecture-independent code registers Heap (malloc/free) . Uses the call x86 Read/Write Segment instruction to call into .data, .bss architecture-dependent code the routine Read-Only Segment Linux kernel __kernel_vsyscall .text, .rodata located in linux- unpriv CPU gate.so 0x00000000 11 L08: Low‐Level I/O, C++ Preview CSE333, Spring 2018 Details on x86/Linux 0xFFFFFFFF Your program IP linux-gate.so Linux kernel stack linux-gate.so is a kernel vdso C standard POSIX . A virtual Stack library SP dynamically linked glibc shared object . Is a kernel-provided Shared Libraries shared library that is plunked into a process’ architecture-independent code address space Heap (malloc/free) . Provides the intricate Read/Write Segment .data, .bss machine code needed to architecture-dependent code trigger a system call Read-Only Segment .text, .rodata Linux kernel unpriv CPU 0x00000000 12 L08: Low‐Level I/O, C++ Preview CSE333, Spring 2018 Details on x86/Linux 0xFFFFFFFF Your program linux-gate.so linux-gate.so SP Linux kernel stack eventually invokes IP kernel the SYSENTER x86 C standard instruction POSIX Stack library . SYSENTER is x86’s “fast system call” glibc instruction • Causes the CPU to Shared Libraries raise its privilege level • Traps into the Linux architecture-independent code kernel by changing the Heap (malloc/free) SP, IP to a previously- Read/Write Segment determined location .data, .bss architecture-dependent code • Changes some Read-Only Segment segmentation-related .text, .rodata Linux kernel registers (see CSE451) priv CPU 0x00000000 13 L08: Low‐Level I/O, C++ Preview CSE333, Spring 2018 Details on x86/Linux 0xFFFFFFFF Your program linux-gate.so The kernel begins SP Linux kernel stack executing code at IP kernel the SYSENTER C standard entry point POSIX Stack library . Is in the architecture- dependent part of Linux glibc . It’s job is to: Shared Libraries • Look up the system call number in a system call dispatch table architecture-independent code • Call into the address Heap (malloc/free) stored in that table Read/Write Segment entry; this is Linux’s .data, .bss architecture-dependent code system call handler Read-Only Segment – For open(), the .text, .rodata Linux kernel handler is named sys_open, and is priv CPU system call #5 0x00000000 14 L08: Low‐Level I/O, C++ Preview CSE333, Spring 2018 Details on x86/Linux 0xFFFFFFFF Your program linux-gate.so The system call SP Linux kernel stack handler executes IP kernel . What it does is C standard system-call specific POSIX Stack library . It may take a long time to execute, especially if glibc it has to interact with hardware Shared Libraries • Linux may choose to context switch the CPU architecture-independent code to a different runnable Heap (malloc/free) process Read/Write Segment .data, .bss architecture-dependent code Read-Only Segment .text, .rodata Linux kernel priv CPU 0x00000000 15 L08: Low‐Level I/O, C++ Preview CSE333, Spring 2018 Details on x86/Linux 0xFFFFFFFF Your program linux-gate.so Eventually, the SP Linux kernel stack system call handler IP kernel finishes C standard POSIX . Returns back to the Stack library system call entry point • Places the system call’s glibc return value in the appropriate register Shared Libraries • Calls SYSEXIT to return to the user-level architecture-independent code code Heap (malloc/free) Read/Write Segment .data, .bss architecture-dependent code Read-Only Segment .text, .rodata Linux kernel priv CPU 0x00000000 16 L08: Low‐Level I/O, C++ Preview CSE333, Spring 2018 Details on x86/Linux 0xFFFFFFFF Your program linux-gate.so SYSEXIT transitions the Linux kernel stack processor back to user- kernel mode code C standard POSIX . Restores the Stack library IP, SP to SP user-land values glibc . Sets the CPU back to IP Shared Libraries unprivileged mode . Changes some architecture-independent code segmentation-related Heap (malloc/free) registers (see CSE451) Read/Write Segment .data, .bss . Returns the processor architecture-dependent code back to glibc Read-Only Segment .text, .rodata Linux kernel unpriv CPU 0x00000000 17 L08: Low‐Level I/O, C++ Preview CSE333, Spring 2018 Details on x86/Linux 0xFFFFFFFF Your program linux-gate.so glibc continues to Linux kernel stack execute kernel . Might execute more C standard system calls POSIX Stack library . Eventually SP returns back to glibc your program code Shared Libraries architecture-independent code Heap (malloc/free) Read/Write Segment .data, .bss architecture-dependent code Read-Only Segment IP .text, .rodata Linux kernel unpriv CPU 0x00000000 18 L08: Low‐Level I/O, C++ Preview CSE333, Spring 2018 strace A useful Linux