Overview
Last Week: ● How to program with directories ● Brief introduction to the UNIX file system Unix System Programming This Week:
● How to program UNIX processes (Chapters 7-9) Processes » Follow the flow of Ch 8. Process control sprinkled with reflections from Ch 7 (e.g., exit, process/program memory layout). ● fork() and exec()
Maria Hybinette, UGA 1 Maria Hybinette, UGA 2
Outline What is a Process?
● What is a process? ● Review: A process is a program in execution (an active ● fork() entity, i.e., it is a running program , this was also on the ● exec() » Basic unit of work on a computer ● wait() » Examples: ● Process Data – compilation process, ● Special Exit Cases – word processing process – a.out process ● Process Ids – Shell process ● I/O Redirection – (we just need to make sure the program is running) ● User & Group ID real and effective (revisit) ● getenv & putenv ● ulimit
Maria Hybinette, UGA 3 Maria Hybinette, UGA 4
What is a Process? What is a Process?
● It has both time, and space ● Each user can run many processes at once (e.g., by using &) » A container of instructions with some resources ● A process: » cat file1 file2 & ● Process reads, and writes (or updates) ● Two processes started on the command line. machine resources » ls | wc -l » e.g., CPU time (CPU carries out the instructions), ● A time sharing system (such as UNIX) run several processes » memory, by multiplexing between them » files, Processes » I/O devices (monitor, printer) to accomplish its task n-1 0 1 2 …
OS (schedules process)
Maria Hybinette, UGA 5 Maria Hybinette, UGA 6 What Makes up a Process? Formal Process Definition size?
● Program code (text) User Mode 3GB » Compiled version of the text Address Space ● Data (cannot be shared) A process is a program in execution , a stack sequential execution characterized by trace. It » global variables – Uninitialized (BSS segment) sometimes has a context (the information or data) and listed separately. – Initialized routine1 this context is maintained as the process var1 ● Process stack (scopes) var2 progresses through the system. » function parameters heap » return addresses arrayB[10] ; » local variables and functions data arrayA[10] = {0} ● <
Info about a process (running and foot print)
0xFFFFFFFF 4GB ● ● longsleepHelloW (binary with long sleep) Example: Process with Kernel Space 4GB Virtual Address Virtual Addresses ● size a.out. (foot print) Space (32 bit ● ps ! architectures) 0xC0000000 3GB » 61542 pts/5 00:00:00 longsleepHelloW! ● User Space (focused on ● cat /proc/61542/status! earlier, lower address ● space) User Space cat /proc/61542/maps! Virtual Addresses ● Kernel Space
● ppp 0x0 0x0
Maria Hybinette, UGA 9 Maria Hybinette, UGA 10
What is needed to keep track of a Process? Process Representation
Process P2 Information System Memory ● Memory information: Memory Limits » Pointer to memory segments needed to run a process, i.e., pointers to the Page tables Memory base Kernel Process Table address space -- text, data, stack Initial P0 segments. Process Number Program counter P0 : HW state: resources ● Process management information: Program Counter Process P3 … » Process state, ID Registers P1 : HW state: resources » Content of registers: Process P P : HW state: resources – Program counter, stack pointer, Process State 2 2 process state, priority, process ID, CPU time used List of opened files P3 : HW state: resources Memory mappings ● File management & I/O information: I/O Devices allocated » Working directory, file descriptors … open, I/O devices allocated Accounting Pending requests Process P1 ● Accounting: amount of CPU used. Process control … Block (PCB)
Maria Hybinette, UGA 11 Maria Hybinette, UGA 12 System Control: OS View: Process Control Block Process Attributes (PCB)
● How does an OS keep track of the state of a ps and top command can be used to look at process? current processes » Keep track of some information in a structure. – Example: In Linux a process information is kept in a ● PID - process ID: each process has a unique ID structure called struct task_struct declared in #include linux/sched.h! ● PPID - parent process ID: The process that – What is in the structure? forked to start the (child) process struct task_struct ● nice value - priority (-20 highest to 19 lowest) pid_t pid; /* process identifier */ long state; /* state for the process */ ● TTY associated with terminal (TTY teletype unsigned int time_slice /* scheduling information */ terminal) struct mm_struct *mm /* address space of this process */ – Where is it defined:
● not in /usr/include/linux – only user level code
● usr/src/kernel/2.6.32-431.29.2.elf6.x86_64/include/linux
Maria Hybinette, UGA 13 Maria Hybinette, UGA 14
Back to user-level Other Process Attributes
● Finding PIDs ● Real user ID » At the shell prompt ● Effective user ID – ps u, ps, ps aux, ● Current directory ● ps no args # your process
● ps –ef # every process ● File descriptor table
● ps -p 77851 # particular process ● Environment – top interative » In a C program: int p = getpid(); // more later ● Pointer to program code, data stack and heap ● Execution priority ● Signal information
Maria Hybinette, UGA 15 Maria Hybinette, UGA 16
Process ID conventions, and the 3 General Process Types in UNIX Process Life Cycle
Interactive – foreground (shell must wait until complete [takes user input], or – background (&) [no user input] ● PID 0 – initiated an controlled terminal session » is usually the scheduler process (swapper), a system process – can accept input form user as it runs and output to the terminal (does not correspond to a program stored on disk, the grandmother of all processes). Daemons – server processes running in the background (e.g., listening to a port) ● init - Mother of all user processes, init is started at – Not associated with the terminal boot time (at end of the boot strap procedure) and is – typically started by init process at boot time responsible for starting other processes – Examples: ftpd, httpd, …, mail » It is a user process with PID 1 – If user wants to creates one, detach it from the terminal, kill its parent. (init adopts) » init uses file inittab and directory /etc/rc?.d Batch (at, cron, batch) » brings the user to a certain specified state (e.g. multiuser) – Jobs that are queued and processed one after another ● – recurrent tasks scheduled to run from a queue getty - login process that manages login sessions – periodic, recurrent tasks run when system usage is low, cron-jobs (administered by the daemon crond). – Examples: backups, experimental runs. » ZombiesMaria Hybinette, UGA… don’t count. 17 Maria Hybinette, UGA 18 Hierarchical Processes Tree on a (historical) UNIX System Display Process Hierarchy
mother of all user processes OS Kernel pstree (processes) ● Syntax: pstree | more (all process) Process 0 Process 1 Process 2 (BSD) ● Syntax: pstree
deamon (e.g. httpd) getty getty ● -d (directories), -a (hidden), -s (size), -p (permissions) ● tree –H . login login
bash ksh
Maria Hybinette, UGA 19 Maria Hybinette, UGA 20
Daemon Processes PID and Parentage
● Print out status information of various processes in the system: ps -axj (BSD) , ps -efjc (SVR4) , switches / flags varies ● A process ID or PID is a positive integer that uniquely identifies a running process and is stored in a variable ● process status (ps) of type pid_t ● Daemons (d) run with root privileges, no controlling terminal, parent process is init ● Example: print the process PID and parent s PID
{atlas:maria} ps -efjc | sort -k 2 -n | more // solaris below UID PID PPID PGID SID CLS PRI STIME TTY TIME CMD {saffron} print-pid root 0 0 0 0 SYS 96 Mar 03 ? 0:01 sched #include
rootMaria Hybinette, 319 UGA 1 319 319 TS 59 Mar 03 ? 6:33 /usr/sbin/syslogd 21 Maria Hybinette, UGA 22
● Linux processes ● pstree ● ps -efjc | sort -k 2 -n | more
● pidstat {nike:maria:125} ps -efjc | sort -k 2 -n | more # linux Oct 2014 below UID PID PPID PGID SID CLS PRI STIME TTY TIME CMD ● top, htop root 1 0 1 1 TS 19 Oct01 ? 00:02:11 /sbin/init root 2 0 0 0 TS 19 Oct01 ? 00:00:04 [kthreadd] ● mpstat root 3 2 0 0 FF 139 Oct01 ? 01:23:55 [migration/0] root 4 2 0 0 TS 19 Oct01 ? 00:01:10 [ksoftirqd/0] ● jobs root 5 2 0 0 FF 139 Oct01 ? 00:00:00 [migration/0] root 6 2 0 0 FF 139 Oct01 ? 00:07:19 [watchdog/0] » ^Z, ^C root 7 2 0 0 FF 139 Oct01 ? 01:14:54 [migration/1] root 8 2 0 0 FF 139 Oct01 ? 00:00:00 [migration/1] ● kill %1 root 9 2 0 0 TS 19 Oct01 ? 00:00:32 [ksoftirqd/1] root 10 2 0 0 FF 139 Oct01 ? 00:07:59 [watchdog/1] root 11 2 0 0 FF 139 Oct01 ? 01:17:56 [migration/2] root 12 2 0 0 FF 139 Oct01 ? 00:00:00 [migration/2] root 13 2 0 0 TS 19 Oct01 ? 00:00:16 [ksoftirqd/2] root 14 2 0 0 FF 139 Oct01 ? 00:07:17 [watchdog/2]
Maria Hybinette, UGA 23 Maria Hybinette, UGA 24 Linux Processes Process Life Cycle
● [] in ps (kernel processes) ● Create, Run, Die » Example: [kthreadd] ● (Creation and Running) In the beginning: root 3 0.0 0.0 0 0 ? S Nov02 4:39 [ksoftirqd/0]! root 6 0.0 0.0 0 0 ? S Nov02 0:00 [migration/0]! » init and it s descendants creates all subsequent root 7 0.0 0.0 0 0 ? S Nov02 0:01 [watchdog/0]! processes by a fork()-exec() mechanism root 8 0.0 0.0 0 0 ? S Nov02 0:00 [migration/1] » fork() creates an exact copy of itself called a child ● ksoftirqd – scheduling process kernel process (per process CPU, soft interrupt handling. » exec() system call places the image of a new ● migration – migrates processes between CPUs program over the newly copied program of the parent ● Watchdog – checks that the system is running OK. ● (Die, Exit) » When a process demises (completion of killed) it sends a signal to it s parent.
Maria Hybinette, UGA 25 Maria Hybinette, UGA 26
fork() a child fork()
#include
pid_t fork( void ); pid = fork() Child (can only ● Creates a child process by making a copy of the parent
Parent have 1 parent) process pid == 5 pid == 0 ● Both the child and the parent continue running ● The return of fork() Shared Copied » depends whether you are the child or the parent process: Data, heap, Program Data, heap – pid == 0 in the child process & stack (read only) & stack – pid ==
Maria Hybinette, UGA 27 Maria Hybinette, UGA 28
Example: parent-child.c Things to Note
#include
Example: talk-to.c Example: talk-to.c
● A simple communications program : #include
ps Output Process Summary
● Process: a program in execution {saffron} ps –l » Time and Space entity UID PID PPID COMMAND 501 3945 371 -ksh » System View : A set of data structures that changes over time. 501 3984 3945 talk-to – Entity that needs system resources (e.g., CPU & Memory, Files). 501 3985 3984 talk-to » Address Space : User / System . . – Stack / Heap / Data (initialized, uninitialized) / Text . – Program pointer, Stack pointer ksh fork() ● Creation/Fork: Identical ‘copy’ of parent initially starting at next instruction after fork talk-to (parent) » logical (separate) copy of parents address space fork() » separate stack and heap talk-to (child) » Caveats: Multi-threaded Processes, Lightweight Processes – Shares ‘more’ (e.g., address space).
Maria Hybinette, UGA 33 Maria Hybinette, UGA 34
Replace Program: w/ exec() Example: tiny-menu.c
Command line arguments: note argv0 is often = file #include
● Process ID does not change across exec calls execlp( cmd[i], cmd[i], (char *) 0 ); » new process is not created, just it s context is replaced. printf( execlp failed\n ); ● The old program is obliterated by the new printf() not executed unless } » ! no return back to the exec caller - unless there is an ERROR there is a problem with execlp()
Maria Hybinette, UGA 35 Maria Hybinette, UGA 36 exec(…) family: execute a file exec(…) family: execute a file (program) (program)
● There are 6 versions of the exec function and they all ● There are 6 versions of the exec function and they all basically do the same thing; they replace the current basically do the same thing; they replace the current program with the text of the new program. program with the text of the new program. ● Main difference is how the parameters are passed: ● Main difference is how the parameters are passed: » Permutations: – pathname/file (p) : #include
● Permutations: pathname/file : vector/list : environment Maria Hybinette, UGA 37 Maria Hybinette, UGA 38
exec(…) Family Tree - Summary
Full Pathname Filename
execl s argument as 1. We created a process the unix way – execl() execle() execlp() list » Forking 2. We communicated 3. And we ran a file/program from a process execv() execvp() execv s argument as vector » Exec d.
Combine these 3 things…. execve()
● Permutations: pathname/file : vector/list : environment ● System call: execve() -> all paths leads to this one ● execve(const char *path, char *const argv[], char *const envp[]); Maria Hybinette, UGA 39 Maria Hybinette, UGA 40
Want: fork() & execv() Terminating processes
Parent ● Problem: Our original menu Make an image of myself Here program only allowed a user to pid = fork() execute Parent Child pid == 0 » Only one command » But now we are forking, couldn’t we pid == 5 fork returns pid == 0 do more? and runs as cloned Original New copy of parent until execv is ● Want: process called Parent Continues » Would like child program to finish before continuing. execv( new program ) » (other instances) perhaps we would like to get result from child before Original New copy of New program continuing process Parent (replacement) WAIT Maria Hybinette, UGA 41 Maria Hybinette, UGA 42 Process control: wait()& waipid() wait()or waitpid()Actions
#include
Maria Hybinette, UGA 43 Maria Hybinette, UGA 44
wait()or waitpid()Example Example: menu-shell.c
int main() {saffron:ingrid:40} menu-shell { 0 ingrid console Apr 4 10:58 ● Example program: menu-shell.c illustrates char *cmd[] = { who , ls , date }; waiting for child 4953...child 4953 int i; finished wait() and includes: 0 = who : 1 = ls : 2 = date pid_t pid; 0 while( 1 ) ingrid console Apr 4 10:58 { waiting for child 4954...child 4954 finished #include
perror( execlp failed\n ); } else { /* parent */ printf( "waiting for child %d...", pid ); wait( (int *) 0 ); printf( "child %d finished\n", pid ); } 45 46 Maria Hybinette, UGA Maria Hybinette, } } UGA
Macros for wait (1) menu-shell Execution samples the status
Enables checking on status of child after wait returns: ● WIFEXITED(status) menu-shell » Returns true if the child exited normally » Checks 8 low order bits, i.e., the most significant eight bits. pid = fork() » If macro is zero then child been stopped by another process via Parent Child a signal. ● WEXITSTATUS(status) execlp( cmd[i] ) » Details on exit status » Evaluates to the least significant eight bits (high order bits) of the menu-shell menu-shell cmd[i] return code of the child which terminated, which may have been set as the argument to a call to exit( ) or as the argument for a return. » This macro can only be evaluated if WIFEXITED returned non- zero. wait()
Maria Hybinette, UGA 47 Maria Hybinette, UGA 48 Macros for wait (2) waitpid():Particular Child
#include
Maria Hybinette, UGA 49 Maria Hybinette, UGA to that of the calling process. 50
waitpid() Macros for waitpid()
● opts : options when pid > 0 ● WIFSTOPPED(status) » Zero or more of the following constants can be » Returns true if the child process which caused the OR’ed: return is currently stopped. – WNOHANG » This is only possible if the call was done using ● Return immediately if no child has exited. WUNTRACED. – WUNTRACED ● Also return for children which are stopped, and whose ● WSTOPSIG(status) status has not been reported (because of a signal). » Returns the signal number which caused the child ● Returns process ID of child which exits, -1 on to stop. error, 0 if WNOHANG was used and no child was » This macro can only be evaluated if WIFSTOPPED available. returned non-zero.
Maria Hybinette, UGA 51 Maria Hybinette, UGA 52
returned if child is stopped and not reported (signal)
Example: waiting.c else {saffron:ingrid:54} waiting { /* parent */ waiting for child 985 while (1) child terminates normal termination with status(0) { #include
{saffron:ingrid:56} kill -TERM 5024
Maria Hybinette, UGA 53 Special Exit Cases Zombies
● Terminated child process, but ● A child exits when its parent is not currently executing still around, waiting for its wait() » the child becomes a zombie parent : to wait() and do the » status data about the child is stored until the parent cleanup. does a wait() ● » Zombie: Terminated process that has not YET been cleaned up. Still take up system Parents are responsible to clean up after their children. Possible resources, memory, and it parent has not YET called wait. will never be schedule since ● A parent exits when 1 or more children are still running it is ‘terminated’ » children are adopted by the system’s init process (/etc/ init) ● Problem: when there are lots – it can then monitor/kill them of zombies, one by itself not – when the adopted child terminates however it does not become a zombie, because init automatically calls wait bad, but a crowd can be a when the child finally terminates problem
Maria Hybinette, UGA 55 Maria Hybinette, UGA http://en.wikipedia.org/wiki/Zombie_(fictional)56
make-zombie.c!
● ps -e -o pid,ppid,stat,cmd | grep zom! ● Child is marked as defunct » Terminated child that has not yet been clean up! ● Parents exits without calling wait, » Zombie child is adopted by init, and now init will clean up after the unclean parent!
Maria Hybinette, UGA 57 Maria Hybinette, UGA 58
Process Data Example: global-example.c
#include
a copy of the parent s data. int globvar = 6; printf( pid = %d, globvar = %d, w = %d\n , char buf[] = stdout write\n ; getpid(), globvar, w );
● A change to a variable in the child will not return 0; int main(void) } /* end main */ change that variable in the parent. { int w = 88; pid_t pid; {saffron:ingrid:62} global-example stdout write write( 1, buf, sizeof(buf)-1 ); Before fork() printf( Before fork()\n ); pid = 5039, globvar = 7, w = 89 if( (pid = fork()) == 0 ) pid = 5038, globvar = 6, w = 88 { /* child */ {saffron:ingrid:63} globvar++; w++; } Maria Hybinette, UGA 59 Maria Hybinette, UGA 60 Caveat: Process File Descriptors Before and after fork()
● While child and parent have (separate) copies of the file descriptors they share system file table entries. » Effect is that the R-W pointer is shared
● This means that a read() or write() in one process will affect the other process since the R-W pointer is ● Un-related processes ● Related processes changed.
Maria Hybinette, UGA 61 Maria Hybinette, UGA https://cs230.wikispaces.com/System-Level+IO+Notes62
int main( void ) {saffron} cat { fileptr.txt int fd; /* file descriptor */ hello Example: file-ptr.c pid_t pid; this is char buf[10]; /* for file data */ the data file
if( (fd = open( file-ptr.txt", O_RDONLY )) < 0 ) perror("open"); read( fd, buf, 10 ); /* move R-W ptr */ {saffron} shared-file printpos( "Before fork", fd ); Before fork: 10 if( (pid = fork()) == 0 ) Child before read: 10 #include
Maria Hybinette, UGA 63
I/O redirection: ls > lis.txt! I/O redirection
● Example implementation shell: (1) Open create file – write mode – » {saffron} ls > lis.txt (2) How do we get stdout of ls to go to the file? » open a new file lis.txt » Redirect standard output to lis.txt using dup2 – Everything that is sent to standard output is also sent to lis.txt ● The trick: you can change where the standard I/O streams are going/coming from after the » Execute ls in the process but before the ● dup2( int fin, int fout ) - copies fin to fout in fork exec file table File table dup2( 3, 1 )
0 stdin STDIN_FILENO 0 1 stdout STDOUT_FILENO 1 2 stderr STDERR_FILENO 2 3 lis.txt 3 lis.txt 4 4 Maria Hybinette, UGA 65 Maria Hybinette, UGA 66 Example: ls > lis.txt User and Group ID (revisit)
#include
Read IDs Change UID and GID (1)
● pid_t getuid( void ); #include
int setuid( uid_t uid ) ● pid_t geteuid( void ); int setgid( gid_t gid ) » Returns the effective user ID of the current process ● Sets the effective user ID of the current process. ● gid_t getgid( void ); ● Superuser process resets the real effective user » Returns the real group ID of the current process IDs to uid. ● Non-superuser process can set effective user ID to uid, only when uid equals real user ID or the ● gid_t getegid( void ); saved set-user ID (set by executing a setuid- » Returns the effective group ID of the current process program in exec). ● In any other cases, setuid returns error.
Maria Hybinette, UGA 69 Maria Hybinette, UGA 70
Change UID and GID (2) Change UID and GID (3)
exec setuid(uid) #include
set from user ID ● Sets real and effective user ID’s of the current process effective user ID unchanged set to uid set to uid of program file ● Un-privileged users may change the real user ID to the effective user ID and vice-versa. saved set-uid copied from euid copied from euid uid unchanged ● It is also possible to set the effective user ID from the saved user ID. ● Supplying a value of -1 for either the real or effective user ID forces the system to leave that ID unchanged. ● Different ways to change the three user IDs (pg 214) ● If the real user ID is changed or the effective user ID is set to a value not equal to the previous real user ID, the saved user ID will be set to the new effective user ID.
Maria Hybinette, UGA 71 Maria Hybinette, UGA 72 Change UID and GID (4) Environment
#include
int seteuid( uid_t uid ); environment pointer environment list environment strings int setregid( gid_t rgid, gid_t egid ) int setegid( gid_t gid ); environ: HOME=/User/ingrid\0
PATH=/bin:/usr/bin\0 ● Functionally equivalent to setreuid( -1, euid ) SHELL=/bin/ksh\0 ● Setuid-root program wishing to temporarily drop root privileges, assume the identity of a non-root user, and then USER=ingrid\0 regain root privileges afterwards cannot use setuid, because setuid issued by the superuser changes all three IDs. One can LOGNAME=ingrid\0
accomplish this with seteuid. NULL
Maria Hybinette, UGA 73 Maria Hybinette, UGA 74
Example: environ.c getenv
#include
Maria Hybinette, UGA 75 Maria Hybinette, UGA 76
putenv Example: getputenv.c
#include
Maria Hybinette, UGA 77 Maria Hybinette, UGA 78