(System V) Inter-process Communication
Emmanuel Fleury B1-201 [email protected]
1 Outline
● Inter-Process Communication
● Semaphores
● Message Queues
● Shared Memory Segments
2 Inter-Process Communication
3 What is (System V) IPC ?
IPC is live communication between processes !
● What is IPC ?
– All processes are active at communication time – Processes resides in different protected domains
● What is NOT IPC ?
– Persistent data communication (files, pipes)
– Process/Kernel communication (signals)
4 What is (System V) IPC ?
Process Process A B
IPC Kernel
Three IPC mechanisms (sys/ipc.h):
● Semaphores (sys/sem.h)
● Message Queues (sys/msg.h)
● Shared Memory Segments (sys/shm.h)
5 IPC Interface
● Each IPC is identified by a unique key (key_t) and a data-structure: - Semaphore ID (semid, semid_ds), - Message Queues ID (msqid, msqid_ds), - Shared Memory Segment ID (shmid, shmid_ds)
● Creation through XXXget() functions: - Semaphore (semget()), - Message Queues (msgget()), - Shared Memory Segment (shmget())
● Destruction through XXXctl() functions: - Semaphore (semctl()), - Message Queues (msgctl()), - Shared Memory Segment (shmctl())
6 Creation of an IPC
My key is 20, I want an ID !
Process Process A Your ID is B 22937 !
IPC
Kernel 22937
1. The user give a key 2. The kernel create an IPC object if necessary 3. The kernel return an ID
7 Creation of an IPC
My key is 20, I want an ID !
Process Process A Your ID is B 22937 !
IPC
Kernel 22937
1. The user give a key 2. The kernel create an IPC object if necessary 3. The kernel return an ID
8 ipcs (IPC Status) ipcs is used to get the status of all the IPC objects of your system
[fleury@hermes]$ ipcs Shared Memory Segments key shmid owner perms bytes nattch status 0x00000000 98305 fleury 600 393216 2 dest Semaphore Arrays key semid owner perms nsems Message Queues key msqid owner perms usedbytes messages
[fleury@hermes]$ ipcs m Shared Memory Segments key shmid owner perms bytes nattch status 0x00000000 98305 fleury 600 393216 2 dest [fleury@hermes]$ ipcs s Semaphore Arrays key semid owner perms nsems
[fleury@hermes]$ ipcs p m Shared Memory Creator/Lastop shmid owner cpid lpid 98305 fleury 4463 5294
9 ipcrm (IPC Remove)
ipcrm is used to remove IPC objects from your system
[fleury@hermes]$ ipcs Shared Memory Segments key shmid owner perms bytes nattch status 0x00000000 98305 fleury 600 393216 2 dest Semaphore Arrays key semid owner perms nsems
Message Queues key msqid owner perms usedbytes messages
[fleury@hermes]$ ipcrm m 98305
[fleury@hermes]$ ipcs Shared Memory Segments key shmid owner perms bytes nattch status
Semaphore Arrays key semid owner perms nsems
Message Queues key msqid owner perms usedbytes messages
10 Creation of an IPC
● #include
11 IPC control operations
#include
● cmd: – IPC_STAT: Copy information from the kernel data structure associated with key into the ipcid_ds structure pointed to by buf – IPC_SET: Write the values of the ipcid_ds structure pointed to by buffer to the kernel data structure associated with this IPC – IPC_RMID: Destroy the IPC
12 Creating an IPC Object
[fleury@hermes]$ ./myipc The key is 0 The identifier is 262144 #include
printf("The key is %i\n", key); printf("The identifier is %i\n", msqid);
exit(0); } 13 Ownership & Access Policy
● Each IPC has an ownership and access data (ipc_perm):
– uid_t uid: Owner's user ID
– gid_t gid: Owner's group ID
– uid_t cuid: Creator's user ID
– gid_t cgid: Creator's group ID
– mode_t mode: Read/write permissions
● At creation time, the values are:
– uid_t uid: Effective user ID of the creating process
– gid_t gid: Effective group ID of the creating process
– uid_t cuid: Effective user ID of the creating process
– gid_t cgid: Effective group ID of the creating process
– mode_t mode: Read/write permissions from the umask of the creating process 14 Creating an IPC (take two)
[fleury@hermes]$ ./myipc The key is 0 #include
printf("The key is %i\n", key); printf("The identifier is %i\n", msqid);
exit(0); }
15 Creating an IPC (given key)
[fleury@hermes]$ ./myipc myipc: No such file or directory The identifier is 1 #include
if ((msqid = msgget(20, 0666)) == 1) { perror("myipc"); exit(1); }
printf("The identifier is %i\n", msqid);
exit(0); }
16 Creating an IPC (given key)
[fleury@hermes]$ ./myipc The identifier is 425984 [fleury@hermes]$ ipcs q #include
if ((msqid = msgget(20, 0666 | IPC_CREAT)) == 1) { perror("myipc"); exit(1); }
printf("The identifier is %i\n", msqid);
exit(0); } 17 Creating an IPC (given key)
[fleury@hermes]$ ./myipc The identifier is 425984 The identifier is 425984 #include
fork();
if ((msqid = msgget(20, 0666 | IPC_CREAT)) == 1) { perror("myipc"); exit(1); }
printf("The identifier is %i\n", msqid);
exit(0); }
18 Deleting an IPC (given key)
#include
if ((msqid = msgget(key, 0666 | IPC_CREAT)) == 1) { perror("myipc"); exit(1); }
printf("The identifier is %i\n", msqid);
if ((msgctl(msqid, IPC_RMID, NULL) == 1)) { perror("myipc"); exit(1); }
exit(0); }
19 Getting an IPC status
#include
if ((msqid = msgget(key, 0666 | IPC_CREAT)) == 1) { perror("myipc"); exit(1); }
printf("The identifier is %i\n", msqid);
if ((msgctl(msqid, IPC_STAT, NULL) == 1)) { perror("myipc"); exit(1); }
printf("Messages in queue: %i\n", (int) status.msg_qnum); printf("Bytes in queue: %i\n", (int) status.msg_qbytes); printf("Last process sending: %i\n", (int) status.msg_lspid); printf("Last process receiving: %i\n", (int) status.msg_lrpid);
exit(0); } 20 Relate an IPC and a File
#include
key_t ftok(const char *pathname, int proj_id);
The function ftok() uses the identity of the file pathname (an already existing and accessible file) and the least significant 8 bits of proj_id (non zero) to generate an IPC key, suitable for use with msgget(), semget(), or shmget().
Note: Previously proj_id was a char, that's why only the least significant 8 bits are taken into account.
21 ipc() (Linux specific)
#include
● Implements any call to an IPC function
● Parameters are depending on which function you are calling (call tell what function is called)
● Don't use this function if portability is required
22 Semaphores
23 Semaphores
Process Process Critical Section A B
Semaphore Kernel
● The IPC semaphore object is a set of semaphores (set of values).
● Each semaphore set is identified by semid (id of the set) and each semaphore within the set by semnum.
● Each operation performed through semop() is atomic.
● The semaphore structure is composed of the following members:
– unsigned short semval: Semaphore value
– unsigned short semncnt: Number of processes waiting for semval to increase.
– unsigned short semzcnt: Number of processes waiting for semval to become 0.
– pid_t sempid: Process ID of last operation.
24 System Wide Limitations
System wide limits on semaphores (/proc/sys/kernel/sem): – SEMMSL: Maximum number of semaphores per set – SEMMNS: Maximum number of semaphores in all sets – SEMOPM: Maximum number of operations specified in semop() – SEMMNI: Maximum number of semaphore identifiers
25 Semaphores API
● semget(): Get a semaphore set's identifier
● semctl(): Control of semaphores informations
● semop(): Semaphore operations
● semtimedop(): Semaphore timed operations
26 semget()
#include
● Get an ID from a key
● Same behaviour as others get() functions with semid and semflg
● nsems: Number of semaphores for this set
27 semget()
#include
/* Creation of the set of semaphores */ if ((semid = semget(20, 5, 0666 | IPC_CREAT)) == 1) { perror("mysems"); exit(1); }
printf("The ID of the semaphore set is: %i\n", semid);
exit(0); }
28 semctl()
#include
● semnu: ID of the semaphore in the set
● cmd: Usual IPC_STAT, IPC_SET, IPC_RMID, and also:
– GETVAL: Get the current value of the semaphore – GETALL: Get the current values for all semaphores
– SETVAL: Set the current value of the semaphore
– SETALL: Set the current value for all semaphores – GETZCNT: Get the number of processes waiting the value to be 0 – GETNCNT: Get the number of processes waiting the value to increase – GETPID: Get the PID of the last process that accessed the semaphore
29 semctl(IPC_RMID)
#include
[fleury@hermes]$ ./mysems int main() { The value of the semaphore 0 is 1 The value of the semaphore 1 is 1 union semun semunion; The value of the semaphore 2 is 2 unsigned short array[5] = {1, 1, 2, 1, 3}; The value of the semaphore 3 is 1 int semid, i; The value of the semaphore 4 is 3 if ((semid=semget(20, 5, 0666 | IPC_CREAT)) == 1) { perror("mysems"); exit(1); Note: The array must } have the exact same size semunion.array = array; if (semctl(semid, 0, SETALL, semunion) == 1) { as the semaphores set. perror("mysems"); exit(1); } if (semctl(semid, 0, GETVAL, semunion) == 1) { perror("mysems"); exit(1); } for (i=0; i<5; i++) { printf("The value of the semaphore %i is %i\n", i, semunion.array[i]); } exit(0); } 33 semun & semid_ds
struct semid_ds { struct ipc_perm sem_perm; /* Ownership and permissions */ time_t sem_otime; /* Last semop time */ time_t sem_ctime; /* Last change time */ unsigned short sem_nsems; /* No. of semaphores in set */ };
union semun { int semval; /* Value for SETVAL */ struct semid_ds *buf; /* Buffer for IPC_STAT, IPC_SET */ unsigned short *array; /* Array for GETALL, SETALL */ struct seminfo *__buf; /* Buffer for IPC_INFO (Linux only) */ };
34 semop()
#include
Atomic operations on semaphores are performed through the semop() system call
● semid: IPC ID of the set of semaphores
● sops: Array of operation(s) to perform atomically !
● nsops: Number of elements in the array sops.
35 sembuf
struct sembuf { ushort sem_num; /* semaphore ID */ short sem_op; /* semaphore operation */ short sem_flg; /* operation flags */ }
● sem_op: Adds this value to the semaphore value
● sem_flg: – IPC_NOWAIT: Operation is performed if it can be done instantly – SEM_UNDO: Automatically undo when the process terminates
36 semop() int lock(int semid, int semnum) { struct sembuf semb; Note: The value of a semaphore
semb.sem_num = semnum; can't be less than 0. If a process semb.sem_op = 1; try to decrease it when the value semb.sem_flg = SEM_UNDO; is 0, then it will hang until the if (semop(semid, &semb, 1) == 1) { value increase again. perror("lock"); return 1; } int unlock(int semid, int semnum) { struct sembuf semb; return 0; } semb.sem_num = semnum; semb.sem_op = 1; semb.sem_flg = SEM_UNDO; [fleury@hermes]$ ./lock_sem0 ^C if (semop(semid, &semb, 1) == 1) { [fleury@hermes]$ ./lock_sem1 perror("unlock"); [fleury@hermes]$ return 1; }
return 0; }
37 semtimedop()
#include
● Look at the manual page... :-)
38 Full Example int main() { union semun semunion; unsigned short array[5] = {1, 1, 1, 1, 1}; int semid; pid_t id; /* Creation of the IPC */ if ((semid = semget(20, 5, 0666 | IPC_CREAT)) == 1) { perror("mysems"); exit(1); } /* Initialization of the semaphores */ semunion.array = array; if (semctl(semid, 0, SETALL, semunion) == 1) { perror("mysems"); exit(1); } [fleury@hermes]$ ./mysems id = fork(); /* Forking */ I'm 0 and I'm in critical section ! I'm 4994 and I'm in critical section ! if (lock(semid, 0)) /* Locking */ exit(1); [fleury@hermes]$ /* Critical section */ printf("I'm %i and I'm in critical section !\n", id); sleep(1); if (unlock(semid, 0)) /* Unlocking */ exit(1); Note: The set of semaphores is exit(0); still here. Think to clean after you. }
39 Message Queues
40 Message Queues
Send Receive Process Process A B
Message Queue Kernel
● Queues are Linked-list of messages (with a maximum number of cells)
● Message size must be known (unlike pipes which are exchanging streams)
● Messages are typed (to avoid confusion when fetching one message)
● Mailbox Mechanism (send()/receive())
41 System Wide Limitations
● MSGMNI: Maximum number of message queues (/proc/sys/kernel/msgmni)
● MSGMAX: Maximum number of messages per queue (/proc/sys/kernel/msgmax)
● MSGMNB: Maximum number of overall messages (/proc/sys/kernel/msgmnb)
42 Message Queues API
● msgget(): Create or open a message queue
● msgctl(): Control message queue informations
● msgsnd(): Send a message to a message queue
● msgrcv(): Receive a message from a message queue
43 msgget()
● #include
● flags: – IPC_CREAT: Create entry if key does not exist – IPC_EXCL: Fail if key exists – IPC_NOWAIT: Fail if request must wait
44 msgctl()
#include
– IPC_STAT: Copy information from the kernel data structure associated with msqid into the msqid_ds structure pointed to by buffer – IPC_SET: Write the values of some members of the msqid_ds structure pointed to by buffer to the kernel data structure associated with this message queue, updating also its msg_ctime member – IPC_RMID: Remove the message queue, awake all waiting reader and writer processes
45 msqid_ds
struct msqid_ds { ipc_perm msg_perm; /* Ownership and permissions */ time_t msg_stime; /* Time of last msgsnd() */ time_t msg_rtime; /* Time of last msgrcv() */ time_t msg_ctime; /* Time of last change */ ulong __msg_cbytes; /* No. of bytes in queue * * (Linux specific) */ msgqnum_tmsg_qnum; /* No. of messages in queue */ msglen_t msg_qbytes; /* Maximum number of bytes * * allowed in queue */ pid_t msg_lspid; /* PID of last msgsnd() */ pid_t msg_lrpid; /* PID of last msgrcv() */ };
46 msgsnd()
● #include
● msgsz: Size of the message (bytes)
● msgflg: - IPC_NOWAIT: Immediate return if no message of the type is in queue - MSG_EXCEPT: If (msgtyp > 0) read the first message in the queue. - MSG_NOERROR: Truncate the message text if longer than msgsz bytes.
47 msgrcv()
● msqid: IPC ID #include
● msgsz: Size of the message (bytes)
● msgtype: Type of the message
● msgflg: - IPC_NOWAIT: Immediate return if no message of the type is in queue - MSG_EXCEPT: If (msgtyp > 0) read the first message in the queue. - MSG_NOERROR: Truncate the message text if longer than msgsz bytes.
48 Full Example struct mymsg { [fleury@hermes]$ ipcs q long mtype; Message Queues char *mtext; key msqid owner perms usedbytes messages }; 0x00000014 2850816 fleury 666 0 0 [fleury@hermes]$ ipcs q int main() { Message Queues int msqid; key msqid owner perms usedbytes messages struct mymsg msg; 0x00000014 2850816 fleury 666 3 1 char buffer[10] = “abcdefghi\0”; [fleury@hermes]$ ipcs q Message Queues msg.mtype = 1; key msqid owner perms usedbytes messages msg.mtext = buffer; 0x00000014 2850816 fleury 666 0 0 /* Creation of the IPC */ if ((msqid = msgget(20, 0666 | IPC_CREAT)) == 1) { perror("mymsg"); exit(1); } /* Sending a message */ [fleury@hermes]$ ./mysmsg msgsnd(msqid, &msg, 3, 0); The message is: abcdefghi sleep(5); [fleury@hermes]$ /* Receiving a message */ msgrcv(msqid, &msg, 3, 1, 0); printf("The message is: %s", msg.mtext); exit(0); }
49 Shared Memory
50 Shared Memory
Attach Attach Process Process A B
Kernel
Shared Memory Memory Memory Process A Process B Memory
51 Shared Memory
Detach Detach Process Process A B
Kernel
Shared Memory Memory Memory Process A Process B Memory
● Allow unrelated processes to share the same logical memory.
● Warning !
– No mechanism preventing race conditions or read/write problems. Accessing this memory should be protected via semaphores. Remember also to clean after you !
– This does not enlarge the logical memory of a process, it only replaces a part of it by a shared memory. 52 System Wide Limitations
System wide limits on shared Memory (/proc/sys/kernel/shm*):
– SHMALL: Maximum number of shared memory pages (/proc/sys/kernel/shmall)
– SHMMAX: Maximum size (bytes) of shared segments (/proc/sys/kernel/shmmax)
– SHMMIN: Minimum size (bytes) of a shared segment (/proc/sys/kernel/shmmin)
– SHMMNI: Maximum number of shared segments (/proc/sys/kernel/shmmni)
53 Shared Memory API
● shmget(): Allocate a memory area and return an identifier
● shmctl(): Control shared memory informations
● shmat(): Attach an IPC shared memory area to the process
● shmdt(): Detach an IPC shared memory area from the process
54 shmget()
● #include
● size: Number of memory pages allocated to the new memory segment
● flags:
– IPC_CREAT: Create entry if key does not exist – IPC_EXCL: Fail if key exists – IPC_NOWAIT: Fail if request must wait
55 shmctl()
● shmid: IPC ID #include
– SHM_LOCK/SHM_UNLOCK (Linux specific): Prevent/Allow the memory to be swapped
56 shmid_ds
struct shmid_ds { struct ipc_perm shm_perm; /* Ownership and permissions */ size_t shm_segsz; /* Size of segment (bytes) */ time_t shm_atime; /* Last attach time */ time_t shm_dtime; /* Last detach time */ time_t shm_ctime; /* Last change time */ pid_t shm_cpid; /* PID of creator */ pid_t shm_lpid; /* PID of last stmat()/shmdt() */ shmatt_t shm_nattch; /* No. of current attaches */ ... };
57 shmat()
● #include
● shmflg: Note: On success shmat() returns – 0: Read/write access. the address of the memory segment and -1 in case of failure. – SHM_RDONLY: Read only access.
– SHM_RND: If (shmaddr != NULL) attach is at shmaddr rounded down to the nearest multiple of SHMLBA (Segment low boundary address multiple).
– SHM_REMAP (Linux specific): The segment replaces any existing mapping in the range starting at shmaddr and continuing for the size of the segment.
58 shmdt()
#include
● shmaddr: Address of the shared memory segment to detach from the process.
● Return:
– 0 on success
– -1 on fail
59 Full Example
#define MEMSIZE 1 #define BUFSIZE 1000 int main() { int shmid, i; char *buffer;
/* Creation of the IPC */ if ((shmid = shmget(20, MEMSIZE, IPC_CREAT | 0666)) == 1) { perror("shared_memory"); exit(1); } [fleury@hermes]$ ./myshm /* Attach to the IPC */ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa if ((int) (buffer = shmat(shmid, NULL, 0)) == 1) { perror("myshm"); aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa exit(1); aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa } aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa /* Use the shared memory */ aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa for (i=0; i exit(0); } 60 Questions ? 61 Next Weeks ● Threads – Creation/Termination – Synchronizations Mechanisms ● Programming CORBA (ORBit2) 62