DOCSLIB.ORG

Distributed Programming I (Socket - Nov'09)

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Distributed Programming I (Socket - Nov'09) Distributed programming I (socket - nov'09) Warning for programmers network programming is dangerously close to O.S. Network programming: sockets kernel, and therefore: It can easily hang the O.S. verify the results of every operation, without assuming anything as granted Antonio Lioy < [email protected] > APIs can vary in details that are minimal but important consider every possible situation to create english version created and modified by “portable” programs Marco D. Aime < [email protected] > we will try to use Posix 1.g Politecnico di Torino Dip. Automatica e Informatica ISO/OSI, TCP/IP, network programming Exercise – copying data copy the content of file F1 (first parameter on the application command line) into file F2 (second parameter on the 7. application details application command line) user 6. presentation (l6: XDR/XML/... process l5: RPC/SOAP/...) 5. session network programming 4. transport TCP UDP SCTP interface 3. network IPv4, IPv6 kernel communication 2. data link device driver details 1. physical and hardware OSI model IP suite ref. UNP Intro copyfile.c Error messages Error functions must contain at least: best to define standard error reporting functions [ PROG ] program name which accept: [ LEVEL ] error level (info, warning, error, bug) a format string for the error [ TEXT ] error signalling, the most specific as possible a list of parameters to be printed (e.g. input file name and line where the problem has UNP, appendix D.4 (D.3 in 3rd edition) occurred) errno? termination? log level [ ERRNO ] system error number and/or name (if applicable) err_msg no no LOG_INFO err_quit no exit(1) LOG_ERR suggested format: err_ret yes no LOG_INFO err_sys yes exit(1) LOG_ERR ( PROG ) LEVEL - TEXT : ERRNO err_dump yes abort( ) LOG_ERR errlib.h errlib.c © A.Lioy - Politecnico di Torino (2009) B-1 Distributed programming I (socket - nov'09) stdarg.h stdarg.h usage example variable list of arguments (ANSI C) create a function named my_printf declared with an ellipsis (. .) as the last argument of … behaving like printf (thus accepting a variable a function number of parameters) arguments used altogether (ap) with special … but printing the string “(MY_PRINTF)” before its functions (e.g. vprintf, vfprintf, vsprintf, vasprintf, output vsnprintf), or separately (va_ arg); but in the second case you must know their number by other means #include <stdarg.h> void va_start (va_list ap, RIGHTMOST); TYPE va_arg (va_list ap, TYPE); void va_end (va_list ap); va_test.c Exchanging data between From 32 to 64 bit: problems heterogeneous nodes migration from 32-bit to 64-bit architecture has problem: changed data sizes when exchanging complex data (i.e., not single ASCII in particular, do not assume anymore that characters), do not assume they are encoded in the | int | = | pointer | same way on different nodes therefore, pay attention to correctly use the data encoding depends on HW + OS predefined types to avoid problems (e. g. size_ t) solution: use a neutral format (the network format) ILP32 LP64 some functions automatically do the conversion… char 88 short 16 16 … but often it is explicit task for the programmer int 32 32 long 32 64 pointer 32 64 datasize.c Sources of data incompatibility “host to network” functions different floating-point formats (IEEE-754 / non-IEEE) to pass parameters to network functions alignment of structs on word boundaries not designed for application data passing byte order of integers (little-endian or big-endian) #include <sys/types.h> #include <netinet/in.h> n = 25810 = 010216 uint32_t htonl (uint32_t hostlong); big-endian little-endian uint16_t htons (uint16_t hostshort); uint32_t ntohl (uint32_t netlong); 01 memory address X 02 uint16_t ntohs (uint16_t netshort); 02 memory address X+1 01 byteorder.c © A.Lioy - Politecnico di Torino (2009) B-2 Distributed programming I (socket - nov'09) Notes on integer types Network addresses sometimes you may still find the old types: IPv4 networks directly use 32-bit addresses u_long (= uint32_t) ... not names (e.g. www.polito.it), that are translated u_short (= uint16_t) into addresses by DNS in old versions of cygwin, they were defined as: ... and neither dotted addresses (e.g. 130.192.11.51) u_ int32_ t (= uint32_t) ... but their 32-bit numerical value u_int16_t (= uint16_t) example: 130.192.11.51 24 16 8 suggestion: = 130*2 + 192*2 + 11*2 + 51 write programs with the types uint32_t and uint16_t = (((130*256) + 192)*256 + 11)*256 + 51 when necessary, map them with a conditioned #define = 130<<24 + 192<<16 + 11<<8 + 51 = 2,193,623,859 Network address conversion inet_aton ( ) for generality, standard functions require numerical converts an IPv4 address … addresses to be expressed in struct format … from “dotted notation” string to convert numerical addresses from/to strings: … to numerical (network) format [IPv4] inet_ntoa( ) and inet_aton( ) returns 0 if the address is invalid [IPv4 / v6] inet_ntop( ) and inet_pton( ) the string may be composed by decimal numbers other functions (e.g. inet_addr) are deprecated (default), octal (start with 0) or hexadecimal (start with 0x) thus 226.000.000.037 is equal to 226.0.0.31 #include <arpa/inet.h> #include <arpa/inet.h> struct in_addr { in_addr_t s_addr }; int inet_aton ( const char *strptr, struct in_addr *addrptr struct in6_addr { uint8_t s6_addr[16]; }; ); inet_ntoa ( ) Example: address validation converts an IPv4 address … write a program that: … from numerical (network) format accepts a dotted notation IPv4 address on the … to “dotted notation” string command line returns the pointer to the string or NULL if the returns its numerical (network) value address is invalid signals an error in case of erroneous format or illegal beware! the pointer being returned points to a static address memory area internal to the function verify: #include <arpa/inet.h> remember that A.B.C.D = A<<24 + B<<16 + C<<8 + D char *inet_ntoa ( struct in_addr addr avrfy.c ); avrfy2.c © A.Lioy - Politecnico di Torino (2009) B-3 Distributed programming I (socket - nov'09) inet_pton( ) inet_ntop( ) converts an IPv4 / IPv6 address … converts an IPv4 / IPv6 address … … from “dotted notation” string … from numerical (network) format … to numerical (network) format … to “dotted notation” string returns 0 if the address is invalid, -1 if the address returns the pointer to the string or NULL if the family is unknown address is invalid address family = AF_INET or AF_INET6 #include <arpa/inet.h> #include <arpa/inet.h> int inet_pton ( char *inet_ntop ( int family, int family, const char *strptr, const void *addrptr, void *addrptr char *strptr, size_t length ); ); Example: address validation Address sizes write a program that: to size the string representation of v4/v6 addresses, accepts an IPv4 or IPv6 address on the command use macros defined in <netinet/in.h> line tells if it is valid in one of the two IP versions #include <netinet/in.h> #define INET_ADDRSTRLEN 16 #define INET6_ADDRSTRLEN 46 avrfy46.c IPv6 addresses Data structure initialization (ANSI) 128 bit = 8 16-bit groups, separated by “ : ” to handle data structures as byte sequences complete form = 1080:0:0:0:8:800:200C:417A … which may include NUL, and therefore cannot be zeros compression = 1080::8:800:200C:417A handled with “str…” functions (strcpy, strcmp, …) #include <string. h> void *memset ( void *dest, int c, size_t nbyte ) void *memcpy ( void *dest, const void *src, size_t nbyte ) int *memcmp ( const void *ptr1, const void *ptr2, size_t nbyte ) © A.Lioy - Politecnico di Torino (2009) B-4 Distributed programming I (socket - nov'09) Data structure initialization (BSD) Signal management (*) pre-ANSI functions, defined in Unix BSD signals are asynchronous events still frequently used every received signal corresponds to an implicit default behaviour often the receiving process is terminated few signals are ignored by default #include <strings. h> to change the response to a signal (i.e. its void bzero ( disposition): void *dest, size_t nbyte ) reset its default behaviour void bcopy ( const void *src, void *dest, size_t nbyte ) ignore it int bcmp ( catch (i.e. intercept) the signal and register a signal const void *ptr1, const void *ptr2, size_t nbyte ) handler Timeout sleep ( ) sometimes timeouts are needed to: starts a timer and suspends the process for the wait in idle state for a fixed time (sleep) selected time know when a certain time elapsed (alarm) if terminates because the selected time has expired returns zero if terminates for being interrupted by a signal returns the time missing to the requested term #include <unistd.h> unsigned int sleep ( unsigned int seconds ); sveglia.c alarm ( ) Signal management starts a timer which generates the SIGALRM signal at pay attention to semantic differences in various expiration time operating systems the process is not suspended for ‘signum’ use the signal logical names (SIGCHLD, note: the default response to SIGALRM is terminating SIGSTOP, …) the receiving process the handler can be: a new call replaces the current timer a user-defined function alarm(0) (e.g. use to clear the current timer) SIG_IGN (ignore the signal) (!) unique timer for all processes in the group SIG_DFL (default behaviour) #include <unistd.h> #include <signal.h> unsigned int alarm ( typedef void Sigfunc(int); unsigned int seconds Sigfunc *signal (int signum, Sigfunc *handler); ); © A.Lioy - Politecnico di Torino (2009) B-5 Distributed programming I (socket - nov'09) signal() function Notes on signals signal() is a pre-POSIX function SIGKILL and SIGSTOP cannot be intercepted nor its behaviour varies across Unix versions, and also ignored varied historically across different releases SIGCHLD and SIGURG are ignored by default the only portable use is setting a signal’s disposition to in Unix, use kill –l to list all signals SIG_DFL or SIG_IGN in Unix, signals can be generated also manually via POSIX has defined the new sigaction() function kill -signal pid most implementations of signal() now call sigaction() for example, to send SIGHUP to process 1234: see UNPv3 chap. 5.8 kill -HUP 1234 Standard signals (POSIX.1) Standard signals (POSIX.1) – cont.
Load more

Recommended publications
  • Beej's Guide to Unix IPC
    Beej's Guide to Unix IPC Brian “Beej Jorgensen” Hall [email protected] Version 1.1.3 December 1, 2015 Copyright © 2015 Brian “Beej Jorgensen” Hall This guide is written in XML using the vim editor on a Slackware Linux box loaded with GNU tools. The cover “art” and diagrams are produced with Inkscape. The XML is converted into HTML and XSL-FO by custom Python scripts. The XSL-FO output is then munged by Apache FOP to produce PDF documents, using Liberation fonts. The toolchain is composed of 100% Free and Open Source Software. Unless otherwise mutually agreed by the parties in writing, the author offers the work as-is and makes no representations or warranties of any kind concerning the work, express, implied, statutory or otherwise, including, without limitation, warranties of title, merchantibility, fitness for a particular purpose, noninfringement, or the absence of latent or other defects, accuracy, or the presence of absence of errors, whether or not discoverable. Except to the extent required by applicable law, in no event will the author be liable to you on any legal theory for any special, incidental, consequential, punitive or exemplary damages arising out of the use of the work, even if the author has been advised of the possibility of such damages. This document is freely distributable under the terms of the Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License. See the Copyright and Distribution section for details. Copyright © 2015 Brian “Beej Jorgensen” Hall Contents 1. Intro................................................................................................................................................................1 1.1. Audience 1 1.2. Platform and Compiler 1 1.3.
    [Show full text]
  • POSIX Signals
    CSE 410: Systems Programming POSIX Signals Ethan Blanton Department of Computer Science and Engineering University at Buffalo Introduction Signals Blocking Concurrency Sending Signals Summary References POSIX Signals POSIX signals are another form of interprocess communication. They are also a way to create concurrency in programs. For these two reasons, they are rather complicated and subtle! Signals provide a simple message passing mechanism. © 2018 Ethan Blanton / CSE 410: Systems Programming Introduction Signals Blocking Concurrency Sending Signals Summary References Signals as Messages POSIX signals are asynchronous messages. Asynchronous means that their reception can occur at any time.1 The message is the reception of the signal itself. Each signal has a number, which is a small integer. POSIX signals carry no other data. 1Almost. We’ll see how to control it later. © 2018 Ethan Blanton / CSE 410: Systems Programming Introduction Signals Blocking Concurrency Sending Signals Summary References Signal Types There are two basic types of POSIX signals: Reliable signals Real-time signals Real-time signals are much more complicated. In particular, they can carry data. We will discuss only reliable signals in this lecture. © 2018 Ethan Blanton / CSE 410: Systems Programming Introduction Signals Blocking Concurrency Sending Signals Summary References Asynchronous Reception From the point of view of the application: Signals can be blocked or ignored Enabled signals may be received between any two processor instructions A received signal can run a user-defined function called a signal handler This means that enabled signals and program code must very carefully manipulate shared or global data! © 2018 Ethan Blanton / CSE 410: Systems Programming Introduction Signals Blocking Concurrency Sending Signals Summary References Signals POSIX defines a number of signals by name and number.
    [Show full text]
  • Programming with POSIX Threads II
    Programming with POSIX Threads II CS 167 IV–1 Copyright © 2008 Thomas W. Doeppner. All rights reserved. Global Variables int IOfunc( ) { extern int errno; ... if (write(fd, buffer, size) == –1) { if (errno == EIO) fprintf(stderr, "IO problems ...\n"); ... return(0); } ... } CS 167 IV–2 Copyright © 2008 Thomas W. Doeppner. All rights reserved. Unix was not designed with multithreaded programming in mind. A good example of the implications of this is the manner in which error codes for failed system calls are made available to a program: if a system call fails, it returns –1 and the error code is stored in the global variable errno. Though this is not all that bad for single-threaded programs, it is plain wrong for multithreaded programs. Coping • Fix Unix’s C/system-call interface • Make errno refer to a different location in each thread – e.g. #define errno __errno(thread_ID) CS 167 IV–3 Copyright © 2008 Thomas W. Doeppner. All rights reserved. The ideal way to solve the “errno problem” would be to redesign the C/system-call interface: system calls should return only an error code. Anything else to be returned should be returned via result parameters. (This is how things are done in Windows NT.) Unfortunately, this is not possible (it would break pretty much every Unix program in existence). So we are stuck with errno. What can we do to make errno coexist with multithreaded programming? What would help would be to arrange, somehow, that each thread has its own private copy of errno. I.e., whenever a thread refers to errno, it refers to a different location from any other thread when it refers to errno.
    [Show full text]
  • UNIT: 4 DISTRIBUTED COMPUTING Introduction to Distributed Programming
    UNIT: 4 DISTRIBUTED COMPUTING Introduction To Distributed Programming: • Distributed computing is a model in which components of a software system are shared among multiple computers. Even though the components are spread out across multiple computers, they are run as one system. This is done in order to improve efficiency and performance. • Distributed computing allows different users or computers to share information. Distributed computing can allow an application on one machine to leverage processing power, memory, or storage on another machine. Some applications, such as word processing, might not benefit from distribution at all. • In parallel computing, all processors may have access to a shared memory to exchange information between processors. In distributed computing, each processor has its own private memory (distributed memory). Information is exchanged by passing messages between the processors. • Distributed computing systems are omnipresent in today’s world. The rapid progress in the semiconductor and networking infrastructures have blurred the differentiation between parallel and distributed computing systems and made distributed computing a workable alternative to high- performance parallel architectures. • However attractive distributed computing may be, developing software for such systems is hardly a trivial task. Many different models and technologies have been proposed by academia and industry for developing robust distributed software systems. • Despite a large number of such systems, one fact is clear that the software
    [Show full text]
  • Introduction to POSIX Signals
    Introduction to POSIX Signals Michael Jantz Prasad Kulkarni Douglas Niehaus Introduction ● This lab is an introduction to signals in Unix systems. – In it, you will learn about some common uses for signals. – You will also construct a small program that uses signals. ● Unpack the starter code, then make and tag it: – bash> tar zxvf eecs678-signals-lab.tar.gz – cd signals/; make; ctags -R EECS 678 Signals Lab 2 Signals ● A signal is a short message that may be sent to a process or a group of processes. ● The only information given to a process is usually the number identifying the signal; there is no room in standard signals for arguments, a message or other accompanying information. ● Signals serve two main purposes: – To make a process aware that a specific event has occurred. – To cause a process to execute a signal handler function included in its code. EECS 678 Signals Lab 3 Interrupts vs. Signals ● Signals and interrupts are very similar in their behavior ● Important difference: interrupts are sent to the operating system by the hardware, signals are sent to the process by the operating system, or other processes through the OS ● Important similarity: both signals and interrupts associate handlers with asynchronous events which interrupt current processing, thus inserting the handler into current code path ● Signals can thus be thought of as an interrupt in software: – However, note that signals have nothing to do with Soft-IRQs. The name seems related, but these are a method for deferring much of the processing associated with a hardware-interrupt into a less restrictive execution context inside the OS.
    [Show full text]
  • POSIX Signal Handling in Java
    Technical Document Series POSIX Signal Handling in Java POSIX Signal Handling In Java Introduction POSIX signals inform a running process of external events, such as the user wishing to kill the process, or the operating system signaling an impending shutdown, or the process being suspended or reinstated; or the process may have violated a resource constraint, such as excessive CPU usage or attempts to access areas outside its permitted memory space, and is asked to shutdown. In short, POSIX signals serve many different purposes. Some are even up to interpretation, such as the HUP (HangUP) signal, which is commonly used to inform a process that something about its environment has changed and the process should adjust accordingly. Some programs may interpret this to mean that the configuration has changed and needs to be reloaded; or the log file has been moved for archiving purposes and a new one should be started. The use of signals is widespread, especially on Unix-based operating systems, but Java provides no standard interface for a Java application to hear and react to them. This document shows you how to get around this limitation. The Good, the Bad, and the Ugly The good news is that there is a way to intercept POSIX signals and react to them in Java. This would allow your Java program to avoid being killable with ^C (SIGINT), for example, even ignore termination requests from the operating system (SIGTERM). Neither of these is necessarily a good idea, of course, unless you know exactly why you would want to catch these signals and either handle them yourself or ignore them altogether.
    [Show full text]
  • Lecture 14: Paging
    Lecture 14: Paging Fall 2018 Jason Tang Slides based upon Operating System Concept slides, http://codex.cs.yale.edu/avi/os-book/OS9/slide-dir/index.html Copyright Silberschatz, Galvin, and Gagne, 2013 "1 Topics • Memory Mappings! • Page Table! • Translation Lookaside Bu$ers! • Page Protection "2 Memory Mapped • Logical address translated not to memory but some other location! • Memory-mapped I/O (MMIO): hardware redirects read/write of certain addresses to physical device! • For example, on x86, address 0x3F8 usually mapped to first serial port! • Memory-mapped file (mmap): OS redirects read/write of mapped memory region to file on disk! • Every call to read() / write() involves a system call! • Writing to a pointer faster, and OS can translate in the background (also see upcoming lecture) "3 Memory-Mapped File • In Linux, typical pattern is to:! • Open file, using open() function! • Optional: preallocate file size, using ftruncate()! • Create memory mapping, using mmap() function! • Do work, and then release mapping using munmap() function! • Kernel might not write data to disk until munmap() "4 mmap() function void *mmap(void *addr, size_t length, int prot, int flags, int fd, off_t offset) • addr is target address of mapping, or NULL to let kernel decide! • length is number of bytes to map! • prot defines what mapping protection (read-only or read/write)! • flags sets other options! • fd is file descriptor that was returned by open()! • offset is o$set into file specified by fd "5 mmap() example part 1 • See man page for each of these functions
    [Show full text]
  • Interprocess Communication 1 Processes • Basic Concept to Build
    Interprocess Communication 1 Processes • Basic concept to build the OS, from old IBM mainframe OS to the most modern Windows • Used to express the requirements to be met by an OS – Interleave the execution of multiple processes, to maximize CPU utilization while providing good response time – Allocate resources to processes using a policy while avoiding deadlocks – Support interprocess communications and user creation of processes to help structuring applications • Background – Computer platform * Collection of hardware resources – CPU, memory, I/O modules, timers, storage devices – Computer applications * Developed to perform some task * Input, processing, output – Efficient to write applications for a given CPU * Common routines to access computer resources across platforms * CPU provides only limited support for multiprogramming; software manages sharing of CPU and other re- sources by multiple applications concurrently * Data and resources for multiple concurrent applications must be protected from other applications • Process – Abstraction of a running program – Unit of work in the system – Split into two abstractions in modern OS * Resource ownership (traditional process view) * Stream of instruction execution (thread) – Pseudoparallelism, or interleaved instructions – A process is traced by listing the sequence of instructions that execute for that process • Modeling sequential process/task – Program during execution – Program code – Current activity – Process stack * Function parameters * Return addresses * Temporary variables –
    [Show full text]
  • Dressing up Data For
    Dressing up data for Hannes Mühleisen DSC 2017 Problem? • People push large amounts of data into R • Databases, Parquet/Feather … • Need native SEXP for compatibility • R has no abstraction for data access • INTEGER(A)[i] * INTEGER(B)[j] etc. • Data possibly never actually used 2 Sometimes lucky • Perfectly compatible bits: • int my_int_arr[100]; • double my_dbl_arr[100]; • Doctor SEXP header in front of data and good to go • Implemented in MonetDBLite with custom allocator • Next version on CRAN will have this https://github.com/hannesmuehleisen/MonetDBLite 3 Zero-Copy in MonetDBLite Page 1 Page 2 Page 3 Page 4 Page 5 addr = mmap(col_file, len, NULL) col_file addr addr1 = mmap(NULL, len + PAGE_SIZE, NULL) addr2 = mmap(col_file, len, addr1 + 4096) addr3 = addr1 + PAGE_SIZE - sizeof(SEXPREC_ALIGN) SEXP res = allocVector3(INTSXP, len/sizeof(int), &allocator); Page 0 Page 1 Page 2 Page 3 Page 4 Page 5 col_file addr1 res & addr3 4 Demo 1 Stock R, MonetDBLite & zero-copy library(“DBI”) con <- dbConnect(MonetDBLite::MonetDBLite(), "/tmp/dscdemo") dbGetQuery(con, "SELECT COUNT(*) FROM onebillion”) # 1 1e+09 system.time(a <- dbGetQuery(con, "SELECT i FROM onebillion”)) # user system elapsed # 0.032 0.000 0.033 .Internal(inspect(a$i)) # @20126efd8 13 INTSXP g0c6 [NAM(2)] (len=1000000000, tl=0) 1,2,3,4,5,... Native R Vector w. zero-copy! 5 Not always so lucky • What if we have to actually convert? • Strings, TIMESTAMP to POSIXct etc. • NULL/NA mismatches • More involved data representations • compressed, batched, hybrid row/col, … • Need to convert all data before handing control over to R. • Can take forever, takes memory, non-obvious wait time 6 ALTREP • Luke Tierney, Gabe Becker & Tomas Kalibera • Abstract vectors, ELT()/GET_REGION() methods • Lazy conversion! static void monetdb_altrep_init_int(DllInfo *dll) { R_altrep_class_t cls = R_make_altinteger_class(/* .
    [Show full text]
  • Signals and Pipes
    Signals and Pipes Signals and Pipes 1/20 Learning Objectives Signals and Pipes I Understand synchronization issues with respect to signals I Learn about the client-server model using pipes and named pipes I Understand basic concepts of servers and daemons 2/20 More on Signals Signals and Pipes I Two main system calls that deal with signals: signal() and sigaction(). The sigaction() call is consistent across various systems whereas signal() is not necessarily consistent across systems but is simpler to use. See man page for details on both. I Examples: I signal-ex1.c I signal-ex2.c 3/20 Signal Handling after Exec Signals and When a program is exec’d the status of all signals is either default or Pipes ignore. The exec functions changes the disposition of any signals that are being caught to their default action (why?) and leaves the status of all other signals alone. For example: if (fork()==0) {// the child process if(signal(SIGINT, SIG_IGN) == SIG_ERR) err_ret ("failed␣to␣set␣SIGINT␣behavior"); if(signal(SIGTSTP, SIG_IGN) == SIG_ERR) err_ret ("failed␣to␣set␣SIGTSTP␣behavior"); execvp(program, argv); // the exec 'd program will ignore the // signalsSIGINT andSIGTSTP 4/20 Signal Handling for an Application Signals and How an application ought to set its signal handling: Pipes An application process should catch the signal only if the signal is not currently being ignored. int sig_int(), sig_quit(); if(signal(SIGINT, SIG_IGN) != SIG_IGN) signal(SIGINT, sig_int); if(signal(SIGQUIT, SIG_IGN) != SIG_IGN) signal(SIGQUIT, sig_quit); See code example: signal-ex3.c 5/20 Other Signal Issues Signals and Pipes I Signals are set to their default after being caught (under Linux and System V UNIX semantics).
    [Show full text]
  • 6.828: Virtual Memory for User Programs
    6.828: Virtual Memory for User Programs Adam Belay <[email protected]> Plan for today • Previously: Discussed using virtual memory tricks to optimize the kernel • This lecture is about virtual memory for user programs: • Concurrent garbage collection • Generational garbage collection • Concurrent checkpointing • Data-compression paging • Persistent stores What primitives do we need? • Trap: handle page-fault traps in usermode • Prot1: decrease the accessibility of a page • ProtN: decrease the accessibility of N pages • Unprot: increase the accessibility of a page • Dirty: returns a list of dirtied pages since previous call • Map2: map the same physical page at two different virtual addresses, at different levels of protection, in the same address space What about UNIX? • Processes manage virtual memory through higher- level abstractions • An address space consists of a non-overlapping list of Virtual Memory Areas (VMAs) and a page table • Each VMA is a contiguous range of virtual addresses that shares the same permissions and is backed by the same object (e.g. a file or anonymous memory) • VMAs help the kernel decide how to handle page faults Unix: mmap() • Maps memory into the address space • Many flags and options • Example: mapping a file mmap(NULL, len, PROT_READ | PROT_WRITE, MAP_PRIVATE, fd, offset); • Example: mapping anonymous memory mmap(NULL, len, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); Unix: mprotect() • Changes the permissions of a mapping • PROT_READ, PROT_WRITE, and PROT_EXEC • Example: make mapping read-only mprotect(addr, len, PROT_READ); • Example: make mapping trap on any acCess mprotect(addr, len, PROT_NONE); Unix: munmap() • Removes a mapping • Example: munmap(addr, len); Unix: sigaction() • Configures a signal handler • Example: get signals for memory access violations act.sa_sigaction = handle_sigsegv; act.sa_flags = SA_SIGINFO; sigemptyset(&act.sa_mask); sigaction(SIGSEGV, &act, NULL); Unix: Modern implementations are very complex e.g.
    [Show full text]
  • Signals and Inter-Process Communication
    2/21/20 COMP 790: OS Implementation COMP 790: OS Implementation Housekeeping Signals and Inter-Process • Paper reading assigned for next class Communication Don Porter 1 2 1 2 COMP 790: OS Implementation COMP 790: OS Implementation Logical Diagram Last time… Binary Memory • We’ve discussed how the OS schedules the CPU Threads Formats Allocators User – And how to block a process on a resource (disk, network) Today’s Lecture System Calls Kernel Process • Today: RCU FileCoordination System Networking Sync – How do processes block on each other? – And more generally communicate? Memory Device CPU Management Drivers Scheduler Hardware Interrupts Disk Net Consistency 3 4 3 4 COMP 790: OS Implementation COMP 790: OS Implementation Outline What is a signal? • Signals • Like an interrupt, but for applications – Overview and APIs – <64 numbers with specific meanings – Handlers – A process can raise a signal to another process or thread – Kernel-level delivery – A process or thread registers a handler function – Interrupted system calls • For both IPC and delivery of hardware exceptions • Interprocess Communication (IPC) – Application-level handlers: divzero, segfaults, etc. – Pipes and FIFOs • No “message” beyond the signal was raised – System V IPC – And maybe a little metadata – Windows Analogs • PID of sender, faulting address, etc. • But platform-specific (non-portable) 5 6 5 6 1 2/21/20 COMP 790: OS Implementation COMP 790: OS Implementation Example Example Pid 300 Pid 300 Pid 400 int main() { int main() { ... ... PC signal(SIGUSR1, &usr_handler);
    [Show full text]