Recall: POSIX I/O: Everything (looks like) a “File” CS162 • Identical interface for: Operating Systems and – Devices (terminals, printers, etc.) Systems Programming – Regular files on disk Lecture 9 – Networking (sockets) – Local interprocess communication (pipes, sockets) Sockets, Networking (Con’t) • Based on open(), read(), write(), and close() Scheduling • Allows simple composition of programs »find | grep | wc … • HOWEVER: Not every thing actually IS a file! February 20th, 2020 – Pipes are only buffered in memory! Prof. John Kubiatowicz – Network sockets only buffered in memory/network! http://cs162.eecs.Berkeley.edu 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.2 Recall: POSIX I/O Design Patterns Recall: Device Drivers • Open before use – Access control check, setup happens here • Device Driver: Device-specific code in the kernel that • Byte-oriented interacts directly with the device hardware – Least common denominator – Supports a standard, internal interface – OS responsible for hiding the fact that real devices may not work this – Same kernel I/O system can interact easily with different device way (e.g. hard drive stores data in blocks) drivers • Explicit close – Special device-specific configuration supported with the ioctl() system call • Reads are buffered – Part of making everything byte-oriented • Device Drivers typically divided into two pieces: – Process is blocked while waiting for device – Top half: accessed in call path from system calls – Let other processes run while gathering result » implements a set of standard, cross-device calls like open(), close(), read(), write(), ioctl(), strategy() • Writes are buffered » This is the kernel’s interface to the device driver – Complete in background (more later on) » Top half will start I/O to device, may put thread to sleep until – Return to user when data is “handed off” to kernel finished • Errors relayed to user in a variety of ways! – Bottom half: run as interrupt routine – Make sure to check them! » Gets input or transfers next block of output » May wake sleeping threads if I/O now complete 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.3 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.4 Recall: Life Cycle of An I/O Request Communication between processes User • Can we view files as communication channels? Program write(wfd, wbuf, wlen); Kernel I/O Subsystem n = read(rfd,rbuf,rmax); Device Driver Top Half • Producer and Consumer of a file may be distinct processes Device Driver – May be separated in time (or not) Bottom Half • However, what if data written once and consumed once? – Don’t we want something more like a queue? Device – Can still look like File I/O! Hardware 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.5 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.6 Communication Across the world looks like file IO Request Response Protocol Client (issues requests) Server (performs operations) write(wfd, wbuf, wlen); write(rqfd, rqbuf, buflen); requests n = read(rfd,rbuf,rmax); n = read(rfd,rbuf,rmax); wait service request • Connected queues over the Internet write(wfd, respbuf, len); – But what’s the analog of open? responses – What is the namespace? – How are they connected in time? n = read(resfd,resbuf,resmax); 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.7 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.8 Request Response Protocol Client-Server Models Client (issues requests) Server (performs operations) Client 1 write(rqfd, rqbuf, buflen); requests Client 2 Server n = read(rfd,rbuf,rmax); *** wait service request write(wfd, respbuf, len); Client n responses n = read(resfd,resbuf,resmax); • File servers, web, FTP, Databases, … • Many clients accessing a common server 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.9 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.10 Client-Server Communication Sockets • Socket: an abstraction of a network I/O queue – Mechanism for inter-process communication • Client “sometimes on” • Server is “always on” – Embodies one side of a communication channel – Initiates a request to the – Services requests from » Same interface regardless of location of other end server when interested many client hosts » Could be local machine (called “UNIX socket”) or remote machine – E.g., Web browser on your – E.g., Web server for the (called “network socket”) laptop or cell phone www.cnn.com Web site – First introduced in 4.2 BSD UNIX: big innovation at time » Now most operating systems provide some notion of socket – Doesn’t communicate – Doesn’t initiate contact with directly with other clients the clients • Data transfer like files – Needs to know the server’s – Needs a fixed, well-known – Read / Write against a descriptor address address • Over ANY kind of network – Local to a machine – Over the internet (TCP/IP, UDP/IP) – OSI, Appletalk, SNA, IPX, SIP, NS, … 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.11 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.12 Silly Echo Server – running example Echo client-server example void client(int sockfd) { Client (issues requests) Server (performs operations) int n; char sndbuf[MAXIN]; char rcvbuf[MAXOUT]; getreq(sndbuf, MAXIN); /* prompt */ gets(fd,sndbuf, …); while (strlen(sndbuf) > 0) { write(sockfd, sndbuf, strlen(sndbuf)); /* send */ requests memset(rcvbuf,0,MAXOUT); /* clear */ n=read(sockfd, rcvbuf, MAXOUT‐1); /* receive */ write(fd, buf,len); write(STDOUT_FILENO, rcvbuf, n); /* echo */ n = read(fd,buf,); getreq(sndbuf, MAXIN); /* prompt */ } } wait print write(fd, buf,); void server(int consockfd) { char reqbuf[MAXREQ]; int n; responses while (1) { memset(reqbuf,0, MAXREQ); n = read(consockfd,reqbuf,MAXREQ‐1); /* Recv */ n = read(fd,rcvbuf, ); if (n <= 0) return; n = write(STDOUT_FILENO, reqbuf, strlen(reqbuf)); print n = write(consockfd, reqbuf, strlen(reqbuf)); /* echo*/ } } 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.13 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.14 What assumptions are we making? Administrivia • Reliable • Midterm 1 is next Thursday (2/27)! – Please answer questions about conflicts by this Thursday – Write to a file => Read it back. Nothing is lost. » New conflict time and room up now. We are trying to handle all – Write to a (TCP) socket => Read from the other side, conflicts on Thursday 2/27 instead of Friday… same. – Topics: All material up to next Tuesday – Like pipes • Review Session: Tuesday (2/25) • In order (sequential stream) – Right here from 6:30-8:00pm (after class) – Write X then write Y => read gets X then read gets Y • When ready? – File read gets whatever is there at the time. Assumes writing already took place. – Like pipes! 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.15 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.16 Socket creation and connection Namespaces for communication over IP • File systems provide a collection of permanent objects • Hostname in structured name space – www.eecs.berkeley.edu – Processes open, read/write/close them • IP address – Files exist independent of the processes – 128.32.244.172 (ipv6?) • Sockets provide a means for processes to • Port Number communicate (transfer data) to other processes. – 0-1023 are “well known” or “system” ports • Creation and connection is more complex » Superuser privileges to bind to one • Form 2-way pipes between processes – 1024 – 49151 are “registered” ports (registry) – Possibly worlds away » Assigned by IANA for specific services • How do we name them? – 49152–65535 (215+214 to 216−1) are “dynamic” or • How do these completely independent programs know “private” that the other wants to “talk” to them? » Automatically allocated as “ephemeral Ports” 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.17 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.18 Socket Setup over TCP/IP Socket Setup over TCP/IP Server Server Socket Socket new socket new socket socket connection socket Connection socket connectionconnection socket Client Server Client Server • Server Socket: Listens for new connections – Produces new sockets for each unique connection – 3-way handshake to establish new connection! • Special kind of socket: server socket • Things to remember: – Has file descriptor – Connection involves 5 values: [ Client Addr, Client Port, Server Addr, Server Port, Protocol ] – Can’t read or write – Often, Client Port “randomly” assigned • Two operations: » Done by OS during client socket setup 1. listen(): Start allowing clients to connect – Server Port often “well known” » 80 (web), 443 (secure web), 25 (sendmail), etc 2. accept(): Create a new socket for a particular client » Well-known ports from 0—1023 connection 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.19 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.20 Web Server using Sockets (in concept) Client-Side of Protocol Client Server char *host_name, port_name; Create Server Socket // Create a socket Create Client Socket Bind it to an Address struct addrinfo *server = lookup_host(host_name, port_name); (host:port) int sock_fd = socket(server‐>ai_family, server‐>ai_socktype, server‐>ai_protocol); Connect it to server (host:port) Listen for Connection // Connect to specified host and port connect(sock_fd, server‐>ai_addr, server‐>ai_addrlen); Accept syscall() Connection Socket Connection Socket // Carry out Client‐Server protocol run_client(sock_fd); write request read request read response write response /* Clean up on termination */ close(sock_fd); Close Client Socket Close Connection Socket Close Server Socket 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.21 2/20/2020 Kubiatowicz CS162 ©UCB Fall 2020 Lec 9.22 Client: getting the server address (as addrinfo) Server Protocol (v1) struct addrinfo *lookup_host(char *host_name, char *port) { // Create socket to listen for client connections struct addrinfo *server; char *port_name; struct addrinfo hints; struct addrinfo *server = setup_address(port_name); int server_socket = socket(server‐>ai_family, server‐>ai_socktype, // Constraints on returned address server‐>ai_protocol); memset(&hints, 0, sizeof(hints)); hints.ai_family = AF_UNSPEC; // Either IPv4 or IPv6 // Bind socket to specific port hints.ai_socktype = SOCK_STREAM; // Reliable stream (i.e.