Chapter 2 Application Layer
A note on the use of these ppt slides: We’re making these slides freely available to all (faculty, students, readers). Computer They’re in PowerPoint form so you see the animations; and can add, modify, and delete slides (including this one) and slide content to suit your needs. Networking: A Top They obviously represent a lot of work on our part. In return for use, we only ask the following: Down Approach v If you use these slides (e.g., in a class) that you mention their source th (after all, we’d like people to use our book!) 6 edition v If you post any slides on a www site, that you note that they are adapted Jim Kurose, Keith Ross from (or perhaps identical to) our slides, and note our copyright of this Addison-Wesley material. March 2012 Thanks and enjoy! JFK/KWR
All material copyright 1996-2012 J.F Kurose and K.W. Ross, All Rights Reserved
Application Layer 2-1 Chapter 2: outline
2.1 principles of network 2.6 P2P applications applications 2.7 socket programming 2.2 Web and HTTP with UDP and TCP 2.3 FTP 2.4 electronic mail § SMTP, POP3, IMAP 2.5 DNS
Application Layer 2-2 Chapter 2: application layer our goals: v learn about protocols by v conceptual, examining popular implementation aspects application-level of network application protocols protocols § HTTP § transport-layer § FTP service models § SMTP / POP3 / IMAP § client-server § DNS paradigm v creating network § peer-to-peer applications paradigm § socket API
Application Layer 2-3 Some network apps v e-mail v voice over IP (e.g., Skype) v web v real-time video v text messaging conferencing v remote login v social networking v P2P file sharing v search v multi-user network games v … v streaming stored video v … (YouTube, Hulu, Netflix)
Application Layer 2-4 Creating a network app application transport network data link write programs that: physical v run on (different) end systems v communicate over network v e.g., web server software communicates with browser software
no need to write software for application transport network-core devices network data link application v physical transport network-core devices do not network data link run user applications physical v applications on end systems allows for rapid app development, propagation
Application Layer 2-5 Application architectures
possible structure of applications: v client-server v peer-to-peer (P2P)
Application Layer 2-6 Client-server architecture
server: v always-on host v permanent IP address v data centers for scaling clients: v communicate with server client/server v may be intermittently connected v may have dynamic IP addresses v do not communicate directly with each other
Application Layer 2-7 P2P architecture v no always-on server peer-peer v arbitrary end systems directly communicate v peers request service from other peers, provide service in return to other peers § self scalability – new peers bring new service capacity, as well as new service demands v peers are intermittently connected and change IP addresses § complex management
Application Layer 2-8 Processes communicating
process: program running clients, servers within a host client process: process that v within same host, two initiates communication processes communicate server process: process that using inter-process waits to be contacted communication (defined by OS) v processes in different hosts communicate by exchanging v aside: applications with P2P messages architectures have client processes & server processes
Application Layer 2-9 Sockets v process sends/receives messages to/from its socket v socket analogous to door § sending process shoves message out door § sending process relies on transport infrastructure on other side of door to deliver message to socket at receiving process
application application socket controlled by process process app developer
transport transport network network controlled by OS link Internet link physical physical
Application Layer 2-10 Addressing processes
v to receive messages, v identifier includes both IP process must have identifier address and port numbers v host device has unique 32- associated with process on bit IP address host. v Q: does IP address of host v example port numbers: on which process runs § HTTP server: 80 suffice for identifying the § mail server: 25 process? v to send HTTP message to § A: no, many processes gaia.cs.umass.edu web can be running on same server: host § IP address: 128.119.245.12 § port number: 80 v more shortly…
Application Layer 2-11 App-layer protocol defines
v types of messages open protocols: exchanged, v defined in RFCs § e.g., request, response v allows for interoperability v message syntax: v e.g., HTTP, SMTP § what fields in messages proprietary protocols: & how fields are delineated v e.g., Skype v message semantics § meaning of information in fields v rules for when and how processes send & respond to messages
Application Layer 2-12 What transport service does an app need? data integrity throughput v some apps (e.g., file transfer, v some apps (e.g., web transactions) require multimedia) require 100% reliable data transfer minimum amount of throughput to be v other apps (e.g., audio) can “ ” tolerate some loss effective v other apps (“elastic apps”) timing make use of whatever throughput they get v some apps (e.g., Internet telephony, interactive security games) require low delay to be “effective” v encryption, data integrity, …
Application Layer 2-13 Transport service requirements: common apps
application data loss throughput time sensitive
file transfer no loss elastic no e-mail no loss elastic no Web documents no loss elastic no real-time audio/video loss-tolerant audio: 5kbps-1Mbps yes, 100’s video:10kbps-5Mbps msec stored audio/video loss-tolerant same as above interactive games loss-tolerant few kbps up yes, few secs text messaging no loss elastic yes, 100’s msec yes and no
Application Layer 2-14 Internet transport protocols services
TCP service: UDP service: v reliable transport between v unreliable data transfer sending and receiving between sending and process receiving process v flow control: sender won’t overwhelm receiver v does not provide: reliability, flow control, v congestion control: throttle sender when network congestion control, overloaded timing, throughput guarantee, security, v does not provide: timing, minimum throughput orconnection setup, guarantee, security v connection-oriented: setup Q: why bother? Why is required between client and there a UDP? server processes
Application Layer 2-15 Internet apps: application, transport protocols
application underlying application layer protocol transport protocol
e-mail SMTP [RFC 2821] TCP remote terminal access Telnet [RFC 854] TCP Web HTTP [RFC 2616] TCP file transfer FTP [RFC 959] TCP streaming multimedia HTTP (e.g., YouTube), TCP or UDP RTP [RFC 1889] Internet telephony SIP, RTP, proprietary (e.g., Skype) TCP or UDP
Application Layer 2-16 Securing TCP
TCP & UDP SSL is at app layer v no encryption v Apps use SSL libraries, v cleartext passwds sent which “talk” to TCP into socket traverse SSL socket API Internet in cleartext v cleartext passwds sent SSL into socket traverse v provides encrypted Internet encrypted TCP connection v See Chapter 7 v data integrity v end-point authentication
Application Layer 2-17 Chapter 2: outline
2.1 principles of network 2.6 P2P applications applications 2.7 socket programming § app architectures with UDP and TCP § app requirements 2.2 Web and HTTP 2.3 FTP 2.4 electronic mail § SMTP, POP3, IMAP 2.5 DNS
Application Layer 2-18 Web and HTTP
First, a review… v web page consists of objects v object can be HTML file, JPEG image, Java applet, audio file,… v web page consists of base HTML-file which includes several referenced objects v each object is addressable by a URL, e.g., www.someschool.edu/someDept/pic.gif
host name path name
Application Layer 2-19 HTTP overview
HTTP: hypertext transfer protocol HTTP request v Web’s application layer protocol PC running HTTP response Firefox browser v client/server model § client: browser that requests, receives, (using HTTP protocol) and “displays” Web server HTTP request running objects Apache Web § server: Web server HTTP response server sends (using HTTP protocol) objects in iphone running response to requests Safari browser
Application Layer 2-20 HTTP overview (continued) uses TCP: HTTP is “stateless” v client initiates TCP v server maintains no connection (creates information about socket) to server, port 80 past client requests v server accepts TCP connection from client aside protocols that maintain v HTTP messages “state” are complex! (application-layer protocol v past history (state) must be messages) exchanged maintained between browser (HTTP v if server/client crashes, their client) and Web server views of “state” may be (HTTP server) inconsistent, must be v TCP connection closed reconciled
Application Layer 2-21 HTTP connections non-persistent HTTP persistent HTTP v at most one object v multiple objects can sent over TCP be sent over single connection TCP connection § connection then between client, server closed v downloading multiple objects required multiple connections
Application Layer 2-22 Non-persistent HTTP
suppose user enters URL: (contains text, www.someSchool.edu/someDepartment/home.index references to 10 jpeg images) 1a. HTTP client initiates TCP connection to HTTP server (process) at 1b. HTTP server at host www.someSchool.edu on port www.someSchool.edu waiting 80 for TCP connection at port 80. “accepts” connection, notifying 2. HTTP client sends HTTP request client message (containing URL) into TCP connection socket. 3. HTTP server receives request Message indicates that client message, forms response wants object message containing requested someDepartment/home.index object, and sends message into its socket time
Application Layer 2-23 Non-persistent HTTP (cont.)
4. HTTP server closes TCP connection. 5. HTTP client receives response message containing html file, displays html. Parsing html file, finds 10 referenced jpeg objects time 6. Steps 1-5 repeated for each of 10 jpeg objects
Application Layer 2-24 Non-persistent HTTP: response time
RTT (definition): time for a small packet to travel from client to server and back
HTTP response time: initiate TCP connection v one RTT to initiate TCP connection RTT request v one RTT for HTTP request file and first few bytes of HTTP time to RTT transmit response to return file v file transmission time file received v non-persistent HTTP response time = 2RTT+ file transmission time time time
Application Layer 2-25 Persistent HTTP non-persistent HTTP issues: persistent HTTP: v requires 2 RTTs per object v server leaves connection v OS overhead for each TCP open after sending connection response v browsers often open v subsequent HTTP parallel TCP connections messages between same to fetch referenced objects client/server sent over open connection v client sends requests as soon as it encounters a referenced object v as little as one RTT for all the referenced objects
Application Layer 2-26 HTTP request message
v two types of HTTP messages: request, response v HTTP request message: § ASCII (human-readable format) carriage return character line-feed character request line (GET, POST, GET /index.html HTTP/1.1\r\n HEAD commands) Host: www-net.cs.umass.edu\r\n User-Agent: Firefox/3.6.10\r\n header Accept: text/html,application/xhtml+xml\r\n Accept-Language: en-us,en;q=0.5\r\n lines Accept-Encoding: gzip,deflate\r\n Accept-Charset: ISO-8859-1,utf-8;q=0.7\r\n carriage return, Keep-Alive: 115\r\n line feed at start Connection: keep-alive\r\n \r\n of line indicates end of header lines
Application Layer 2-27 HTTP request message: general format
method sp URL sp version cr lf request line header field name value cr lf header ~ ~ ~ lines
header field name value cr lf cr lf
~ entity body ~ body
Application Layer 2-28 Uploading form input
POST method: v web page often includes form input v input is uploaded to server in entity body URL method: v uses GET method v input is uploaded in URL field of request line:
www.somesite.com/animalsearch?monkeys&banana
Application Layer 2-29 Method types
HTTP/1.0: HTTP/1.1: v GET v GET, POST, HEAD v POST v PUT v HEAD § uploads file in entity § asks server to leave body to path specified requested object out in URL field of response v DELETE § deletes file specified in the URL field
Application Layer 2-30 HTTP response message status line (protocol status code HTTP/1.1 200 OK\r\n status phrase) Date: Sun, 26 Sep 2010 20:09:20 GMT\r\n Server: Apache/2.0.52 (CentOS)\r\n Last-Modified: Tue, 30 Oct 2007 17:00:02 GMT\r\n header ETag: "17dc6-a5c-bf716880"\r\n lines Accept-Ranges: bytes\r\n Content-Length: 2652\r\n Keep-Alive: timeout=10, max=100\r\n Connection: Keep-Alive\r\n Content-Type: text/html; charset=ISO-8859- 1\r\n \r\n data, e.g., data data data data data ... requested HTML file
Application Layer 2-31 HTTP response status codes v status code appears in 1st line in server-to- client response message. v some sample codes: 200 OK § request succeeded, requested object later in this msg 301 Moved Permanently § requested object moved, new location specified later in this msg (Location:) 400 Bad Request § request msg not understood by server 404 Not Found § requested document not found on this server 505 HTTP Version Not Supported
Application Layer 2-32 Trying out HTTP (client side) for yourself
1. Telnet to your favorite Web server:
telnet cis.poly.edu 80 opens TCP connection to port 80 (default HTTP server port) at cis.poly.edu. anything typed in sent to port 80 at cis.poly.edu
2. type in a GET HTTP request:
GET /~ross/ HTTP/1.1 by typing this in (hit carriage Host: cis.poly.edu return twice), you send this minimal (but complete) GET request to HTTP server 3. look at response message sent by HTTP server! (or use Wireshark to look at captured HTTP request/response) Application Layer 2-33 User-server state: cookies
example: many Web sites use cookies v Susan always access Internet four components: from PC 1) cookie header line of v visits specific e-commerce HTTP response site for first time message v when initial HTTP requests 2) cookie header line in arrives at site, site creates: next HTTP request § unique ID message § entry in backend 3) cookie file kept on database for ID user’s host, managed by user’s browser 4) back-end database at Web site
Application Layer 2-34 Cookies: keeping “state” (cont.)
client server
ebay 8734 usual http request msg Amazon server cookie file creates ID usual http response backend set-cookie: 1678 1678 for user create ebay 8734 entry database amazon 1678 usual http request msg cookie: 1678 cookie- access specific usual http response msg action one week later: access ebay 8734 usual http request msg amazon 1678 cookie: 1678 cookie- specific usual http response msg action Application Layer 2-35 Cookies (continued) aside what cookies can be used cookies and privacy: for: v cookies permit sites to v authorization learn a lot about you v shopping carts v you may supply name and v recommendations e-mail to sites v user session state (Web e-mail) how to keep “state”: v protocol endpoints: maintain state at sender/receiver over multiple transactions v cookies: http messages carry state
Application Layer 2-36 Web caches (proxy server) goal: satisfy client request without involving origin server v user sets browser: Web accesses via cache v browser sends all HTTP HTTP requestproxy requests to cache server clientHTTP response § object in cache: cache HTTP request origin returns object HTTP response server § else cache requests
object from origin HTTP request server, then returns object to client HTTP response client origin server
Application Layer 2-37 More about Web caching
v cache acts as both why Web caching? client and server v reduce response time § server for original for client request requesting client § client to origin server v reduce traffic on an v typically cache is institution’s access link installed by ISP v Internet dense with (university, company, caches: enables “poor” residential ISP) content providers to effectively deliver content (so too does P2P file sharing)
Application Layer 2-38 Caching example: assumptions: v avg object size: 100K bits origin v avg request rate from browsers to servers origin servers:15/sec public v avg data rate to browsers: 1.50 Mbps Internet v RTT from institutional router to any origin server: 2 sec v access link rate: 1.54 Mbps 1.54 Mbps consequences: access link v LAN utilization: 15% problem! institutional network v access link utilization = 99% 1 Gbps LAN v total delay = Internet delay + access delay + LAN delay = 2 sec + minutes + usecs
Application Layer 2-39 Caching example: fatter access link assumptions: v avg object size: 100K bits origin v avg request rate from browsers to servers origin servers:15/sec public v avg data rate to browsers: 1.50 Mbps Internet v RTT from institutional router to any origin server: 2 sec v access link rate: 1.54 Mbps 1.54 Mbps 154 Mbps 154 Mbps consequences: access link v LAN utilization: 15% institutional network v access link utilization = 99% 9.9% 1 Gbps LAN v total delay = Internet delay + access delay + LAN delay = 2 sec + minutes + usecs msecs Cost: increased access link speed (not cheap!) Application Layer 2-40 Caching example: install local cache assumptions: v avg object size: 100K bits origin v avg request rate from browsers to servers origin servers:15/sec public v avg data rate to browsers: 1.50 Mbps Internet v RTT from institutional router to any origin server: 2 sec v access link rate: 1.54 Mbps 1.54 Mbps consequences: access link v LAN utilization: 15% institutional network v access link utilization = 100%? 1 Gbps LAN v total delay = Internet? delay + access delay + LAN delay local web How to compute link cache = 2 secutilization, + minutes + delay? usecs Cost: web cache (cheap!) Application Layer 2-41 Caching example: install local cache
Calculating access link utilization, delay with cache: origin v suppose cache hit rate is 0.4 servers § 40% requests satisfied at cache, public 60% requests satisfied at origin Internet v access link utilization: § 60% of requests use access link v data rate to browsers over access link 1.54 Mbps = 0.6*1.50 Mbps = .9 Mbps access link § utilization = 0.9/1.54 = .58 institutional network v total delay 1 Gbps LAN § = 0.6 * (delay from origin servers) +0.4 * (delay when satisfied at cache) local web § = 0.6 (2.01) + 0.4 (~msecs) cache § = ~ 1.2 secs § less than with 154 Mbps link (and cheaper too!) Application Layer 2-42 Conditional GET
client server v Goal: don’t send object if cache has up-to-date cached version HTTP request msg If-modified-since:
2.1 principles of network 2.6 P2P applications applications 2.7 socket programming § app architectures with UDP and TCP § app requirements 2.2 Web and HTTP 2.3 FTP 2.4 electronic mail § SMTP, POP3, IMAP 2.5 DNS
Application Layer 2-44 FTP: the file transfer protocol
file transfer FTP FTP FTP user client server interface user at host remote file local file system system
v transfer file to/from remote host v client/server model § client: side that initiates transfer (either to/from remote) § server: remote host v ftp: RFC 959 v ftp server: port 21
Application Layer 2-45 FTP: separate control, data connections
TCP control connection, v FTP client contacts FTP server server port 21 at port 21, using TCP v client authorized over control TCP data connection, connection FTP server port 20 FTP client server v client browses remote directory, sends commands over control connection v server opens another TCP data connection to transfer v when server receives file another file transfer command, server opens 2nd TCP data v control connection: “out of connection (for file) to client band” v after transferring one file, v FTP server maintains server closes data connection “state”: current directory, earlier authentication
Application Layer 2-46 FTP commands, responses
sample commands: sample return codes v sent as ASCII text over v status code and phrase (as control channel in HTTP) v USER username v 331 Username OK, v PASS password password required v 125 data v LIST return list of file in current directory connection already open; v RETR filename transfer starting retrieves (gets) file v 425 Can’t open v STOR filename stores data connection (puts) file onto remote v 452 Error writing host file
Application Layer 2-47 Chapter 2: outline
2.1 principles of network 2.6 P2P applications applications 2.7 socket programming § app architectures with UDP and TCP § app requirements 2.2 Web and HTTP 2.3 FTP 2.4 electronic mail § SMTP, POP3, IMAP 2.5 DNS
Application Layer 2-48 outgoing Electronic mail message queue user mailbox Three major components: user agent v user agents v mail servers mail user server agent v simple mail transfer SMTP protocol: SMTP mail user server agent User Agent SMTP v a.k.a. “mail reader” SMTP user agent v composing, editing, reading mail server mail messages user v e.g., Outlook, Thunderbird, agent iPhone mail client user agent v outgoing, incoming messages stored on server Application Layer 2-49 Electronic mail: mail servers
mail servers: user agent v mailbox contains incoming messages for user mail user server agent v message queue of outgoing SMTP (to be sent) mail messages mail user v SMTP protocol between server agent mail servers to send email SMTP messages SMTP user agent § client: sending mail mail server server § “ ” user server : receiving mail agent server user agent
Application Layer 2-50 Electronic Mail: SMTP [RFC 2821]
v uses TCP to reliably transfer email message from client to server, port 25 v direct transfer: sending server to receiving server v three phases of transfer § handshaking (greeting) § transfer of messages § closure v command/response interaction (like HTTP, FTP) § commands: ASCII text § response: status code and phrase v messages must be in 7-bit ASCI
Application Layer 2-51 Scenario: Alice sends message to Bob
1) Alice uses UA to compose 4) SMTP client sends Alice’s message “to” message over the TCP [email protected] connection 2) Alice’s UA sends message 5) Bob’s mail server places the to her mail server; message message in Bob’s mailbox placed in message queue 6) Bob invokes his user agent 3) client side of SMTP opens to read message TCP connection with Bob’s mail server
1 user mail user mail agent agent server server 2 3 4 6 5 Alice’s mail server Bob’s mail server Application Layer 2-52 Sample SMTP interaction
S: 220 hamburger.edu C: HELO crepes.fr S: 250 Hello crepes.fr, pleased to meet you C: MAIL FROM:
Application Layer 2-53 Try SMTP interaction for yourself:
v telnet servername 25 v see 220 reply from server v enter HELO, MAIL FROM, RCPT TO, DATA, QUIT commands
above lets you send email without using email client (reader)
Application Layer 2-54 SMTP: final words v SMTP uses persistent comparison with HTTP: connections v HTTP: pull v SMTP requires message (header & body) to be in v SMTP: push 7-bit ASCII v both have ASCII v SMTP server uses command/response CRLF.CRLF to interaction, status codes determine end of message v HTTP: each object encapsulated in its own response msg v SMTP: multiple objects sent in multipart msg
Application Layer 2-55 Mail message format
SMTP: protocol for exchanging email msgs header blank RFC 822: standard for text line message format: v header lines, e.g., § To: body § From: § Subject: different from SMTP MAIL FROM, RCPT TO: commands! v Body: the “message” § ASCII characters only
Application Layer 2-56 Mail access protocols
mail access user SMTP SMTP user agent protocol agent (e.g., POP, IMAP)
sender’s mail receiver’s mail server server
v SMTP: delivery/storage to receiver’s server v mail access protocol: retrieval from server § POP: Post Office Protocol [RFC 1939]: authorization, download § IMAP: Internet Mail Access Protocol [RFC 1730]: more features, including manipulation of stored msgs on server § HTTP: gmail, Hotmail, Yahoo! Mail, etc.
Application Layer 2-57 POP3 protocol S: +OK POP3 server ready C: user bob authorization phase S: +OK C: pass hungry v client commands: S: +OK user successfully logged on § user: declare username § pass: password C: list S: 1 498 v server responses S: 2 912 § +OK S: . § -ERR C: retr 1 transaction phase, client: S:
Application Layer 2-58 POP3 (more) and IMAP
more about POP3 IMAP v previous example uses v keeps all messages in one POP3 “download and place: at server ” delete mode v allows user to organize § Bob cannot re-read e- messages in folders mail if he changes v keeps user state across client sessions: v POP3 “download-and- § names of folders and keep”: copies of messages mappings between on different clients message IDs and folder v POP3 is stateless across name sessions
Application Layer 2-59