DOCSLIB.ORG

HTTP Transactions

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
HTTP Transactions 05 4547 ch03 1/24/03 8:44 AM Page 29 3 HTTP Transactions HTTP TRAFFIC CONSISTS OF REQUESTS AND RESPONSES. All HTTP traffic can be associated with the task of requesting content or responding to those requests. Every HTTP message sent from a Web browser to a Web server is classified as an HTTP request, whereas every message sent from a Web server to a Web browser is classified as an HTTP response. HTTP is often referred to as a stateless protocol.Although this is accurate, it does little to explain the nature of the Web.All this means, however, is that each transaction is atomic, and there is nothing required by HTTP that associates one request with another. A transaction refers to a single HTTP request and the corresponding HTTP response. Another fundamental topic related to the nature of the Web is the topic of connections. Connections When I speak of a connection in HTTP,I refer to a TCP connection.As illustrated in Figure 3.1, a TCP connection requires three separate messages. TCP SYN TCP SYN + ACK TCP ACK Figure 3.1 A TCP connection requires three messages. SYN and ACK are two flags within the TCP segment of a packet. Because TCP is such a common transport layer protocol to be used in conjunction with IP,the combined pack- et of an IP packet containing a TCP segment is sometimes called a TCP/IP packet, even though it would best be described as a packet within a packet. By this example, you can see that a connection is unlike what you might otherwise expect.After this exchange, both computers simply consider themselves connected. In terms of HTTP,this simply means the server is ready to receive requests from this specific client.There is no real 05 4547 ch03 1/24/03 8:44 AM Page 30 30 Chapter 3 HTTP Transactions active connection in the traditional sense. It is better described as an understanding between the two computers that they are connected. An example of this type of connection is a two-way radio. If you and a friend both have two-way radios, you can establish a similar method for ensuring that you are both able to send and receive messages properly.To do this, you can send a message (by talking into the radio) asking to establish a connection.Your friend sends back a confirmation message acknowledging your request and agreeing to the connection.At this point, you feel confident that each of you can both send and receive messages, but your friend can- not be assured of this without knowing whether you received the confirmation.You send back a final message acknowledging the receipt of your friend’s confirmation.At this point, you both have confidence in your ability to communicate with these radios. This series of events is very similar to a TCP connection. Note A single connection can support multiple HTTP transactions. In many cases, multiple HTTP transactions are required to properly render a URL in a Web browser due to images and other associated content. Example HTTP Request It is probably easiest to get an idea about what HTTP is by looking at a few examples. Using my Galeon 1.2.0 browser, I type in the URL http://127.0.0.1/ and press Enter.This is actually a request to the Web server running on my local computer (127.0.0.1 is a special IP address called the loopback address).The request that my browser sends is as follows: GET / HTTP/1.1 Host: 127.0.0.1 User-Agent: Mozilla/5.0 Galeon/1.2.0 (X11; Linux i686; U;) Gecko/20020326 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9, text/plain;q=0.8,video/x-mng,image/png,image/jpeg,image/gif;q=0.2, text/css,*/*;q=0.1 Accept-Language: en Accept-Encoding: gzip, deflate, compress;q=0.9 Accept-Charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66 Keep-Alive: 300 Connection: keep-alive 05 4547 ch03 1/24/03 8:44 AM Page 31 Connections 31 Example HTTP Response In this example, my Web server gives the following response: HTTP/1.1 200 OK Date: Tue, 21 May 2002 12:34:56 GMT Server: Apache/1.3.22 (Unix) (Red-Hat/Linux) mod_python/2.7.8 Python/1.5.2 mod_ssl/2.8.5 OpenSSL/0.9.6b DAV/1.0.2 PHP/4.0.6 mod_perl/1.26 mod_throttle/3.1.2 Last-Modified: Thu, 01 Nov 2001 20:51:45 GMT ETag: “df6b0-b4a-3be1b5e1” Accept-Ranges: bytes Content-Length: 2890 Connection: close Content-Type: text/html <!DOCTYPE HTML PUBLIC “-//W3C//DTD HTML 3.2 Final//EN”> <html> <head> <title>Test Page for the Apache Web Server on Red Hat Linux</title> </head> <body bgcolor=”#ffffff”> (...) </body> </html> The majority of the response is HTML (omitted for brevity). Only the first few lines are HTTP.Thus, as intended, HTTP does not have much overhead. Lower-level protocols such as TCP and IP have even less overhead than HTTP,however, due mostly to the fact that HTTP is intentionally readable.This makes it easy to study and comprehend. Example Transaction A good example transaction to review is a search on Google. Being one of the most popular sites on the Web, most people have interacted with this site at one time or another.When performing a search on HTTP (see Figure 3.2), you enter HTTP into the form field and click the button labeled Google Search. When using my Web browser to perform this search, the following HTTP request is sent when I click the button: GET /search?hl=en&q=HTTP&btnG=Google+Search HTTP/1.1 Host: www.google.com User-Agent: Mozilla/5.0 Galeon/1.2.0 (X11; Linux i686; U;) Gecko/20020326 Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9, text/plain;q=0.8,video/x-mng,image/png,image/jpeg,image/gif;q=0.2, text/css,*/*;q=0.1 Accept-Language: en Accept-Encoding: gzip, deflate, compress;q=0.9 05 4547 ch03 1/24/03 8:44 AM Page 32 32 Chapter 3 HTTP Transactions Accept-Charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66 Keep-Alive: 300 Connection: keep-alive Figure 3.2 Searching Google for the term “HTTP”. Google’s Web site responds: HTTP/1.1 200 OK Server: GWS/2.0 Date: Tue, 21 May 2002 12:34:56 GMT Transfer-Encoding: chunked Content-Encoding: gzip Content-Type: text/html Cache-control: private Set-Cookie: PREF=ID=58c005a7065c0996:TM=1021283456:LM=1021283456:S=OLJcXi3RhSE; domain=.google.com; path=/; expires=Sun, 17-Jan-2038 19:14:07 GMT (Web content compressed with gzip) Of interest in this response is that the Web content is of a format that cannot be printed, binary. Because my browser specified in its request that it accepts gzip (GNU zip) encoding, Google chose to encode the response with gzip.This is a popular compression algorithm that allows for a quicker transfer due to the smaller size of the HTTP response. My browser decompresses the content in order to reveal the HTML it needs to render the Web page (the results of my search). 05 4547 ch03 1/24/03 8:44 AM Page 33 Summary 33 Summary This should give you a good introduction to the basics of HTTP transactions. It is important to remember that an HTTP transaction consists of a single HTTP request and the corresponding HTTP response, but that multiple transactions are sometimes required for a single Web page to be ultimately rendered in your Web browser. In the next chap- ter, I explain some Web development techniques that leverage knowledge of HTTP to make Web development easier and of higher quality. 05 4547 ch03 1/24/03 8:44 AM Page 34.
Load more

Recommended publications
  • Browser Versions Carry 10.5 Bits of Identifying Information on Average [Forthcoming Blog Post]
    Browser versions carry 10.5 bits of identifying information on average [forthcoming blog post] Technical Analysis by Peter Eckersley This is part 3 of a series of posts on user tracking on the modern web. You can also read part 1 and part 2. Whenever you visit a web page, your browser sends a "User Agent" header to the website saying what precise operating system and browser you are using. We recently ran an experiment to see to what extent this information could be used to track people (for instance, if someone deletes their browser cookies, would the User Agent, alone or in combination with some other detail, be enough to re-create their old cookie?). Our experiment to date has shown that the browser User Agent string usually carries 5-15 bits of identifying information (about 10.5 bits on average). That means that on average, only one person in about 1,500 (210.5) will have the same User Agent as you. On its own, that isn't enough to recreate cookies and track people perfectly, but in combination with another detail like an IP address, geolocation to a particular ZIP code, or having an uncommon browser plugin installed, the User Agent string becomes a real privacy problem. User Agents: An Example of Browser Characteristics Doubling As Tracking Tools When we analyse the privacy of web users, we usually focus on user accounts, cookies, and IP addresses, because those are the usual means by which a request to a web server can be associated with other requests and/or linked back to an individual human being, computer, or local network.
    [Show full text]
  • Longitudinal Characterization of Browsing Habits
    Please cite this paper as: Luca Vassio, Idilio Drago, Marco Mellia, Zied Ben Houidi, and Mohamed Lamine Lamali. 2018. You, the Web, and Your Device: Longitudinal Charac- terization of Browsing Habits. ACM Trans. Web 12, 4, Article 24 (November 2018), 30 1 pages. https://doi.org/10.1145/3231466 You, the Web and Your Device: Longitudinal Characterization of Browsing Habits LUCA VASSIO, Politecnico di Torino IDILIO DRAGO, Politecnico di Torino MARCO MELLIA, Politecnico di Torino ZIED BEN HOUIDI, Nokia Bell Labs MOHAMED LAMINE LAMALI, LaBRI, Université de Bordeaux Understanding how people interact with the web is key for a variety of applications – e.g., from the design of eective web pages to the denition of successful online marketing campaigns. Browsing behavior has been traditionally represented and studied by means of clickstreams, i.e., graphs whose vertices are web pages, and edges are the paths followed by users. Obtaining large and representative data to extract clickstreams is however challenging. The evolution of the web questions whether browsing behavior is changing and, by consequence, whether properties of clickstreams are changing. This paper presents a longitudinal study of clickstreams in from 2013 to 2016. We evaluate an anonymized dataset of HTTP traces captured in a large ISP, where thousands of households are connected. We rst propose a methodology to identify actual URLs requested by users from the massive set of requests automatically red by browsers when rendering web pages. Then, we characterize web usage patterns and clickstreams, taking into account both the temporal evolution and the impact of the device used to explore the web.
    [Show full text]
  • RSA Adaptive Authentication (On-Premise) 7.2 Integration Guide
    RSA® Adaptive Authentication (On-Premise) 7.2 Integration Guide Contact Information Go to the RSA corporate website for regional Customer Support telephone and fax numbers: www.emc.com/domains/rsa/index.htm Trademarks RSA, the RSA Logo, BSAFE and EMC are either registered trademarks or trademarks of EMC Corporation in the United States and/or other countries. All other trademarks used herein are the property of their respective owners. For a list of EMC trademarks, go to www.emc.com/legal/emc-corporation-trademarks.htm#rsa. License agreement This software and the associated documentation are proprietary and confidential to EMC, are furnished under license, and may be used and copied only in accordance with the terms of such license and with the inclusion of the copyright notice below. This software and the documentation, and any copies thereof, may not be provided or otherwise made available to any other person. No title to or ownership of the software or documentation or any intellectual property rights thereto is hereby transferred. Any unauthorized use or reproduction of this software and the documentation may be subject to civil and/or criminal liability. This software is subject to change without notice and should not be construed as a commitment by EMC. Note on encryption technologies This product may contain encryption technology. Many countries prohibit or restrict the use, import, or export of encryption technologies, and current use, import, and export regulations should be followed when using, importing or exporting this product. Distribution Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.
    [Show full text]
  • Lab Exercise – HTTP
    Lab Exercise – HTTP Objective HTTP (HyperText Transfer Protocol) is the main protocol underlying the Web. HTTP functions as a re- quest–response protocol in the client–server computing model. A web browser, for example, may be the client and an application running on a computer hosting a website may be the server. The client submits an HTTP request message to the server. The server, which provides resources such as HTML files and other content, or performs other functions on behalf of the client, returns a response message to the client. The response contains completion status information about the request and may also contain requested con- tent in its message body. A web browser is an example of a user agent (UA). Other types of user agent include the indexing software used by search providers (web crawlers), voice browsers, mobile apps, and other software that accesses, consumes, or displays web content. HTTP is designed to permit intermediate network elements to improve or enable communications between clients and servers. High-traffic web- sites often benefit from web cache servers that deliver content on behalf of upstream servers to improve response time. Web browsers cache previously accessed web resources and reuse them when possible to reduce network traffic. HTTP is an application layer protocol designed within the framework of the Inter- net protocol suite. Its definition presumes an underlying and reliable transport layer protocol and Trans- mission Control Protocol (TCP) is commonly used. Step 1: Open the http Trace Browser behavior can be quite complex, using more HTTP features than the basic exchange, this trace will show us how much gets transferred.
    [Show full text]
  • Working with User Agent Strings in Stata: the Parseuas Command
    JSS Journal of Statistical Software February 2020, Volume 92, Code Snippet 1. doi: 10.18637/jss.v092.c01 Working with User Agent Strings in Stata: The parseuas Command Joss Roßmann Tobias Gummer Lars Kaczmirek GESIS – Leibniz Institute GESIS – Leibniz Institute University of Vienna for the Social Sciences for the Social Sciences Abstract With the rising popularity of web surveys and the increasing use of paradata by survey methodologists, assessing information stored in user agent strings becomes inevitable. These data contain meaningful information about the browser, operating system, and device that a survey respondent uses. This article provides an overview of user agent strings, their specific structure and history, how they can be obtained when conducting a web survey, as well as what kind of information can be extracted from the strings. Further, the user written command parseuas is introduced as an efficient means to gather detailed information from user agent strings. The application of parseuas is illustrated by an example that draws on a pooled data set consisting of 29 web surveys. Keywords: user agent string, web surveys, device detection, browser, paradata, Stata. 1. Introduction In recent years, web surveys have become an increasingly popular and important data collec- tion mode, and today, they account for a great share of the studies in the social sciences. Web surveys have been used to complement traditional offline surveys as a less expensive form of pre-test or a mode of choice for respondents who are not willing to use “traditional” modes, such as face-to-face or telephone interviews. Along with the rising popularity of web surveys, survey methodologists have taken an increas- ing interest in paradata, which are collected as a byproduct of the survey process (Couper 2000).
    [Show full text]
  • Why Is XHTML Needed? Isn't HTML Good Enough?
    Why is XHTML needed? Isn't HTML good enough? HTML is probably the most successful document markup language in the world. But when XML was introduced, a two-day workshop was organised to discuss whether a new version of HTML in XML was needed. The opinion at the workshop was a clear 'Yes': with an XML-based HTML other XML languages could include bits of XHTML, and XHTML documents could include bits of other markup languages. We could also take advantage of the redesign to clean up some of the more untidy parts of HTML, and add some new needed functionality, like better forms. What are the advantages of using XHTML rather than HTML? If your document is just pure XHTML 1.0 (not including other markup languages) then you will not yet notice much difference. However as more and more XML tools become available, such as XSLT for tranforming documents, you will start noticing the advantages of using XHTML. XForms for instance will allow you to edit XHTML documents (or any other sort of XML document) in simple controllable ways. Semantic Web applications will be able to take advantage of XHTML documents. If your document is more than XHTML 1.0, for instance including MathML, SMIL, or SVG, then the advantages are immediate: you can't do that sort of thing with HTML. Can I just put the XML declaration on top of existing HTML documents? Can I intermix HTML 4.01 and XHTML documents? No. HTML is not in XML format. You have to make the changes necessary to make the document proper XML before you can get it accepted as XML.
    [Show full text]
  • Economic and Technical Drivers of Technology Choice: Browsers
    Economic and Technical Drivers of Technology Choice: Browsers (Working paper: please do not cite without permission from authors) Timothy F. Bresnahan and Pai-Ling Yin First Draft: Nov 05, 2003 Abstract The diffusion of new technologies is their adoption by different economic agents at different times. A classical concern in the diffusion of technologies (Griliches, 1957) is the importance of raw technical progress versus economic forces. We examine this classical issue in the modern market of web browsers. Using a new data source, we study browser brand shares and the diffusion of new browser versions. Features of that market also generate novel questions in the economics of diffusion. We find that browser distribution via the expanding market for a complementary technology, personal computers (PCs), had a larger effect on the rate and direction of technical change than technical browser improvements. Timothy F. Bresnahan Pai-Ling Yin Landau Professor in Technology and the Economy Harvard Business School Stanford University Morgan Hall 241 Department of Economics Soldiers Field 579 Serra St. Boston MA 02163 Stanford, CA, 94305 USA [email protected] [email protected] 1 1) Introduction A new invention creates a technological opportunity. The diffusion of the new technology to the economic agents who will use it determines the rate and direction of realized technical change in the economy. We have known since the work of Griliches (1957) that both economic and technical forces shape diffusion. Invention is only the beginning of technical progress. Inventors introduce new technologies into the field and start their diffusion. The movement of the overall economy toward realizing a technological opportunity depends on the behavior of the adopters of the technology.
    [Show full text]
  • Information Systems Modeling
    Information systems modeling Tomasz Kubik Design Pattern “Each pattern describes a problem which occurs over and over again in our environment and then describes the core of the solution to that problem, in such a way that you can use this solution a million times over, without ever doing it in the same way twice” Ch.W. Alexander, S. Ishikawa, M. Silverstein, M. Jacobson, I. Fiksdahl-King, S. Angel: A Pattern Language: Towns, Buildings, Construction, 1977 Design patterns are “descriptions of communicating objects and classes that are customized to solve a general design problem in a particular context.” E. Gamma, R. Helm, R. Johnson, J. Vlissides (Gang of Four): Design Patterns: Elements of Reusable Object-Oriented Software, 1994 Design patterns can be expressed at various abstraction levels – OO programming (like GoF patterns) • Structure, Creational, and Behavioral – Design and implementation of the multi-tier software • Presentation, Business and Integration T.Kubik: ISM Design Pattern “All well-structured systems are full of patterns. A pattern provides a good solution to a common problem in a given context. A mechanism is a design pattern that applies to a society of classes; a framework is typically an architectural pattern that provides an extensible template for applications within a domain. You use patterns to specify mechanisms and frameworks that shape the architecture of your system. You make a pattern approachable by clearly identifying the slots, tabs, knobs, and dials that a user of that pattern may adjust to apply the pattern
    [Show full text]
  • HTML, CSS, & Javascript Mobile Development for Dummies
    Mobile Development/HTML ™ Making Everything Easier! Master the art of designing web pages for mobile devices — Open the book and find: a site for small screens! & JavaScript CSS HTML, • Why you should know WURFL Mobile Development HTML, CSS ® When designing a web page for mobile devices, the big thing • A system for keeping your site is — think small! Your objective is to provide what the mobile up to date user wants without losing the “wow” in your website. This • All about bitmap and vector book shows you how to do it using three key technologies. & JavaScript images Soon you’ll be building mobile pages with forms, quizzes, appropriate graphics, shopping carts, and more! • Easy ways to adjust your site for different devices Mobile Development • Think mobile — consider screen size, lack of a mouse, duel • Powerful SEO ideas to get your orientation screens, and mobile browsers site noticed • Know your audience — understand how people use the mobile • Tips for creating a mobile web and how their habits differ from those of desktop users shopping cart • Get interactive — optimize multimedia files and develop contact • How to take your blog theme forms that encourage visitors to interact with your site mobile ® • Latest and greatest — maximize the new features of HTML5 and CSS3, automate your site with JavaScript, and use WebKit Extensions • Ten mobile CSS-friendly apps and widgets • Be sure they find you — make your mobile site both easily searchable and search engine-friendly Learn to: • Use standard web tools to build sites Go to Dummies.com® for iPhone®, iPad®, BlackBerry®, and Visit the companion website at www.wiley.com/go/ for videos, step-by-step examples, Android ™ platforms htmlcssjscriptmobiledev for code samples you can how-to articles, or to shop! use when creating your mobile sites.
    [Show full text]
  • Economic and Technical Drivers of Technology Choice: Browsers
    Economic and Technical Drivers of Technology Choice: Browsers Timothy F. Bresnahan and Pai-Ling Yin First Draft: Nov 05, 2003 This Draft: August 12, 2005 Abstract The diffusion of new technologies is their adoption by different economic agents at different times. A classical concern in the diffusion of technologies (Griliches 1957) is the importance of raw technical progress versus economic forces. We examine this classical issue in a modern market, web browsers. Using a new data source, we study the diffusion of new browser versions. In a second analysis, we study the determination of browser brand shares. Both analyses let us examine the important of technical progress vs. economic forces. We find that the critical economic force was browser distribution with a complementary technology, personal computers (PCs). In both of our analyses, distribution had a larger effect on the rate and direction of technical change than technical browser improvements. This shows an important feedback. Widespread use of the Internet spurred rapid expansion of the PC market in the late 1990s. Our results show that the rapid expansion in PCs in turn served to increase the pace of diffusion of new browsers and thus move the economy toward new mass market commercial Internet use. Timothy F. Bresnahan Pai-Ling Yin Landau Professor in Technology and the Economy Harvard Business School Stanford University Morgan Hall 241 Department of Economics Soldiers Field 579 Serra St. Boston MA 02163 Stanford, CA, 94305 USA [email protected] [email protected] 1) Introduction A new invention creates a technological opportunity. The diffusion of the new technology to the economic agents who will use it determines the rate and direction of realized technical change in the economy.
    [Show full text]
  • 3 User-Agent 17 3.1 Format
    Masaryk University Faculty of Informatics Detection of network attacks using HTTP related information Master’s Thesis Lenka Kuníková Brno, Spring 2017 Masaryk University Faculty of Informatics Detection of network attacks using HTTP related information Master’s Thesis Lenka Kuníková Brno, Spring 2017 This is where a copy of the official signed thesis assignment and a copy ofthe Statement of an Author is located in the printed version of the document. Declaration Hereby I declare that this paper is my original authorial work, which I have worked out on my own. All sources, references, and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source. Lenka Kuníková Advisor: RNDr. Pavel Minařík PhD. i Acknowledgement I would like to thank to my advisor RNDr. Pavel Minařík, PhD. and to Mgr. Martin Juřen for guidance and useful advices. iii Abstract This thesis deals with extended HTTP network flows and their ap- plication for the detection of various attacks and anomalies on the network. It highlights advantages of extended HTTP flows on cho- sen attacks, it implements and tests existing detection methods and suggests numerous improvements. Furthermore the thesis analyses in detail User-Agent request header. It describes possibilities of how this field can by used for anomaly detection and explains problems related to User-Agent analysis. iv Keywords Flow monitoring, HTTP, Anomaly detection, User-Agent, Brute-force attack, SQL injection v Contents 1 Introduction 1 2 HTTP 3 2.1 Basic concepts .........................3 2.2 URI ..............................4 2.3 Message format ........................5 2.3.1 HTTP request .
    [Show full text]
  • The Complete Guide to User Agents
    THE COMPLETE GUIDE TO USER AGENTS WHAT THEY ARE, HOW TO READ THEM, AND HOW TO USE THEM. CONTENTS 03 WHAT IS A USER AGENT? 04 HOW DOES USER AGENT PARSING WORK? 05 WHAT CAN YOU DO WITH USER AGENTS? 06 BUILDING A REGEX DEVICE DETECTION SOLUTION 07 THIRD-PARTY CLOUD BASED SOLUTION 08 LOCALLY INSTALLED DEVICE DETECTION 09 MOST COMMON USER AGENTS The Complete Guide To User-Agents What is a User-Agent? A User-Agent (UA) is an alphanumeric string that identifies the ‘agent’ or program making a request to a web server for an asset such as a document, image or web page. It is a standard part of web architecture and is passed by all web requests in the HTTP headers. The User-Agent string is very useful because it gives you information about the software and hardware running on the device making the request. You can make important decisions on how to handle web traffic based on the User-Agent string, ranging from simple segmentation and redirection, to more complex content adaptation and device targeting decisions. Even more information, such as carrier ID, screen resolution, CPU and storage capacity can be returned when the User-Agent string is mapped to an additional set of data, returned in real-time. This is the advantage of using DeviceAtlas for your detection requirements. Anatomy of a User-Agent Use of the User-Agent string is specified in the standards on HTTP here - RFC 1945. In fact, the UA string has been part of the HTTP standard since the very first version, and has been retained in every update since, right up to HTTP/2.
    [Show full text]