Moving Edge-Side Includes to the Real Edge—the Clients
y z Michael Rabinovichy Zhen Xiao Fred Douglis
Chuck Kalmaneky
yAT&T Labs – Research
zIBM T.J. Watson Research Center
Abstract 1.1 Fragment-based Technologies
Edge-Side Includes (ESI) is an open mark-up lan- As an example of fragment usage, consider AT&T’s www.att.com guage that allows content providers to break their pages home page, , shown in Figure 1. One into fragments with individual caching characteristics. can identify two natural fragments: one includes the A page is reassembled from ESI fragments by a con- newsroom headlines; another encompasses the stock tent delivery network (CDN) at an edge server, which se- prices for AT&T and AT&T Wireless and the date/time lectively downloads from the origin content server only of these quotes. The rest of the page can be viewed as the those fragments that are necessary (as opposed to the template. The stock quote fragment changes very fre- entire page). This is expected to reduce the load and quently, probably every minute when the stock market bandwidth requirements of the content server. is open. The headlines fragments changes much more This paper proposes an ESI-compliant approach in slowly, perhaps a few times each day. The rest of the which page reconstruction occurs at the browser rather page changes even less often. than the CDN. Unlike page assembly at the network If the entire page is considered as an indivisible edge, CSI optimizes content delivery over the last mile, whole, its lifetime in the cache is limited to the life- which is where the true bottleneck often is. We call the time of the fastest-changing content, one minute in our client-based approach Client-Side Includes,orCSI. example. With ESI, each portion of content is treated individually according to its own properties: the tem- plate will usually be accessed using the cached copy, the headlines fragment will be validated/refetched every few 1 Introduction hours, and only the small stock quote fragment will be refetched every minute. We discuss ESI in greater detail in the next section. As the use of the Internet increases, caching is an im- The above example represents a typical organization portant tool in coping with the rate of requests to Internet of Web content, where the general “look and feel” of the servers. Caching can be client-centric (proxy caching) page remains the same for a long time and only certain or server-centric (reverse proxy caching or CDNs). Re- segments within this general page framework change. gardless, a significant limitation of caching is that it is Several studies have found benefits from such page frag- mostly oriented toward static content. This limitation mentation [6, 21, 5]. The next question is where on the has been recognized and there have been a number of processing path from the origin server to the browser to approaches to extend caching to handle other types of reassemble a fragmented page? Historically, page as- content. sembly was first done at origin sites using technologies These approaches include specialized tools to gen- like Active Server Pages [1], Java Server Pages [11],
erate pages with dynamic content on a client (e.g., PHP Hypertext Preprocessor [18], and Server-Side In- > HPP [6] and While existing work used their own ad-hoc frag- the browser. We found that assembly in the browser can mentation languages, ours is to our knowledge the dramatically reduce the user response time. Depending first paper that implements client-side page assem- on the nature of the page, we observed a significant re- bly using an existing widely supported language for duction in the page display time for dial-up users with page fragmentation, which was independently pro- 56Kbps modems. Because page assembly occurs on the posed for a different purpose. This is important be- client, we call our approach Client-Side Includes,orCSI. cause legacy content that already uses this language Our mechanism requires no modification of browsers can benefit from our approach, whereas previous or configuration of the browser machine. In particular, work required re-authoring the content. it does not require such typical extension techniques as configuring a new browser plug-in, or co-locating a cus- A significant concern with client-side implemen- tom proxy at the browser machine. At the same time, tations is the need to support the implementation our mechanism implements most of the language spec- across a vast number of different browser types and ification of ESI 1.0 and thus enables content providers versions that co-exist on the Internet. We demon- using ESI to switch to our mechanism without changing strate that this concern can be effectively addressed their content.1 without modifying the server code and without the An important advantage of CSI is that, unlike edge- need to maintain multiple versions of the content. based page assembly, CSI does not require the presence of an edge server. CSI can be implemented between the To our knowledge, ours is the first paper that raises origin server and the browser directly. The existence of and addresses an issue of assessing whether and the edge server, and indeed the use of a CDN by the which pages should use fragmentation on a Web Web site, becomes an orthogonal issue. When a CDN is site. 1Our current implementation excludes support for the optional “in- Finally, we provide details of our prototype imple- lining” feature and provides incomplete support for ESI variables. mentation. While obviously dependent on the cur- rent technology context, the lessons from our im- The > 24. 25. Please click on a 3 Our Approach > < > 26. 27. There are a number of ways to assemble a page from < /HTML> various segments of information. The most common way is to do it at the origin server as shown in Figure 3a. Figure 2: An example of ESI usage. With ESI, Akamai and others moved page assembly to CDNs’ edge servers (Figure 3b). By reusing unchanged Figure 2 shows an example of an ESI template. content from edge servers’ caches, this approach reduces onodtms 4ad6 seconds. 65 and 44 Wednes- times, the high download extremely two for had which except Italy, with servers experiment day times local there download and median shows, remote in from 2 difference Table discernible As no is CDN- delivery. of scenario content lo- possible the LAN facilitated best from a the downloading emulated and Thus, server building cal dialing server. a local modem shared the 56Kbps that with bank a lo- modem used the a client and into The sites client remote dial-up server. the a from cal takes objects it these time fetch idle local to the a compared to 1) and Table in server (listed we sites remote indication, some preliminary from a As a used intuition. this verify link. dial-up slow the the is because case this operation in normal factor determining during users dial-up times for download reduce not would intu- However, assembly edge sites. itively, can origin of assembly capacity con- edge peak the bandwidth load, improve as reducing server effect possibly By opposite and sumption an 4). have Section also in discussed may server it reduce (although may on assembly load depending edge content, Also, of nature providers. the content connec- for associated costs and tivity sites origin from outflow traffic eet h ifrn ereo opesblt,snemdm s hard- use modems since compressibility, of degree different the reflects 2 eaaesuywudb eddt conclusively to needed be would study separate A h ag ifrnei onodtmsbtentetreobjects three the between times download in difference large The iue3 aeasml lentvsasmn nyfamn rg utb rfthdit h cache. the into prefetched be must Frag1 fragment only assuming alternatives assembly Page 3: Figure wget olt onodojcso rud100K around of objects download to tool Browser Browser Browser GET /frag1.html GET /www.att.com GET /www.att.com Full page Full page 2 Frag1 eseuaethat speculate We (b) ESIwithedge−sidepageassembly (c) ESIwithclient−sidepageassembly (a) NoESI Edge Edge o hs e ie htd s CDNs. use do that costs sites not Web CDN-related those or reduces for it whether and of servers, CDN regardless use sites CSIthey Web assembly, to edge applicable unlike low- dis- is Introduction, already other the as in or cussed Furthermore, dial-up the improves with connections. also those bandwidth it for But improves experience capacity. possibly user peak and site provider origin content the the for costs trans- is the that to and client the Internet. server connecting origin link last-mile the the over of mitted out downloads flows traf- that client the fic both the reducing thereby 3c, fragments, changed Figure only in illustrated proach, aeasml,a motn usini o content a how is question important an assembly, page aecmrsin sa niaino opesblt,teojc size object the compressibility, of after indication an As compression. ware ESI? not or ESI 4 our In itself. client the – edge network timate servers. ori- gin by connections dropped reflect downloads two these ymr hntefco f3 ojs vr3K o h eklyobject. Berkeley the for 30K, over reduced just and to object, 3, of Cornell factor the the for than 99K more over by to slightly reduced ject, Edge server server server utlk deasml,CIrdcsconnectivity reduces CSI assembly, edge like Just eoew xlr ifrn lentvsfrESI for alternatives different explore we Before epooet uhpg sebyfrhr oteul- the to further, assembly page push to propose We gzip GET /www.att.com GET /frag1.html GET /frag1.html opeso eandams nhne o h tla ob- Italian the for unchanged almost remained compression Full page Frag1 Frag1 Origin Origin Origin server server server CSI ap- Location URL Size Italy 131.114.9.184/ luigi/rlc99.ps.gz 96983 Berkeley 169.229.60.105/ helenjw/papers/icc.ps 96176 Cornell 128.84.154.132/Info/Projects/Spinglass/public pdfs/Randomized%20Error.pdf 115788 Table 1: Objects used in the comparison of download times. To factor out DNS latency, host names have been pre-resolved into IP addresses. Friday Wednesday Object local remote reduction local remote reduction 17:1 Italy 17.5 17.8 2 18.4 22.2 1 4:1 Berkeley 8.8 8.7 9.3 9.7 0 Cornell 19.9 19.9 0 21.0 21.0 Table 2: Median download times (in seconds) from remote sites and a local server. The table also shows the percent- age of improvements due to proximity of the local server. provider could decide if ESI encoding would be benefi- ated by a request includes a fixed component that does cial in its case. The answer depends on what the provider not depend on response size and a variable component is trying to optimize. that is proportional to the response size. The relative contribution of these components depends on the nature If the goal is to improve the end user experience, of content (e.g., static files vs. CGI scripts in C vs. CGI then ESI encoding is beneficial if it allows the con- scripts in Perl vs. fast CGI or servlets), and whether tent provider to move a substantial portion of con- or not persistent connections are used. As one indica- tent to templates or fragments with long TTLs. For tion, we tested the request throughput of Apache 2.0.40 poorly connected clients, the benefits are greatest on a 733MHz NetBSD machine and, using static files if CSI assembly is used rather than ESI assembly, with persistent connections, the throughput for a 20K since traffic is reduced over the bottleneck link. page was reduced by only a factor of 2 as compared with If the goal is to reduce bandwidth consumption the throughput for a 2K page – 1150 vs. 2274 requests on the link from the Web server to the network, per second.3 So, in our example, with this workload, the condition that a substantial portion of content ESI would prove detrimental for effective server capac- belong to ESI objects with long TTL will ensure ity even though it reduces the number of bytes served. that ESI is beneficial, regardless of the assembly method. 4.1 Guidelines Finally, if the goal is to increase the effective server To generalize the above example, we propose the fol- capacity, the answer depends on the nature of the lowing procedure to infer the overall effect of reduced content. The following example elaborates upon bandwidth consumption and increased request rate on this issue. the origin server capacity. We will use Figure 4 to illus- As an extreme example of a situation where ESI trate the procedure, and we will explain various elements would be detrimental to server capacity, consider a 20K in this figure as we move along. page that has three mutable regions of 2K each that change every minute. According to the ESI literature, Stress-test the server to see how its request through- this page seems like a good candidate for ESI encod- put depends on the size of the response and plot ing. However, for every HTTP request that reached the the result. For example, the descending solid line server without ESI, there will be either four requests (if on Figure 4 shows the throughput vs. response it is a brand new client) or three (if is it a repeat client size curve obtained in our experiment with static with a cached template). Overall, in the best-case sce- files mentioned earlier. Call this a capacity curve. nario for ESI, when all requests are due to repeat clients, In obtaining the capacity curve, use resource types the bandwidth consumption out of the server reduces by that approximate resources to be used in practice (e.g., Figure 4 was obtained using static files, which K= 3 2K = 3:3 a factor of three (20 ), but the number of requests to the server increases three-fold. would be appropriate for sites like att.com, where If the bottleneck is the server load, the overall effect 3A very similar trend can be seen in results by Nahum et al. for a can easily be detrimental. Indeed, the server load gener- variety of Web servers – see Table II in [16]. 2600 max T = K T that is equal to af ter .Ifthere- or ig 2400 max T = T quest rate af ter , then the server will re- ESI 2200 main fully utilized after ESI-encoding, and its ef- T < fective capacity will remain the same. If af ter 2000 max T , the server utilization will be below capac- ESI 1800 max ity and hence it could serve some additional re- x (|F|,TESI ) 1600 quests. In other words, ESI encoding would in- max x 1400 (|F|,KT orig ) crease the effective capacity of the server. Finally, max max throughput (req/sec) T > T if af ter , ESI encoding will reduce the 1200 (|P|,Torig ) ESI x effective capacity. Graphically, we plot the point 1000 max j F j;K T on the coordinate plane. If this or ig 800 point is above the throughput curve, ESI encod- 600 ing of page P will reduce the effective server ca- 0 5 10 15 20 25 30 35 40 size (KB) pacity, otherwise the capacity will increase. For max example, in Figure 4, T is 1150 requests per or ig = 1:25 Figure 4: Effect of ESI encoding on server load. second. Assuming K , the same demand that drove the server to capacity would result in 1:25 = 1437:5 1150 requests per second for ESI even the most mutable fragments are only updated objects, which falls below the capacity curve and periodically; for other resources, servlets or CGI thus indicates that ESI encoding of this page would scripts might be more appropriate). be beneficial from the perspective of server capac- ity. jP j Let be the size of the original page in ques- tion, that is, the page for which we must decide F j K 4.2 Estimating j and whether to use ESI encoding. Estimate (we will discuss how later) the average size of ESI objects, F j K To estimate j and , the most general technique is F j j , that would be shipped from the server if the a trace simulation using the Web server access log. The page were ESI-encoded and by how many times the simulator should translate each request for page P in the K T request rate would increase, .Thatis,if or ig is log into requests for the template and every fragment, KT the request rate for the original page, or ig will and then filter out repeated requests from the same client be the total request rate for the page template and that would hit in the client’s cache, taking into account F j fragments after it is ESI-encoded. Note that j is TTLs of individual ESI objects. The remaining requests the average over server responses rather than the F j could then be used to calculate j and the new request number of fragments on the page, so a more fre- rate. quently requested fragment contributes more to the However, some special cases, such as the case where average. every fragment has a non-zero TTL, allow analytical es- timations. Consider a Web site that uses a CDN and a Draw two vertical lines in the throughput vs. object P jP j n F ; :::; F n size plane, corresponding to the original and aver- page of size that contains fragments, 1 , F t i i with each fragment having a lifetime of seconds P j jF j age ESI-encoded response size, j and .For jF j t m instance, the two dashed vertical lines in Figure 4 and size i .Let be the smallest lifetime of all frag- t correspond to the case where the original page size ments, with m greater than 0. is 20K and the average ESI-encoded response size With ESI, in the best case of all repeat clients, the P max max template is (almost) never sent. Assume that page is jP j;T j F j;T will be 10K. Let and be or ig ESI the points where these vertical lines intersect with popular enough so that the request rate for it at each of max the CDN’s edge servers is much higher than fragment the capacity curve. T represents the server ca- or ig max lifetimes. (After all, unpopular pages do not matter from pacity for the original page and T gives the ESI server capacity for the ESI-encoded page. the perspective of load.) Then, without ESI, an edge 1=t server will send m requests per second. With ESI, the P n 1=t edge server will send i requests per second. =1 We know that, whatever the request rate for the i original page was, the request rate for the ESI ob- Since the same is true for all edge servers, the request rate is increased by a factor of jects after ESI encoding will be K times higher. In particular, the maximum sustainable demand for n X max the original page (corresponding to T request or ig 1=t ]t K =[ m rate) results in the request rate for the ESI objects i i=1 > plete support of this feature in MSIE [7]. The imple- >< > If your browser does not support Javascript please click support CSI (such as different versions of MSIE). In ei- > < >