var∙nish: A deceptively attractive external appearance; an outward show. var∙nished, var∙nish∙ing: To give a smooth and glossy finish to. We will talk about... What is a Reverse Proxy Cache? Architecture of Varnish Installation & Basic Configuration VCL by example Tools Varnish & Rails Misc tips & tricks ehcaC yxorP esreveR A P P R A P P P C A P P What? = Reverse‐Proxy ... à la HAProxy, Pound, mod_proxy_balancer etc. + Cache ... only proxy to backend if necessary a.k.a.: „HTTP Accelerator“ (= BS Bingo) Other „HTTP Accelerators“: Web Cache 10g BIG‐IP Users search.twitter.com hulu.com wikia.com pcwelt.de creativecommons.org ... Architecture: Cache Store Squid Varnish Mem‐Store Disk‐Store VMM (OS) VMM (OS) RAM HDD RAM HDD • one file per object (pre 2.7) • one big file mapped to VM • book keeping (disk vs. memory) • VMM often „smarter“ http://varnish.projects.linpro.no/wiki/ArchitectNotes Architecture: VCL Varnish Configuration Language DSL, compiled to C code (srsly!) allows inline C code C{ syslog(LOG_INFO, “Just served the 1000000th page. Hooray!"); }C hooks into a requests lifecycle Backends, ACLs, LB‐strategies defined here can be hot‐loaded into a running varnishd hot‐switching between multiple versions/profiles Architecture: Logging Not your daddy‘s log file Logs straight to shared memory Enables all kinds of fancy tools: varnishtop varnishstat varnishhist (= geek pr0n) Use varnishlog/varnishncsa to generate old school logs Installation Debian/Ubuntu: apt-get –t unstable install varnish OS X via MacPorts: sudo port install varnish From source: ./configure && make && make install Interesting files: /etc/default/varnish /etc/varnish/*.vcl Configuration Zero configuration in a perfect world (= all origin servers perfect HTTP citizens, setting correct cache control headers, conservative use of cookies) Varnish won't cache anything "private" or carrying a cookie by default The real world sucks: Tracking cookies (Google Analytics) Session cookies although no data in session "Cache‐control: private" by default (Rails) * ... (* which is a sensible default, btw.) VCL: Backends & Probes backend default { .host = "10.0.0.12"; .port = "80"; } backend slow_j2ee_app { .host = "10.0.0.13"; .port = "8080"; .connect_timeout = 1s; .first_byte_timeout = 10s; .between_bytes_timeout = 5s; .probe = { .url = "/check.jsp"; .timeout = 1s; } } VCL: Directors for simple load‐balancing requirements director d1 random { .retries = 3; { .backend = "default"; .weight = 10; } { .backend = "other_host"; .weight = 5; } } director d2 round-robin { ... } VCL: ACLs customize behaviour for different clients acl admins { "localhost"; "10.0.0.0"/24; ! "10.0.0.3"; # intern's laptop } ... if (client.ip ~ admins) { set req.http.x-magic-auth = "1"; } else { unset req.http.x-magic-auth; } VCL: Hooks Most important: vcl_recv Request comes in, decide what to do vcl_fetch Fetched obj from backend, allows tweaking vcl_deliver Object is about to be delivered to client vcl_hash Calculate hash key for lookup, defaults to full URL Other hooks: vcl_miss, vcl_hit, vcl_error, vcl_discard, vcl_timeout, vcl_pipe, vcl_pass http://varnish.projects.linpro.no/wiki/VCL VCL: Functions & Variables regsub(), regsuball(), purge_hash(), purge_url() own subroutines (not functions) with sub foo { ... } include "other.vcl"; to split files into parts req.* Request resp.* Response bereq.* Backend Request obj.* requested Object client.*, server.* set / unset for variables, remove additionally for headers http://varnish.projects.linpro.no/wiki/VCL Example: Choose backend sub vcl_recv { if (req.host ~ "slowapp.com$") { set req.backend = slow_j2ee_app; } else { set req.backend = other_backend; } } Example: Serve static assets sub vcl_recv { if (req.url ~ "^/(images|javascripts|styles)/") { remove req.http.cookie; } } sub vcl_fetch { if (req.url ~ "^/(images|javascripts|styles)/") { remove obj.http.set-cookie; } } Example: Remove certain cookies sub vcl_recv { set req.http.cookie = regsuball( req.http.cookie, "__utm.=[^;]+(; )?", "" ); set req.http.cookie = regsub(req.http.cookie, "; $", ""); if (req.http.cookie ~ "^ *$") { remove req.http.cookie; } } Example: "Stale while revalidate" Serve slightly stale content while a fresh version is fetched => better user experience + no thread pileup sub vcl_recv { set req.grace = 2m; } sub vcl_fetch { set obj.grace = 2m; } http://www.rfc‐editor.org/internet‐drafts/draft‐nottingham‐http‐stale‐controls‐00.txt Example: Backend is down Serve cachable (outdated) content even when the backend is on fire sub_recv { if (req.backend.healthy) { set req.grace = 30s; } else { set req.grace = 1h; } } sub_fetch { set obj.grace = 1h; } Tools: varnishtop Most popular Browser / Agent: varnishtop -i RxHeader -I \^User-Agent 2667.43 RxHeader User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9 459.54 RxHeader User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9 372.66 RxHeader User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9 369.90 RxHeader User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1) 353.06 RxHeader User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www 341.84 RxHeader User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; 323.87 RxHeader User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; 317.88 RxHeader User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9 250.55 RxHeader User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; 231.82 RxHeader User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; 173.69 RxHeader User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Most popular URLs: varnishtop –i RxUrl Traffic sources: varnishtop –i RxHeader –I \^Referer Tools: varnishhist | | | Hits | | | | | | | | | | | | | || ||| ||| ||| ||| ||| ||| ||| ||| ||| |||| |||| Misses |||| ||||| ||||| |||||| ## # #| +-------------+-------------+-------------+-------------+-------------+-------------+-------------+-------------+------------- |1e-6 |1e-5 |1e-4 |1e-3 |1e-2 |1e-1 |1e0 |1e1 |1e2 More Tools: varnishlog: Generate (customized) logs varnishncsa: Generate Apache compatible logs varnishadm: Manipulate a running varnishd varnishadm -T localhost:6082 purge.url "^/images/" varnishadm –T localhost:6082 vcl.load newconf /etc/my.vcl varnishreplay: Parses a log generated by varnishlog and replays the traffic! Varnish & Rails Proper use of expires_in instead of page caching Only use session if really necessary Purging of content possible with: `varnishadm –T #{hostport} purge.url #{url2purge}` net/telnet klarlack: http://github.com/schoefmax/klarlack !secure the connection to varnish's admin interface! (ssh tunnel, iptables etc.) Varnish & Rails: Sweepers # environment.rb config.gem "schoefmax-klarlack", :lib => 'klarlack', :source => 'http://gems.github.com' VARNISH = Varnish::Client.new('1.2.3.4:6082') # app/sweepers/blog_sweeper.rb class BlogSweeper < ActionController::Caching::Sweeper observe Post include ActionController::UrlWriter after_save(post) expire_post(post) end after_destroy(post) expire_post(post) end private def expire_post(post) VARNISH.purge :url, post_path(post) VARNISH.purge :url, latest_posts_path end end Misc: Edge Side Includes (ESI) Invented by Akamai & Co. <esi:include src="http://example.com/friend_feed"/> http://www.w3.org/TR/esi‐lang fragment_fu‐plugin for Rails (part of mongrel‐esi) Header, TTL: 15 min Activity‐ Nav, Article, Feed, TTL: TTL: 5 min TTL: 60 min 2 min Misc: Fine tuning your setup Use a non‐journaling file system (e.g. ext2) for storage file Pre‐create storage file (minimizes fragmentation). 4GB: dd if=/dev/zero of=storage.bin bs=4M count=1024 Tweak varnish's various startup settings – Twitters are: http://projects.linpro.no/pipermail/varnish‐dev/2009‐February/000968.html Misc: Monitoring with munin Thank you. • http://www.varnish‐cache.org • http://github.com/schoefmax/klarlack • http://varnish.projects.linpro.no/wiki/VCL • http://varnish.projects.linpro.no/wiki/ArchitectNotes • http://www.rfc‐editor.org/internet‐drafts/draft‐nottingham‐http‐stale‐controls‐00.txt • http://projects.linpro.no/pipermail/varnish‐dev/2009‐February/000968.html • http://www.w3.org/TR/esi‐lang .