Varnish Administrator documentation Release @VERSION@ Varnish Cache Project January 17, 2015 Contents 1 Varnish Installation 3 1.1 Prerequisites...............................................3 1.2 Installing Varnish.............................................3 1.3 Compiling Varnish from source.....................................4 1.4 Getting help...............................................5 1.5 Reporting bugs..............................................7 1.6 Platform specific notes..........................................9 2 The Varnish Tutorial 11 2.1 The fundamentals of web proxy caching with Varnish......................... 11 2.2 Supported platforms........................................... 12 2.3 About the Varnish development process................................. 12 2.4 Getting in touch............................................. 12 2.5 Starting Varnish............................................. 12 2.6 Put Varnish on port 80.......................................... 13 2.7 Restarting Varnish again......................................... 14 2.8 Backend servers............................................. 14 2.9 Peculiarities............................................... 14 2.10 Now what?................................................ 15 3 The Varnish Users Guide 17 3.1 The Big Varnish Picture......................................... 17 3.2 Starting and running Varnish....................................... 18 3.3 VCL - Varnish Configuration Language................................. 28 3.4 Reporting and statistics.......................................... 45 3.5 Varnish and Website Performance.................................... 46 3.6 Content composition with Edge Side Includes.............................. 56 3.7 Doing ESI on JSON and other non-XML’ish content.......................... 57 3.8 Troubleshooting Varnish......................................... 57 4 The Varnish Reference Manual 61 4.1 VCL................................................... 61 4.2 Varnish Processing States........................................ 66 4.3 Varnish CLI............................................... 69 4.4 varnishadm................................................ 73 4.5 varnishd................................................. 75 4.6 varnishhist................................................ 78 4.7 varnishlog................................................ 79 i 4.8 varnishncsa................................................ 80 4.9 varnishstat................................................ 82 4.10 varnishtest................................................ 85 4.11 varnishtop................................................ 87 4.12 VSM: Shared Memory Logging and Statistics.............................. 88 4.13 VMOD - Varnish Modules........................................ 89 4.14 VSL.................................................... 94 4.15 Varnish VSL Query Expressions..................................... 95 5 What’s new in Varnish 4.0 99 5.1 Changes in Varnish 4........................................... 99 5.2 Upgrading to Varnish 4.......................................... 99 6 Poul-Hennings random outbursts 105 6.1 A persistent message........................................... 105 6.2 Raking in the dough on Free and Open Source Software........................ 106 6.3 Wanton destruction for the greater good................................. 109 6.4 What SPDY did to my summer vacation................................. 110 6.5 Why HTTP/2.0 does not seem interesting................................ 112 6.6 Varnish Does Not Hash......................................... 116 6.7 The Tools We Work With........................................ 118 6.8 Thoughts on the eve of Varnish 3.0................................... 121 6.9 Why no SSL ?.............................................. 121 6.10 How GZIP, and GZIP+ESI works in Varnish.............................. 122 6.11 VCL Expressions............................................. 125 6.12 IPv6 Suckage............................................... 126 6.13 What do you mean by ‘backend’ ?.................................... 126 6.14 Picking platforms............................................. 128 6.15 Security barriers in Varnish....................................... 129 6.16 What were they thinking ?........................................ 131 6.17 Did you call them autocrap tools ?................................... 132 6.18 Why Sphinx and reStructuredText ?................................... 133 7 Varnish Glossary 135 8 Indices and tables 137 ii Varnish Administrator documentation, Release @VERSION@ Varnish Cache is a web application accelerator also known as a caching HTTP reverse proxy. You install it in front of any server that speaks HTTP and configure it to cache the contents. Varnish Cache is really, really fast. It typically speeds up delivery with a factor of 300 - 1000x, depending on your architecture. To get started with Varnish-Cache we recommend that you read the installation guide Varnish Installation. Once you have Varnish up and running we recommend that you go through our tutorial - The Varnish Tutorial, and finally the The Varnish Users Guide. If you need to find out how to use a specific Varnish tool, the The Varnish Reference Manual contains detailed doc- umentation over the tools. Changes from previous versions are located in the What’s new in Varnish 4.0 chapter. In closing, we have Poul-Hennings random outbursts, a collection of blog posts from Poul-Henning Kamp related to Varnish and HTTP. Conventions used in this manual include: service varnish restart A command you can run, or a shortkey you can press. Used either in the terminal or after starting one of the tools. /usr/local/, varnishadm, sess_timeout A utility, Varnish configurable parameter or path. http://www.varnish-cache.org/ A hyperlink. Longer listings like example command output and VCL look like this: $ /opt/varnish/sbin/varnishd -V varnishd (varnish-4.0.0-tp1 revision ddd00e1) Copyright (c) 2006 Verdens Gang AS Copyright (c) 2006-2011 Varnish Software AS Contents 1 Varnish Administrator documentation, Release @VERSION@ 2 Contents CHAPTER 1 Varnish Installation This section covers installation prerequisites, a step-by-step installation procedure, how and where to get help, and how to report bugs. It also contains a set of platform specific notes to aid you when installing Varnish on certain platforms. 1.1 Prerequisites In order for you to install Varnish you must have the following: • A recent, preferably server grade, computer. • A fairly modern and 64 bit version of either - Linux - FreeBSD, or - Solaris (x86 only). • Root access. Varnish can be installed on other UNIX systems as well, but it is not extensively or systematically tested by us on other systems than the above. Varnish is, from time to time, said to work on: • 32 bit versions of the before-mentioned systems, • OS X, • NetBSD, • OpenBSD, and • Windows with Cygwin. 1.2 Installing Varnish With open source software, you can choose to install binary packages or compile it yourself from source code. To install a package or compile from source is a matter of personal taste. If you don’t know which method to choose, we recommend that you read this whole section and then choose the method you feel most comfortable with. 1.2.1 Source or packages? Installing Varnish on most relevant operating systems can usually be done with with the specific systems package manager, typical examples being: 3 Varnish Administrator documentation, Release @VERSION@ 1.2.2 FreeBSD Binary package: pkg_add -r varnish From source: cd /usr/ports/varnish && make install clean 1.2.3 Red Hat / CentOS We try to keep the latest version available as prebuilt RPMs (el5 and el6) on repo.varnish-cache.org. See the online Red Hat installation instructions for more information. Varnish is included in the EPEL repository, however due to incompatible syntax changes in newer versions of Varnish, only older versions are available. We therefore recommend that you install the latest version directly from our repository, as described above. 1.2.4 Debian/Ubuntu Varnish is distributed with both Debian and Ubuntu. In order to get Varnish up and running type sudo apt-get install varnish. Please note that this might not be the latest version of Varnish. If you need a later version of Varnish, please follow the online installation instructions for Debian or Ubuntu. 1.3 Compiling Varnish from source If there are no binary packages available for your system, or if you want to compile Varnish from source for other reasons, follow these steps: Download the appropriate release tarball, which you can find on http://repo.varnish-cache.org/source/. Alternatively, if you want to hack on Varnish, you should clone our git repository by doing. git clone git://git.varnish-cache.org/varnish-cache 1.3.1 Build dependencies on Debian / Ubuntu In order to build Varnish from source you need a number of packages installed. On a Debian or Ubuntu system these are: • automake • autotools-dev • libedit-dev • libjemalloc-dev • libncurses-dev • libpcre3-dev • libtool • pkg-config • python-docutils • python-sphinx 4 Chapter 1. Varnish Installation Varnish Administrator documentation, Release @VERSION@ 1.3.2 Build dependencies on Red Hat / CentOS To build Varnish on a Red Hat or CentOS system you need the following packages installed: • autoconf • automake • jemalloc-devel • libedit-devel • libtool • ncurses-devel • pcre-devel • pkgconfig • python-docutils • python-sphinx 1.3.3 Compiling Varnish The configuration