Concepts of Server Load Balancing
By PANKAJ SHARMA
1 Concepts of Server Load Balancing
Introduction of Load balancing and clustering with Liferay Load balancing is one of the most popular in the world due to its impressive ease-of-use. load balancing is techniques to distributed load on multiple system . Server Load Balancing (SLB) could mean many things, for the purpose of this book it is defined as a process and technology that distributes site traffic among several servers using a network-based device. This device intercepts traffic destined for a site and redirects that traffic to various servers .
The load-balancing process is completely transparent to the end user. it’s configured optimally for a multiple server environment. If one server isn’t sufficient to serve the high traffic needs of your site, Liferay scales to the size you need. In Figure 1-1, we see the simplest representation of SLB. Apache server based Load Balancing:
Simply add load balancing to your current multiple tomcat server configuration in order to achieve high availability and build your own web cluster without burning your entire hosting budget. Load balancing can be implemented quickly and easily as an add-on to your current server solution to share the load between your web servers, using a simple script to replicate the data on the server . A load balancer performs the following functions:
Intercepts network-based traffic (such as web traffic) destined for a site.
2 Concepts of Server Load Balancing
Splits the traffic into individual requests and decides which servers receive individual requests.
Maintains a watch on the available servers, ensuring that they are responding to traffic. If they are not, they are taken out of rotation.
Provides redundancy by employing more than one unit in a fail-over scenario. Offers content-aware distribution, by doing things such as reading URLs, inter-cepting cookies, and XMLparsing.
3 Concepts of Server Load Balancing
.
Figure 1-1 Apache server based Load Balancing Concept The Concept of Architecture Load Balancing:
The objective here is that the solution should distribute the load among the servers in the cluster to provide the best possible response time to the end user. In a typical clustering solution, this involves use of a load distribution algorithm, like a simple round robin algorithm or more sophisticated algorithms, that distributes requests to the servers in the cluster by keeping track of the load and available resources on the servers. In shown figure1-2
4 Concepts of Server Load Balancing
5 Concepts of Server Load Balancing
Figure 1-2 Apache server based Architecture Load Balancing
Types of Load Balancing : DNS-Based Load Balancing:
With DNS round robin, it is possible to give multiple IP addresses to hostname,distributing traffic more or less evenly to the listed IP addresses. For instance, let's say you had three web servers with IP addresses of 208.185.43.202, 208.185.43.203, and 208.185.43.204 that we wanted to SLB was a technology or a viable product, site administrators would (andsometimes still do)employ a load balancing process known as DNS round robin.DNS round robin uses a function of DNS that allows more than one IP address to associate with a hostname. Every DNS entry has what is known as an A record,which maps a hostname to an IP address (such as In the Beginning 208.185.43.202).Usually only one IP address is given for a hostname. Under ISO's DNS server, BIND 8, this is what the DNS entry for www.vegan.net would look like: IN 208.185.43.202 , share the load for the site www.vegan.net. The configuration in the DNS server for the three IP addresses would look like this: IN A 208.185.43.20
IN A 208.185.43.203
IN A 208.185.43.204
6 Concepts of Server Load Balancing
You can check the effect using a DNS utility known as look up, which would show the following for www.vegan.net Server: ns1.vegan.net Address: 198.143.25.15 Name: www.vegan.net Addresses: 208.185.43.202, 208.185.43.203, 208.185.43.204 The end result is that the traffic destined for www.vegan.net is distributed between the three IP addresses listed .as shown in Figure 1-3.
Figure 1-3 Traffic distribution by DNS-based load balancing
7 Concepts of Server Load Balancing
we have four front-end servers, {FE1, FE2, FE3, FE4}. The user retrieves this list of servers from DNS, and then randomly connects to one in the list. In our example, let’s assume that the client connects to FE3. Upon connecting, the client presents its SIP URI from which a hash is generated by the front-end server. From this hash, the server determines the location of the registrar assigned to it. Note When a user is first enabled on (or moved to) a pool, a hash is generated to determine which front-end server is the primary registration database for the user, along with the order in which the remaining front- end servers will be attempted (as the backup registrar services). For our example, our user hash results in {FE4, FE2, FE1, FE3}. This is then interpreted as the order in which the clients will attempt to register. The client attempts to register with FE3, but because it’s not the primary registrar assigned to the user, FE3 redirects the client to FE4 as the correct registrar to connect to. The client successfully registers with FE4.
SLB has several benefits:
Flexibility: SLB allows the addition and removal of servers to a site at any time, and the effect is immediate. Among other advantages, this allows for the maintenance of any machine, even during peak hours with little or no impact to the site. A load balancer can also intelligently direct traffic using cookies, URL parsing, static and dynamic algorithms, and much more.
High availability: SLB can check the status of the available servers, take any non responding servers out of the rotation, and put them in rotation when they are functioning again. This is automatic, requiring no 8 Concepts of Server Load Balancing
intervention by an administrator. Also, the load balancers themselves usually come in a redundant configuration, employing more than one unit in case any one unit fails.
Scalability: Since SLB distributes load among many servers, all that is needed to increase the serving power of a site is to add more servers. This can be very economical, since many small- to medium-sized servers can be much less expensive than a few high-end servers. Also, when site load increases, servers can be brought up immediately to handle the increase in traffic.Load balancers started out as PC-based devices, and many still are, but now load balancing functions have found their way into switches and routers as well. firewall Load Balancing:
Firewall load balancing balances traffic flows to one or more firewall farms. A firewall farm is a group of firewalls that are connected in parallel or that have their “inside” (protected) and “outside” (unprotected interfaces connected to common network segments.
Firewall load balancing requires a load-balancing device (IOS SLB) to be connected to each side of the firewall farm. A firewall farm with “inside” and “outside” interfaces would then requir two load- balancing devices, each making sure that traffic flows are directed toward the same firewall for the duration of the connection.
Firewall load balancing is performed by computing a hash value of each new traffic flow (source and destination IP addresses and ports). This is called a route lookup. The firewall load-balancing device then masquerades as the IP 9 Concepts of Server Load Balancing
address for all firewalls in the firewall farm. Firewall load balancing can detect a firewall failure by monitoring probe activity. The HSRP can be used to provide a “stateless backup redundancy for multiple firewall load-balancing devices. If one device fails, a redundant device can take over its function. Multiple firewall load-balancing devices can also use “stateful- backup” for redundancy. Backup devices keep state information dynamically and can take over immediately if a failure occurs.
Figure 1-4 Firewall Load-Balancing Concept
10 Concepts of Server Load Balancing
What About Load Balancing: First, let's be clear on what "load balancing" is. Load balancing - a technique to distribute workload across resources. It is but one component in a high-availability cluster. Liferay Portal’s case, we are load balancing the volume of requests across multiple app servers, which may or may not be on physically separate hardware. Initially, this may seem sufficient, until you realize some of the components that the portal uses.
What is mod_jk: mod_jk is a replacement to the elderly mod_jserv. It is a completely new Tomcat-Apache plug-in that handles the communication between Tomcat and Apache. The mod_jk connector is an Apache HTTPD module that allows HTTPD to communicate with Apache Tomcat instances over the AJP protocol. The module is used in conjunction with Tomcat's AJP Connector component. shown in figure1-5 . Now the configure is much easier and more consistent . ProxyPass /servlets ajp://tc.example.com:8089 Easier when Apache needs to proxy both HTTP and AJP . Leaverage improvements proxy module .
11 Concepts of Server Load Balancing
12 Concepts of Server Load Balancing
Figure 1-5 Apache + mod_jk Balancing Concept
About Connectors: Apache Tomcat uses Connector components to allow communication between a Tomcat instance and another party, such as a browser, server, or another Tomcat instance that is part of the same network. For example, the HTTP connector listens for requests over the HTTP/1.1 protocol on various TCP ports, and forwards them to the Engine associated with processing the request. Using the AJP connector, Apache Tomcat instances can exchange data with mod_jk enabled instances of Apache HTTPD, using the AJP protocol. Implementations of mod_jk are also available for integration with IIS and NES/iPlanet/Sun, but are less widely used.
About AJP: AJP, an acronymn for Apache Jserv Protocol, is a binary version of HTTP that is optimized for communication between Apache HTTPD and Apache Tomcat over a TCP connection. The current version of the AJP protocol is 1.3, referred to by the standard name ajp13. ajp13 extends the earlier mod_jserv and ajp12 modules, offering significant speed improvements and SSL support. Other than the data format, differences between the standard HTTP and AJP protocols include more persistent connections (to avoid unnecessary socket creation) and a focus on connection reuse over a series of request/response cycles. 1. Apache can now talk Ajp with Tomcat directly . 2. mod_proxy_ajp is magic mojo . 3. Other proxy improvements make this even more exciting . 4. Mod_jk alternative . 13 Concepts of Server Load Balancing
Apache Module mod_proxy: Apache has a mod_proxy module available for almost all versions of apache .However, prior to apache 2.2, only reverse proxy features were available and mod_proxy_balancer was not available for load balancing.This module implements a proxy/gateway for Apache. It implements proxying capability for AJP13 (Apache JServe Protocol version 1.3), FTP, CONNECT (for SSL), HTTP/0.9, HTTP/1.0, and HTTP/1.1. The module can be configured to connect to other proxy modules for these and other protocols. Apache's proxy features are divided into several modules in addition to mod_proxy: mod_proxy_http, mod_proxy_ftp, mod_proxy_ajp, mod_proxy_balancer, and mod_proxy_connect. Thus, if you want to use one or more of the particular proxy functions, load mod_proxy and the appropriate module(s) into the server (either statically at compile-time or dynamically via the LoadModule directive).shown in figure1-6 Documentation for mod_proxy is available for: 1. Apache 1.3 2. Apache 2.0 3. Apache 2.1 4. Apache 2.2
The proxy module (and other proxy extension used) must be loaded:
LoadModule proxy_module modules/mod_proxy.so 14 Concepts of Server Load Balancing
15 Concepts of Server Load Balancing
Figure1-6 Apache + mod_proxy Concept Configure mod_proxy: Now enable mod_proxy in Apache, and proxy requests to the application server by adding the example below to your Apache httpd.conf (note: the files may be different on your system; the JIRA docs describe the process for Ubuntu/Debian layout): # Put this after the other LoadModule directives
LoadModule proxy_module/usr/lib/apache2/modules/mod_proxy.so
LoadModule proxy_http_module/usr/lib/apache2/modules/mod_proxy_http.so
# Put this in the main section of your configuration (or desired virtual host, if using Apache virtual hosts)
ProxyRequests Off
ProxyPreserveHost On
16 Concepts of Server Load Balancing
Order deny,allow
Allow from all
ProxyPass /confluence http://www.example.com/confluence
ProxyPassReverse /confluence http://www.example.com/confluence
Order allow,deny
Allow from all
Apache Mod]ule mod_proxy_ajp: mod_proxy_ajp is an Apache module which can be used to forward a client HTTP request to an internal Tomcat application server using the AJP protocol. This module requires the service of mod_proxy. It provides support for the Apache JServ Protocol version 1.3 (hereafter AJP13). Thus, in order to get the ability of handling AJP13 protocol, mod_proxy and mod_proxy_ajp have to be present in the server.
This module is used to reverse proxy to a backend application server (e.g. Apache Tomcat) using the AJP13 protocol. The usage is similar to an HTTP reverse proxy, but uses the ajp:// prefix:
17 Concepts of Server Load Balancing
Simple Reverse Proxy:
enableLookups="false" redirectPort="8443" protocol="AJP/1.3" URIEncoding="UTF-8" /> However we want to run Tomcat on an AJP connector, so: 1. To directory of server.xml file configuration Ubuntu path :>liferay-portal-6.1.0-ce-ga1/tomcat-7.0.23/conf/server.xml 2. Comment out the normal connector above, and uncomment the AJP connector: enableLookups="false" redirectPort="8443" protocol="AJP/1.3" URIEncoding="UTF-8" /> 3. Adjust the port if you wish and add in the URIEncoding="UTF-8" attribute. 4. Start Tomcat normally. Configure Apache: We now need to tell Apache to talk to Tomcat over AJP. You should note that since in ubuntu OS.the Apache config and modules have been relocated .. This guide is for Ubuntu and for Apache 2.2. Also note that in Ubuntu , the necessary modules are already installed into /usr/libexec/apache2. Thanks Apple! 1. Navigate to your Apache directory: ubuntu path : 2. Open httpd.conf and scroll to the large LoadModule section. Ensure you have the following uncommented: LoadModule proxy_module libexec/apache2/mod_proxy.so LoadModule proxy_ajp_module libexec/apache2/mod_proxy_ajp.so 3. In my httpd.conf, right at the bottom there is a line: In my httpd.conf, right at the bottom there is a line: 4. Create an ajp.conffile. You could put the configuration for AJP in the main httpd.conf file but I prefer to keep things separated. 19 Concepts of Server Load Balancing touch other/ajp.conf 5. Open ajp.confand paste in the following: ProxyRequests Off Order deny,allow Deny from all Allow from localhost ProxyPas/ ajp://localhost:8009/ ProxyPassReverse / ajp://localhost:8009/ This will forward all requests to Tomcat. You can optionally pass a list of contexts that you want forwarded, like so: ProxyRequests Off ProxyPass /somecontext ajp://localhost:8009/somecontext ProxyPassReverse/somecontextajp://localhost:8009/somecontext Adjust the port to be whatever the port is in your Tomcat AJP connector. Save and close. 6. Once again, ensure you have the line in httpd.conf that is going to load this ajp.conf file. 20 Concepts of Server Load Balancing 7. Restart Apach2e: 8. Open your terminal and type the bellow command 9. sudo /etc/init.d/apache2 start 10. sudo /etc/init.d/apache2 restart 11. sudo /etc/init.d/apache2 stop 12. sudo /etc/init.d/apache2 reload 13. #sudo/home/pankaj/pankajsharma/Softwares/LifeRay/Apache _server_cluster/liferay-portal sh.catalina.sh run Apache Module mod_proxy_balancer: This module requires the service of mod_proxy. It provides load balancing support for HTTP, FTP and AJP13 protocols Thus, in order to get the ability of load balancing, mod_proxy and mod_proxy_balancer have to be present in the server. The module mod_proxy_balancer implements stickyness on top of two alternative means: cookies and URL encoding. Providing the cookie can be either done by the back-end or by the Apache web server itself. The URL encoding is usually done on the back-end. Shown in figure1-7 With apache 2.2 mod_proxy is able to use the extension mod_proxy_balancer. Configuraion: The configuration of mod_proxy_balancer is similar to pure mod_proxy, except that balancer:// URLs may be used as a protocol instead of 21 Concepts of Server Load Balancing http:// when specifying destinations (workers) in ProxyPass element # map to cluster with session affinity (sticky sessions) ProxyPass /balancer ! BalancerMember ajp://localhost:8009/ route=jvm1 BalancerMember ajp://localhost:8010/ route=jvm2 ProxyPass balancer://mycluster/ stickysession=JSESSIONID nofailover=On 22 Concepts of Server Load Balancing Figure1-7 Apache + mod_proxy_balancer Concept How Do You Know You’re Done? Liferay clustering is not just pointing a load balancer to two(or more) Liferay nodes.You are not done.Why? Because there are certain application-level components that need to be either centrally managed or synchronized. Let’s go through a checklist. Configuring multiple Tomcat application servers with Apache HTTP server/ Load Balancing with Apache WebServer: we can configure multiple Tomcat instances with single Apache Web Server. There are many solutions posted on the web, but it is the simplest and easiest way to do this. 23 Concepts of Server Load Balancing Prerequisite: Apache HTTP Serve 2. Tomcat 6.1.x mod_jk.so module for Apache Web server Any text editor, i.e. Notepad, etc. We will required to two tomcat server 1 and tomcat server 2 fo version 6.1.x if you do not have Apache HTTP server goto http://httpd.apache.org/ and download the appropriate version and install on your machine. if you do not have Apache Tomcat Server goto http://tomcat.apache.org/ and download the appropriate version and install on your machine. if you do not have mod_jk.so module, goto http://tomcat.apache.org/download- connectors.cgi and download the appropriate connector for Apache Web Server. Given below steps follows : Step 1: Configure Apache 2.x: Httpd.conf configuration: Path in ubuntu:->/etc/apache2/httpd.conf Path in windows:->Apache-home/apache2.2/conf/httdp.conf Add below code ServerName localhost LoadModule proxy_module modules/mod_proxy.so 24 Concepts of Server Load Balancing LoadModule proxy_ajp_module modules/mod_proxy_ajp.so LoadModule proxy_balancer_module modules/mod_proxy_balancer.so LoadModule status_module modules/mod_status.so ServerName localhost ServerName 1localhost ProxyRequests Off Order deny,allow Allow from all ProxyPass / ajp://localhost:8080/ ProxyPass / ajp://localhost:8080/ ProxyPassReverse / ajp://localhost:8080/ ProxyPassReverse / ajp://localhost:8080/ BalancerMember ajp://localhost:8010/ route=jvm2 ProxyPass balancer://mycluster/ stickysession=JSESSIONID Step 2: Configure Tomcat: you have to configure each tomcat instance for different port to run successfully. To configure Tomcat’s port, goto conf directory of any tomcat instance you have and open the server.xml file in the text editor. In Tomcat-server1: Path:->home-LifeRay/Apache_server_cluster/Tomcat-server1/liferay-portal- 6.1.0-ce- ga1/tomcat/conf Every liferay-tomcat's server.xml files change the ports. Given At localhost change the port below shutdown : 8005 connector : 8081 26 Concepts of Server Load Balancing redirectport : 8445 ajp-connector: 8009 Set jvmRoute to support load-balancing via AJP ie . so it should look like this Add to enableLookups="false" at AJP 1.3 Connector on port 8009 . Uncomment below line for clustering tag . In Tomcat-server2: Given At localhost change the port below shutdown :8006 connector :8082 Connector URIEncoding="UTF-8" connectionTimeout="20000" port="8082" protocol="HTTP/1.1" redirectPort="8446"/> redirectport : 8446 ajp-connector :8010 Set jvmRoute to support load-balancing via AJP ie 27 Concepts of Server Load Balancing so it should look like this Add to enableLookups="false" at AJP 1.3 Connector on port 8009 Uncomment below line for clustering tag # lists the workers by name worker.list=tomcat1, tomcat2, loadbalancer #Tomcat1 configuration worker.tomcat1.port=8009 //AJP1.3 Connector’s port worker.tomcat1.host=localhost //IP of machine where tomcat1 is running worker.tomcat1.type=ajp13 //AJP Connector type. worker.tomcat1.lbfactor=100 //Load Balancing factor 28 Concepts of Server Load Balancing #Tomcat2 configuration worker.tomcat2.port=8010 //AJP1.3 Connector’s port worker.tomcat2.host=localhost //IP of machine where tomcat1 is running worker.tomcat2.type=ajp13 //AJP Connector type. worker.tomcat2.lbfactor=100 //Load Balancing factor #Load Balance worker configuration worker.loadbalancer.type=lb worker.loadbalancer.balanced_workers=tomcat1,tomcat2 Step 4:Configure JK module: Copy the downloaded JK module file (mod_jk.so) to the modules directory of Apache Web Server. now goto conf directory of Apache Server, and open the httpd.conf file in the text editor. and add these lines to the end of httpd.conf LoadModule jk_module modules/mod_jk.so JkWorkersFile conf/workers.properties JkLogFile logs/mod_jk.log kLogLevel info now you have done all the configuration, now you need to configure the JK Module to redirect the requests to specific tomcats. It can be done by the following: To redirect specific request to specific Tomcat add this line to httpd.conf: JkMount /abc/* tomcat1 or JkMount /*.do tomcat2 Or, if you want to configure requests to be served by any of the tomcat then add 29 Concepts of Server Load Balancing this: JkMount /*.jsp loadbalancer if any of the tomcat is stopped then all the request will be served by other tomcat and if both the tomcat are running then the requests are redirected to tomcat servers according to the lbfactor specified in the workers.properties. Now start all the Tomcat Servers and Apache Server, and its done. Step 5 To configuratin of Web.xml file: The directory of web.xml file Ubuntu path:->liferay-home/tomcat-7.0.23/webapps/ROOT/WEB- INF/wex.xml To Insert tag Add these below lines into Catalina.sh file . Add these below lines into Catalina.bat file for windows OS at starting . These below lines are add In both Liferay Apache tomcat server . JAVA_OPTS="-Dsli.env=ci -Djava.net.preferIPv6Addresses=true -Djava.net.preferIPv4Stack=true 30 Concepts of Server Load Balancing -Djava.net.preferIPv6Stack=true -Xms128m -Xmx512m" Step 7 To start apache, tomcatA, tomcatB and navigate to http://localhost/ : Open your terminal and type the below commands such as sudo /etc/init.d/apache2 start sudo /etc/init.d/apache2 restart 31 Concepts of Server Load Balancing Checklist (Logical View): Load Balancer – it can be software (i.e. - Apache), or hardware (F5), orwhatever you wish, really. All it is doing is redirecting requests. Centralized Database- Hopefully, you have gotten off of HSQL and are using a real DB server. This is a JBDC connection. It is abstracted from Liferay's point of view. Any level of redundancy you have behind that JDBC connection is up to you and your DBA. Just as an example, you may choose to configure a MySQL cluster, or Oracle RAC, for DB high availability. 32 Concepts of Server Load Balancing Ehcache – This is what Liferay uses out-of-the-box for it's Hibernate level 2 cache. This needs to be configured to sync, else you will see inconsistencies depending on what node the load balancer redirects end users to. You are not forced to use Ehcache, it is simply what it ships with. You could use something like Terracotta, for example. If you do not do this, you will most definitely see inconsistencies depending on the node the end user is on, due to stale caches. Lucene – This needs to be centralized. This can be done: a) via JDBC (canwork, but there may be issues with speed and table locks), b) swapped out for something like SOLR (runs as a webapp in Tomcat), or c) using the cluster link feature that can be turned on where each node maintains its own replicated cache. If you do not do this, you will see inconsistencies with search and other indexed data that is returned from the DB. Document Library – This needs to be centralized. This is because each node keeps the assets locally on the file system by default. While the meta-data is in the DB, the files serve up faster this way (vs. BLOBS in a DB). So,you need to either a) point the content repository to be stored in the DB (can work but performance may suffer) via JCRHook in portal properties, or b) mount a path a shared storage (i.e. SAN or other file system storage) and configure each node to connect to this common place via AdvancedFileSystemHook in portal properties. If you do not do this, the meta-data regarding your documents will be in the DB, but when you try to retrieve them, the node may or may not find the document there, physically. 33 Concepts of Server Load Balancing Checklist (Detailed File View): 34 Concepts of Server Load Balancing How Do I Know It’s Working?: A quick test: 1. Bring up the load balancing 2. Bring up Node 1 3. Bring up Node 2 35 Concepts of Server Load Balancing 4. Open Browser 1 and go to Node 1 directly (i.e. – http://192.168.1.101:8080 5. On Browser 1 go to page 1 (i.e. http://192.168.1.101:8080/web/guest/home)logged in as Admin 6. Open Browser 2 (different browser session) and go to Node 2 directly (i.e-http://192.168.1.102:8080) 7. On Browser 2 go to page 1 (i.e. http://192.168.1.102:8080/web/guest/home logged in as Admin 8. In Browser 1 add a portlet to the page 9. Shut down Node 1 10. Go to Browser 2 and refresh the page (i.e. – CTRL F5) Quick Test Result: If Browser 2 refreshes and shows the added portlet fromBrowser 1, Liferay clustering in most likely configured correctly. This is because this behavior demonstrates that not only the load balancer is redirected to the live node, but that cache replication is occurring between nodes. 36 Concepts of Server Load Balancing 37 Concepts of Server Load Balancing