LDM7 Deployment Guidance
Total Page:16
File Type:pdf, Size:1020Kb
LDM7 Deployment Guidance Steve Emmerson, University Corporation for Atmospheric Research ([email protected]) Malathi Veeraraghavan, University of Virginia ([email protected]) Multicast network test To install LDM7, you will need a Linux/Unix machine and a copy of the LDM7 software package. Also, you need to make sure the network that LDM7 is going to run on supports and has enabled multicast. To test whether multicast is supported, please follow the link https://github.com/shawnsschen/CCNIEToolbox/tree/master/generic/mcast_test and obtain the two C files. One is for multicast sender and the other for receiver. If there are more than one network interfaces in the sender’s system, you have to manually update the routing table with one new entry: route add 224.0.0.1 dev eth1. Then use route n to check the updated routing table. You will be able to see something like: Note that the entry we added is only matching a particular multicast address 224.0.0.1, not a subnet. If the multicast address changes, you need to manually change the configuration or use a subnet address. Now you have a multicast network (though not sure if it works), next step is to upload the mcast_send.c onto the sender system and mcast_recv.c onto all the receiver systems (more than one in a multicast network). Use the following command to compile them, respectively: gcc o send mcast_send.c gcc o recv mcast_recv.c send is a multicast senderside test program, it sends out a “hello multicast world” message every second to multicast group 224.0.0.1:5173. recv is a multicast receiverside program, which receives the “hello multicast world” message from the multicast group 224.0.0.1:5173. Both send and recv take one input argument, the interface IP address, to inform the OS which interface they would like to use for multicast. For example, we usually use eth1 in GENI because it is the dataplane interface. Only on the same plane will this test program work. send listening on eth0 and recv listening on eth1 will not work since the management plane and data plane are physically disconnected. You can do a ping to test the reachability between interfaces. Once everything has settled, execute ./send <dataplane interface IP> on the sender system, and execute ./recv <dataplane interface IP> on the receiver system. If working well, you will be able to see “hello multicast world” on all systems. LDM7 Installation As the multicast network itself is working well, we can proceed and start installing LDM7 on all the nodes in this network. According to UCAR, here are the platform requirements before installing: https://www.unidata.ucar.edu/software/ldm/ldm6.12.14/basics/platform.html. In addition, pax, sudo, libyamldev(el), sysstat, ntp, gcc 4.8, g++ 4.8 (gccc++ for CentOS), make, autoconf, m4, and automake are needed for compiling. Ubuntu: sudo aptget install libxml2dev libpngdev zlib1gdev sudo aptget install pax libyamldev sysstat ntp autoconf m4 automake (Check gnuplot separately, vmstat, netstat, top, etc. should already be installed) Centos: yum install libxml2devel yum install libpngdevel yum install zlibdevel Note that the development packages will be found in your package managers as dev in Ubuntu/Debian and devel in CentOS. Special Notes: zlib package can be found as zlibdev(el) in package manager. For CentOS 6, the default gcc/g++ version provided by the system is too outdated. You have to manually upgrade it to 4.8. To do this: $ wget http://people.centos.org/tru/devtools2/devtools2.repo O /etc/yum.repos.d/devtools2.repo $ yum install devtoolset2gcc devtoolset2binutils devtoolset2gccc++ $ mv /usr/bin/gcc /usr/bin/gcc_old $ mv /usr/bin/g++ /usr/bin/g++_old $ ln s /opt/rh/devtoolset2/root/usr/bin/gcc /usr/bin/gcc $ ln s /opt/rh/devtoolset2/root/usr/bin/g++ /usr/bin/g++ $ gcc v You will see gcc is upgraded to 4.8. For CentOS: To give sudo access to users in the wheel group, execute visudo and search for the following line: # %wheel ALL=(ALL) ALL For Ubuntu: Look for the following line (it may not be commented) # %sudo ALL=(ALL) ALL Uncomment it by removing the hash mark if commented. If not commented do nothing. And save and quit. Now, upload LDM7 package onto the nodes. Next, create an ldm user if it does not exist. To create the user, execute: Centos: adduser ldm Ubuntu: adduser ldm Then use passwd ldm to set a password. We also add ldm into the sudo list by executing the following command. Centos: usermod aG wheel ldm Ubuntu: usermod aG sudo ldm (Can check if the following command was successful with: $ groups ldm) After the account is created, all the following operations will be done as user ldm. We switch to ldm home and proceed: su ldm. Move your uploaded LDM7 package to ldm home directory, you should expect something like ldm6.12.15.42.tar.gz. We unpack it with: gunzip c ldm6.12.15.42.tar.gz | pax r 's:/:/src/:' Then we will compile it. Go into the source folder: cd /home/ldm/ldm6.12.15.42/src. Use the following commands to compile (Note that if make distclean returns “No rule to make target…”, it is fine, just proceed to next step): Centos: $ sudo /etc/init.d/rsyslog start (CentOS 6) $ sudo systemctl start rsyslog.service (CentOS 7) Ubuntu: $ sudo start rsyslog (restart if already running) $ make distclean $ find exec touch \{\} \; $ ./configure withdebug withmulticast disablerootactions CFLAGS=g CXXFLAGS=g $ make install $ sudo make rootactions If no error is reported, you will have a fresh installed LDM7. We are going to configure it. For the sender side (upstream LDM7), find the configuration file: vi $HOME/etc/ldmd.conf. Delete everything inside and add the following lines: MULTICAST ANY 224.0.0.1:38800 1 10.10.1.1 ALLOW ANY ^.*$ EXEC "insert.sh" EXEC "cpu_mon.sh" EXEC "tc_mon.sh" 10.10.1.1 is the dataplane IP address for the sender, you can replace it with the actual IP. 224.0.0.1 is the multicast address we are going to use, you should make sure it is the same as what you set in the routing table. For the receivers, open the same configuration file and delete everything. Add the following line: RECEIVE ANY 10.10.1.1 10.10.1.x 10.10.1.1 is the IP of the sender, 10.10.1.x is the IP of the receiver’s dataplane interface. We need to configure the LDM7 product queue settings as well. For LDM7 sender, execute regutil s 5G /queue/size in ldm home. For receivers, execute regutil s 2G /queue/size instead. Then execute regutil s 35000 /queue/slots. Finally, execute regutil s 10.10.1.x /hostname, where 10.10.1.x is the corresponding IP on that node. Some scripts for measurement are also needed. Go to https://github.com/shawnsschen/CCNIEToolbox/tree/master/generic/misc and get util folder. Put util onto the sender’s ldm home. Run chmod +x util/* to make sure all scripts are executable. Run chown R ldm.ldm util to change the permission. Then open /home/ldm/.bash_profile and add a new line at the bottom: export PATH=$PATH:$HOME/util. Then touch /home/ldm/.bash_profile. Put the github repo containing the test input in ldm home: git clone https://github.com/shawnsschen/LDM6LDM7comparison.git. For bandwidth measurement, we need to use sysstat. Configure it to log every minute: sed i e \'s/*\/10/*\/1/g\' /etc/cron.d/sysstat rm /var/log/sa/* And make sure it is running: service crond start service sysstat start The last thing we need to do is to configure tc htb on the sender. Run the following commands with root permission (assuming eth1 is the dataplane interface): tc qdisc del dev eth1 root tc qdisc add dev eth1 root handle 1: htb default 2 tc class add dev eth1 parent 1: classid 1:1 htb rate 20mbit ceil 20mbit tc qdisc add dev eth1 parent 1:1 handle 10: bfifo limit 600mb tc class add dev eth1 parent 1: classid 1:2 htb rate 20mbit ceil 20mbit tc qdisc add dev eth1 parent 1:2 handle 11: bfifo limit 600mb tc filter add dev eth1 protocol ip parent 1:0 prio 1 u32 match ip dst 224.0.0.1/32 flowid 1:1 tc filter add dev eth1 protocol ip parent 1:0 prio 1 u32 match ip dst 0/0 flowid 1:2 20 Mbps (perclass sending rate) and 600 Mb (perclass BFIFO size) are the key parameters for LDM7, as the experimental settings change, you can replace them with new values. So far, we have finished all the settings and are good to go! But wait a second, open util/insert.sh on the sender. See the sleep xxx line? Change it with the value: m*40, where m is your number of receivers. This is to make sure the sender would not start sending before all receivers are connected. You can have 40 seconds to start a receiving LDM. You can also set it to a larger value to buy yourself more time.