Connection and VPN Bonding

From Hack Sphere Labs Wiki

Contents

1 Objective 2 Ideas 2.1 Notes 3 Custom 3.1 Configuration 3.1.1 OpenVPN 3.1.2 Bonding Script 3.1.3 Cleanup Script 3.2 Client Configuration 3.2.1 USB Drive/Modem CD Rom Eject 3.2.2 Testing Modem with wvdial 3.2.3 ppd and chat scripts 3.2.4 OpenVPN on the Client 3.2.5 Bonding Script 3.2.6 Cleanup Script 3.3 NAT Forwarding as Internet Gateway 4 Notes 5 pfSense 5.1 Notes 6 ZeroShell 6.1 Notes 7 udev/cdcontrol Creation

Objective

Perferablly bond multiple 3G modems together to create a stable faster connection. I am trying to aggregate 3 unstable connections into one. Ideas

Bond 2-3 OpenVPN tun interfaces. LAGG Kernel Bonding LACP (Stable connections, same BW) Linux Advanced Routing & Traffic Control - http://lartc.org/

Notes

http://serverfault.com/questions/171333/how-exactly-specifically-does-layer-3-lacp-destination-address-hashing-work It should have been easy: http://evilprojects.org/2009/09/howto-setup-openvpn-channel-bonding-on-multiple-umts-uplinks.html Route VPNs through interfaces using port numbers. Custom Linux

I tested with . Once it get greater then two modems I will post some real results. I was able to get what I wanted working but it did not seem that much faster over the 2x 3g modems. I do not know if it was the latent speed or whatnot but we will see.

I would like to try 3 to 4 of them and really see what happends.

Server Configuration

I used a Debian VPS because I wanted to route all my traffic out to the internet through the bond.

OpenVPN su - aptitude update aptitude upgrade aptitude install tap configuration is a bit different then tun configuration. Since it works via layer two you do not need to worry about layer 3 stuff like IPs in the config file.

Setup a CA, Certs, ta.key: http://wiki.hackspherelabs.com/index.php?title=OpenVPN#Setup but here are some commands for reference: mkdir /etc/openvpn/easy-rsa cp /usr/share/doc/openvpn/examples/easy-rsa/2.0/* /etc/openvpn/easy-rsa cd /etc/openvpn/easy-rsa/ source ./vars ./clean-all ./build-ca ./build-key-server servername ./build-dh cd keys openvpn --genkey --secret ta.key cd .. ./build-key-pkcs12 clientx

You need some openvpn config files in /etc/openvpn/ and here is an example of a tap server openvpn config file: openvpntapserver.conf proto udp dev tap1 port 36214 ca /etc/openvpn/easy-rsa/keys/ca.crt cert /etc/openvpn/easy-rsa/keys/server.crt key /etc/openvpn/easy-rsa/keys/server.key dh /etc/openvpn/easy-rsa/keys/dh1024.pem tls-auth /etc/openvpn/easy-rsa/keys/ta.key 0 keepalive 2 10 server 10.123.148.0 255.255.255.0 tls-server client-to-client max-clients 10 tun-mtu 1500

#daemon verb 3 cipher AES-256-CBC comp-lzo

#status /var/log/openvpn-status35214.log persist-key persist-tun log-append openvpn.36214.log

You need a vpn server for each modem that you want to bond. You will need to configure a different port and ip for each one while also a different tap interface.

You need to stop and disable openvpn from starting because the bonding.sh script will call openvpn.

/etc/init.d/openvpn stop update-rc.d openvpn disable

Bonding Script

You also need the utilities that this script calls aptitude install uml-utilities ifenslave

bonding.sh

#!/bin/bash modprobe bonding mode=0 miimon=100 modprobe tun ifconfig tap0 down ifconfig tap1 down tunctl -u root -g root -t tap0 tunctl -u root -g root -t tap1 ifconfig tap0 up ifconfig tap1 up openvpn /etc/openvpn/server1.conf & openvpn /etc/openvpn/server2.conf & sleep 10 ifconfig bond0 hw ether 00:11:22:33:44:55 ifconfig bond0 10.10.0.1 netmask 255.255.255.252 broadcast 10.10.0.3 up ifenslave bond0 tap0 tap1 ip addr add 10.10.10.1/24 dev tap0 scope link ip addr add 10.10.11.1/24 dev tap1 scope link

Take note of the 'modprobe bonding mode=0 miimon=100' line. I use zero because I would like to try and combine the BW of both modems. The bonding mode is important depending on what you would like to do:

Possible values are:

balance-rr or 0 - Round-robin policy: Transmit packets in sequential order from the first available slave through the last. This mode provides load balancing and fault tolerance. active-backup or 1 - Active-backup policy: Only one slave in the bond is active. A different slave becomes active if, and only if, the active slave fails. The bond's MAC address is externally visible on only one port (network adapter) to avoid confusing the switch. balance-xor or 2 - XOR policy: Transmit based on the selected transmit hash policy. The default policy is a simple ( {source} \oplus {destination} ) % n_{slaves} - Alternate transmit policies may be selected via the xmit_hash_policy option. This mode provides load balancing and fault tolerance. broadcast or 3 - Broadcast policy: transmits everything on all slave interfaces. This mode provides fault tolerance. 802.3ad or 4 - IEEE 802.3ad Dynamic link aggregation. Creates aggregation groups that share the same speed and duplex settings. Utilizes all slaves in the active aggregator according to the 802.3ad specification. (Need Switch Support) balance-tlb or 5 - Adaptive transmit load balancing: channel bonding that does not require any special switch support. The outgoing traffic is distributed according to the current load (computed relative to the speed) on each slave. Incoming traffic is received by the current slave. If the receiving slave fails, another slave takes over the MAC address of the failed receiving slave. Need Switch Support) balance-alb or 6 - Adaptive load balancing: includes balance-tlb plus receive load balancing (rlb) for IPV4 traffic, and does not require any special switch support. The receive load balancing is achieved by ARP negotiation.

More info here: http://www.linuxfoundation.org/collaborate/workgroups/networking/bonding

Cleanup Script

You will also want to cleanup the bonding script after exit or when you need to: cleanup.sh

#!/bin/bash ip addr del 10.10.0.1/32 dev lo ip addr del 10.10.10.1/24 dev tap1 ip addr del 10.10.11.1/24 dev tap0 killall -9 openvpn rmmod bonding rmmod tap

Client Configuration

Configuring the modems, vpns, bonding, forwarding, to connect adn route to the vpn server.

USB Drive/Modem CD Rom Eject

I have some novatel wireless usb modems. They have built in mini sd card holders and emulated cdrom drives on them for drivers. To get them to work in linux or bsd you need to eject the drive.

The first step is to disble some settings for the usb devices in windows. You cannot get around it. I had to disable "Enable Removable Disk" and Enable CD- ROM Disk in my software card manager that I installed in windows. (VZAccess Manager)

You would think that disabling the drive would disable it all the way. It does not. You have to make the OS eject it on plugin.

The next step is to get the OS to eject the CD drive:

After you plug the device in edit /etc/udev/70-persistent-cd.rules find your device (Novatel_Mass_Storage) and add:

, RUN+="/usr/bin/eject %k"

You will have to do this for each of these type of modems.

Testing Modem with wvdial

I used wvdial to test the modem. It looks like pppd accepts .chat scripts too. Here is my wvdial conf script:

/etc/wvdial.conf

[Dialer Defaults] Init1 = ATZ #Init2 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0 Init2 = ATQ V1 E1 S0=0 &C1 &D2 +FCLASS=0 Init3 - ATQ V1 E1 S0=0 &C1 &D2 +FCLASS=0 #? - Init5 = AT+CGDCONT=1,"IP","" Carrier Check = yes Dial Command = ATX1DT Modem Type = Analog Modem Baud = 460800 New PPPD = yes Modem = /dev/ttyUSB0 ISDN = 0 Phone = #777 Password = JustAnyOldPW Username = [email protected]

Replace 5555555555 with your devices phone number. I cant remember what the no auth option is. It was actually working with any number...so... ppd and chat scripts

We are going to use pppd to manage and connect to the modems.

I used the commands: pppd updetach defaultroute usepeerdns noipdefault debug noauth asyncmap 0 ipcp-accept-local ipcp-accept-remote modem crtscts noauth connect '/usr/sbin/chat -t5 -v -e -E -f /etc/ppp/peers/vzw_chat' 100 460800 /dev/ttyUSB0 pppd updetach defaultroute usepeerdns noipdefault debug noauth asyncmap 0 ipcp-accept-local ipcp-accept-remote modem crtscts noauth connect '/usr/sbin/chat -t5 -v -e -E -f /etc/ppp/peers/vzw_chat' 100 460800 /dev/ttyUSB4

I put the command in a pppup.sh file along with an & char and they would not finish running. I have to look into it.

This is the chat script I use for verizon:

/etc/ppp/peers/vzw_chat

ABORT BUSY ABORT 'NO CARRIER' ABORT ERROR REPORT CONNECT TIMEOUT 10 "" "ATZ" OK 'ATQ V1 E1 S0=0 &C1 &D2 +FCLASS=0' OK 'ATQ V1 E1 S0=0 &C1 &D2 +FCLASS=0' SAY "Dialing" TIMEOUT 30 OK "ATX1DT#777" CONNECT ' '

OpenVPN on the Client

Install openvpn: aptitude install openvpn

Disable automatic startup:

/etc/init.d/openvpn stop update-rc.d openvpn disable

You need your ta.key and client.p12 file from the server. Get them to the client.

You need a client config for each modem: openvpnexampleclient.conf

remote so.me.ip.add 55555 client dev tap1 proto udp ping 2 ping-restart 10 resolv-retry infinite cipher AES-256-CBC tls-client persist-key #persist-tun tun-mtu 1500 pkcs12 /etc/openvpn/thenameofyourcert.p12 tls-auth /etc/openvpn/ta.key 1 ns-cert-type server

#comp-lzo verb 3 txqueuelen 10 no-replay

Once again you will have to change the tap interface number and remote server port.

Make a scripts dir in /etc/openvpn and put these two files in it: entap0.sh

#!/bin/bash ip addr add 10.45.46.2/24 dev tap0 ifenslave bond0 tap0 }}

{{File|name=entap0.sh|content= #!/bin/bash ip addr add 10.45.47.2/24 dev tap1 ifenslave bond0 tap1

Bonding Script

Here is the script that you will use to connect to the server and bond the taps. You will first need to install some commands called by it: aptitude install ipcalc uml-utilities

bonding.sh

#!/bin/bash ip route del default ip route del default ppp_ip0=`ip addr show ppp0 | grep inet | awk '{ print $2 }'` ppp_ip1=`ip addr show ppp1 | grep inet | awk '{ print $2 }'` ppp_pip0=`ip addr show ppp0 | grep inet | awk '{ print $4 }'` ppp_pip1=`ip addr show ppp1 | grep inet | awk '{ print $4 }'` ppp_gwip0=`ipcalc "$ppp_pip0" | grep Address | awk '{ print $2 }'` ppp_gwip1=`ipcalc "$ppp_pip1" | grep Address | awk '{ print $2 }'` echo "$ppp_ip0" "$ppp_ip1" "$ppp_pip0" "$ppp_pip1" "$ppp_gwip0" "$ppp_gwip1" ip route add "$ppp_gwip0" dev ppp0 src "$ppp_ip0" scope link table 100 ip route add "$ppp_gwip1" dev ppp1 src "$ppp_ip1" scope link table 101 ip route add default via "$ppp_gwip0" dev ppp0 table 100 ip route add default via "$ppp_gwip1" dev ppp1 table 101 ip rule add from "$ppp_ip0" table 100 prio 20000 ip rule add from "$ppp_ip1" table 101 prio 20001 modprobe bonding mode=0 miimon=100 modprobe tun tunctl -d tap0 tunctl -d tap1 echo "$ppp_ip0" "$ppp_ip1" openvpn --config /etc/openvpn/client.21456.conf --local "$ppp_ip0" --script-security 2 --up /etc/openvpn/scripts/entap0.sh & openvpn --config /etc/openvpn/client.21466.conf --local "$ppp_ip1" --script-security 2 --up /etc/openvpn/scripts/entap1.sh & ifconfig bond0 hw ether 55:44:33:22:11:00 ifconfig bond0 10.10.0.2 netmask 255.255.255.252 broadcast 10.10.0.3 up ip route add default via 10.10.0.1 dev bond0

Edit it with your openvpn config file locations.

Cleanup Script

To clean the client vpn/routes/bond: cleanup.sh

#!/bin/bash ip addr del 10.10.0.2/32 dev lo ip addr del 10.10.10.2/24 dev tap1 ip addr del 10.10.11.2/24 dev tap0 killall -9 openvpn killall -9 openvpn ppp_ip0=ip addr show ppp0 | grep inet | awk '{ print $2 }' ppp_ip1=ip addr show ppp1 | grep inet | awk '{ print $2 }' ip addr del "$ppp_ip0" dev ppp0 ip addr del "$ppp_ip1" dev ppp1 killall -9 pppd killall -9 pppd

#Still need a way te reset the modems #pccardctl eject #pccardctl insert ip route flush table 100 ip route flush table 101 rmmod bonding rmmod tap

NAT Forwarding as Internet Gateway

The entire reason I wanted to do this was to forward internet traffic through multiple modems. So on my debian box: nano /etc/sysctl.conf

Uncomment: #net.ipv4.ip_forward=1 echo 1 > /proc/sys/net/ipv4/ip_forward

You then can forward incoming traffic with: -t nat -A POSTROUTING -o bond0 -j MASQUERADE

nano iptables.nat.sh

And put the line in it so we can call the file later when we want.

On the server you want to forward traffic from bond0 and on the client from whatever interface your want to share from/turn into a gateway.

If you use ufw you will need to do it in the ufw files. Notes

http://mailman.ds9a.nl/pipermail/lartc/2007q3/021307.html - OpenVPN failover bond tun0 http://wiki.leipzig.freifunk.net/ChannelBonding http://www.enterprisenetworkingplanet.com/linux_unix/article.php/3850636/Understanding-NIC-Bonding-with-Linux.htm http://stackoverflow.com/questions/9357365/bonding-two-tun-device-connected-to-openvpn http://ubuntuforums.org/showthread.php?t=1418377 http://forums.debian.net/viewtopic.php?f=10&t=67794 http://code.google.com/p/vmdialer/ pfSense http://skear.hubpages.com/hub/Dual-Wan-Router-How-To-Build-One-On-a-Budget

With the USB760 modem you need to eject the drive before it works. cdcontrol -f /dev/cd0

This needs to be automated. This thread suggests a devd rule: http://forum.pfsense.org/index.php/topic,43285.0.html bsd //more

The modem worked great once I ejected it. I did not see the type of bonding I wanted. I am 99% sure this can do layer 3 loadbalancing though.

I did not go further atm.

With the UML290: the dial command is not #777 but instead *99***3# Username is: devicephone#@vzw4g.com Password is: vzw

Notes

http://blog.martinshouse.com/2012/01/multi-wan-multi-lan-no-nat-routing-with.html - Failover - Different then LAGG http://doc.pfsense.org/index.php/LAGG_Interfaces http://forum.pfsense.org/index.php/topic,16923.0.html http://forum.pfsense.org/index.php/topic,43285.0.html http://skear.hubpages.com/hub/Dual-Wan-Router-How-To-Build-One-On-a-Budget ZeroShell Zeroshell is a for servers and embedded devices aimed at providing the main network services a LAN requires. It is available in the form of Live CD or Compact Flash image and you can configure and administer it using your web browser.

Says it supports VPN Bonding 3G support (+It is linux)

I was able to get ZeroShell to work. It has a nice interface but is made to be run off a . I need the flexibility of a custom router.

To get my modem to work I had to move the eject command from a 32bit fedora rpm to the system.

ZeroShell does vpn bonding just like the one that I made in this artical.

Notes

http://www.zeroshell.net/eng/forum/viewtopic.php?t=2969&sid=1a794e9b20193f03e3c5306f93faf4e9 - However I tried doing this using various permutations, over multiple 3G connections and whilst it did work, if one of the connections in the bond fails, it doesn't fail particularly gracefully. http://www.zeroshell.net/eng/forum/viewtopic.php?t=1487&sid=3743889f4a3a308582ee5ac6f4899dba - I am trying to get aggregate bandwidth by bonding multiple vpn connections over 3g dongles and keep running into the following issues http://www.zeroshell.net/eng/load-balancing-failover/#vpn-bonding http://www.ieee754.org/?p=4 http://beusergroup.co.uk/technotes/index.php/VPN_Bonding http://www.zeroshell.net/eng/UMTS-HSDPA-Mobile-Router/ http://www.linuxplanet.com/linuxplanet/tutorials/6799/1 http://digilander.libero.it/smasherdevourer/schede/linux/zeroshellEN.pdf http://oss.oetiker.ch/smokeping/ udev/cdcontrol Creation

http://wiki.debian.org/udev http://ubuntuforums.org/showthread.php?t=1002262 http://forums.gentoo.org/viewtopic-t-747416-start-0.html http://forum.eeeuser.com/index.php?showtopic=52086

Retrieved from "https://wiki.hackspherelabs.com/index.php?title=Connection_and_VPN_Bonding&oldid=2313"

This page was last modified on 16 June 2015, at 08:26. Content is available under Attribution 3.0 Unported unless otherwise noted.