Internet Mail Local Mail
Web Based smtpd Anti - Spam Control Panel on port 25 pickup (Apache - PHP)
with Anti UCE MySQL smtpd Interface cleanup on port 1025
Postfix LMTP Queue
qmgr Perl DBI
Amavisd-new on port 1024 smtp local
Unix Socket mailbox Perl Module Cal
Clamav
LMTP=Local Mail Transfer Protocol SMTP=Simple Mail Transfer Protocol smtpd=Simple Mail Transfer Protocol Daemon qmgr=Queue Manager
Overview
Postfix is a mail transport agent written by security researcher Wietse Venema. Not surprisingly, Postfix is designed from the ground up to be a highly secure system. It consists of several components, each of which runs with least privilege and none of which trust data from the other without validating it themselves. Despite the extensive security emphasis in the system's architecture, Postfix is capable of very good performance in normal conditions; because of architectural decisions, it is also fault tolerant and capable of good performance under adverse conditions such as resource starvation.
Postfix supports three content inspection methods, ranging from light - weight one-line-at-a-time scanning before mail is queued, to heavy duty machinery that does sophisticated content analysis after mail is queued. Each approach serves a different purpose.
The method shown in this article inspects mail AFTER it is stored in the queue, and uses standard protocols such as SMTP. After-queue inspection allows you to use content filters of arbitrary complexity without causing timeouts while receiving mail, and without running out of memory resources under a peak load.
The SpamAssassin system is software for analyzing email messages, determining how likely they are to be spam, and reporting its conclusions. It is a rule-based system that compares different parts of email messages with a large set of rules. Each rule adds or removes points from a message's Spam score. A message with a high enough score is reported to be spam.
Amavisd-New is a high-performance and reliable interface between mailer (MTA) and one or more content checkers: virus scanners, and SpamAssassin. It is written in Perl, assuring high reliability, portability and maintainability. It talks to MTA via (E)SMTP or LMTP, or by using helper programs. No timing gaps exist in the design, which could cause a mail loss.
It is normally positioned at or near a central mailer, not necessarily where user's mailboxes and final delivery takes place. When calling of Mail::SpamAssassin (SA) is enabled, it calls SA only once per message (regardless of the number of recipients), and tries very hard to correctly honors per-recipient preferences, such as pass/reject, and inserting spam-related mail header fields.
Document assumes a fresh Linux install with at least 'Editors' and 'Base'. Use your favorite editor if you don't want to use vim. A few commands in this document will wrap - be sure you place the entire command on one line. This document does not cover a lot thing like configuring the firewall or disabling unnecessary daemons. Its focus is getting you to the point you can log on to Maia. This document is not a tutorial for any of the programs listed here. You can edit this document to customize it. The host name I use is 'ahost'. The domain name used is 'example.com' and the administrator's email name is 'admin' # Root's MySQL password is 'roots_password' # Do a search and replace on those four items.
# Optional yum update
# Note however that if you don't run 'yum update' you will need to at least update perl: yum update perl # see https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=218916 vi /etc/sysconfig/i18n
# and change to a non UTF8 Lang. e.g. "en_US.UTF-8" is changed to "en_US"
# a mail server should keep accurate time yum -y install ntp ntpdate clock.fmt.he.net ntpdate ntp1.tummy.com chkconfig --level 235 ntpd on hwclock --systohc service ntpd start adduser -r -d /var/amavisd -c "amavis user" -s /bin/sh amavis
# add yourself (as the maia administrator) adduser admin passwd admin vi /etc/hosts # in the /etc/hosts file. I do it more like this, adding the actual IP and moving our host name there (this assumes you are using a static IP address): 127.0.0.1 localhost.localdomain localhost 192.168.1.222 ahost.example.com ahost # In this test setup, we don't have DNS MX records set up but we want to be able to locally send test mail to [email protected], so we are going to temporarily add the domain to the interface: 127.0.0.1 localhost.localdomain localhost 192.168.1.222 ahost.example.com ahost example.com reboot mkdir /var/virusmails chown amavis:amavis /var/virusmails mkdir /var/amavisd mkdir /var/amavisd/db mkdir /var/amavisd/var mkdir /var/amavisd/tmp mkdir /var/amavisd/maia mkdir /var/amavisd/maia/scripts chown -R amavis:amavis /var/amavisd chmod -R 750 /var/amavisd yum -y install mysql mysql-server perl-DBD-MySQL perl-DBI subversion patch db4 db4-devel db4-utils yum -y install perl-Crypt-Blowfish perl-Crypt-CBC uuid-perl perl-Template-Toolkit perl-libwww-perl perl-Digest-SHA1 yum -y install perl-HTML-Parser perl-Unix-Syslog perl-IO-stringy perl-Net-Server yum -y install perl-MailTools perl-Convert-UUlib perl-Convert-TNEF perl- Compress-Zlib perl-MIME-tools yum -y install perl-Archive-Zip perl-Archive-Tar perl-BerkeleyDB perl-Digest- SHA1 perl-Digest-HMAC perl-Net-DNS yum -y install php php-pear php-mysql php-gd php-Smarty pax cabextract cpio lha lzop freeze tmpwatch bzip2 yum -y install nomarch binutils gzip spamassassin php-bcmath pyzor gcc bison zlib zlib-devel gmp-devel yum -y install perl-Authen-SASL perl-LDAP perl-URI perl-DBD-SQLite perl- Razor-Agent perl-Mail-SPF-Query
# It wouldn't hurt to run those installs again to make sure we got them all. optional - depends if you want/need them for authentication or encryption or if you decide to use Postgre instead of MySQL yum install php-imap php-ldap php- mcrypt php-pgsql
# I list packages to a file so I can easily grep it later yum list > 1
# read the note before you run 'svn checkout' mkdir /usr/local/src/maia cd /usr/local/src/maia svn checkout https://www.maiamailguard.com/svn/branches/1.0 # answer (p)ermanently? # please make a note of the SVN revision that you get # if you need to download again, first move or remove /usr/local/src/maia then start over # I got 1160 (which I believe is the bundled version of Maia 1.0.2) # note that you can specify particular revision: # svn -r 1160 checkout https://www.maiamailguard.com/svn/branches/1.0
# If you picked a version less than 1184, you need to apply a security patch: cd 1.0/php cp xlogin.php xlogin.php~ cp login.php login.php~ cp internal-init.php internal-init.php~ wget http://www200.pair.com/mecham/spam/maiasecuritypatch1184.txt patch -p0 < maiasecuritypatch1184.txt
# If the patch was applied successfully: rm -f xlogin.php~ rm -f login.php~ rm -f internal-init.php~
# I don't use sendmail yum -y install postfix cyrus-sasl rpm -e sendmail alternatives --config mta # then select /usr/sbin/sendmail.postfix
# set up pyzor and razor chmod -R a+rX /usr/share/doc/pyzor-0.4.0 /usr/bin/pyzor /usr/bin/pyzord chmod -R a+rX /usr/lib/python2.4/site-packages/pyzor su amavis -c 'pyzor discover' echo "82.94.255.100:24441" > /var/amavisd/.pyzor/servers su amavis -c 'razor-admin -create' su amavis -c 'razor-admin -create' su amavis -c 'razor-admin -register'
# (may have to run that one twice)
# disable razor logging (debuglevel = 0) cp /var/amavisd/.razor/razor-agent.conf /tmp/8 sed 's/= 3/= 0/g' /tmp/8 > /var/amavisd/.razor/razor-agent.conf cat /var/amavisd/.razor/razor-agent.conf
# create initial bayes database su amavis -c 'sa-learn --sync' ls -l /var/amavisd/.spamassassin/
###################### # install DCC cd /usr/local/src wget http://www.dcc-servers.net/dcc/source/dcc-dccproc.tar.Z tar xzf dcc-dccproc.tar.Z
# adjust for current version: cd dcc-dccproc-1.3.59 ./configure --with-uid=amavis && make && make install chown -R amavis:amavis /var/dcc ln -s /var/dcc/libexec/cron-dccd /usr/bin/cron-dccd crontab -e # and insert (at the top): 43 11 * * * /usr/bin/cron-dccd ######################
# enable DCC in v310.pre (if you abide by the license) cp /etc/mail/spamassassin/v310.pre /etc/mail/spamassassin/v310.pre-mybackup cp /etc/mail/spamassassin/v310.pre /tmp/7 sed 's/#loadplugin Mail::SpamAssassin::Plugin::DCC/loadplugin Mail::SpamAssassin::Plugin::DCC/g' /tmp/7 > /etc/mail/spamassassin/v310.pre head -30 /etc/mail/spamassassin/v310.pre
# test - look for pyzor, razor, dcc wget http://www200.pair.com/mecham/spam/sample-spam.txt su amavis -c 'spamassassin -D # May want to run those again to make sure you got them all. There is a bug in Pie.php provided with Image_Graph-0.7.2. If you have installed this version of Image_Graph, grab a file from me to fix the bug: cd /usr/share/pear/Image/Graph/Plot/ mv Pie.php Pie.php.orig wget http://www200.pair.com/mecham/Pie.php.txt mv Pie.php.txt Pie.php cd /var/amavisd/maia/scripts cp /usr/local/src/maia/1.0/scripts/* . cp /etc/maia.conf /etc/maia.conf-backup cp /usr/local/src/maia/1.0/maia.conf.dist /etc/maia.conf cp -r /usr/local/src/maia/1.0/templates /var/amavisd/maia/templates chown -R amavis:amavis /var/amavisd chmod -R 750 /var/amavisd chmod 640 /var/amavisd/maia/templates/*.tpl vi /etc/maia.conf # and edit the base_url - I use /mail $base_url = "http://ahost.example.com/mail/"; # you will probably need to make an entry in the hosts file on any computer that you are going to use to browse the server unless a DNS record exists. chkconfig --level 235 mysqld on service mysqld start # If you have not yet added a password: # (substitute a real password for roots_password) mysql SET PASSWORD FOR 'root'@'localhost' = PASSWORD('roots_password'); SET PASSWORD FOR 'root'@'ahost.example.com' = PASSWORD('roots_password'); # literally use the password of 'passwd' here CREATE DATABASE maia; USE maia; SOURCE /usr/local/src/maia/1.0/maia-mysql.sql; GRANT CREATE, DROP, ALTER, SELECT, INSERT, UPDATE, DELETE ON maia.* TO amavis@localhost IDENTIFIED BY 'passwd'; FLUSH PRIVILEGES; quit # You didn't have to have every possible program, but make sure nothing is broken here cd /var/amavisd/maia/scripts/ ./configtest.pl # Setup and run sa-update to get the latest spamassassin rules (make sure there are no error messages here) # you should run sa-update on occasion (and then run a Maia 'load-sa-rules.pl' script to load them into Maia) cd /etc/mail/spamassassin wget http://spamassassin.apache.org/released/GPG-SIGNING-KEY gpg --import GPG-SIGNING-KEY sa-update -D # make sure we lint Ok su amavis -c 'spamassassin --lint' cd /var/amavisd/maia/scripts ./load-sa-rules.pl mkdir /var/www/html/mail cp -r /usr/local/src/maia/1.0/php/* /var/www/html/mail cp /var/www/html/mail/config.php.dist /var/www/html/mail/config.php chgrp amavis /var/www/html/mail/themes/*/compiled chmod 775 /var/www/html/mail/themes/*/compiled # create a symlink so Maia can find the Smarty stuff ln -s /usr/share/Smarty /var/www/html/mail/libs/Smarty # add apache to the amavis group gpasswd -a apache amavis # Sorry, lots of apache things don't work with this setup when SELinux is enabled # and I am not an SELinux expert, so I disable SELinux for the httpd service # You might consider going one step further and disable (or set it to 'permissive', which is what I do) for the whole system # http://fedora.redhat.com/docs/selinux-faq-fc5/#id2962490 setsebool -P httpd_disable_trans 1 # (start or restart) chkconfig --level 235 httpd on service httpd start # (from a client) test out the install: http://ahost.example.com/mail/admin/configtest.php # remember to restart httpd if you make changes # if IMMAPProtocol.php patch is needed: cd /usr/share/pear/Net/ cp IMAPProtocol.php IMAPProtocol-orig.php wget http://www200.pair.com/mecham/spam/IMAPProtocol.php.patch.txt patch IMAPProtocol.php < IMAPProtocol.php.patch.txt # note that patching IMAPProtocol.php will not turn the error message off # also note that we have already patched Image_Graph-0.7.2 cd /etc/init.d wget http://www200.pair.com/mecham/redhat/amavisd_init.sh mv amavisd_init.sh amavis chmod +x amavis chkconfig --add amavis # copy amavisd into place, make backups of current ones if you have them test -e /usr/local/sbin/amavisd && cp /usr/local/sbin/amavisd /usr/local/sbin/amavisd-mybackup test -e /usr/sbin/amavisd && cp /usr/sbin/amavisd /usr/sbin/amavisd-mybackup cp /usr/local/src/maia/1.0/amavisd-maia /usr/sbin/amavisd hash -r # get my custom amavisd.conf - or at least use many of these settings in yours # also get the 2.2.1 amavisd.conf-sample cd /etc mv amavisd.conf amavisd.conf-myoriginal wget http://www.xmission.com/~jmcrc/2.2.1/amavisd.conf-sample wget http://www200.pair.com/mecham/redhat/amavisd.conf-maia cp amavisd.conf-maia amavisd.conf # edit amavisd.conf and set at least $mydomain # include all your domains in @local_domains_maps. e.g.: # @local_domains_maps = ( [".$mydomain", '.example.com', '.example.org'] ); # I change $DO_SYSLOG = 1; so amavisd logs to /var/log/maillog # insure: # @lookup_sql_dsn = ( ['DBI:mysql:maia:localhost', 'amavis', 'passwd'] ); vi /etc/amavisd.conf ###################### # clamd adduser -r -d /var/lib/clamav -c "clamav user" -s /sbin/nologin clamav gpasswd -a clamav amavis # install clamav from source cd /usr/local/src # Download location and version are examples only (but valid when I wrote this), # see http://sourceforge.net/project/showfiles.php?group_id=86638 wget http://easynews.dl.sourceforge.net/sourceforge/clamav/clamav-0.91.2.tar.gz tar xzf clamav-0.91.2.tar.gz cd clamav-0.91.2 ./configure && make && make install cp contrib/init/RedHat/clamd /etc/init.d/ chmod +x /etc/init.d/clamd chkconfig --add clamd mkdir /var/log/clamav/ chown -R clamav:clamav /var/log/clamav/ mkdir /var/run/clamav/ chown -R clamav:clamav /var/run/clamav/ mkdir /var/lib/clamav/ chown -R clamav:clamav /var/lib/clamav/ # cheat and use my conf files for this particular setup (if you want this stuff to work): cd /usr/local/etc/ mv clamd.conf clamd.conf.old mv freshclam.conf freshclam.conf.old wget http://www200.pair.com/mecham/clamav9/clamd.conf wget http://www200.pair.com/mecham/clamav9/freshclam.conf freshclam #then start clamd service clamd start # check for errors: cat /var/log/clamav/clamav.log ######### # For freshclam updates you can either use this script: cd /etc/init.d wget http://www200.pair.com/mecham/redhat/freshclam chmod +x freshclam chkconfig --add freshclam service freshclam start # (or) make a crontab entry: crontab -e # and make the following entry. # change NN here to a number between 1 and 59 (three minutes from now) NN * * * * /usr/local/bin/freshclam --quiet ######### # keep an eye on the freshclam.log to make sure updates are happening # if there is no freshclam.log we are in trouble cat /var/log/clamav/freshclam.log ###################### # Start amavisd in debug mode and check for severe errors. "INFO: no optional modules:" is not a problem, Use [Ctrl]+c to cancel: # Note that there will be modules we are not using. This is not a problem either. Hopefully you get: "Net::Server: Parent ready for children." amavisd debug # If everything looks Ok, cancel debug mode and start amavisd-maia service amavis start ###################### # minimal Postfix configuration:- you are on your own here, this is not a Postfix HOWTO # this delivers locally to one domain (using mbox format) # # If you are relaying mail instead of storing it locally, you may wish to read my Debian document # http://www200.pair.com/mecham/spam/spamfilter20060630.html#maincf # Also, you would not want to have the bare domain in the hosts file, which I placed there for this test. cd /etc/postfix cp main.cf main.cf-myoriginal wget http://www200.pair.com/mecham/spam/amavisd-master.txt cp master.cf master.cf-backup cp master.cf master.cf-myoriginal grep 'smtp-amavis' master.cf || cat master.cf-myoriginal amavisd-master.txt > master.cf cat master.cf postconf -e "alias_maps = hash:/etc/aliases" newaliases postconf -e "myorigin = example.com" postconf -e "mydomain = example.com" postconf -e "myhostname = ahost.example.com" postconf -e "content_filter = smtp-amavis:[127.0.0.1]:10024" postconf -e "inet_interfaces = all" postconf -e "mynetworks = 127.0.0.0/8, 192.168.0.0/16" chkconfig postfix on # system dependent, default is $myhostname, localhost.$mydomain, localhost # this is for local mail delivery postconf -e "mydestination = \$myhostname, localhost.\$mydomain, localhost, \$mydomain" # we escape the $ sign when using postconf - the backslash does not actually go in main.cf # service postfix stop service postfix start postfix flush # did Postfix die? Is so, why?: tail -30 /var/log/maillog | grep postfix ###################### reboot # When the system comes back up, make sure clamd and amavis (and freshclam?) are running ps aux | egrep '(clam|amavis)' | grep -v grep # send a simple test message echo "test" | sendmail [email protected] # the result should be in: tail -30 /var/spool/mail/adminguy # I got (notice the processing by amavisd): ###################### From [email protected] Sat Aug 5 13:37:43 2006 Return-Path: test ###################### # Note that the administrator you create here (by running internal-init.php and login.php?super=register) must be real address because you will get an email at that address with your password. I hope the mail system is functioning enough to actually deliver mail to that user. I am going to use internal authentication, so first: here is a sample of what we are after next: http://www200.pair.com/mecham/spam/authentication4.htm http://ahost.example.com/mail/internal-init.php # If all goes well, you will get a "250 Ok" response on the screen when you submit this page # and your password in a message in your inbox: tail -30 /var/spool/mail/adminguy # Then log in with your email address and password, the first person to log in (hopefully you) to this script will be the super user. http://ahost.example.com/mail/login.php?super=register # You will want to change your password. Send eicar.com.txt virus as an attachment in order to test clamd once you have enabled virus scanning in Maia. # This is just the beginning. Please read the Maia documentation to continue. Make sure you set up maintenance scripts and move the admin directory out of http # http://www.maiamailguard.com/maia/wiki/MaintenanceScripts # The SQL based Bayes and AWL tables have also been created for you. # You can enable them by placing this in /etc/mail/spamassassin/local.cf: bayes_store_module Mail::SpamAssassin::BayesStore::SQL bayes_sql_dsn DBI:mysql:maia:localhost bayes_sql_username amavis bayes_sql_password passwd # Change ::SQL to ::MySQL if SpamAssassin >= 3.1.0 and MySQL >= 4.1 bayes_sql_override_username amavis auto_whitelist_factory Mail::SpamAssassin::SQLBasedAddrList user_awl_dsn DBI:mysql:maia:localhost user_awl_sql_username amavis user_awl_sql_password passwd # You will need to reload amavisd after making that change # If you have not used SQL based Bayes before (you are currently using the default BDB # based Bayes) and you need to migrate Bayes data or AWL data to SQL, see http://www200.pair.com/mecham/spam/fc5-maia-spamassassin-sql.html # Suggested reading: http://www.mysqlperformanceblog.com/2006/09/29/what-to-tune-in-mysql-server- after-installation/ http://www.mysql.com/news-and-events/newsletter/2003-11/a0000000269.html # Out of the box MySQL is poorly tuned for use with InnoDB tables. I suggest at least configuring # innodb_buffer_pool_size and innodb_log_file_size. For example, increasing innodb_buffer_pool_size # (from the default of 8M) to a conservative 128M can make a hugh difference in performance. # In order to change innodb_log_file_size you will need to shut down the server, rename the old # log files and then let MySQL create new ones when it starts back up: http://dev.mysql.com/doc/refman/5.0/en/adding-and-removing.html wget http://spamassassin.apache.org/gtube/gtube.txt sa-learn --spam gtube.txt sa-learn --dump magic ###################### # SELinux problems will drive you crazy (grep denied /var/log/messages) # stuff will not work and you will not know why - first check 'grep denied /var/log/messages' # see http://fedora.redhat.com/docs/selinux-faq-fc5/#id2962490 # and 'man selinux' # http://fedoraproject.org/wiki/SELinux/apache # If you disable SELinux, then enable it, stuff may stop working. # # If you value your sanity, turn it off and forget about it # or try setting it to 'permissive' if you like # or you could spend the next few months figuring it all out. If you do, tell the rest of us how to deal with it. # # I would also suggest adding (purchasing) an additional virus scanner - something like Kaspersky or Bitdefender http://www.maiamailguard.com/docs.php # Suggested reading (settings like innodb_buffer_pool_size go in the [mysqld] section of /etc/my.cnf): # run find /usr/share/doc -name my-*.cnf for samples