Design and Management of Email Service Source : homepage.ntu.edu.tw/~jsc/2005-mail.ppt Outline Introduction to the architecture and operation of SMTP Design of a suitable email system – Webmail solutions Postfix and simple configuration samples Spam and virus filtering Conclusion 2 Overview Electronic mail service has already evolved into one of the major Internet applications. It is not only fundamental, but also a must. Users may become impatient when mails were delayed, not to mention failed to access their emails. – Imagine we meet the situation of power failure or cut of water supply 3 Architecture of a Simple Mail System Consists of the following components – MTA - Mail transfer agent Sending and forwarding emails Server end – MDA - Mail delivery agent Delivering emails to recipients’ mailbox Server end – Pop3/Imap4 Daemons For users to download their mailboxs Server end – MUA - Mail user agent Reading and composing emails 4 Client end Architecture of a Simple Mail System Protocols Used for Mail System Protocols – For computer programs to communicate with each other – Similar to languages that human beings speak SMTP – Simple Mail Transfer Protocol – Too simple to provide any “advanced features” Authentication Authorization POP3 – Post Office Protocol version 3 – Simple IMAP4 – Internet Message Access Protocol version 4 – Fully compatible with internet message standards, e.g. MIME. – Allow messages to be accessed from more than one computer. – Provide support for online, offline, and disconnected modes. 6 – Multiple and share folders. Mail Forwarding Between Servers How to Find the Way to the Destination? How do we find the way to [email protected]? 8 DNS: The Key to All Internet Services Query DNS server by the address part of email address.([email protected]) 1. Query for MX records 2. Query for A records 3. Give up! Not required to exist a real machine hosting the address.(ntu.edu.tw) Adding MX records to improve query performance and to reduce unnecessary query overhead. 9 DNS: Query Results Precedence value of MX records – Lower values with higher priority 10 DNS: Query Results When MX record is absent, second query is required to issue. 11 Features Required By Modern Mail Systems This is what end users care. Ease of use interface – IMAP4 – Webmail Security – Authentication – Encrypted transmission Spam and virus filtering – The only effective way to prevent from virus infection is to remind and force your users to install antivirus softwares. 12 – Stop using M$ systems is alternative solution. Make the Right Decision For the Promise of Performance MTA – Postfix Our suggestion! – Sendmail Not scalable! Lack of new features. Not efficiency in resource management. Rich of documents and tutorials – Qmail Termination of development. – Exim Immature. – M$ Exchange Seeking trouble for yourself? POP3/IMAP4 – UW-IMAP – Courier-IMAP Supports Maildir format. – Dovecot Supports both mbox and Maildir format. Still under development. – Cyrus Proprietary spool format. 13 Not easy to convert between other formats. Webmail: The Trend of Browser Based Mail Reader Your determination depends on the system architecture. IMAP4 based – Access the mailbox via IMAP4 protocol locally or remotely. – Flexible, scalable – Speed may be limited by protocol overhead and network transmission. – Horde IMP project Locally direct access – Access the mailbox locally, proprietary protocol. – Could only run on local mail storage server. – NFS could be applied when deploying multiple servers – Extremely fast!! iff system not overloaded. – Not scale well when you want to expand. – Openwebmail Excellent project developed by NCKU, Taiwan Marvelous support to Chinese encodings Only support mbox format currently 14 Webmail Architecture – IMAP4 based Webmail Architecture – Locally Direct Access Postfix: The Modern and Advance MTA http://www.postfix.org Developed by Wietse Venema Features – Easy replacement of existing sendmail system by binaries with same file name and functionality . e.g. sendmail, newaliases – Human readable configuration file – Multiple small programs with limited execution privilege – Multiple queues to avoid block of normal mails – Backend database lookup table supported – Better support for virtual host – Virtual user system supported – Content filtering interface 17 – Simple spam check/block mechanism Postfix - the Big Picture http://www.postfix.org/big-picture.html – Yellow ellipsoids are mail programs. – Yellow boxes are mail queues or files. – Blue boxes are lookup tables. 18 Postfix Installation Debian Linux – apt-get install postfix-tls libsasl7 libsasl-modules-plain courier-imap Redhat/Fedora Linux – rpm -ivh postfix-2.2.x.i386.rpm – rpm -ivh cyrus-sasl-2.1.21.i386.rpm BSD – Use the ports system – /usr/ports/mail/postfix – /usr/ports/security/cyrus-sasl2 19 – /usr/ports/mail/courier-imap Post Installation Postfix Configuration – master.cf Similar to inetd.conf Control the behavior of small programs – In contrast against sendmail, with one binary and one config file – main.cf The main configuration of the mail system – In general cases, no modification is required for a simple setup. Commands to control the postfix program – postfix start – postfix stop 20 – postfix reload Test For Receiving Email Send an email to yourself. Check system logs – /var/log/maillog An simple test from [email protected] to [email protected] 21 Frequently Used Configuration Options In main.cf myhostname = mail.cc.ntu.edu.tw home_mailbox = Maildir/ mydomain = cc.ntu.edu.tw message_size_limit = 20971520 myorigin = $mydomain smtpd_recipient_limit = 1000 mydestination = $myhostname, localhost.$mydomain – On a mail domain gateway, you should also include $mydomain. mynetworks_style = host relay_domains = $mydestination 22 SMTP/Authentication Traditional mail servers could only determine their relay policy based on connected ip addresses. Insufficient in face of the modern network – DHCP, NAT – ADSL and dialup users – Roaming users Mobile, wireless SMTP Authentication – Username/password authentication for each mail delivery. – Reduce the chance to relay spam mails for anonymous. 23 Email Headers Envelope header – Negotiated between mail servers – Typical tricks that SPAM mail plays Why do I receive mails that are not destined to me? Content Text – Mail servers will not examine the content in general cases. 24 Real World Snail Mail Example Postman could only deliver the mail by envelope information. Sender address is not verified when email delivery. – Trick of SPAM mails. Content may be irrelevant or conflict with envelope. 25 Virus and Spam Mail Virus mail – Contain virus infected attachment – How do we determine if the mail is virus infected? Simple and easy. Base on the inspection and judgment of antivirus software. Spam mail – UCE/UBE Unsolicited Commercial Email Unsolicited Bulk Email – How do we determine if the mail is a Spam mail? Difficult. It’s not a yes/no problem. 26 One person’s meat may be another’s poison. Issues When Dealing With SPAM Do not block or discard users’ mail unless you have the authorization and delegation. The only safe and acceptable assistant you can provide is to score and tag each mail and let your users to do the rest themselves. To warn or not to warn the senders/recipients is a dilemma! – Virus and spam senders nowadays always fake the from address. – System notification mails may be another kind of SPAM. The result will be blamed by your users and blocked by 27 other mail servers. Greylisting Recently proposed (2003) method to block spam and virus mails. RFC requires MTA to be capable to support queue and retry of temporarily undeliverable mails. – Spam and virus always fire and forget. – For efficiency concern, usually they do not retry. How it works? – In brief, block every delivery at its first try. – For each mail delivery, check if the the triplet(source ip, sender, recipient) was seen recently. If yes, pass the mail. If not, reject the delivery with “450 Please retry later.” 28 Greylisting Drawback – Mail delivery will suffer a delay ranging from several minutes to hours. Depending on the remote MTA configuration. Risk – Sites that do not retry their mail delivery. Some bank notifications were observed to try only once. Mail sending programs that process the SMTP transaction themselves without calling the “sendmail” MTA interface. Suggestion – Whitelisted known hosts to prevent from delay and loss of mail. 29 Greylisting Benefit – Effectively reduce the volume of spam and virus. – Effectively reduce both cpu and disk load of servers. 30 Softwares To Block/Filter Spam and Virus Postfix – Content filter interface Postgrey – Greylisting policy server Amavisd-new – http://www.ijs.si/software/amavisd – http://www.ijs.si/software/amavisd/README.postfix Clam Antivirus – http://www.clamav.net – A GPL virus scanner SpamAssassin – http://spamassassin.apache.org 31 Software Architecture for Filtering 32 Amavisd-new A high performance interface between MTA and content checkers. – Calling external antivirus programs to do virus scanning. – Calling external spamassassin program to do spam level determination. – CPU intensive workloads. – Can be flexibly configured to pass, discard, or quarantine mails based on user defined policy. Pass spam mails with score > 10 with subject prepended the *** SPAM *** keyword. Quarantine spam mails with score > 20. Discard spam mails with score > 30. Quarantine