How Internet Email Works “Everything you never wanted to know about email but were afraid to ask...” Dave O'Neill <dmo@{dmo.ca,roaringpenguin.com}> So, you use email... ● ... but do you know how it works? ● If you don't care how it works, leave now, or you'll be bored silly. How many of you... ● know what SMTP is? ● know what an MX is? ● can understand your email headers? ● Know what R$* < @ [ $+ ] : $- : $*> $* $#$3 $@ $4 $: $1 < @ [$2] > $5 means? The Whole Mess ● What a typical email system looks like The Whole Mess Mail User Agents ● Reading mail ● Managing folders ● Composing mail ● Examples: – Mutt – Thunderbird – Outlook Mail Transfer Agents ● Transfer mail to another user or system ● Examples: – Sendmail – Postfix – Qmail – Exchange Mail Delivery Agents ● Deliver mail to an individual mailbox ● Examples: – procmail – maildrop – mail.local Mail Retrieval Agents ● Retrieve mail on a remote machine ● Pretty much the opposite of an MTA ● Examples: – fetchmail – getmail Okay, but... ● Much of the time, those four theoretical components don't exist separately ● Even when they do, the lines get a bit blurry ● Mail software usually incorporates two (or more) of MUA, MTA, MRA, MDA Mozilla Thunderbird ● Is a MUA ● Also acts as an MRA, reading mail via IMAP and POP3 ● Also acts as a minimal MTA, sending to your SMTP server ● Most GUI mail clients have MRA functionality Sendmail ● Is an MTA ● But it comes bundled with an MDA ● Most Sendmail people use a different MDA, like Procmail MS Exchange ● Exchange provides an MTA, MRA, MDA, and MUA ● Many people think it does none of these well The MUA ● Reads mail delivered by an MDA, or accessed via an MRA ● Sends via an MTA ● If it reads and writes email, it's an MUA. The MTA ● An MTA takes an incoming mail message and determines where and how to transfer it ● Internet email is sent via SMTP over port 25 What's port 25? ● TCP port 25 is where SMTP is usually sent and received ● If you want to receive email on the internet, you need to have a server listening on port 25 ● If you want to direct mail for a particular domain to a particular server, you publish an MX record in your DNS What's an MX record? ● Mail eXchange record ● Tells senders what server, or servers, accept mail for a domain ● Listed in order of priority – lower number is preferred host ● Multiple MX hosts can be good for reliability MX Information $ host -t mx dmo.ca dmo.ca mail is handled by 10 mail.dmo.ca. dmo.ca mail is handled by 15 home.dmo.ca. What's SMTP ● It's Simple Mail Transfer Protocol ● It's RFC 2821 ● It's not entirely simple anymore ● Here's a simple example anyway... SMTP In Action dmo@bowmore:~$ telnet mail.dmo.ca 25 Trying 66.11.168.35... Connected to colo.dmo.ca. Escape character is ©^]©. 220 colo.dmo.ca ESMTP Sendmail (No UCE/UBE) logging access from: home.dmo.ca(OK)-home.dmo.ca [209.217.122.203] HELO dmo.ca 250 colo.dmo.ca Hello home.dmo.ca [209.217.122.203], pleased to meet you MAIL FROM:<[email protected]> 250 2.1.0 <[email protected]>... Sender ok RCPT TO:<[email protected]> 250 2.1.5 <[email protected]>... Recipient ok DATA 354 Enter mail, end with "." on a line by itself Subject: This is a minimal message From: Dave O©Neill <[email protected]> To: Dave O©Neill <[email protected]> Date: Sun, 05 Dec 2004 20:57:00 -0500 Here©s the body text . 250 2.0.0 iB61u9ds019688 Message accepted for delivery Other Transfer Protocols ● UUCP ● X.400 ● MAPI ● Evil Hacks – Mail::Queue::DB The MDA ● The MTA gives the MDA a message ● The MDA delivers it to a user mailbox ● And maybe it does other things... Procmail ● An MDA that does other things ● Sort email from mailing lists ● Client-side spam filtering ● Call other programs Fun With Procmail MAILDIR=$HOME/mail LOGFILE=$HOME/log/procmail NEWDATE="Á/bin/date +%Y-%mÁ" THREADKILLER=/home/dmo/bin/killthread.pl ## Backup all messages first :0 c: archive/backup-$NEWDATE # First, killfile an individual or two: :0 * ^From:.*[email protected] | $THREADKILLER kill :0 * ^List-Id:.*oclug.lists.oclug.on.ca { :0 hW | $THREADKILLER check :0: oclug/list/$NEWDATE } Sieve ● RFC 3028 ● A filtering language like procmail, but not quite as confusing, destructive, or powerful ● Designed for GUI filter editors require "fileinto"; if header :contains "From" "[email protected]" { discard; } elsif header :contains "List-Id:" "oclug.lists.oclug.on.ca" { fileinto "oclug/list"; } The MRA ● Retrieves mail from another server ● Gives it to an MTA, MDA, or MUA ● Common protocols are POP3 and IMAP POP3 ● Post Office Protocol, version 3 ● Defined in RFC 1939 ● Limited to one remote folder per account IMAP ● Internet Mail Access Protocol ● Latest version defined in RFC 3501 ● Allows multiple remote folders, shared folders, updating of flags on messages, etc ● More feature-ful than POP3. Also more resource-intensive Reading Headers ● Know where your mail comes from ● Know where your spam comes from ● Figure out which address you used to subscribe to that list Let's see that message Received: from localhost (bowmore [127.0.0.1]) by bowmore.i.dmo.ca (8.12.11.Beta0/8.12.11.Beta0/Debian-1) with ESMTP id i79JfKjn002191 for <dmo@localhost>; Mon, 9 Aug 2004 15:41:20 -0400 Received: from mail.dmo.ca by localhost with IMAP (fetchmail-6.2.4) for dmo@localhost (single-drop); Mon, 09 Aug 2004 15:41:20 -0400 (EDT) Received: from tux.oclug.on.ca (tux.oclug.on.ca [204.225.221.10]) by mail.dmo.ca (8.12.11/8.12.11/Debian-3) with ESMTP id i79JTlse016302 for <[email protected]>; Mon, 9 Aug 2004 15:29:48 -0400 Received: from localhost (localhost [127.0.0.1]) by tux.oclug.on.ca (Postfix) with ESMTP id 25D7D47837 for <[email protected]>; Mon, 9 Aug 2004 15:29:31 -0400 (EDT) Received: from colo.roaringpenguin.com (colo.roaringpenguin.com [nnn.nnn.nnn.nnn]) by tux.oclug.on.ca (Postfix) with ESMTP id 5B24347836 for <[email protected]>; Mon, 9 Aug 2004 15:29:30 -0400 (EDT) Headers, continued Received: from (hydrogen.roaringpenguin.com [nnn.nnn.nnn.nnn]) by colo.roaringpenguin.com (8.13.1/8.13.0) with ESMTP id i79JTS2i010158 for <[email protected]>; Mon, 9 Aug 2004 15:29:28 -0400 Received: from oxygen.roaringpenguin.com (oxygen.roaringpenguin.com [192.168.10.8]) by hydrogen.roaringpenguin.com (8.12.10/8.12.10) with ESMTP id i79JTSCl014641 for <[email protected]>; Mon, 9 Aug 2004 15:29:28 -0400 Email Really Sucks ● No security ● Designed for plain ASCII text ● Unicast ● and everyone used to be so trustworthy... Email Security ● SMTP used to have no notion of authentication ● SMTP, IMAP, and POP3 protocols are all cleartext ● Mail relays can see/modify anything SSL ● SMTP, IMAP, and POP3 can all work over SSL, assuming your client and server support it ● At least then nobody in the middle can snoop your data or your passwords ● If you're on a public network, you should probably be using this SMTP AUTH ● SMTP servers used to let anyone send to anyone ● Now, most relays only allow you to – send to addresses they handle mail for – send from addresses within their networks ● What if you're remote, and need to send mail? ● SMTP has an AUTH extension now Email Encryption ● Keep your data secret from all but the intended recipient ● Prove authenticity of sender's identity ● Beyond the scope of this talk MIME ● Multipurpose Internet Mail Extensions ● Lets you send arbitrary binary data via email ● Simple idea, complicated implementation Aliases and Lists ● Email is mainly a one-to-one medium ● Discussion groups are a hack added on through aliases and listservs ● An alias is an MTA feature to resend inbound mail to one or more addresses ● A listserv is a program that expands upon this to provide other features Spam / UCE ● Unwanted email advertisements ● Filtering is now a necessity ● Filtering is a big enough topic to require its own talk Client-Side Filtering ● Typically performed by the MUA or MDA ● Hides or deletes unwanted messages ● You've already wasted bandwidth, but at least you don't waste your time viewing them Server-Side Filtering ● Typically performed by MTA or MDA ● Lets you reject messages before the end user has to download it ● May even be able to reject before SMTP conversation is over (RBLs, inline filtering milter plugins, etc) Discussion and Arguments.