SMTP Authentication [Tutorial] (last edit: 2011-08-19 -- this page exists since 2003) Introduction ...................................................................................................................................................................... 1 Request For Comments ..................................................................................................................................................... 2 Consequences for the ESMTP procedure ......................................................................................................................... 3 Authentication Framework ............................................................................................................................................... 5 Authentication and Transport Layer Security ................................................................................................................. 12 Usernames & Realms ...................................................................................................................................................... 13 Summary & Conclusion ................................................................................................................................................... 16 Introduction SMTP Authentication is a scheme which was introduced in 1999 by J. Myers of Netscape Communications and fiGnally released as RFC 2554 ("SMTP Service Extension for Authentication"). It is partly based on the SMTP Service Extensions as defined in RFC 1869. Most modern SMTP implementations support SMTP Authentication, whereas Qmail 1.03 does not (without a patch). On the other hand, a lot of Mail User Agents (MUAs) - which include a SMTP Client - make SMTP Authentication available (e.g. Outlook, Eudora, Netscape, Mozilla, The Bat! ....). SMTP Authentication is advertised by the SMTP Authentication server, requires a client to authenticate, while finally both parties have to mutually accept and support the chosen authentication procedure. Originally invented as a Host-to-Host protocol, with SMTP Authentication, a User has to identify itself and after successful authentication, reception/transmission of his/her emails is granted. RFC 2554 does not explicitly state, what advantages/benefits a user has being SMTP authenticated, except that optionally a "security layer" for subsequent protocol interactions may be chosen. However, in common sense, an authenticated user is allowed for email transmission not only to the target system (the SMTP server) but rather anywhere. In Qmail terminology, this is equivalent to a 'relayclient'. SMTP Authentication takes some ideas of the Simple Authentication and Security Layer (SASL) and does not fit well into the SMTP scheme, as will be outlined in this document. 1 Request For Comments In order to understand SMTP Authentication, one has to work through several RFC, which seem to be unrelated in the first place. On the other hand, the most recent SMTP RFC 5321 and it's predecessor RFC 2821 (by John Klensin) now at least mentions the existance of SMTP extensions and - by the same token - requiring the 'EHLO' command introducing a SMTP transaction. Even after all those years, it really would be time, to have more coherent SMTP RFCs; see also the comments of Dan Bernstein about the " Klensin RFC". (E)SMTP is not that such a difficult protocol to not at least cover the basics in one document - while removing obsolete commands like VRFY and EXPN. RFC 1869 defines a protocol extension (ESMTP) for the SMTP dialog, in order to indicate extended capabilities by the SMTP Server and/or to transmit additional SMTP protocol information by the SMTP client. SMTP servers, supporting ESMTP, have to use the keyword 'EHLO' in the SMTP greeting. The SMTP client may only use those extensions the server offers. By construction, the server may send the offered extensions as ESMTP verb anywhere in the SMTP dialog or as part of the 'MAIL FROM: ' or 'RCPT TO: ' command. A typical use is 'MAIL FROM: <[email protected]> SIZE=1512'. In this sample, 'SIZE' is the ESMTP keyword, '1512' is the ESMTP value and the whole term 'SIZE=1512' is the ESMTP parameter (RFC 1870 " SMTP Service Extension for Message Size Declaration"). RFC 1869 employes two different schemes to promote the ESMTP value: As ESMTP verb, it uses "SIZE xxxxx", whereas in the 'MAIL FROM: <[email protected]> SIZE=1512' command, the ESMTP keyword and it's value are joined by a "=" equal sign. Every ESMTP keyword has to be registered at the IANA. In this scope, RFC 2554 describes SMTP Authentication with the particular ESMTP keyword 'AUTH'. In the text passages and samples of RFC 2554, the ESMTP Auth values 'CRAM-MD5', 'DIGEST-MD5', and 'PLAIN' are mentioned (which correspond to particular authentication methods or mechanisms) but no reference to any of those is provided. The attempt, to find the meaning of the above mentioned ESMTP AUTH mechanisms in RFC 2222 "Simple Authentication and Security Layer (SASL)" fails. In this RFC (also published by John Myers), only the overall SASL mechanism is outlined and how to register a new "SASL mechanism name". However, the SASL mechanisms 'KERBEROS_V4', 'GSSAPI', and 'SKEY' are defined. In order to succeed, one has to dig out RFC 1731 "IMAP4 Authentication Mechanisms" and RFC 2195 "IMAP/POP Authorize Extension for Simple Challenge/Response". Here, some practical samples for authentication are given based upon the POP3 and IMAP4 protocol. Those RFC are originated as well by John Myers. RFC 2060 "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1" (John Myers, sic) tells about the IMAP4 'LOGIN' command. Too bad; this has nothing in common with the ESMTP 'AUTH LOGIN' method. 2 The way the actual ESMTP Auth values are en-/decoded, corresponds to the BASE64 scheme, which was first described in RFC 1113 "Privacy Enhancement for Internet Electronic Mail: Part I -- Message Encipherment and Authentication Procedures"; though not explicitly declared as BASE64 here (but later called it that). RFC 2045 "Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies" gives an identical outline of the BASE64 "alphabet" in section 6.8. If in addition the Challenge/Response authentication mechanism is used, one has to become familiar with the so- called HMAC procedure from RFC 2104 "HMAC: Keyed-Hashing for Message Authentication" and in addition according to RFC 1321 with "The MD5 Message-Digest Algorithm" as an en-/decryption scheme. Until recently, there was no common understanding, how to propagate the SMTP Authentication information in the email's header. With RFC 3848 however, there exist at least a minimal scheme, including a particular keyword ESMTPA in the last included "Received:" header line in case of an authenticated SMTP session. Setting up MTAs thus they will accept only SMTP authenticated emails on a dedicated port has been poured into RFC 4409 which defines the Mail SUBMISSION port 587. Just recently, R. Siemborski from Google and A. Melnikov from ISODE (wow, they still exist) have updated Meyer's SMTP Auth RFC: RFC 4945. Essentially, there is not much new information in this RFC compared to RFC 2554, however, it hooks together SMTP Authentication with Transport Layer Security (TLS) as defined in RFC 3207. Consequences for the ESMTP procedure While SMTP Authentication has been introduced solely as a service extension, it actually touches the (E)SMTP protocol substantially, which is not yet fully documented/discussed. ESMTP Authentication turns SMTP from a host-to-host to a user-to-host protocol. In consequence, some protocol features, which make sense while two MTAs communicate need to be refined or dropped, like mailinglist expansion (VRFY and EXPN). More subtle, SMTP Authentication (as well as STARTTLS RFC 3207) move ESMTP from a transaction oriented protocol now into a both session and transaction aware protocol. ESMTP session state While within SMTP basically only a transaction is usefully defined, we now need to care about an ESMTP session: A SMTP transaction starts with the client's MAIL FROM: command, and finishes with the client's final .<CRLF> command as last LINE during the DATA phase, acknowledged by the server with the 250 reply code. An ESMTP session begins with the EHLO command, includes STARTTLS and AUTH commands, as well as any SMTP transactions and finishes with the server's final QUITcommand. Thus, RFC 2821 requires from the ESMTP server to preserve a certain session state. Further, sessions states are ordered: The STARTTLS state needs to be established before the AUTH state is processed. 3 In contrast, some session state informations need to be cleared by the server, in case the ESMTP client issues a RST command. The current Klensin ESMTP draft RFC 5321 takes partially care of this. Obviously, Klensin did not read his own RFC carefully, because it mixes in the attached sample (taken almost unaltered from RFC 821) happily the terminology 'transaction' and 'session' (Appendix D.1.). However, the conceptual change is more severe. The problem here becomes virulent in case of an ESMTP reply code. Does the server's response now belong to a transaction, or the entire session ? One particular problem is the ESMTP error code 552: 552 Requested mail action aborted: exceeded storage allocation 552 Too much mail data (deprecated) Clearly, the first case is mailbox (and thus transaction) specific, while the second case is a policy limit as discussed further in RFC 5321: 4.5.3.1.10. Too Many Recipients Code RFC 821 [1] incorrectly