Sendmail Evolution: 8.10 and Beyond

Home , Delivermail, Eric Allman, Sendmail

Sendmail,Inc. 6603Shellmound Emeryville,California

Originally presented at the 1999 USENIX Annual Technical Conference FREENIX Track Monterey, CA

ABSTRACT SendmailTM has been the de facto mail transfer agent implementation since the dawn of the Internet. Today, sendmail development is still drivenbyacontinually changing set of network requirements and user demands. Lately,two new driving forces have also contributed to sendmail development. First, as more open source mail transfer agents, such as Exim and Postfix,become available, a newfriendly competition has developed in which the authors of the various MTAs share their ideas via open source and help to advance open standards as opposed to advancing their own particular implementation. Second, a new“hybrid” company, Sendmail, Inc., has been created to offer commercial versions of the open source software while continuing to fuel open source development. This paper will briefly discuss the evolution of sendmail;the influences which drive sendmail development; and howthe creation of Sendmail, Inc. has contributed to the open source version. The paper will also describe the newfeatures appearing in the next ‘‘functionality release’’ofopen source sendmail.Inparticular,changes in queueing and newprotocol support are discussed. Finally,the authors will speculate on future directions for sendmail.

Copyright (c) 1999 Sendmail, Inc. All rights reserved. 1 1. Introduction shipped in 1983 with 4.1c BSD—one of the initial The sendmail mail transfer agent (MTA) is used operating systems to support TCP/IP. Sendmail on most UNIXTM systems today.Recent changes have accomplished twoimportant goals. First, it provided a influenced sendmail development, notably the creation reference implementation of the Network Working of a new“hybrid” companydedicated to supporting Group (later the Internet Engineering Task Force, or both the open source code as well as a commercial IETF) mail standard [Cost97]. Second, the version. configuration was read at run time to allow reconfiguration for different networks without Section 2 givesabrief history of sendmail. recompilation. Because of the wide variety of networks Section 3 describes the forces acting to influence supported, the configuration was designed to be changes in sendmail.Section 4 outlines Sendmail, friendly toward non-conforming addresses. Instead of Inc.’s effects on the open source. Section 5 discusses rejecting messages that were not acceptable to the changes appearing in sendmail 8.10.Future directions standard, it tried to repair them; this broad acceptance that sendmail may takeare laid out in section 6. of inputs maximized interoperability with other Finally,asummary and concluding remarks are networks available at the time, such as UUCP. presented in section 7. By late 1986, Allman’sinv olvement with sendmail had tapered off, and several other people picked up development. The most important version .. was IDAsendmail from Lennart Lovstrand of the .. University of Linkoping in Sweden, with later 2. History maintenance by Neil Rickert of Northern Illinois To understand the continuing evolution of University and Paul Pomes of the University of Illinois sendmail,you must first look at its history.Likemany [Cost97]. The most important feature added by IDA successful open source projects, sendmail started as a wasthe concept of external databases in DBM format. “scratch your itch” solution to a problem. Shortly thereafter,Paul Vixie, then at Digital Equipment, created KJS (King James Sendmail),an 2.1. In the Beginning... attempt to unify the divergent versions, but this version wasnot widely adopted. Sendmail had effectively Sendmail started out as delivermail,written by splintered. Eric Allman, then a graduate student and staffmember at the University of California at Berkeley. Delivermail 2.2. Sendmail 8Emerges solved the problem of routing mail between three different networks running on the Berkeleycampus at In late 1989, Allman returned to U.C. Berkeley, the time: the ARPAnet, UUCP,and BerkNet. The first and not long thereafter was drawn back into sendmail public version was distributed in 1979 as part of the development. By July of 1991, serious work on what Fourth BerkeleySoftware Distribution (4BSD) and would become sendmail 8 had begun. Manyideas were later as part of 4.1BSD [Allm85]. taken from IDAsendmail and KJS,although most were generalized. For example, external databases were Although delivermail solved the immediate added, but in such a way that formats other than DBM problem faced by Berkeley, itwas not generic enough were available. Sendmail 8.1 wasreleased with 4.4 to solvethe problems of other custom networks in BSD in mid-1993. Sendmail 8 quickly became a operation. Since the instructions for talking among the unifying influence, as vendors converted from their networks were part of the C source code, it was not hacked versions to the newer version. Some features easy for sites to reconfigure delivermail for their from vendor versions were also included in the new specific needs. The configuration was also not flexible release, for example, NIS support from Sun enough to handle complexmail environments. Microsystems. These additions are just one of many At the same time the ARPAnet was transitioning examples of the success of open source software: to the newInternet protocol, TCP/IP.Part of the new sendmail 8 wasfertilized with ideas from other open protocol suite included extracting mail transmission out source and vendor versions. of the file transfer protocol (FTP) into its own protocol, Another important change that occurred the Simple Mail Transport Protocol (SMTP) [RFC821]. concurrent with sendmail 8 wasthat versions were The user demand for a customizable program and controlled more carefully.The previous major release the network requirements created by the newmail (sendmail 5)had no fewer than 143 “dot” releases (that protocol led to the creation of sendmail,which first is, 5.1 through 5.143), often more than one in a single

Copyright (c) 1999 Sendmail, Inc. All rights reserved. 2 day.Some of those were intended for public years. Unfortunately,with the growth of spam on the consumption, some were test releases. With version 8, Internet, this default is no longer acceptable. sendmail switched to a policyofclearly labeling test The increasing use of email as a vector of viruses releases, producing production releases less often, and has heightened the need for MTAs to include content clearly identifying newfunctionality releases from bug- checking. An SMTP server running on a firewall must fix releases. This change in release frequencywas be prepared to vet the data it is handling. Because of essential to the wide acceptance of sendmail 8 by the this need, 8.9 included message header checking and community.The downside of this change is that people 8.10 will include a mail filter API for more advanced who liketobeonthe “bleeding edge” have towait header and body filtering. longer,and newfeatures are not tested immediately. We viewthis loss of quick feedback as being an Changes in Internet standards from the IETF also acceptable tradeoff. have a major impact on sendmail.During 1998, the IETF accepted 22 newRFCs that involved electronic An unfortunate effect of the success of sendmail mail in one form or another.Atleast one of these 8 wasthat Allman quickly became overloaded with RFCs has a direct effect on MTAs such as sendmail. answering questions. This overload was the impetus RFC 2476, Message Submission, specifies a separate behind the establishment of the Sendmail Consortium, protocol for initial insertion of a newmessage into the aloosely-knit group of volunteers providing free message delivery system using SMTP [RFC2476]. support for sendmail.Gregory Shapiro was invited to join that group during the 8.8 cycle, and by 8.8.6 was As of April 1999, three more MTA-related RFCs doing a large part of development and most of the have already appeared. RFCs 2487 [RFC2487] and release engineering, although Allman continued to 2554 [RFC2554] provide encryption and authentication reviewand approve changes. for an SMTP session. RFC 2505 [RFC2505] is a set of recommendations for features that MTAs should In 1997, Allman found that evenwith the help of provide to combat spam. Additionally,the Detailed an extremely capable volunteer staff, he was unable to Revision/Update of Message Standards (drums) IETF keep up with the support load and continue to move Working Group is preparing to release the long-awaited sendmail forward. After exploring several other update to RFCs 821 and 822, probably later in 1999 approaches for adding resources for sendmail [SMTPUPD, MSGFMT]. Clearly,the messaging development, he finally settled on founding a “hybrid” standards landscape is not static. business model companytoproduce a commercial version of sendmail while continuing to support and 3.2. User Demands extend the open source version. By using the “hybrid” approach, he was able to protect the interests of the By far,howev er, most feature requests come open source community while creating a viable from sendmail users. It is common for the Sendmail business model. Consortium to receive three to fivefeature requests per week, some complete with the patches necessary to implement the feature. These feature requests produced a list of 320 requests before 8.10 development evenbeg an. When deciding which features to implement and 3. Driving Forces howtheyshould be implemented, we try to balance As can be seen in the preceding section, sendmail backwards compatibility with change. By introducing has responded to both changing network requirements radical changes gradually,wegiv e sendmail sites a and user demands. In addition to these demands, new chance to prepare for the changes. Acombination of a open source MTAalternativeshelp in driving sendmail huge user population and 20 years of sendmail forward. availability prevents us from doing radical changes without advanced warning. For example, the 8.9 3.1. Network Requirements documentation included a notice warning users that The network requirements come both from the configuration file names would be changing in 8.10. changing face of the Internet and from newInternet Also in 8.10, the LDAP map class will be changed drafts and RFCs from the IETF.For example, up until from ldapx to ldap,thereby dropping the class version 8.9, sendmail allowed third party,promiscuous name’sconnotation as an experimental map. The old relaying by default. This willingness to relay had been name will continue to work (and print a warning) in an acceptable, evendesirable, default for more than 15 8.10, but will be removedina subsequent release.

Copyright (c) 1999 Sendmail, Inc. All rights reserved. 3 Some of the other open source mail transfer agents, create a complete engineering team to work on such as Postfix and qmail,are not yet so constrained. sendmail,including software engineers, quality assurance (QA) engineers, and technical writers. 3.3. AlternativestoSendmail There are currently twofull-time engineers At the same time, these other open source MTAs working on the next version of the open source, also drive sendmail development. The open source Gregory Shapiro and Claus Assmann, with more to be alternatives, such as Postfix and Exim,giv e sendmail a added as theycan be hired. These newengineers will (for the most part) friendly form of competition. This help deal with the growing complexity of new competition promotes both innovation and sharing for standards and respond to newuser requests as they all of the MTAs. For example, Wietse Venema, author arrive.Additionally,other engineers are working on of Postfix,not only asked about sendmail behavior commercial products, and selected features from those during his development of Postfix (to maintain products are being included in the open source compatibility), but also made contributions to sendmail. distribution. With this mostly friendly competition and The presence of a QA department has an cooperation among open source authors, everyone additional impact. Previously,formal testing was wins. Without multiple open source implementations, minimal; in particular,formal testing tools, such as there would be no choice for the user,nor much code coverage tools, were not applied by sendmail core pressure to move the existing implementation forward developers. The Sendmail, Inc. QA department now or adhere to standards issues. With multiple provides the first line of formal testing before release to implementations, users are free to chose the open outside testers. source MTAwith which theyare most comfortable. The technical writers provide professional Since the MTAs are all based on open standards instead writing and editing resources to improve and expand of commercial, proprietary standards, theyare able to the available documentation. Theywill be able to help interoperate and prevent the Internet from becoming clean up and augment the available documentation for proprietary and vendor specific. the open source sendmail distribution. Beyond people, Sendmail, Inc. has made commercial development and software testing tools (such as memory leak detectors and code coverage monitors) available to the engineers. These tools were 4. Enter Sendmail, Inc. previously too expensive for the volunteer developers 1. As one might imagine, the creation of Sendmail, The companyisalso able to afford a variety of Inc. represents a major change in the development of hardware platforms giving development and QA the open source version of sendmail.Now,there is a engineers the chance to test portability in-house before commercial entity behind the development—a releasing a distribution for testing. companythat is completely committed to the open This “hybrid” approach to sendmail development source. The development of sendmail would have introduced some newconcepts for the developers, for continued without the creation of Sendmail, Inc., but at example, project schedules and structured code aslower pace and with fewer resources. Sendmail, Inc. reviews. Previously, sendmail development was not wasable to release its first commercial product, done according to a schedule and there were no hard Sendmail ProTM quickly and successfully thanks to the deadlines2.Releases were made when theywere ready already available and provenopen source sendmail.In instead of on a predetermined date. The addition of return, Sendmail, Inc. can contribute both financial and commercial influences does not mean that releases will human resources to open source development. These be made before theyare ready.Instead, givenafuture contributions can be found in the manyplaces within release date, the number of features that can be the company. 1It would seem a “good thing” if producers of software devel- 4.1. Engineering opment tools would consider donating copies of their software to es- As detailed in the history section, up until the tablished open source development groups that could not otherwise afford them. formation of Sendmail, Inc., all sendmail development 2There were targets, for example, sendmail releases were often and support was done by volunteers in their spare time. targeted to precede USENIX conferences. This development model limited the total energy that could be exerted. Sendmail, Inc. has been able to

Copyright (c) 1999 Sendmail, Inc. All rights reserved. 4 implemented in that time frame are determined. Of Times [Mark98]. course, if there are problems with the release as the With the addition of a business development unit, release date nears, either those problematic features Sendmail, Inc. is in a better position to partner with will be removedbefore the release or the schedule will other companies to provide enhanced services to the slip. No version will be released before it is ready. sendmail community.For example, third-party As more engineers work on the code, more commercial virus and spam checkers are planned for structured code reviews are planned. Previously code availability with sendmail 8.10.Since the hooks reviewwas done on an ad hoc basis; for example, needed for these third-party plug-ins will be in the open Allman reviewed most newly contributed code and source release as well as the commercial release, open someone (often Shapiro) reviewed most of Allman’s source users will be able to takeadvantage of these new code. Nowall code check-ins are mailed to a list of filtering technologies. core team members. This mailing has the effect of Finally,Sendmail, Inc. provides the legal keeping everyone “in sync” and catching many resources necessary to research and complete the problems immediately.For wholly newcode, more necessary paperwork for the open source distribution, formal code reviews will be instituted. such as licenses and government export approvalfor features likeSMTP authentication and secure SMTP 4.2. Support (discussed below). The support infrastructure of Sendmail, Inc. collects and reports problems, analyzes trends of incoming questions, and provides feedback to the developers. Sendmail, Inc. consultants and engineers nowvisit customer sites, allowing them to see sendmail in use in the field and discuss the customer’sneeds and 5. The Present: Sendmail 8.10 expectations. These visits will all lead to improved Even though coding for sendmail 8.10 beganthe features and clearer documentation not only for the first week of February,1999, plans for the version commercial customers, but for the open source users as started evenbefore 8.9 released. The 8.9 release was to well. be the anti-spam release and there was great demand to Another unexpected benefit has been acceptance get these features out to the users as soon as possible. by companies that refuse to use “free software” because This time pressure forced us to limit newfunctionality of fears that theywill not be able to get support. to spam fighting features and defer others for the next Particularly for “mission critical” code such as the mail release. Wealso did not want to obsolete the “Bat system, manycompanies require a commercial support Book” [Cost97]; which would be a disservice to our organization that has contractual obligations to answer users and the open source customers. From the 320 questions within a certain time frame, and as a last customer requests, we picked more than 100 features resort, an entity willing and able to stakeits reputation for inclusion in 8.10; another 80 were selected as on the ability to provide solutions to customers’ potential features if time permits in the release cycle. problems. Sendmail, Inc. can provide guaranteed 24/7 As with past releases, 8.10 has a “theme”. support coverage. Although manyofthe other changes are important, we plan to highlight SMTP authentication and a newmail 4.3. Other Expertise filter API as the premier features for 8.10. Sendmail, Inc. also provides specialized resources for handling other tasks unrelated to the 5.1. SMTP Authentication software itself, freeing up the developers to do what Our hopes are to have SMTP authentication theydobest. Although sendmail has always had some [RFC2554] as part of the 8.10 release. SMTP form of ‘‘marketing’’toentice users to upgrade, it has authentication provides a method for the mail user not had a marketing organization to spread the gospel agent (MUA) to authenticate the user to the mail and inform trade press about the features of new transfer agent and carry that authentication with the versions. That has all changed with the creation of message as it passes between mail servers toward the Sendmail, Inc. and a true marketing department. For final destination. example, with the formation of the newcompanyand Formessage submission from an MUAinto a the public beta release of sendmail 8.9 in March of site’smail server,SMTP authentication provides a 1998, information about the release and its newanti- mechanism for recognizing users as trusted for that site. spam features made the front page of the NewYork

Copyright (c) 1999 Sendmail, Inc. All rights reserved. 5 This feature can be used to allowrelaying based on the implementation for use with sendmail when 8.10 is submitting user instead of the submitting host, a feature released. This feedback of changes is another example especially useful for roaming users submitting mail of Sendmail, Inc.’s commitment to the open source from untrusted sites. community. Although the authentication information is carried in the message envelope until reaching the final 5.2. Third Party Mail Filter API delivery host, remote sites should not trust this The other major functionality enhancement for information as it may have been altered by a “man-in- 8.10 is a third party mail filter API. This API will the-middle” attack. As the RFC notes, SMTP allowsystem administrators and third party companies authentication is “generally useful only within a trusted to provide message filtering using hooks in the enclave”[RFC2554]; it is not meant as an end-to-end sendmail code. This newplug-in architecture will authentication or security mechanism. allowfor better spam and virus monitoring as well as Initially, sendmail will use the message give administrators the ability to accept, reject, discard, submission authentication to override the relaying modify,orarchive messages. checks. It will also provide the authentication Briefly, sendmail will have a compile flag that information to user rulesets as macros. will implement callouts to user-supplied routines that Unfortunately,there is potentially a major road will be called to process envelope information, headers, block that would prevent us from including SMTP and the message body.The filter can request that new authentication—the United States government’s headers be added or the entire message body be cryptographyexport policy. Although authentication is replaced. In addition, portions of the envelope can be claimed to be acceptable for exporting, the Bureau of modified—in particular,recipients can be added or Export Administration may reject our application if the deleted. This API provides exceptional flexibility. bureau feels the authentication hooks in sendmail can be easily converted to provide encryption, eventhough 5.3. Other NoteworthyChanges enabling encryption is not the purpose of the hooks. Beyond these twonew major additions, 8.10 will The definition of “easily converted” is unclear. include manyother newfeatures. Although it is Surprisingly,the distribution of sendmail in source impossible to describe them all in this paper,wewill code form hurts our chances of getting approval. mention some of the high points. Products that do not ship with source code, such as the TM IPv6 support. In response to network changes, Netscape Messaging Server,are able to ship SMTP sendmail 8.10 includes IPv6 support using the interface authentication. In such cases, those products are able described in RFC 2553, Basic SocketInterface to limit the use of the routines to an authentication Extensions for IPv6 [RFC2553]. This support allows model that is weak enough to be accepted by the United sites that are moving to IPv6 the ability to include States government. Also, in binary form, it would be sendmail in their transition plans and testing. nearly impossible to convert the authentication routines into encryption routines. New RFC support. Other RFCs under consideration for possible sendmail 8.10 inclusion are Assuming we are able to distribute 8.10 with the message submission protocol [RFC2476], enhanced SMTP authentication, there are still some outstanding SMTP status codes to provide more precise error issues. In creating the extension, we needed an reporting [RFC2034], and anti-spam recommendations implementation of the Simple Authentication and [RFC2505]. Security Layer (SASL) [RFC2222] library to provide the framework for the different authentication methods. Sendmail 8.9 is already compliant with RFC The only open source implementation currently 2505, Anti-Spam Recommendations for SMTP MTAs. available (that we are aware of) is the Cyrus SASL However, weare investigating including a suggested library.Early attempts using this library were not change so that the MTAcan limit the maximum encouraging because of portability and implementation number of messages that can be sent from a particular problems. Since we do not currently have the time or user in a specified interval. This feature will help to resources to create our own SASL implementation, we reduce the damage that can be performed by “hit-and- have decided to use the Cyrus library and contribute all run” spammers. our bug fixes and portability changes back to the Cyrus Improvedvirtual hosting capabilities. The development group. These changes are being most requested enhancements has been for better incorporated into the base Cyrus release, making them virtual hosting support. Sendmail 8.10 will include available to others and providing a stronger better control overthe virtual user table, which

Copyright (c) 1999 Sendmail, Inc. All rights reserved. 6 provides a domain-specific form of aliasing, allowing Other newanti-spam features include multiple virtual domains to be hosted on one machine. FEATURE(`require_fqdn´),which requires a Anew class is available for triggering virtual user table fully qualified domain name for sender addresses lookups to match the functionality of the generics table, unless the connection comes from a local system, and the feature used to rewrite local addresses into a generic FEATURE(`relay_mail_from´),which allows form. “Plus detail” information, the portion of the mail relaying if the mail sender is listed as RELAY in the address used to carry additional information about the access map. user address that precedes the plus sign (for example, The ability to delay anti-spam checks until the user+detail@host), will also be made available SMTP RCPT command has been added using for both generics and virtual user table lookups. FEATURE(`delay_checks´).This feature allows Additionally,8.10 maintains information sites to permit mail to certain addresses, such as regarding the incoming connection in a newmacro. postmaster,reg ardless of the results of other anti-spam Forexample, hosts having multiple IP addresses on checks. different virtual interfaces always advertise themselves New macros, rulesets, and options. The 8.10 as the primary host name in 8.9. In 8.10, theywill be release also introduces newnamed macros and rulesets able to identify themselves as the virtual host for controlling other facets of the daemon. Examples throughout the transaction. The SMTP greeting and of the newmacros include ${rcpt_mailer}, Received: headers will use the virtual host name ${rcpt_host},and ${rcpt_addr},which and outgoing IP connections will be bound to the represent the resolved triplet that delivers the mail to address of the customer instead of the hosting ISP (so this recipient. These macros can be used to simplify the “next hop” SMTP server will log the appropriate matching in custom check_* rulesets. Three new host name in its Received: lines). ruleset calls, check_etrn, check_expn,and Forsites providing queueing services, 8.10 will check_vrfy have been added to restrict the ETRN, offer a newmailer flag for queueing mail until delivery EXPN,and VRFY SMTP services. Instead of globally is explicitly requested via either a queue run with turning these services offvia the PrivacyOptions pattern matching (-qR, -qS, -qI)orvia ETRN,the option, administrators can nowuse the rulesets to allow SMTP service extension for remote message queue these commands for certain sites. starting [RFC1985]. This feature provides better Newoptions have been added for general mail support for ISPs that provide queueing for dial-up server policyand protection. These options include the customers, as queue runs are no longer held up waiting popular MaxHeadersLength and for the dial-up server connection attempt to time out. MaxMimeHeaderLength options, which protect Improvedanti-spam features. To allowusers against “denial-of-service” attacks and buffer overflows more fine grain control, 8.10 introduces more detailed in some MUAs. specification for the access database. Tags on the key Better LDAP integration. The 8.10 release of access database entries can limit the lookups to offers tighter integration with the Lightweight specific anti-spam checks. Forexample, specifying Directory Access Protocol (LDAP), which has proven To:friend.example.com instead of to be the directory service of choice at manysites. friend.example.com in the access database, Support for multiple entry/attribute LDAP value allows relaying to friend.example.com without searches, LDAP authentication, and LDAP-based alias permitting mail relaying from that site. maps will appear in sendmail 8.10.Weare also Anew DNS-based blacklist feature (dnsbl) monitoring the IETF LDAP Schema for E-mail Routing supersedes the Realtime Blackhole List (rbl)feature “birds-of-a-feather” group for a standard schema for available with 8.9. The newfeature takes the name of alias specification using LDAP [LASER]. the blacklist server as well as an optional rejection Improvedperformance. In an effort to improve message. The blacklist server is queried with the IP sendmail’s performance, 8.10 includes code donated by address of each incoming connection and, if the query Exactis.com (formerly InfoBeatTM)that provides is successful and the IP address is blacklisted, the support for multiple queues in sendmail.The new connection is rejected. This newfeature can be Exactis.com donation also includes code that extends included multiple times to allowsites to subscribe to the queue file name, making it unique for a 32-year multiple servers. FEATURE(`dnsbl´) replaces period. This change reduces the amount of file locking FEATURE(`rbl´) to prevent the possible confusion and renaming necessary for instantiating a queued between the Realtime Blackhole List and other DNS message. In addition, the newqueue file naming based blacklist servers.

Copyright (c) 1999 Sendmail, Inc. All rights reserved. 7 system makes it possible to move items between potentially dozens of files in /etc with obscure file different queues easily and quickly. names, such as sendmail.cw (nowknown as Exactis.com also donated the code necessary to /etc/mail/local-host-names), and allows implement memory-buffered files on systems that that directory to be owned and managed by the user include the Chris Torek stdio library,such as the specified in the TrustedUser option. The files BSD family.Ifyour operating system can take affected include maps, aliases, and classes, as well as advantage of this newcode, sendmail will be able to the error header,help, service switch, and statistics reduce file system overhead by not creating temporary files. files on disk. In combination with the newqueue file Although the newfile names will make naming system and multiple queue support, file system configuration and support easier in the future and users bottlenecks will be greatly reduced. were warned of the upcoming change in 8.9, this Features from Sendmail Pro. Beyond funding change will probably be the most traumatic 8.10 for development from Sendmail, Inc., the open source change for users upgrading from earlier versions. version also benefited by receiving newMTA features Beyond the MTA. Outside the sendmail MTA from Sendmail Pro, the commercial product. These itself, the open source distribution includes other changes were released in open source evenbefore utilities, such as mail.local,the local delivery Sendmail Pro was released3.Two ofthese changes are agent; makemap,the map generation tool; and newdaemon control functionality and trusted user praliases,the tool that converts an alias database support. back to its textual form. These utilities have also had minor updates to improve ease of use. Although The first, newdaemon control functionality, invisible to the end user,code sharing between allows an external program to control and query status sendmail and these utilities has increased by using from the running sendmail daemon via a named socket, portability and utility libraries. By sharing code and similar to the ctlinnd interface of the INN news breaking the utility routines out of the MTA, 8.10 server [Salz92]. Although only a fewcommands movesusastep closer to splitting up the monolithic (restart, shutdown,and status)are available in sendmail process into multiple programs in future this first version, the framework is in place for releases. extending this functionality to control and query different facets of the daemon. Since access to this Anew enhanced version of the vacation auto- interface is controlled by the UNIX file permissions on responder is a standard part of the sendmail distribution the named socket, the file permissions provide beginning with 8.10. Some of the newfeatures planned administrators a means of controlling the daemon via for the revised vacation include newcommand line external interfaces without requiring root privileges. A options for specifying alternate databases and alternate Perl program (contrib/smcontrol.pl)is messages, as well as a method for getting the sender provided in the distribution as example code to take out of band. The new vacation will also support an advantage of the control socket. exclusion list of addresses to which an automatic response should not be generated. The new TrustedUser option allows the administrator to specify a user name that will be 5.4. Maybe Next Time considered equivalent to the superuser for permission checks and other operations normally reserved only for As with anylarge software project, there are root. For example, the TrustedUser is allowed to enhancements we had planned on including in 8.10 but start the daemon as well as own maps, files, and were unable to tackle because of resource constraints. directories without sendmail marking them as At the current time, the twobiggest casualties were TM untrusted. This change is another step in the migration Windows NT portability and support for secure toward a sendmail daemon that does not heavily rely on SMTP (i.e., encryption) [RFC2487]. While we superuser privileges. continue to design with Windows NT portability in mind, the extensive changes required have lead us to Consistent file names. As of 8.10, the default postpone this change until a future version. location for all sendmail configuration files will be /etc/mail/.This change avoids sprinkling SecureSMTP with TLS. Although secure SMTP is an extremely important feature, arguably just 3Theywere disabled by default as theyhad not been fully test- as important as secure web service, the United States ed at the time of the open source release. government is not expected to allowrelease of the source code for an encrypting mail server to the world.

Copyright (c) 1999 Sendmail, Inc. All rights reserved. 8 It is unfortunate that eventhough this encryption is sockets. In particular,Windows NT does not have the widely available in other countries and freely available fdopen(3) call, and BeOS sockets are not inherited by for download from international servers, the United forked children. We expect to hide most of the I/O States still has not recognized that the people being hurt behind another compatibility layer,possibly sfio most by these export restrictions on encryption are its [Korn91]. owncitizens and businesses. After 8.10 is released, we expect to do Encryption patches for sendmail are available considerable work on performance enhancements and from one of a number of sites outside the United States. tuning, including memory pools for more efficient As an example, one can look at the ssmail patches at memory allocation, support for threaded delivery,and http://www.home.aone.net.au/qualcomm/ the use of shared memory for saving long-term state. [Rose99]. However, this patch does not use the published TLS extension4. We will continue to investigate methods of making secure SMTP with TLS available for sendmail. Forexample, we might produce a ‘‘domestic’’version 7. Summary of sendmail with TLS. As sendmail development continues, it is affected by four driving forces: continually changing network requirements, user requests, available development resources, and competition. None of these factors are particularly newtoany popular network server 6. The Future software, but the substance of these factors for sendmail are unique. The paper has laid out these There remain several major factors to research factors both historically and in the present, including and goals to accomplish in future versions. Sendmail some of the newfeatures for 8.10 brought about by will need to move tow ard a threaded model to improve these four forces. portability for Windows NT.This change will require significant changes to the MTAinboth its use of global The most notable event in the evolution of variables and memory management. Anyservices, sendmail development is undoubtedly the creation of such as DNS and system libraries, that sendmail uses Sendmail, Inc., a “hybrid” business model company will also need to be thread safe. This change may producing both the open source and commercial improve ormay degrade performance for UNIX versions of sendmail. Sendmail, Inc. helps drive systems depending on the thread implementation of the network changes by participating more fully in the operating system and howitcompares to forking, IETF.More directly,Sendmail, Inc. provides far more which has become quite efficient on some systems. development resources—in the form of funding, people, and tools—to the sendmail open source than Apopular trend in newer open source MTA were previously available. For example, the company implementations has been to break up the tasks into has paid for conference calls between members of the separate programs. We will be studying the Sendmail Consortium and plans to host meetings for performance trade-offs of making these changes to the group. sendmail and breaking tasks offasappropriate. This approach has its benefits as “[i]t has been observed that This arrangement benefits both the companyand one of the great successes of UNIX is that each tool the open source distribution. The open source gains does only one job, and therefore can do that job well” newfeatures and enhancements, while the commercial [Allm85]. It will also allowustoimprove security by products reap the benefits of an active open source securing smaller portions of privileged code. community contributing both newideas and testing. As we make sendmail portable to non-UNIX The future promises some exciting times for both platforms, we will have toreconsider the I/O the open source distribution of sendmail and the subsystem. For example, Windows NT and BeOSTM commercial products as both growtogether. sockets do not have the same semantics as UNIX The latest open source version of sendmail is available from http://www.sendmail.org/. 4This limitation may not be a disadvantage; the ssmail authors More information about Sendmail, Inc. can be found at argue that the overhead of TLS is too high for routine use. http://www.sendmail.com/.

Copyright (c) 1999 Sendmail, Inc. All rights reserved. 9 8. Acknowledgments January,1999. The authors would liketothank David Conrad of [RFC2505] Lindberg, Gunnar, Anti-Spam the Internet Software Consortium, Sendmail, Inc.’s Recommendations for SMTP MTAs, RFC Mark Delanyand Nick Christenson, and Kirk 2505, February,1999. McKusick for their insights and suggestions regarding [RFC2553] Gilligan, Robert E., Susan Thomson, Jim the paper.Wealso appreciate the hard work of Claus Bound, and W.Richard Stevens, Basic Assmann, Katie Calvert, and Cathe Ray,for reviewing SocketInterface Extensions for IPv6, and improving the paper.Finally,wewould liketo March, 1999. offer our sincere appreciation to the members of the Sendmail Consortium for all of their efforts and [RFC2554] Myers, John G., SMTP Service Extension contributions to sendmail. for Authentication, RFC 2554, March, 1999. REFERENCES [Rose99] Bentley, Damien, GregRose, and Tara [Allm85] Allman, Eric, and Miriam Amos, Whalen, ssmail:Opportunistic Encryption Sendmail Revisited, Proceedings of the in sendmail, Draft. Summer 1985 USENIX Conference, pp. 547-555, 1985. [Salz92] Salz, Rich, InterNetNews: Usenet transport for Internet sites, Proceedings [Cost97] Costales, Bryan, and Eric Allman, of the Summer 1992 USENIX sendmail. O’Reilly & Associates, Inc., Conference, pp. 93-98, June 1992. Second Edition, 1993. [SMTPUPD] Klensin, John C., Simple Mail Transfer [Korn91] Korn, David G., and Kiem-Phong Vo, Protocol, Internet Draft draft-ietf-drums- Sﬁo: Safe/Fast String/File IO, smtpupd-10, February,1999. Proceedings of the Summer 1991 USENIX Conference, pp. 235-256, 1991. [LASER] Freed, Ned, LDAP Schema for E-mail Routing (laser) bof, Meeting Report, Proceedings of the Forty-Third Internet Engineering Task Force, December,1998. [Mark98] Markoff, John, Battling Junk Email May Become Easier, NewYork Times, March 17, 1998. [MSGFMT] Resnick, Peter W., Internet Message Format Standard, Internet Draft draft-ietf- drums-msg-fmt-07, January,1999. [RFC821] Postel, Jonathan B., Simple Mail Transfer Protocol, RFC 821, August, 1982. [RFC1985] De Winter,Jack, SMTP Service Extension for Remote Message Queue Starting, RFC 1985, August, 1996. [RFC2034] Freed, Ned, SMTP Service Extension for Returning Enhanced Error Codes, RFC 2034, October,1996. [RFC2222] Myers, John G., Simple Authentication and Security Layer (SASL), RFC 2222, October,1997. [RFC2476] Gellens, Randall, and John C. Klensin, Message Submission, RFC 2476, December,1998. [RFC2487] Hoffman, Paul, SMTP Service Extension for SecureSMTP over TLS, RFC 2487,