In the good old days of Un*x when men were real men, women were real women and timesharing systems were real timesharing systems, networks were not commonplace and the "pipe it into sendmail" approach was a good one. Sendmail can handle many diverse means of mail transport and can parse and translate addresses and message headers to suit the underlying transport's characteristics. It is convenient for programmers, just shell out to sendmail and pipe the message into sendmail's standard input. Not much code is required to do this which is good, less code means fewer bugs.

However, things have changed over the years. Nowadays, personal workstations are normal and RFC 5322 format mail transported using the SMTP protocol is ubiquitous. Frequently the MTA that is supposed to handle mail submission is remote from the workstation. There are many other MTAs to choose from than sendmail, and the alternatives may be much easier to install and maintain.

Unfortunately, the MUAs have not changed much in the way they work. They still shell out to sendmail. This means that MTA authors must provide a sendmail command to wrap the way their program really works. Users on workstations need a sendmail-like MTA on the local host which can be an administrative nightmare. All those command line options which must be accepted but do nothing. Useful SMTP extensions that are not supported via the command line options (or just not supporetd). Where is sendmail really installed on this system? Does the emulation implement every obscure feature an MTA might need?

The solution is simple, an MUA uses SMTP to submit its mail to the MTA. Why not? After all, SMTP is defined in open standards that are easy to obtain, all the MTAs implement it and it breaks the dependency on an external program's idiosyncrasies and calling conventions. Sendmail talks SMTP so its ability to transport mail using other mechanisms is not lost. Furthermore, many Linux and Un*x users are on workstations and the MTA is on a remote system. Using SMTP directly surely must be the solution.

Well, interacting by the use of standardised protocols is the solution for the modern networked environment but writing the client side of even a simple protocol like SMTP is not that trivial. Quite a lot of code is needed just to implement the communications between client and server, never mind the protocol itself. There are many SMTP extensions which are desirable but not so easy to implement. There are restrictions on the format of many of the protocol elements, for example, the domain of an email address should not reference a CNAME in the DNS. MUA authors are caught between the Devil and the deep blue sea. Either they opt for the simple solution which requires installation of a complex piece of MTA software or they choose the protocol solution which is difficult to implement.

The list of difficulties goes on but the outlook is not all bleak. It is possible to start small and extend once the underlying communications is in place. By having a library that implements an electronic mail submission API, it becomes as easy for programmers to use SMTP for mail submission as it is to use sendmail. The library is isolated from the application since it interfaces only via the API. It can be extended and debugged independently of the application. SASL and TLS can be added. On modern systems implementing dynamic linking, the library can be upgraded without disturbing the application.

If the API is simple and intuitive and the implementation is robust, then many programmers will wish to adopt it. With widespread use the library will be tested by many and become robust as a consequence. In the longer term direct use of SMTP by the submitting program will eliminate the necessity for sendmail or any other MTA to be present on the local machine. A mail submitting program will not need to provide an alternative "pipe into sendmail" interface since SMTP is supported by all modern MTAs (doubtless many will provide the sendmail interface until confidence is gained with the new API). This reduces the number of configuration options the application will need or, at least, means there is no increase in the amount of application configuration.

libESMTP is my attempt at implementing just such a library.