docs/diploma

annotate thesis/tex/3-MailTransferAgents.tex @ 143:70201774b457

work on market share analysis
author meillo@marmaro.de
date Mon, 15 Dec 2008 13:40:25 +0100
parents 002fd18820cc
children 93a47593a493
rev   line source
meillo@89 1 \chapter{Mail transfer agents}
meillo@89 2
meillo@117 3 After having analyzed the market for electronic mail and trends for it in the last chapter, this chapter takes a look at \mta{}s, the intelligent most important parts of the email infrastructure. \MTA{}s will be grouped by similarities, and the four most popular \freesw\ \mta{}s, will be presented to the reader in a short overview and with the most important facts. At the end of this chapter a comparison of these programs in several disciplines will be done.
meillo@89 4
meillo@117 5
meillo@89 6
meillo@89 7
meillo@120 8 \section{Types of MTAs}
meillo@117 9 ``Mail transfer agent'' is a term covering a variety of programs. One thing is common to them: they transfer email from one machine to another.
meillo@89 10
meillo@143 11 This is how Bryan \person{Costales} defines a \mta\ in \cite{costales97}:
meillo@117 12 \begin{quote}
meillo@117 13 A mail transfer agent (MTA) is a highly specialized program that delivers mail and transports it between machines, like the post office.
meillo@117 14 \end{quote}
meillo@117 15 \name{The Free Dictionary} is a bit more concrete on the term: \citeweb{website:thefreedictionary}
meillo@117 16 \begin{quote}
meillo@117 17 Message Transfer Agent - (MTA, Mail Transfer Agent): Any program responsible for delivering e-mail messages. Upon receiving a message from a Mail User Agent or another MTA, [...] it [...] delivers it to any local addressees and/or forwards it to other remote MTAs (routing) for delivery to remote recipients.
meillo@117 18 \end{quote}
meillo@89 19
meillo@117 20 Common to all \MTA{}s is the transfer of mail to other machines; this is the actual job. Besides this similarity, \MTA{}s can be very different. Some of them have \NAME{POP3} and/or \NAME{IMAP} servers included. Some can fetch mails through these protocols. Others have have all features you can think of. And maybe there are some that do nothing else but transporting email.
meillo@89 21
meillo@117 22 Following is a classification of \mta{}s into groups of similar programs, regarding what is viewable from the outside.
meillo@117 23
meillo@117 24
meillo@120 25 \subsubsection*{Relay-only MTAs}
meillo@89 26 \label{subsec:relay-only}
meillo@89 27 This is the most simple kind of \MTA. It transfers mail only to defined \name{smart hosts}\footnote{\name{smart host}s are \MTA{}s that receives email and route it to the actual destination}. \name{Relay-only} \MTA{}s do not receive mail from outside the system, and they do not deliver locally.
meillo@89 28
meillo@89 29 Most \MTA{}s can be configured to act as such a \name{forwarder}. But this is usually an additional functionality.
meillo@89 30
meillo@89 31 One would use such a program to give a system the possibility to send mail, without the need to do lots of configuration. In a local network, usually the clients are set up with a \name{relay-only} \MTA, while there is one \name{mail server} that acts as a \name{smart host}. The ``dumb'' clients send mail to this one \name{mail server} which does all the work.
meillo@89 32
meillo@89 33 Examples for that group are: \name{nullmailer}, \name{ssmtp} and \name{esmtp}.
meillo@89 34
meillo@89 35
meillo@117 36 \subsubsection*{Groupware}
meillo@124 37 Normally the term ``groupware'' does not mean one single program, but a suite of programs. They build a framework which is then populated with various modules that provide actual functionality. Modules for mail transfer, file storage, calendars, resource management, instant messaging, etc., are commonly available.
meillo@89 38
meillo@124 39 One would use one of these program suites if the main work to do is not mail transfer, but providing integrated communication facilities and team working support for a group of people. The most common scenario are companies. They have \name{groupware} running to provide adequate services for their teams to work efficiently. But one may use \name{groupware} on the home server for his family members also.
meillo@89 40
meillo@89 41 Examples are: \name{Lotus Notes}, \name{Microsoft Exchange}, \name{OpenGroupware.org} and \name{eGroupWare}.
meillo@89 42
meillo@89 43
meillo@120 44 \subsubsection*{``Real'' MTAs}
meillo@117 45 There is a third type of \mta{}s in between the minimalistic \name{relay-only} \MTA{}s and the bloated \name{groupware}. Those programs may be named ``real \MTA{}s'', or ``proper \MTA{}s'', though there is no common name. They are what is meant with the term ``\mta''---programs that transfer mail between hosts.
meillo@89 46
meillo@132 47 Common to them is their focus on transferring email, while being able to act as \name{smart host}. Their variety ranges from ones mostly restricted to mail transfer (\qmail) to others already having interfaces for adding further mail processing modules (\postfix). They cover everything in between the other two groups. %FIXME: are postfix and qmail good examples?
meillo@89 48
meillo@117 49 This group is of importance in this document. All programs selected for the comparison in the following section are ``real \MTA{}s''. \masqmail\ is one too.
meillo@89 50
meillo@89 51
meillo@117 52 \subsubsection*{Other segmenting}
meillo@124 53 \name{Mail transfer agents} can also be split in other ways.
meillo@117 54 \begin{itemize}
meillo@117 55 \item
meillo@124 56 Due to \sendmail's significance---described in section \ref{sec:sendmail}---compatibility interfaces for \sendmail\ are of importance for \unix\ \MTA{}s. Being not \emph{sendmail-compatible} does not need to matter for some fields of action, but makes the program ineligible for serving as a general purpose \MTA\ on \unix\ systems. Hence being sendmail-compatible is a major property of a \mta. %todo: how many MTAs are sendmail-compatible?
meillo@124 57 \MTA{}s not having a \emph{sendmail-compatible} interface or not offering it as a compatibility add-on, will not be covered here. One example for such a program is \name{Apache James}. %FIXME: check if correct
meillo@89 58
meillo@117 59 \item
meillo@117 60 Another separation can be done between \freesw\ programs and proprietary software. Many of the \MTA{}s for \unix\ systems are \freesw. Only these are regarded in the following sections, because comparing \freesw\ with proprietary or commercial software is not what typical users of programs like \masqmail\ do. %fixme: what are typical users?
meillo@117 61 Comparison with those non-free programs may be a point for large \freesw\ projects, trying to step into the business world. Small projects, mostly used by individuals at home, %fixme: is this the right target field? see chap02
meillo@119 62 need to be compared against other projects of similar shape. The document should be seen from \masqmail's point of view---an \MTA\ for a \unix\ system on home servers, workstations, or maybe embedded platforms---so non-free software is out of the way.
meillo@117 63 \end{itemize}
meillo@89 64
meillo@89 65
meillo@89 66
meillo@89 67
meillo@120 68 \section{Popular MTAs}
meillo@89 69
meillo@130 70 This section introduces a selection of popular \MTA{}s; they are the most likely substitutes for \masqmail. All are \emph{sendmail-compatible} ``smart'' \freesw\ \MTA{}s that focus on mail transfer, as is \masqmail.
meillo@89 71
meillo@143 72 The programs chosen are: \sendmail, \exim, \qmail, and \postfix. They are the most important representatives of the regarded group. Although \MTA\ statistics are rare, differ, and good data is hard to collect, these programs tend to stay near the top.
meillo@117 73
meillo@143 74 Table \ref{tab:mta-market-share} shows the most used \MTA{}s determined by three different statistics. The first done Ken \person{Simpson} and Stas \person{Bekman} published by \name{O'ReillyNet} in 2007 \citeweb{oreillynet:mta-stats} , the second by \name{MailRadar.com} with unknown date \citeweb{mailradar:mta-stats} , and the third by Daniel~J.\ \person{Bernstein} (the author of \qmail) done in 2001 \citeweb{djb:mta-stats}.
meillo@117 75
meillo@130 76 \begin{table}
meillo@130 77 \begin{center}
meillo@130 78 \input{input/mta-market-share.tex}
meillo@130 79 \end{center}
meillo@130 80 \caption{Market share of \MTA{}s}
meillo@130 81 \label{tab:mta-market-share}
meillo@130 82 \end{table}
meillo@89 83
meillo@143 84 All surveys show high market shares for the four \MTA{}s---\sendmail, \exim, \qmail, \postfix. Only the \name{Microsoft} mail servers and \name{IMail} have comparable large shares. Other \freesw\ \mta{}s are: \name{smail}, \name{zmailer}, \name{MMDF}, and \name{courier-mta}. They all are less important and rarely used.
meillo@130 85
meillo@143 86 The three surveys base on different data. \person{Bernstein} took 1,000,000 randomly chosen \NAME{IP} addresses, containing 39,206 valid hosts; 958 of them accepted a connection. \name{MailRadar}'s data source is unspecified, as well as the time of data collection. The \name{O'Reilly} survey used only domains with companies behind them; in total it were 400,000 hosts.
meillo@130 87
meillo@143 88 Remarkable are the 22 percent of ``mail security layers''\footnote{mail security layers are software guards between the network and the \mta\ that filter unwanted mail before it reaches the \MTA. This increases security by blocking attacks against the \MTA\ and by filtering malicious content. It also decreases the work load of the guarded \MTA.} in the \name{O'Reilly} survey. This large share may be a result of only regarding business mail servers. The problem concerning the survey is the deguise of the \mta\ working behind the security layer. It seems wrong to assume equal shares for the \MTA{}s behind the guards as for the unguarded \MTA{}s, because they will be more often used to guard weak \MTA{}s, as strong ones might not need them. This needs to be kept in mind when using the \name{O'Reilly} survey.
meillo@143 89
meillo@143 90
meillo@143 91 Now follows a small introduction to the five programs chosen for comparison, except \masqmail\ which already was introduced in chapter \ref{chap:introduction}. Longer introductions, including analysis and comparison, were written by Jonathan de \person{Boyne Pollard} \citeweb{jdebp}.
meillo@89 92
meillo@117 93
meillo@117 94
meillo@120 95 \subsubsection*{sendmail}
meillo@89 96 \label{sec:sendmail}
meillo@117 97 \sendmail\ is the most popular \mta, since it was one of the first and was shipped as default \MTA{}s by many vendors of \unix\ systems. %fixme: ref
meillo@89 98
meillo@143 99 The program was written by Eric \person{Allman} as the successor of his program \name{delivermail}. \sendmail\ was first released with \NAME{BSD} 4.1c in 1983. Allman was not the only one working on the program. Other people developed own versions of it and a variety of flavors came up, especially in the late eighties when Allman was inactive. %fixme: ref
meillo@89 100
meillo@124 101 \sendmail\ is focused on transferring mails between different protocols and networks, this lead to a very flexible (though complex) configuration.
meillo@89 102
meillo@89 103 The latest version is 8.14.3 from May 2008. The program is distributed under the \name{Sendmail License} as both, \freesw\ and proprietary software of \name{Sendmail, Inc.}.
meillo@89 104
meillo@128 105 Further development will go into the project \name{MeTA1} (the former name was \name{sendmail X}) which succeeds \sendmail.
meillo@89 106
meillo@130 107 More information can be found on the \sendmail\ homepage \citeweb{sendmail:homepage}.
meillo@89 108
meillo@89 109
meillo@117 110
meillo@120 111 \subsubsection*{exim}
meillo@117 112 \label{sec:exim}
meillo@143 113 \exim\ was started in 1995 by Philip \person{Hazel} at the \name{University of Cambridge}. It is forked of \name{smail-3}, and inherited the monolithic architecture, similar to \sendmail's. But having no separation of the individual components of the system, like \qmail\ and \postfix\ have, did not hurt. Its security is comparably good. %fixme: ref
meillo@117 114
meillo@132 115 \exim\ is highly configurable, especially in the field of mail policies. This makes it easy to specify how mail is routed through the system and who is allowed to send email to whom. Also interfaces for integration of virus and spam check programs are provided by design. %fixme: ref
meillo@117 116
meillo@117 117 The program is \freesw, released under the \GPL. The latest stable version is 4.69 from December 2007.
meillo@117 118
meillo@132 119 One finds \exim\ on its homepage \citeweb{exim:homepage}.
meillo@117 120
meillo@117 121
meillo@117 122
meillo@120 123 \subsubsection*{qmail}
meillo@89 124 \label{sec:qmail}
meillo@132 125 \qmail\ is seen by its community as ``a modern SMTP server which makes sendmail obsolete''.%fixme: ref
meillo@143 126 It was written by Daniel~J.\ \person{Bernstein} starting in 1995. His primary goal was to create a secure \MTA\ to replace the popular, but vulnerable, \sendmail. %fixme: ref
meillo@89 127
meillo@132 128 \qmail\ first introduced many innovative concepts in \mta\ design and is generally seen as the first security-aware \MTA\ developed. %fixme:ref
meillo@117 129 %fixme: what about mmdf?
meillo@89 130
meillo@132 131 Since November 2007, \qmail\ is released in the \name{public domain} which makes it \freesw. The latest release is 1.03 from July 1998.
meillo@89 132
meillo@143 133 The programs homepages are \citeweb{qmail:homepage1} and \citeweb{qmail:homepage2}. Further information about \qmail\ is available with Dave \person{Sill}'s ``Life with qmail'' \citeweb{lifewithqmail}.
meillo@89 134
meillo@89 135
meillo@117 136
meillo@120 137 \subsubsection*{postfix}
meillo@89 138 \label{sec:postfix}
meillo@143 139 The \postfix\ project was started in 1999 at \name{IBM research}, then called \name{VMailer} or \name{IBM Secure Mailer}. Wietse \person{Venema}'s program ``attempts to be fast, easy to administer, and secure. The outside has a definite Sendmail-ish flavor, but the inside is completely different.''\citeweb{postfix:homepage} In fact, \postfix\ was mainly designed after qmail's architecture to gain security. But in contrast to \qmail\ it aims much more on being fast and full-featured.
meillo@89 140
meillo@132 141 Today \postfix\ is taken by many \unix\ systems and \gnulinux\ distributions as default \MTA.
meillo@89 142
meillo@132 143 The latest stable version is numbered 2.5.5 from August 2008. \postfix\ is covered by the \name{IBM Public License 1.0} which is a \freesw\ license.
meillo@89 144
meillo@132 145 Additional information can be retrieved from the program's homepage \citeweb{postfix:homepage}.
meillo@89 146
meillo@89 147
meillo@89 148
meillo@89 149
meillo@89 150
meillo@89 151
meillo@120 152 \section{Comparison of MTAs}
meillo@89 153
meillo@143 154 This section does not try to provide an overall \MTA\ comparison, because this is already done by others. Remarkable comparisons are the one by Dan \person{Shearer} \cite{shearer06} and a discussion on the mailing list \name{plug@lists.q-linux.com} \citeweb{plug:mtas}. Tabulary overviews may be found at \citeweb{mailsoftware42}, \citeweb{wikipedia:comparison-of-mail-servers}, and \citeweb[section 1.9]{lifewithqmail}.
meillo@89 155
meillo@128 156 Here provided is an overview on a selection of important properties, covering the four previously introduced programs. The data comes from the above stated sources and is collected in table \ref{tab:mta-comparison}.
meillo@126 157
meillo@117 158 \begin{table}
meillo@126 159 \begin{center}
meillo@126 160 \input{input/mta-comparison.tex}
meillo@126 161 \end{center}
meillo@126 162 \caption{Comparison of MTAs}
meillo@126 163 \label{tab:mta-comparison}
meillo@117 164 \end{table}
meillo@89 165
meillo@89 166
meillo@132 167 \subsection{Architecture}
meillo@89 168
meillo@132 169 Architecture is most important when comparing \MTA{}s. Many other properties of a program depend on its architecture. %fixme: add ref?
meillo@143 170 Munawar \person{Hafiz} \cite{hafiz05} discusses in detail on \mta\ architecture, comparing \sendmail, \qmail, \postfix, and \name{sendmail X}. Jonathan de \person{Boyne Pollard}'s \MTA\ review \citeweb{jdebp} is a source too.
meillo@89 171
meillo@132 172 Two different architecture types show off: monolithic and modular \mta{}s.
meillo@130 173
meillo@132 174 Monolithic \MTA{}s are \sendmail, \name{smail}, \exim, and \masqmail. They all consist of one single \emph{setuid root}\footnote{\emph{setuid root} lets a program run with the rights of its owner, here root. This is considered a security risk.} binary which does all the work.
meillo@130 175
meillo@132 176 Modular \MTA{}s are \NAME{MMDF}, \qmail, \postfix, and \name{MeTA1}. They consist of several programs, each doing a part of the overall job. The different programs run with the least permissions the need, and \emph{setuid root} needs not to be used.
meillo@130 177
meillo@143 178 The architecture does not directly define the program's security, but ``[t]he goal of making a software secure can be better achieved by making the design simple and easier to understand and verify''\cite[chapter 6]{hafiz05}. \exim, though being monolithic, has a fairly clean security record. But it is very hard to keep the security up, as the program growth. Wietse \person{Venema} (the author of \postfix) says, the architecture enabled \postfix\ to grow without running into security problems. \citeweb[page 13]{venema:postfix-growth}
meillo@130 179
meillo@140 180 The modular design, with each sub-program doing one part of the overall job, is applied \name{Unix Philosophy}. The Unix Philosophy \cite{gancarz95} demands ``small is beautiful'' and ``make each program do one thing well''. Monolithic \MTA{}s fail here.
meillo@130 181
meillo@132 182 Today modular \mta\ architectures are the state-of-the-art.
meillo@89 183
meillo@89 184
meillo@89 185
meillo@132 186 \subsection{With focus on the future}
meillo@89 187
meillo@132 188 Section \ref{sec:what-will-be-important} tried to figure out the importances for future \MTA{}s. The four programs are compared on these (possible) future requirements now.
meillo@89 189
meillo@140 190 The first trend was provider independence, requiring easy configuration. \postfix\ seems to do best here. It used primary two configuration files (\path{master.cf} and \path{main.cf}) which are easy to manage. \sendmail\ appears to have a bad position. Its configuration file \path{sendmail.cf} is very complex, including Turing-completeness, thus it needs simplification wrappers around it to provide easier configuration. There exist the \name{m4} macros to generate \path{sendmail.cf}, but adjusting the generated result by hand seems to be nessesary for non-trivial configurations. \qmail's configuration files are simple, but the whole system is complex to set up; it requires various system users and is hardly usable without applying several patches to add basic functionality. \name{netqmail} is the community effort to help here. \exim\ has only one single configuration file (\path{exim.conf}), but it suffers most from its flexibility, like \sendmail. Flexibility and easy configuration are contrary.
meillo@89 191
meillo@132 192 As second trend, the decreasing nessesarity for high performance was identified. This goes along with the move of \MTA{}s from service providers to home servers. \postfix\ focuses much on performance, this might not be an important point then. Of course there still will be the need for high performance \MTA{}s, but a growing share of the market will not require high performance. Performance is related to simplicity, which effects security. Increasing performance does in most times decrease the other two. Simple \mta{}s not aiming for highest performance are what is needed in future. The simple of \qmail, still being fast enough, seems to be a good example.
meillo@126 193
meillo@132 194 The third trend, even more security awareness, is addressed by each of the four programs. It seems as if all widely used \mta{}s provide good security nowadays---even \sendmail\ can be considered secure today. %fixme:ref
meillo@132 195 The modular architecture, used by \qmail\ and \postfix, is generally seen to be conceptually more secure, however.%fixme: ref
meillo@132 196 \sendmail's creators have started \name{MeTA1}, a modular \MTA\ merging the best of \qmail\ and \postfix, to replace the old \sendmail. It will be interesting to watch \exim's future---will it become modular too?
meillo@126 197
meillo@126 198
meillo@93 199
meillo@89 200
meillo@89 201
meillo@117 202
meillo@117 203
meillo@132 204 %todo: my own poll (?)
meillo@117 205
meillo@117 206
meillo@132 207 %<< complexity >> << security >> << simplicity of configuration and administration >> << flexibility of configuration and administration >> << code size >> << code quality >> << documentation (amount and quality) >> << community (amount and quality) >> << used it myself >> << had problems with it >>
meillo@117 208
meillo@117 209
meillo@132 210 %<< quality criteria >> << standards of any kind >> << how to compare? >> << (bewertungsmatrix) objectivity >> << how many criteria for ``good''? >>
meillo@133 211