1.1 --- a/thesis/tex/2-MailTransferAgents.tex	Fri Nov 14 18:13:14 2008 +0100
     1.2 +++ b/thesis/tex/2-MailTransferAgents.tex	Fri Nov 14 18:16:26 2008 +0100
     1.3 @@ -1,7 +1,5 @@
     1.4  \chapter{Mail transfer agents}
     1.5  
     1.6 -\section{\unix\ \MTA{}s}
     1.7 -
     1.8  After having read about the history of electronic mail and the basics of \mta{}s in the last chapter, this chapter introduces a group of \mta{}s. Among them, the already mentioned \sendmail. The selected group will be delimited against other groups of \MTA{}s, which are described as well.
     1.9  
    1.10  The chosen programs will be presented to the reader in a short overview and with the most important facts. The next chapter will show a comparison of these programs in several disciplines.
    1.11 @@ -14,7 +12,7 @@
    1.12  
    1.13  Following are groups of \mta{}s that will \emph{not} be regarded further.
    1.14  
    1.15 -\subsection{Relay-only \MTA{}s}
    1.16 +\subsection*{Relay-only \MTA{}s}
    1.17  \label{subsec:relay-only}
    1.18  This is the most simple kind of \MTA. It transfers mail only to defined \name{smart hosts}\footnote{\name{smart host}s are \MTA{}s that receives email and route it to the actual destination}. \name{Relay-only} \MTA{}s do not receive mail from outside the system, and they do not deliver locally.
    1.19  
    1.20 @@ -25,7 +23,7 @@
    1.21  Examples for that group are: \name{nullmailer}, \name{ssmtp} and \name{esmtp}.
    1.22  
    1.23  
    1.24 -\subsection{Groupware}
    1.25 +\subsection*{Groupware}
    1.26  Normally the term ``groupware'' does not mean one single program, but a suite of programs. They build a framework which is then populated with various modules that provide actual funktionality. Modules for mail transfer, file storage, calendars, resource management, instant messaging, etc., are commonly available.
    1.27  
    1.28  One would use one of these program suites if the main work to do is not mail transfer, but providing integrated communication facilities and team working support for a group of people. The most common scenario are companies. They have \name{groupware} running to provide adequate services for their teams to work efficently. But one may use \name{groupware} on the home server for his family members also.
    1.29 @@ -33,7 +31,7 @@
    1.30  Examples are: \name{Lotus Notes}, \name{Microsoft Exchange}, \name{OpenGroupware.org} and \name{eGroupWare}.
    1.31  
    1.32  
    1.33 -\subsection{``Real'' \MTA{}s}
    1.34 +\subsection*{``Real'' \MTA{}s}
    1.35  There is a third type of \mta{}s in between the minimalistic \name{relay-only} \MTA{}s and the bloated \name{groupware}. Those programs may be named ``real \MTA{}s'', or ``proper \MTA{}s'', though there is no common name. They are what is meant with the term ``\mta''.
    1.36  
    1.37  Common to them is their focus on transfering email, while being able to act as \name{smart host}. Their variety ranges from ones mostly restricted to mail transfer (\name{qmail}) to others already having interfaces for adding further mail processing modules (\name{postfix})---thus everything in between the other two groups.  %FIXME: are postfix and qmail good examples?
    1.38 @@ -42,11 +40,11 @@
    1.39  
    1.40  
    1.41  
    1.42 -\subsection{Programs to sort out}
    1.43 +\subsection*{Programs to sort out}
    1.44  
    1.45  \name{Mail transfer agent}s can be segmented in various ways, apart from the classification above. Groups of programs wiproperties significantly different from \masqmail\ will be sorted out now.
    1.46  
    1.47 -\subsection{Non-\emph{sendmail-compatible} \MTA{}s}
    1.48 +\subsection*{Non-\emph{sendmail-compatible} \MTA{}s}
    1.49  Due to \sendmail's significance---described in section \ref{sec:sendmail}---compatiblity interfaces for \sendmail\ are of importance for \unix\ \MTA{}s. Being not \emph{sendmail-compatible} does not need to matter for some fields of action, but makes the program ineligible for serving as a general purpose \MTA\ on \unix\ systems.
    1.50  
    1.51  Hence all \MTA{}s not having a \emph{sendmail-compatible} interface or not offering it as a compatibility addon, will not be covered here.
    1.52 @@ -54,7 +52,7 @@
    1.53  An Examples here is \name{Apache James}.  %FIXME: check if correct
    1.54  
    1.55  
    1.56 -\subsection{Non-free software}
    1.57 +\subsection*{Non-free software}
    1.58  Only programs being \freesw\ are regarded, because comparing \freesw\ with proprietary or commercial software is not what typical users of programs like \masqmail\ do. Comparison with those non-free programs may be a point for large \freesw\ projects, trying to step into the business world. Small projects, mostly used by individuals at home, need to be compared against other projects of similar shape.
    1.59  
    1.60  The comparison should be seen from \masqmail's point of view, so non-free software is out of the way.
    1.61 @@ -72,7 +70,7 @@
    1.62  
    1.63  Following is a small introduction to each of the five programs chosen for comparision.
    1.64  
    1.65 -\subsection{\sendmail}
    1.66 +\subsection*{\sendmail}
    1.67  \label{sec:sendmail}
    1.68  \sendmail\ is the most popular \mta. Since it was one of the first \MTA{}s and was shipped by many vendors of \unix\ systems.
    1.69  
    1.70 @@ -87,7 +85,7 @@
    1.71  More information can be found on the \sendmail\ homepage \citeweb{sendmail:homepage} and on \citeweb{wikipedia:sendmail} and \citeweb{jdebp}.
    1.72  
    1.73  
    1.74 -\subsection{\name{qmail}}
    1.75 +\subsection*{\name{qmail}}
    1.76  \label{sec:qmail}
    1.77  \name{qmail} is seen by its community as ``a modern SMTP server which makes sendmail obsolete''. It was written by Daniel~J.\ Bernstein starting in 1995. His primary goal was to create a secure \MTA\ to replace the popular, but vulnerable, \sendmail.
    1.78  
    1.79 @@ -98,7 +96,7 @@
    1.80  The programs homepages are \citeweb{qmail:homepage1} and \citeweb{qmail:homepage2}. Further information about \name{qmail} is available on \citeweb{lifewithqmail}, \citeweb{wikipedia:qmail} and \citeweb{jdebp}.
    1.81  
    1.82  
    1.83 -\subsection{\name{postfix}}
    1.84 +\subsection*{\name{postfix}}
    1.85  \label{sec:postfix}
    1.86  The \name{postfix} project was started in 1999 at \name{IBM research}, then called \name{VMailer} or \name{IBM Secure Mailer}. Wietse Venema's program ``attempts to be fast, easy to administer, and secure. The outside has a definite Sendmail-ish flavor, but the inside is completely different.''\citeweb{postfix:homepage} In fact, \name{postfix} was mainly designed after qmail's architecture to gain security. But in contrast to \name{qmail} it aims much more on being fast and full-featured.
    1.87  
    1.88 @@ -109,7 +107,7 @@
    1.89  Additional information is available on the program's homepage \citeweb{postfix:homepage}, on \citeweb{jdebp} and \citeweb{wikipedia:postfix}.
    1.90  
    1.91  
    1.92 -\subsection{\name{exim}}
    1.93 +\subsection*{\name{exim}}
    1.94  \label{sec:exim}
    1.95  \name{exim} was started in 1995 by Philip Hazel at the \name{University of Cambridge}. Its age is about the same as \name{qmail}'s, but the architecture is totally different.
    1.96  
    1.97 @@ -122,7 +120,7 @@
    1.98  One finds \name{exim} on its homepage \citeweb{exim:homepage}. More information about it can be retrieved from \citeweb{wikipedia:exim} and \citeweb{jdebp}.
    1.99  
   1.100  
   1.101 -\subsection{\masqmail}
   1.102 +\subsection*{\masqmail}
   1.103  \label{sec:masqmail}
   1.104  The \masqmail\ program was written by Oliver Kurth, starting in 1999. His aim was to create a small \mta\ which is especially focused on computers with dial-up connections to the internet. \masqmail\ is easy configurable for situations which are rarely solveable with the common \MTA{}s.
   1.105  
   1.106 @@ -185,6 +183,9 @@
   1.107  
   1.108  masqmail: monolitic
   1.109  
   1.110 +Like its anchestor \sendmail, \masqmail\ is a monolitic program. It consists of only one \emph{setuid root}\footnote{Runs as user root, no matter which user invoked it.}\index{setuid root} binary file, named \path{masqmail}. All functionality is included in it; of course some more comes from dynamic libraries linked.
   1.111 +
   1.112 +
   1.113  exchange: (unknown)
   1.114  
   1.115  
   1.116 @@ -240,126 +241,20 @@
   1.117  
   1.118  
   1.119  
   1.120 -\section{The future of communication}
   1.121 -\label{chap:future-of-communication}
   1.122 -As globalization proceeds, long distance communication becomes more and more important. This chapter tries to locate trends in communication methods and their impact on the future for communication. The insights gathered from the analysis will be applied to \masqmail, afterwards.
   1.123  
   1.124  
   1.125 -\subsection{Communication methods}
   1.126 -\label{sec:communication-methods}
   1.127 -Today's long distance communication methods are either written or spoken information. And on the other side, they can be classified by the time between responses.
   1.128  
   1.129 -A classification of long distance communication methods is shown in figure %\ref{fig:}.
   1.130 -% slow     |              |             |
   1.131 -%          |              | letter      | days
   1.132 -%          |              |             |
   1.133 -%          |              |             |
   1.134 -%          | answering    | email       |
   1.135 -%          |   machine    | telefax     | few seconds
   1.136 -%          |              | SMS         |
   1.137 -% fast     |              |             |
   1.138 -%          | telephone    | IM          | real time
   1.139 -% -----------------------------------------------------
   1.140 -% response | spoken       | written     | delivery time
   1.141 +\subsubsection*{masqmail stuff}
   1.142  
   1.143 -% TODO: find reference literature
   1.144 +The \masqmail\ executable can be called under various names for \name{sendmail-compatibility} reasons. This is commonly organized by creating symbolic links with with different names to the \masqmail\ executable. These are \path{/usr/lib/sendmail} and \path{/usr/sbin/sendmail} because many programs expect a \mta\ to be located there. Further more \sendmail\ provides shortcuts by calling it with a different name instead of supplying command line arguments. The best known of it is \path{mailq}, which is equivilent to calling the \MTA\ with the argument \verb+-bq+. \masqmail\ reacts to the names \path{mailq}, \path{smtpd}, \path{mailrm}, \path{runq}, \path{rmail}, and \path{in.smtpd}. The last four are an addition to \sendmail. Not implemented is the name \path{newaliases} because it is not relevant to \masqmail. To provide the command nonetheless, one may write a shell script located at \path{/usr/bin/newaliases}, that simply invokes \verb+masqmail -bi+.
   1.145  
   1.146 -\paragraph{Speed}
   1.147 -Communication gets faster in general. Slow mediums as letters get substituted by electronic mail, which is delivered within seconds. Also communication becomes more transmitted through digital channels. This can be seen at the telephone which's information is now more and more transported in bits over the internet link. Also telefaxes are succeeded by email or are transported within email. Instant messaging can be seen as the written couterpart to the telephone; not to substitute it completely, but to be used if it is more useful for the information to transmit.
   1.148 +%masqmail: mailq, mailrm, runq, rmail, smtpd/in.smtpd
   1.149 +%sendmail: hoststat, mailq, newaliases, purgestat, smtpd
   1.150  
   1.151 -Many of the digital communication methods gained success by beeing cheaper than their counterparts. One example here is instant messaging in contrast to the telephone. As phoning costs fell, it became more popular again. The last years showed, that communication cost degreased dropped generally, caused by the transport through digital channels. And nothing to see, that would make them rise again.
   1.152 +\masqmail\ is written in the \NAME{C} programming language. The program, as of version 0.2.21, consists of 34 source code and eight header files, containing about 9,000 lines of code\footnote{Measured with \name{sloccount} by David A.\ Wheeler.}. Additionally, it includes a \name{base64} implementation (about 300 lines) and \name{md5} code (about 150 lines). For systems that do not provide \name{libident}, this library is distributed as well (circa 600 lines); an available shared library however has higher precedence in linking.
   1.153  
   1.154 -It seems as if in future will be low-cost communication methods available, which will be digitally transmitted.
   1.155 +The only mandatory dependency is \name{glib}---a cross-platform software utility library, originated in the \NAME{GTK+} project. It provides safer replacements for many standard library functions. (The unsafe \verb+sprintf()+ is one example.) Also it offers handy data containers, easy-to-use implementations of data structures, and much more.
   1.156  
   1.157 -\paragraph{Variety}
   1.158 -Regarding the variety of communication methods shows a change, too. Communication systems are more easy to establish today, so more get established. This leads to more methods a person uses. But not only in the amount, also in parallel. For example when two people talk to each other on the phone, one might send a URI\footnote{Uniform Resource Identifier} by email meanwhile, because oral communication is not well suited to exchange such data. Another example for in parallel used communication channels is video chatting. Ony typically sees the other person, talks to it, and additionally has a instant messaging facility for exchanging written information.
   1.159 +With \masqmail\ comes the small tool \path{mservdetect}; it helps setting up a configuration that uses the \name{mserver} system to detect the online state. Two other binaries get compiled for testing purposes: \path{readtest} and \path{smtpsend}. All three programms use \masqmail\ source code; they only add a file with a \verb+main()+ function each.
   1.160  
   1.161 -Parallel usage of different kinds of communication channels will be important in future. The most common combinations are one for spoken and one for written information. But one for dialogs and one for sending documents will be important too.
   1.162 -
   1.163 -\paragraph{Hardware}
   1.164 -Next about the hardware needed for communicating. On the one side stands the telephone, now available as the mobile phone. It provides spoken dialog by calling, spoken messages with the included answering machine and written messages in form of short message service. On the other side stands the letter and its relatives. They need pen and paper, a telefax machine or in most today's cases a computer. They typically send documents, only instant messaging is focused on dialog.
   1.165 -
   1.166 -The last years finally brought the two groups together, with \name{smart phones} being the merging element. Smart phones are computers in the size of mobile phones. They provide both functions, using it as telephones and as computers.
   1.167 -
   1.168 -It matches well the requirements of telephoning and short message service, for which it was designed of course. Also providing being suitable for instant messaging in what is needed additionally to the telephone and short message service. The only problem is the minimal keyboard available to insert text. This also affects writing documents in case of email. It can be done but not very comfortably. Further communication methods include voice and video messages.
   1.169 -
   1.170 -This leaves us with the need for ordinary computers for the field of exchanging documents, and as better input hardware for all written input.
   1.171 -
   1.172 -
   1.173 -
   1.174 -\subsection{Trends for electronic mail}
   1.175 -\label{sec:email-trends}
   1.176 -The previous section stated that electronic mail will still be important in future to complete the communication methods provided by phone and instant messaging.
   1.177 -
   1.178 -But will emailing in future not be the same as emailing now. This will mainly affect how email is transfered.
   1.179 -
   1.180 -\paragraph{Provider oriented emailing}
   1.181 -Today's email structure is heavily dependent on email providers. This means, most people have email addresses from some provider. These can be the provider of their online connection (e.g.\ \NAME{AOL}, \name{T\~Online}), freemail provider (e.g.\ \NAME{GMX}, \name{Yahoo}, \name{Hotmail}) or provider that offer enhanced mail services that one needs to pay for. Outgoing mail is send either with the webmail client of the provider or using \name{mail user agent}s sending it to the provider for relay. Incoming mail is read with the webmail client or retrieved from the provider via \NAME{POP3} or \NAME{IMAP} to the local computer to be read in the \name{mail user agent}. This means all mail sending and receiving work is done by the provider.
   1.182 -
   1.183 -The reason therefor is originated in the time when people used dial-up connections to the internet. A mail server needs to be online to receive email. Sending mail is no problem, but receiving it is hardly possible with an \MTA\ being few time online. Internet service providers had servers running all day long connected to the internet. So they offered email service.
   1.184 -
   1.185 -\paragraph{Provider independence}
   1.186 -Nowadays, dial-up internet access is rare; the majority has broadband internet access paying a flat rate for it. So being online or not does not affect costs anymore, even traffic is unlimited. Today it is possible to have an own mail server running at home. The last technical problem remaining are the changing \NAME{IP} addresses one gets assigned every 24 hours. But this is easily solvable with one of the dynamic \NAME{DNS} services around; they provide the mapping of a fixed domain name to the changing \NAME{IP} addresses.
   1.187 -
   1.188 -Home servers become popular in these days, for central data storage and multi media services. Being assembled of energy efficient elements, power consumption is no big problem anymore. These home servers will replace video recorders and music collections in the near future. It is also realistic that they will manage heating systems and intercoms too. Given the future leads to this direction, it is a logical step to have email and other communication will be provided by the (or one of) the own server aswell.
   1.189 -
   1.190 -After \mta{}s have not been popular for users in the last time, the next years might bring them back to them. Maybe in a few years nearly everyone will have one running at home \dots\ possibly without knowing about it.
   1.191 -
   1.192 -\paragraph{Is email future-safe?}
   1.193 -It seems as if electronic mail or a similar technology has good chances to survive the next decades. This bases on the assumption that it always will be important to send information messages. These can be notes from other people, or notifications from systems (like a broken or full hard drive in the home server, or the coffee machine ran out of coffee beans). Other communication technologies are not as suitable for this kind of messages, as email, short message service, voice mail, and the like. Telephone talks are more focused on dialog and normally interrupt people. These kind of messages should not interrupt people, unless urgent, and they do not need two-way information exchange. The second argument appies to instant messaging too. If only one message is to be send, one does not need instant messaging. Thus, one type of one-way message sending technology will survive.
   1.194 -
   1.195 -Whether email will be the one surviving, or short message service, or another one, does not matter. Probably it will be \name{unified messaging}, which includes all of the other ones in it, anyway. \MTA{}s are a kind of software needed for all of these messaging methods---programs that transfer and receive messages.
   1.196 -
   1.197 -\paragraph{Pushing versus polling}
   1.198 -The retrieval of email is a field that is about to change now. The old way is to fetch email by polling the server that holds the personal mail box. This polling is done in regular intervals, often once every five to thirty minutes. The mail transfer from the mail box to the \name{mail user agent} is initiated from the mail client side. The disadvantage herewith is the delay between mail actually arriving on the server and the user finally having the message on his screen.
   1.199 -
   1.200 -To remove this disadvantage, \name{push email} was invented. Here the server is not polled every few minutes about new mail, but the server pushes new mail directly to the client on arrival. The transfer is initiated by the server. This concept became popular with the smart phones; they were able to do emailing, but the traffic caused by polling the server often was expensive. The concept workes well with mobile phones where the provider knows about the client, but it seems not to be a choice for computers since the provider needs to have some kind of login to push data to the computer.
   1.201 -
   1.202 -The push concept, however could swap over to computers when using a home server and no external provider. A possible scenario is a home server receiving mail from the internet and pushing it to computers and smart phones. The configuration could be done by the user through some simple interface, like one configures his telephone system to have different telephone numbers ring on specified phones.
   1.203 -%FIXME: add reference to push email
   1.204 -
   1.205 -\paragraph{Internet Mail 2000}
   1.206 -Another concept to redesign the electronic mail system, but this time focused on mail transfer is named ``Internet Mail 2000''. It was proposed by Daniel J.\ Bernstein, the creater of \name{qmail}. Similar approaches were independently introduced by others too.
   1.207 -
   1.208 -As main change it makes the sender have the responsibility of mail storage; only a notification about a mail message gets send to the receiver, who can fetch the message then from the sender's server. This is in contrast to the \NAME{SMTP} mail architecture, where mail and the responsibility for it is transfered from the sender to the receiver.
   1.209 -
   1.210 -\name{Mail transfer agent}s are still important in this mail architecture, but in a slightly different way. Their job is not transfering mail anymore---this makes the name missleading---they are used to transport the notifications about new mail to the destinations. This is a quite similar job as they do in the \NAME{SMTP} model. The real transfer of the mail can be done in any way, for example via \NAME{FTP} or \NAME{SCP}.
   1.211 -
   1.212 -%FIXME: add references for IM2000
   1.213 -
   1.214 -
   1.215 -\section{Market analysis}
   1.216 -
   1.217 -\subsection{\NAME{SWOT} analysis}
   1.218 -%TODO
   1.219 -
   1.220 -
   1.221 -
   1.222 -\subsection{What will be important}
   1.223 -\label{sec:important-for-mtas}
   1.224 -Now that it is explained why email will survive (in some changed but related form), it is time to think about the properties required for \mta{}s in the next years. As the fields and kinds of usage change, the requirement change too.
   1.225 -
   1.226 -Provider independence through running an own mail server at home asks for easy configuration of the \MTA. Providers have specialists to configure the systems, but ordinary people do not. Solutions are either having some home service system for computer configuration established with specialists coming to one's home to set up the systems; like it is already common for problems with the power supply or water supply system. Or configuration needs to be easy and fool-prove, to be done by the owner himself. The latter solution depends on standardized parts that fit together seamlessly. The technology itself must not be a problem itself. Only settings custom to the users environment should be left open for him to set. This of course needs to be doable on a simple configuration interface like a web interface; non-technical educated users should be able to configure the system.
   1.227 -
   1.228 -\sendmail\ and \name{qmail} appear to have bad positions at this point. Their configuration is complex, thus they would need simplification wrappers around them to provide easy configuration.
   1.229 -
   1.230 -The approach of wrappers around the main program to make it look easier to the outside is a good concept in general. %FIXME: add ref
   1.231 -It still lets the specialist do complex and detailed configuration, and also offering a simple configuration interface to novices. Further more is it well suited to provide various wrappers with different user interfaces (e.g.\ a graphical program, a website, a command line program; all of them either in a questionaire style or iteractive).
   1.232 -
   1.233 -When \MTA{}s become popular on home servers and maybe even on workstations and smart phones, then performance will be less important. Providers need \mta{}s that process a large amount of mail in short time. Home servers or workstations however, do not see that much mail; they need to handle tens or hundrets of email messages per hour. Thus performance will probably not be a main requirement for an \MTA\ in the future, if they mainly run on private machines.
   1.234 -
   1.235 -\name{postfix} focuses much on performance, this might not be an important point then.
   1.236 -
   1.237 -New mailing concepts and architectures like push email or \name{Internet Mail 2000} will, if they succeed, require \mta{}s to adopt the new technology. \MTA{}s that are not able to change are going to be sorted out by evolution. Thus it is important to not focus too much on one use case, but to stay flexible. Allman saw this property of \sendmail\ one reason for its huge success (see section \ref{sec:sendmail}).
   1.238 -
   1.239 -Another important requirement for all kinds of software will be security. There is a constant trend going from completely non-secured software from the 70s and 80s over growing security awareness in the 90s to security being a primary goal now. This leads to the conclusion that software security will even more important in the next years. As more clients get connected to the internet and especially more computers are waiting for incoming connections (like an \MTA\ in a home server), there are more possibilities to break into systems. Securing software systems will be done with increasing effort in future.
   1.240 -
   1.241 -``Plug-and-play''-able hardware with preconfigured software running can be expected to become popular. Like someone buys a set-top box to watch Pay-TV today, he might be buying a box acting as mail server in a few years. He plugs the power cable in, inserts his email address in a web interface and selects the clients (workstation computers or smart phones) to which mail should be send and from which mail is accepted to receive. That's all. It would just work then, like everyone expects it from a set-top box today.
   1.242 -
   1.243 -Containing secure and robust software is a pre-requisite for such boxes to make that vision possible.
   1.244 -
   1.245 -It seems as if all widely used \mta{}s provide good security nowadays. \name{qmail}'s architecture, also used in \name{postfix}, is generally seen to be conceptually more secure, however.
   1.246 -
   1.247 -In summary: easy configuration, aswell as the somehow opposed flexibility will be important for future \mta{}s. Also will it be security, but not performance. \MTA{}s might become more commodity software, like web servers already are today, with the purpose to include it in many systems and the need of minimal configuration.
   1.248 -
   1.249 -
   1.250 +\masqmail\ does not provide an interface for modules with additional functionality. There exists no add-on or module system. But the code is separated by function to the various source files, and some functional parts can be included or excluded by defining symbols. This means adding some argument (like \verb+--enable-maildir+) to the \verb+configure+ call. Thus the concerning code gets not removed by the preprocessor.