1.1 --- a/thesis/tex/1-Introduction.tex	Fri Feb 06 21:08:49 2009 +0100
     1.2 +++ b/thesis/tex/1-Introduction.tex	Fri Feb 06 21:09:21 2009 +0100
     1.3 @@ -94,7 +94,7 @@
     1.4  
     1.5  \person{Kurth} abandoned \masqmail\ after 2005 and no one adopted the project since then. Thus, the author of this thesis decided to take over responsibility for \masqmail\ now. He received \person{Kurth}'s permission to do so in private telephone conversation with \person{Kurth} on September 4, 2008.
     1.6  
     1.7 -The program's new homepage \citeweb{masqmail:homepage} includes a collection of available information about this \MTA.
     1.8 +The program's new homepage\index{masqmail!homepage} \citeweb{masqmail:homepage} includes a collection of available information about this \MTA.
     1.9  
    1.10  
    1.11  
    1.12 @@ -102,7 +102,7 @@
    1.13  \subsection{Target field}
    1.14  \label{sec:masqmail-target-field}
    1.15  
    1.16 -\person{Kurth}'s intention when creating \masqmail\ is best told in his own words:
    1.17 +\person{Kurth}'s intention when creating \masqmail\ is best told in his own words:\index{masqmail!design intention}
    1.18  
    1.19  \begin{quote}
    1.20  MasqMail is a mail server designed for hosts that do not have a permanent internet connection eg. a home network or a single host at home. It has special support for connections to different \NAME{ISP}s. It replaces sendmail or other \MTA{}s such as qmail or exim.
    1.21 @@ -111,7 +111,7 @@
    1.22  
    1.23  It is intended to cover a specific niche: non-permanent Internet connection and different \name{Internet Service Providers} (short: \NAME{ISP}s).
    1.24  
    1.25 -Although it can basically replace other \MTA{}s it is not \emph{generally} aimed to do so. The package description of \masqmail\ within \debian\ states this more clearly by changing the last sentence to:
    1.26 +Although it can basically replace other \MTA{}s it is not \emph{generally} aimed to do so. The package description of \masqmail\ within \debian\ states this more clearly by changing the last sentence to:\index{debian!masqmail package}
    1.27  
    1.28  \begin{quote}
    1.29  In these cases, MasqMail is a slim replacement for full-blown \MTA{}s such as sendmail, exim, qmail or postfix.
    1.30 @@ -120,9 +120,9 @@
    1.31  
    1.32  The program is a good replacement ``in these cases'' but not generally, since it lacks essential features for running on publically accessable mail servers. It is primarily not secure enough for being accessible from untrusted locations.
    1.33  
    1.34 -\masqmail\ is best used in home networks which are non-permanently connected to the Internet. It is easy configurable for situations which are rarely solvable with the common \MTA{}s. Such include different handling of mail to local or remote destination and respecting different routes of online connection. These features are explained in more detail in section~\ref{sec:masqmail-features}.
    1.35 +\masqmail\ is best used in home networks which are non-permanently connected to the Internet\index{non-permanent}. It is easy configurable for situations which are rarely solvable with the common \MTA{}s. Such include different handling of mail to local or remote destination and respecting different routes of online connection. These features are explained in more detail in section~\ref{sec:masqmail-features}.
    1.36  
    1.37 -While many other \MTA{}s are general purpose \MTA{}s, \masqmail\ aims on special situations. Nevertheless, it can be used as general purpose \MTA\ too. Especially this was a design goal of \masqmail: To be a replacement for \sendmail\ or similar \MTA{}s.
    1.38 +While many other \MTA{}s are general purpose \MTA{}s, \masqmail\ aims on special situations. Nevertheless, it can be used as general purpose \MTA\ too. Especially this was a design goal of \masqmail: To be a replacement for \sendmail\ or similar \MTA{}s.\index{masqmail!sendmail replacement}
    1.39  
    1.40  \masqmail\ is designed to run on workstations and on servers in small networks, like they are common in \NAME{SOHO}s (\name{Small Offices/Home Offices}).
    1.41  
    1.42 @@ -130,7 +130,7 @@
    1.43  
    1.44  \subsubsection*{Typical usage scenarios}
    1.45  
    1.46 -This section describes three common setups that make sensible use of \masqmail. The first two are shown in figure~\ref{fig:masqmail-typical-usage}.
    1.47 +This section describes three common setups that make sensible use of \masqmail. The first two are shown in figure~\ref{fig:masqmail-typical-usage}.\index{masqmail!common setups}
    1.48  
    1.49  \begin{figure}
    1.50  	\begin{center}
    1.51 @@ -147,27 +147,38 @@
    1.52  \label{scenario1}
    1.53  If no server is present, every workstation would be equipped with \masqmail. Mail transfer within the same machine or within the local net works straight forward using direct transfer. Outgoing mail to the Internet is sent to an \name{Internet Service Provider} (short: \NAME{ISP}) for relaying whenever the router goes online. The configuration of \masqmail\ would be the same on every computer; only host names would differ.
    1.54  To receive mail from the Internet requires a mailbox on the \NAME{ISP}'s mail server. Mail needs to be fetched from the \NAME{ISP}'s server onto the workstation using the \NAME{POP3} or \NAME{IMAP} protocol.
    1.55 +\index{isp}
    1.56 +\index{pop3}
    1.57 +\index{imap}
    1.58  
    1.59  \item[Scenario 2:]
    1.60  \label{scenario2}
    1.61  In the same network but with a server, one could have \masqmail\ running on the server and using simple forwarders (see section~\ref{subsec:relay-only}) on the workstations to transfer mail to the server. The server would then, dependent on the destination of the message, deliver locally or relay to an \NAME{ISP}'s server for further relay. This setup does only support mail transfer to the server but not back to a workstation. However, this can be solved by mounting the user's mailbox from the server to the workstation or by using \NAME{POP3} or \NAME{IMAP}. Mail transfer from the \NAME{ISP} to the local server needs \NAME{POP3} or \NAME{IMAP} as well.
    1.62 +\index{isp}
    1.63 +\index{pop3}
    1.64 +\index{imap}
    1.65  
    1.66  \item[Scenario 3:]
    1.67  \label{scenario3}
    1.68  A third scenario is unrelated as it is about notebooks. Notebooks are usually used as mobile workstations. One uses them to work at different locations. With the increasing popularity of wireless networks this becomes more and more common. Different networks demand for different setups: In one network it is best to send mail to an \NAME{ISP} for relay. In another network it might be preferred to use a local mail server. A third network may have no Internet access at all, hence using a local mail server is required. All these different setups can be configured once and then used by simply telling the online state to \masqmail, even automatically within a network setup script.
    1.69 +\index{isp}
    1.70 +\index{notebook}
    1.71  \end{description}
    1.72  
    1.73  
    1.74  In general, all kinds of usage scenarios within a trusted network are possible. Important to notice is that mail can not be sent from outside into the trusted network then. For using \masqmail\ on notebooks it is suggested to only accept mail from local users because notebooks are often in untrusted environments.
    1.75 +\index{untrusted environments}
    1.76  
    1.77  
    1.78  
    1.79  
    1.80  \subsubsection*{Limitations}
    1.81 +\index{masqmail!limitations}
    1.82  
    1.83  Although \masqmail\ is seen as a replacement for other general purpose \MTA{}s, it should not be used on large mail servers. The reasons are that it implements only a basic subset of features and that its performance and security is not as good as needed for such usage.
    1.84  
    1.85  The author, \person{Kurth}, warns on the old project's website about using \masqmail\ to accept connections from the Internet because of the risk of being an open relay:
    1.86 +\index{open relay}
    1.87  
    1.88  \begin{quote}
    1.89  MasqMail is not designed to run on a host with a permanent internet connection. It does not have the ability to check for spam mail and it will relay everything from everywhere to everywhere. Use another mail server such as exim for permanent connections.
    1.90 @@ -175,6 +186,7 @@
    1.91  \end{quote}
    1.92  
    1.93  The actual problem is not the permanent Internet connection but listening for incoming mail on it. If a firewall is closed for incoming mail, then the permanent Internet connection is no problem. To use \masqmail\ for permanent Internet connections it needs to be secured with care.
    1.94 +\index{firewall}
    1.95  
    1.96  The Internet is the common example for an untrusted network but other networks may be untrusted too.
    1.97  
    1.98 @@ -197,12 +209,23 @@
    1.99  \subsubsection*{The source code}
   1.100  
   1.101  \masqmail\ is written in the C programming language. The program, as of version 0.2.21, consists of 34 source code and eight header files which contain about 9\,000 lines of code\footnote{Measured with \name{sloccount} by David A.\ Wheeler \citeweb{sloccount}.}. Additionally, it includes a \name{base64} implementation (about 300 lines) and \name{md5} code (about 150 lines). For systems that do not provide \name{libident}, this library is distributed as well (circa 600 lines); an available shared library has higher precedence in linking, though.
   1.102 +\index{c}
   1.103 +\index{lines of code}
   1.104 +\index{base64}
   1.105 +\index{md5}
   1.106 +\index{libident}
   1.107  
   1.108  The only mandatory dependency is \name{glib}---a cross-platform software utility library, originated in the \NAME{GTK+} project. It provides safe replacements for many standard library functions, especially for the string functions. It also offers handy data containers, easy-to-use implementations of data structures, and much more.
   1.109 +\index{glib}
   1.110 +\index{masqmail!dependencies}
   1.111  
   1.112  Some parts of \masqmail's functionality can be included or excluded at compile time by defining symbols. To enable maildir support for example, one has to add \verb_--enable-maildir_ to the configure call. Otherwise the concerning code gets removed during preprocessing.
   1.113 +\index{exclude code}
   1.114 +\index{maildir}
   1.115  
   1.116  With \masqmail\ comes the small tool \path{mservdetect}; it helps setting up a configuration that uses the \name{mserver} system for online state detection. Two other binaries get compiled for testing purposes: \path{readtest} and \path{smtpsend}. These three additional programs use parts of \masqmail's source code; they only add a file with a \verb+main()+ function each.
   1.117 +\index{mserver}
   1.118 +\index{test program}
   1.119  
   1.120  
   1.121  
   1.122 @@ -210,19 +233,29 @@
   1.123  \label{sec:masqmail-features}
   1.124  
   1.125  \masqmail\ supports two channels for incoming mail:
   1.126 +\index{masqmail!incoming channels}
   1.127  
   1.128  \begin{enumerate}
   1.129 -\item Standard input which is used when \path{masqmail} (or the \path{sendmail} link) is executed on the command line
   1.130 -\item A \NAME{TCP} socket which is used by local or remote clients that talk \SMTP
   1.131 +	\item Standard input which is used when \path{masqmail} (or the \path{sendmail} link) is executed on the command line
   1.132 +	\item A \NAME{TCP} socket which is used by local or remote clients that talk \SMTP
   1.133  \end{enumerate}
   1.134 +\index{sendmail!command}
   1.135 +\index{tcp socket}
   1.136  
   1.137  The outgoing channels for mail are:
   1.138  
   1.139  \begin{enumerate}
   1.140 -\item Direct delivery to local mailboxes (in \name{mbox} or \name{maildir} format)
   1.141 -\item Local pipes to pass mail to a program (e.g.\ to \MDA{}s or to gateways to \NAME{UUCP} or fax)
   1.142 -\item \NAME{TCP} sockets to transfer mail to other \MTA{}s using the \SMTP\ protocol
   1.143 +	\item Direct delivery to local mailboxes (in \name{mbox} or \name{maildir} format)
   1.144 +	\item Local pipes to pass mail to a program (e.g.\ to \MDA{}s or to gateways to \NAME{UUCP} or fax)
   1.145 +	\item \NAME{TCP} sockets to transfer mail to other \MTA{}s using the \SMTP\ protocol
   1.146  \end{enumerate}
   1.147 +\index{tcp socket}
   1.148 +\index{local delivery}
   1.149 +\index{mbox}
   1.150 +\index{maildir}
   1.151 +\index{uucp}
   1.152 +\index{fax}
   1.153 +\index{gateway}
   1.154  
   1.155  Figure~\ref{fig:masqmail-channels} shows this as a picture. (The ``online state'' input is explained a bit later.)
   1.156  
   1.157 @@ -232,36 +265,48 @@
   1.158  	\end{center}
   1.159  	\caption{Incoming and outgoing channels of \masqmail}
   1.160  	\label{fig:masqmail-channels}
   1.161 +	\index{figure!incoming and outgoing channels of \masqmail}
   1.162  \end{figure}
   1.163  
   1.164  Outgoing \SMTP\ connections feature \SMTP-\NAME{AUTH} and \SMTP-after-\NAME{POP} authentication but incoming connections do not. Using wrappers for outgoing connections is supported. This allows encrypted communication through a gateway application like \name{openssl}.
   1.165 +\index{auth!smtp-auth}
   1.166 +\index{auth!smtp-after-pop}
   1.167  
   1.168  Mail queuing is essential for \masqmail\ and thus supported of course, alias expansion is also supported.
   1.169 +\index{alias expansion}
   1.170  
   1.171  The \masqmail\ executable can be called by various names for sendmail-compatibility reasons. As many programs expect the \MTA\ to be located at \path{/usr/lib/sendmail} or \path{/usr/sbin/sendmail}, symbolic links are pointing from there to the \masqmail\ executable. Furthermore does \sendmail\ support calling it with a different name instead of supplying command line arguments. The best known of these shortcuts is \path{mailq} which is equivalent to calling it with the argument \verb+-bq+. \masqmail\ recognizes the shortcuts \path{mailq}, \path{smtpd}, \path{mailrm}, \path{runq}, \path{rmail}, and \path{in.smtpd}. The first two are inspired by \sendmail. Not implemented yet is the shortcut \path{newaliases} because \masqmail\ does not generate binary representations of the alias file.\footnote{A shell script named \path{newaliases} that invokes \texttt{masqmail -bi} can provide the command to satisfy strict requirements.} \path{hoststat} and \path{purgestat} are missing for complete sendmail-compatibility.
   1.172 -%masqmail: mailq, mailrm, runq, rmail, smtpd/in.smtpd
   1.173 -%sendmail: hoststat, mailq, newaliases, purgestat, smtpd
   1.174 +\index{sendmail!compatibility}
   1.175 +\index{symbolic link}
   1.176 +\index{shortcuts}
   1.177  
   1.178 -Additional to the \MTA\ job, \masqmail\ also offers mail retrieval services by acting as a \NAME{POP3} client. It can fetch mail from different remote locations, also dependent on the active online connection. Such functionality is especially useful in a setup like \name{Scenario 2} on page \pageref{scenario2}.
   1.179 +Additional to the \MTA\ job, \masqmail\ also offers mail retrieval services by acting as a \NAME{POP3} client. It can fetch mail from different remote locations, also dependent on the active online connection. Such functionality is especially useful in a setup like \name{Scenario 2} on page~\pageref{scenario2}.
   1.180 +\index{pop3}
   1.181  
   1.182  
   1.183  
   1.184  \subsubsection*{Online detection and online routes}
   1.185  \label{sec:masqmail-routes}
   1.186 +\index{masqmail!online routes}
   1.187  
   1.188  \masqmail\ focuses on handling different non-permanent online connections, thus a concept of online routes is used. One may configure any number of routes to send mail. Each route can have criteria to determine if some message is allowed to be sent over it. Mail to destinations outside the local network gets queued until a suitable online connections is available.
   1.189 +\index{non-permanent}
   1.190  
   1.191  The idea behind this concept is sending mail to the Internet through the mail server of the same \NAME{ISP} over which one had dialed in. It was quite common that \NAME{ISP}s accepted mail for relay only if it came from a online connection they managed. This means, it was not possible to relay mail through the mail server of one \NAME{ISP} while being online through the connection of another \NAME{ISP}. \masqmail\ is a solution to the wish of switching the relaying mail server easily.
   1.192 +\index{isp}
   1.193  
   1.194  Related is \masqmail's ability to rewrite the sender's email address dependent on which \NAME{ISP} is used. This prevents mail from being likely classified as spam.
   1.195 +\index{spam}
   1.196  
   1.197  To react on the different situations, \masqmail\ needs to query the current online state. Is an online connection available? And if it is: Which one? Three methods are implemented:
   1.198 +\index{online state}
   1.199  
   1.200  \begin{enumerate}
   1.201 -\item Reading from a file
   1.202 -\item Reading the output of a command
   1.203 -\item Querying an \name{mserver} system
   1.204 +	\item Reading from a file
   1.205 +	\item Reading the output of a command
   1.206 +	\item Querying an \name{mserver} system
   1.207  \end{enumerate}
   1.208 +\index{mserver}
   1.209  
   1.210  Each method may return a string naming the route that is online or returning nothing to indicate offline state.
   1.211  
   1.212 @@ -276,14 +321,21 @@
   1.213  
   1.214  
   1.215  \section{Why \masqmail\ is worth it}
   1.216 +\index{masqmail!reasons to revive}
   1.217  
   1.218  First of all, \masqmail\ is better suited for its target field of operation (multiple non-permanent online connections) than every other \MTA. Especially is such usage easy to set up because \masqmail\ was designed for that. Many alternative \MTA{}s were not designed for those scenarios at all as the following two example show: ``Exim is designed for use on a network where most messages can be delivered at the first attempt.'' \cite[page~30]{hazel01}. ``qmail was designed for well-connected hosts: those with high-speed, always-on network connectivity.'' \cite[page9]{sill02}.
   1.219 +\index{non-permanent}
   1.220 +\index{qmail}
   1.221 +\index{exim}
   1.222  
   1.223  %fixme: hikernet
   1.224  
   1.225  Additionally does \masqmail\ make it easy to run an \MTA\ on workstations or notebooks. There is no need to do complex configuration or to be a mail server expert. Only a handful of options need to be set; the host name, the local networks, and one route for relaying are sufficient in most times. %fixme: is that true?
   1.226 +\index{notebook}
   1.227  
   1.228  Probably users say it best; in this case \person{Derek Broughton}:
   1.229 +\index{masqmail!users}
   1.230 +
   1.231  \begin{quote}
   1.232  No kidding. The whole point is that you \_have\_ to have an \MTA\ and you don't
   1.233  want to configure Postfix/Exim/Sendmail/Qmail (almost all of which I've
   1.234 @@ -300,16 +352,21 @@
   1.235  Not to forget \masqmail's size. \masqmail\ is much smaller than full-blown \MTA{}s like \sendmail, \postfix, or \exim, and still smaller than \qmail. (See section~\ref{sec:mta-comparison} for details.) This makes \masqmail\ a good choice for workstations or even embedded computers.
   1.236  
   1.237  Again words of a user who chose \masqmail\ as \MTA\ on his old laptop with a 75 megahertz processor and eight megabytes of \NAME{RAM}:
   1.238 +
   1.239  \begin{quote}
   1.240  Masqmail appears to be a great sendmail replacement in this case. It's small and is built to support sending mail ``off-line'', and to connecting to the \SMTP\ servers of several \NAME{ISP}s.
   1.241  \hfill\citeweb{stosberg:low-mem-laptop}
   1.242  \end{quote}
   1.243 +\index{isp}
   1.244 +\index{notebook}
   1.245  
   1.246  
   1.247  
   1.248  Although the development on \masqmail\ has been stopped in 2003, \masqmail\ still has its users. Having users is already reason enough for further development and maintenance. This applies especially when the software covers a niche and when requirements for such software in general changed. Both is the case for \masqmail.
   1.249  
   1.250  It is difficult to get numbers about users of Free Software because no one needs to tell anyone when he uses some software. \debian's \name{popcon} statistics \citeweb{popcon.debian} are a try to provided numbers. For January 2009, the statistics report 60 \masqmail\ installations of which 49 are in active use. If it is assumed that one third of all \debian\ users report their installed software\footnote{One third is a high guess as it means there would be only about 230 thousand \debian\ installations in total. But according to the \name{Linux Counter} \citeweb{counter.li.org} between 490 thousand and 12 million \debian\ users can be estimated.}, there would be in total around 150 active \masqmail\ installations in \debian. \name{Ubuntu} which also does \name{popcon} statistics \citeweb{popcon.ubuntu}, counts 82 installations with 13 active ones. If here also one third of all systems submit their data, 40 active installations can be added. Including a guessed amount of additional 30 installations on other \unix\ operating systems makes about 220 \masqmail\ installations in total. Of course one person may have \masqmail\ installed on more than one computer, but a total of 150 different users seems to be realistic.
   1.251 +\index{debian!popcon}
   1.252 +\index{masqmail!users}
   1.253  
   1.254  %The increasing number of systems using \masqmail, as it is shown on the \name{popcon} graph \citeweb{popcon.debian:masqmail}, seems to be impressive in the beginning as \masqmail\ was not developed during that time. But it might come from the increasing popularity of \name{popcon} over the time.
   1.255  
   1.256 @@ -327,8 +384,10 @@
   1.257  
   1.258  
   1.259  \section{Problems to solve}
   1.260 +\index{masqmail!problems}
   1.261  
   1.262  A program that is neglected for more than five years in a field of operation that changed during this time surely needs improvement. Security and spam have highly increased in importance since 2003. Dial-up connections became rare, instead broadband flat rates are common now. Other \MTA{}s evolved in respect to theses changes---\masqmail\ did not.
   1.263 +\index{dial-up connections}
   1.264  
   1.265  The current market situation and trends for the future need to be identified. Looks at other \MTA{}s need to be taken. Required work on \masqmail\ needs to be defined in combination with the evaluation of strategies to do this work. And a plan for further development should be created.
   1.266  
   1.267 @@ -341,8 +400,10 @@
   1.268  This thesis is neither a installation guide for \masqmail\ nor a detailed explanation of \masqmail's source code. Installation and setup guides can be found on \masqmail's homepage \citeweb{masqmail:homepage}.
   1.269  
   1.270  The \NAME{POP3} functionality of \masqmail\ receives few regard in this document because it is not directly related to the core of \masqmail\ which is being an \MTA.
   1.271 +\index{pop3}
   1.272  
   1.273  The \name{mserver} system to query the online state is also only mentioned but not regarded further. It seems best to move this functionality into a separate program which is run through the shell command interface, anyway.
   1.274 +\index{mserver}
   1.275  
   1.276  
   1.277