1.1 --- a/thesis/tex/4-MasqmailsFuture.tex	Tue Feb 03 17:53:03 2009 +0100
     1.2 +++ b/thesis/tex/4-MasqmailsFuture.tex	Tue Feb 03 18:01:33 2009 +0100
     1.3 @@ -42,10 +42,10 @@
     1.4  \label{rf1}
     1.5  \sendmail-compatible \MTA{}s must support at least two incoming channels: mail submitted using the \sendmail\ command, and mail received on a \NAME{TCP} port. Thus it is common to split the incoming channels into local and remote. This is done by \qmail\ and \postfix. The same way is \person{Hafiz}'s view \cite{hafiz05}.
     1.6  
     1.7 -\SMTP\ is the primary mail transport protocol today, but with the increasing need for new protocols (see section \ref{sec:what-will-be-important}) in mind, support for more than just \SMTP\ is good to have. New protocols will show up, maybe multiple protocols need to be supported then. This leads to multiple remote channels, one for each supported protocol as it was done in other \MTA{}s. Best would be interfaces to add further protocols as modules.
     1.8 +\SMTP\ is the primary mail transport protocol today, but with the increasing need for new protocols (see section~\ref{sec:what-will-be-important}) in mind, support for more than just \SMTP\ is good to have. New protocols will show up, maybe multiple protocols need to be supported then. This leads to multiple remote channels, one for each supported protocol as it was done in other \MTA{}s. Best would be interfaces to add further protocols as modules.
     1.9  
    1.10  
    1.11 -Outgoing mail is commonly either sent using \SMTP, piped into local commands (for example \path{uucp}), or delivered locally by appending to a mailbox. Outgoing channels are similar for \qmail, \postfix, and \name{sendmail X}: All of them have a module to send mail using \SMTP, and one for writing into a local mailbox.
    1.12 +Outgoing mail is commonly either sent using \SMTP, piped into local commands (for example \path{uucp}), or delivered locally by appending to a mailbox. Outgoing channels are similar for \qmail, \postfix, and \name{sendmail~X}: All of them have a module to send mail using \SMTP, and one for writing into a local mailbox.
    1.13  
    1.14  %fixme: is the def of MTA: transfer between machines, or transfer between users?
    1.15  Local mail delivery is a job that uses root privilege to be able to switch to any user in order to write to his mailbox. It is possible to deliver without being root privilege, but delivery to user's home folders is not generally possible then. Thus even the modular \MTA{}s \qmail\ and \postfix\ use root privilege for it. As mail delivery to local users is \emph{not} included in the basic job of an \MTA{} and introduces a lot of new complexity, why should the \MTA\ bother? In order to keep the system simple, reduce privilege, and to have programs that do one job well, the local delivery job should be handed over to a specialist: the \NAME{MDA}. \NAME{MDA}s know about the various mailbox formats and are aware of the problems of concurrent write access and the like. Hence passing the message, and the responsibility for it, over to an \NAME{MDA} seems to be best.
    1.16 @@ -60,7 +60,7 @@
    1.17  	\label{fig:mta-channels}
    1.18  \end{figure}
    1.19  
    1.20 -An overview on in and outgoing channels required for an \MTA, gives figure \ref{fig:mta-channels}.
    1.21 +An overview on in and outgoing channels required for an \MTA, gives figure~\ref{fig:mta-channels}.
    1.22  
    1.23  %fixme: write about submission (port 587)
    1.24  
    1.25 @@ -94,7 +94,7 @@
    1.26  
    1.27  \paragraph{\RF5: Route management}
    1.28  \label{rf5}
    1.29 -One key feature of \masqmail\ is its ability to send mail out over different routes. The online state defines the active route to be used. A specific route may not be suited for all messages, thus these messages are hold back until a suiting route is active. For more information on this concept see section \ref{sec:masqmail-routes}.
    1.30 +One key feature of \masqmail\ is its ability to send mail out over different routes. The online state defines the active route to be used. A specific route may not be suited for all messages, thus these messages are hold back until a suiting route is active. For more information on this concept see section~\ref{sec:masqmail-routes}.
    1.31  
    1.32  
    1.33  
    1.34 @@ -131,7 +131,7 @@
    1.35  
    1.36  The common way to encrypt \SMTP\ dialogs is using \name{Transport Layer Security} (short: \TLS, the successor of \NAME{SSL}). \TLS\ encrypts the datagrams of the \name{transport layer}. This means it works below the application protocols and can be used with any of them \citeweb{wikipedia:tls}.
    1.37  
    1.38 -Using secure tunnels that are provided by external programs, should be preferred over including encryption into the application, because the application needs not to bother with encryption then. Outgoing \SMTP\ connections can get encrypted using a secure tunnel, created by an external application (like \name{openssl}). But incoming connections can not use external secure tunnels, because the remote \NAME{IP} address is hidden then; all connections would appear to come from localhost instead. Figure \ref{fig:stunnel} depicts the situation of using an application like \name{stunnel} for incoming connections. The connection to port 25 comes from localhost and this information reaches the \MTA. Authentication based on \NAME{IP} addresses and many spam prevention methods are useless then.
    1.39 +Using secure tunnels that are provided by external programs, should be preferred over including encryption into the application, because the application needs not to bother with encryption then. Outgoing \SMTP\ connections can get encrypted using a secure tunnel, created by an external application (like \name{openssl}). But incoming connections can not use external secure tunnels, because the remote \NAME{IP} address is hidden then; all connections would appear to come from localhost instead. Figure~\ref{fig:stunnel} depicts the situation of using an application like \name{stunnel} for incoming connections. The connection to port 25 comes from localhost and this information reaches the \MTA. Authentication based on \NAME{IP} addresses and many spam prevention methods are useless then.
    1.40  
    1.41  \begin{figure}
    1.42  	\begin{center}
    1.43 @@ -149,7 +149,7 @@
    1.44  
    1.45  \paragraph{\RF8: Spam handling}
    1.46  \label{rf8}
    1.47 -Spam is a major threat nowadays, but it is a war that is hard to win. The goal is to provide state-of-the-art spam protection, but not more (see section \ref{sec:swot-analysis}).
    1.48 +Spam is a major threat nowadays, but it is a war that is hard to win. The goal is to provide state-of-the-art spam protection, but not more (see section~\ref{sec:swot-analysis}).
    1.49  
    1.50  As spam is, by increasing the amount of mail messages, not just a nuisance for end users, but also for the infrastructure---the \MTA{}s---they need to protect themselves.
    1.51  
    1.52 @@ -171,7 +171,7 @@
    1.53  
    1.54  In any way should malware checking be performed by external programs that may be invoked by the \MTA. But \NAME{MDA}s are better points to invoke content scanners.
    1.55  
    1.56 -A popular email filter framework is \name{amavis} which integrates various spam and malware scanners. The common setup includes a receiving \MTA\ which sends it to \name{amavis} using \SMTP, \name{amavis} processes the mail and sends it then to a second \MTA\ that does the outgoing transfer. Having interfaces to such scanners is nice to have, though. (This setup with two \MTA\ instances is discussed in more detail in section \ref{sec:current-code-security}).
    1.57 +A popular email filter framework is \name{amavis} which integrates various spam and malware scanners. The common setup includes a receiving \MTA\ which sends it to \name{amavis} using \SMTP, \name{amavis} processes the mail and sends it then to a second \MTA\ that does the outgoing transfer. Having interfaces to such scanners is nice to have, though. (This setup with two \MTA\ instances is discussed in more detail in section~\ref{sec:current-code-security}).
    1.58  
    1.59  
    1.60  
    1.61 @@ -195,7 +195,7 @@
    1.62  
    1.63  
    1.64  \paragraph{\RG1: Security}
    1.65 -\MTA{}s are critical points for computer security, as they are accessible from external networks. They must be secured with high effort. Properties like the need for high privilege level, from outside influenced work load, work on unsafe data, and demand for reliability, increase the need for security. This is best done by modularization, also called \name{compartmentalization}, as described in section \ref{sec:discussion-mta-arch}.
    1.66 +\MTA{}s are critical points for computer security, as they are accessible from external networks. They must be secured with high effort. Properties like the need for high privilege level, from outside influenced work load, work on unsafe data, and demand for reliability, increase the need for security. This is best done by modularization, also called \name{compartmentalization}, as described in section~\ref{sec:discussion-mta-arch}.
    1.67  
    1.68  \masqmail\ needs to be secure enough for its target field of operation. \masqmail\ is targeted to workstations and private networks, with explicit warning to not use it on permanent online hosts \citeweb{masqmail:homepage2}. But as non-permanent online connections and trustable environments become rare, \masqmail's security should be so good, that it is usable with permanent online connections and in unsafe environments. For example should mails with bad content not break \masqmail.
    1.69  
    1.70 @@ -256,7 +256,7 @@
    1.71  \sendmail\ provides now, with its \name{milter} interface, standardized connection channels to external modules.
    1.72  \masqmail\ has none of them; it is what \sendmail\ was in the beginning: a single large block.
    1.73  
    1.74 -Figure \ref{fig:masqmail-arch} is a call graph generated from \masqmail's source code, excluding logging functions. It gives a impression of how interweaved the internals are. There are no compartments existent.
    1.75 +Figure~\ref{fig:masqmail-arch} is a call graph generated from \masqmail's source code, excluding logging functions. It gives a impression of how interweaved the internals are. There are no compartments existent.
    1.76  
    1.77  \begin{figure}
    1.78  	\begin{center}
    1.79 @@ -269,7 +269,7 @@
    1.80  	\label{fig:masqmail-arch}
    1.81  \end{figure}
    1.82  
    1.83 -\sendmail\ improved its old architecture by adding the milter interface, to include further functionality by invoking external programs. \exim\ was designed, and is carefully maintained, with a modular-like code structure in mind. \qmail\ started from scratch with a ``security-first'' approach, \postfix\ improved on it, and \name{sendmail X}/\name{MeTA1} tries to adopt the best of \qmail\ and \postfix\ to completely replace the old \sendmail\ architecture. \person{Hafiz} describes this evolution of \MTA\ architecture very well \cite{hafiz05}.
    1.84 +\sendmail\ improved its old architecture by adding the milter interface, to include further functionality by invoking external programs. \exim\ was designed, and is carefully maintained, with a modular-like code structure in mind. \qmail\ started from scratch with a ``security-first'' approach, \postfix\ improved on it, and \name{sendmail~X}/\name{MeTA1} tries to adopt the best of \qmail\ and \postfix\ to completely replace the old \sendmail\ architecture. \person{Hafiz} describes this evolution of \MTA\ architecture very well \cite{hafiz05}.
    1.85  
    1.86  Every one of these programs is more modular, or became more modular over time, than \masqmail\ is. Modern requirements like spam protection and future requirements like---probably---the use of new mail transport protocols demand for modular designs in order to keep the software simple. Simplicity is a key property for security. ``the essence of security engineering is to build systems that are as simple as possible.'' \cite[page 45]{graff03}.
    1.87  
    1.88 @@ -303,7 +303,7 @@
    1.89  
    1.90  
    1.91  \paragraph{\RF1: In/out channels}
    1.92 -The incoming and outgoing channels that \masqmail\ already has (depicted in figure \ref{fig:masqmail-channels} on page \pageref{fig:masqmail-channels}) are the ones required for an \MTA{}s at the moment. Support for other protocols seems not to be necessary at the moment, although new protocols and mailing concepts are likely to appear (see section \ref{sec:email-trends}). Today, other protocols are not needed, so \masqmail\ is regarded to fulfill \RF1. But as \masqmail\ has no support for adding further protocols, delaying the work to support them until they are widely used, appears to be the best strategy anyway.
    1.93 +The incoming and outgoing channels that \masqmail\ already has (depicted in figure~\ref{fig:masqmail-channels} on page \pageref{fig:masqmail-channels}) are the ones required for an \MTA{}s at the moment. Support for other protocols seems not to be necessary at the moment, although new protocols and mailing concepts are likely to appear (see section~\ref{sec:email-trends}). Today, other protocols are not needed, so \masqmail\ is regarded to fulfill \RF1. But as \masqmail\ has no support for adding further protocols, delaying the work to support them until they are widely used, appears to be the best strategy anyway.
    1.94  
    1.95  %fixme: << smtp submission >> %fixme
    1.96  
    1.97 @@ -323,7 +323,7 @@
    1.98  Static authentication, based on \NAME{IP} addresses, can be achieved with \person{Venema}'s \NAME{TCP} \name{Wrapper} \cite{venema92}, by editing the \path{hosts.allow} and \path{hosts.deny} files. This is only relevant to authenticate host that try to submit mail into the system. Dynamic (secret-based) \SMTP\ authentication is already supported in form of \NAME{SMTP-AUTH} and \SMTP-after-\NAME{POP}, but only for outgoing connections. For incoming connections, only address-based authentication is supported.
    1.99  
   1.100  \paragraph{\RF7: Encryption}
   1.101 -Similar is the situation for encryption which is also only available for outgoing channels; here a wrapper application like \name{openssl} is needed. This creates a secure tunnel to send mail trough, but state-of-the-art is using \NAME{STARTTLS}, which is not supported. For incoming channels, no encryption is available. The only possible setup to provide encryption of incoming channels is using an application like \name{stunnel} to translate between the secure connection to the remote host and the \MTA. Unfortunately, this suffers from the problem explained on page \pageref{fig:stunnel} in figure \ref{fig:stunnel}. Anyway, this would still be no \NAME{STARTTLS} support.
   1.102 +Similar is the situation for encryption which is also only available for outgoing channels; here a wrapper application like \name{openssl} is needed. This creates a secure tunnel to send mail trough, but state-of-the-art is using \NAME{STARTTLS}, which is not supported. For incoming channels, no encryption is available. The only possible setup to provide encryption of incoming channels is using an application like \name{stunnel} to translate between the secure connection to the remote host and the \MTA. Unfortunately, this suffers from the problem explained on page \pageref{fig:stunnel} in figure~\ref{fig:stunnel}. Anyway, this would still be no \NAME{STARTTLS} support.
   1.103  
   1.104  \paragraph{\RF8: Spam handling}
   1.105  \masqmail\ nowadays does not provide special support for spam filtering. Spam prevention by not accepting spam during the \SMTP\ dialog is not possible at all. Spam filtering is only possible by using two \masqmail\ instances with an external spam filter in between. The mail flow is from the receiving \MTA\ instance, which accepts mail, to the filter application that processes and possible modifies it, to the second \MTA\ which is responsible for further delivery of the mail. This is a concept that works in general. And it is a good concept in principle to separate work with clear interfaces. But the need of two instances of the same \MTA (each for only half of the job) with doubled setup, is more a work-around. Best is to have this data flow respected in the \MTA\ design, like in \postfix. But the more important part of spam handling, for sure, is done during the \SMTP\ dialog in completely refusing unwanted mail.
   1.106 @@ -389,7 +389,7 @@
   1.107  
   1.108  \section{Work to do}
   1.109  
   1.110 -After the requirements for modern \MTA{}s were identified in section \ref{sec:mta-requirements} and \masqmail's features were set against them in section \ref{sec:fulfilled-requirements}, here the work that is left to do is identified. Table \ref{tab:requirements} lists all requirements with importance and the work needed to achieve them. The column ``Focus'' shows the attention a work task should get. The focus depends on the task's importance and the amount of work it includes.
   1.111 +After the requirements for modern \MTA{}s were identified in section~\ref{sec:mta-requirements} and \masqmail's features were set against them in section~\ref{sec:fulfilled-requirements}, here the work that is left to do is identified. Table~\ref{tab:requirements} lists all requirements with importance and the work needed to achieve them. The column ``Focus'' shows the attention a work task should get. The focus depends on the task's importance and the amount of work it includes.
   1.112  
   1.113  \begin{table}
   1.114  	\begin{center}
   1.115 @@ -415,7 +415,7 @@
   1.116  
   1.117  
   1.118  \subsubsection*{\TODO3: Security (\RG1)}
   1.119 -\masqmail's security is bad, thus the program is forced into a limited field of operation. This field of operation even shrinks as security becomes more important and networking and interaction increases. Save and trusted environment become rare. Thus improving security is an important thing to do. The focus should be on adding compartments to split \masqmail\ into separate modules. (See section \ref{sec:discussion-mta-arch}.) Further more should \masqmail's security be tested throughout to get a definitive view how good it really is and where the weak spots are.
   1.120 +\masqmail's security is bad, thus the program is forced into a limited field of operation. This field of operation even shrinks as security becomes more important and networking and interaction increases. Save and trusted environment become rare. Thus improving security is an important thing to do. The focus should be on adding compartments to split \masqmail\ into separate modules. (See section~\ref{sec:discussion-mta-arch}.) Further more should \masqmail's security be tested throughout to get a definitive view how good it really is and where the weak spots are.
   1.121  
   1.122  
   1.123  \subsubsection*{\TODO4: Reliability (\RG2)}
   1.124 @@ -457,7 +457,7 @@
   1.125  
   1.126  The requirements are now regarded each on its own, and are linked to the development strategy that is preferred to reach each specific requirement. If some requirement is well achievable by using different strategies then it is linked to all of them. Implementing encryption (\TODO1) and authentication (\TODO2), for example, are limited to a narrow region in the code. Such features are addable to the current code base without much problem. In contrast can quality properties like reliability (\TODO4), extendability (\TODO6), and maintainability hardly be added to code afterwards---if at all. Security (\TODO3) is addable in a new design, of course, but also with wrappers or interposition filters.
   1.127  
   1.128 -This linking of strategies to the requirements is shown in table \ref{tab:strategies}. The requirements are ordered by their focus.
   1.129 +This linking of strategies to the requirements is shown in table~\ref{tab:strategies}. The requirements are ordered by their focus.
   1.130  
   1.131  \begin{table}
   1.132  	\begin{center}
   1.133 @@ -487,7 +487,7 @@
   1.134  
   1.135  \subsubsection*{Quality improvements}
   1.136  
   1.137 -Most quality properties can hardly be added to a software afterwards. Hence, if reliability, extendability, or maintainability shall be improved, a redesign of \masqmail\ is the best way to take. The wish to improve quality inevitably point towards a modular architecture. Modularity with internal and external interfaces is highly preferred from the architectural point of view (see section \ref{sec:discussion-mta-arch}). The need for further features, especially ones that require changes in \masqmail's structure, support the decision for a new design too. Hence a rewrite is favored if \masqmail\ should become a modern \MTA, with good quality properties.
   1.138 +Most quality properties can hardly be added to a software afterwards. Hence, if reliability, extendability, or maintainability shall be improved, a redesign of \masqmail\ is the best way to take. The wish to improve quality inevitably point towards a modular architecture. Modularity with internal and external interfaces is highly preferred from the architectural point of view (see section~\ref{sec:discussion-mta-arch}). The need for further features, especially ones that require changes in \masqmail's structure, support the decision for a new design too. Hence a rewrite is favored if \masqmail\ should become a modern \MTA, with good quality properties.
   1.139  
   1.140  
   1.141  
   1.142 @@ -552,7 +552,7 @@
   1.143  
   1.144  Besides these advantages of existing code, one must not forget that further work on it is often repair work. Small bug fixes are not the problem, but adding something for which the software originally was not designed for are problems. Such work often destroys the clear concepts of the software, especially in interweaved monolithic code.
   1.145  
   1.146 -Repair strategies are useful, but only in the short-time view and in times of trouble. If the future is bright, however, one does best by investing into a software. As shown in section \ref{sec:market-analysis-conclusion}, the future for \MTA{}s is bright. This means it is time to invest into a redesigning to build up a more modern product.
   1.147 +Repair strategies are useful, but only in the short-time view and in times of trouble. If the future is bright, however, one does best by investing into a software. As shown in section~\ref{sec:market-analysis-conclusion}, the future for \MTA{}s is bright. This means it is time to invest into a redesigning to build up a more modern product.
   1.148  
   1.149  In the author's view is \masqmail\ already needing a redesign since about 2003 when the old design was still quite suitable \dots\ it already delayed too long.
   1.150  
   1.151 @@ -568,7 +568,7 @@
   1.152  
   1.153  A new design does protect against such dead ends.
   1.154  
   1.155 -Changing requirements are one possible dead end if the software does not evolve with them. A famous example is \sendmail, which had an almost monopoly for a long time. But when security became important \sendmail\ was only repaired instead of removing the problem sources---its insecure design. Thus security problems reappeared and over the years \sendmail's market share shrank as more secure \MTA{}s became available. \sendmail's reaction to the new requirements, in form of \name{sendmail X} and \name{MeTA1}, came much to late---the users already switched to other \MTA{}s. Redesigning a software as requirements change helps keeping it alive. % fixme: add quote: ``one thing surely remains: change'' (something like that)
   1.156 +Changing requirements are one possible dead end if the software does not evolve with them. A famous example is \sendmail, which had an almost monopoly for a long time. But when security became important \sendmail\ was only repaired instead of removing the problem sources---its insecure design. Thus security problems reappeared and over the years \sendmail's market share shrank as more secure \MTA{}s became available. \sendmail's reaction to the new requirements, in form of \name{sendmail~X} and \name{MeTA1}, came much to late---the users already switched to other \MTA{}s. Redesigning a software as requirements change helps keeping it alive. % fixme: add quote: ``one thing surely remains: change'' (something like that)
   1.157  
   1.158  Another danger is the dead end of complexity which is likely to appear by constantly working on the same code base. It is even more likely if the code base has a monolithic architecture. A good example for simplicity is \qmail\ which consists of small independent modules, each with only about one thousand lines of code. Such simple code makes it obvious to understand what it does. The \name{suckless} project \citeweb{suckless.org} for example advertises such a philosophy of small and simple software by following the thoughts of the \unix\ inventors \cite{kernighan84} \cite{kernighan99}. Simple, small, and clear code avoids complexity and is thus also a strong prerequisite for security.
   1.159  
   1.160 @@ -687,5 +687,5 @@
   1.161  
   1.162  %A program's structure is primary its architecture. Which is the most influencing design decision, and has the greatest impact on the program's future capabilities. The architecture defines what the program can do, and how it can be used. If the architecture does not fit to the requirements, development will reach a dead end \dots\ further work then will make everything worse. The only good solution then is to change the architecture, which, sadly but most likely, means a redesign from scratch.
   1.163  
   1.164 -%This plan is similar to the change from \sendmail\ to \name{sendmail X}/\name{MeTA1}, except the \sendmail\ change was much too late.
   1.165 +%This plan is similar to the change from \sendmail\ to \name{sendmail~X}/\name{MeTA1}, except the \sendmail\ change was much too late.
   1.166