1.1 --- a/thesis/tex/4-MasqmailsFuture.tex	Thu Dec 18 11:32:31 2008 +0100
     1.2 +++ b/thesis/tex/4-MasqmailsFuture.tex	Thu Dec 18 13:39:23 2008 +0100
     1.3 @@ -74,7 +74,7 @@
     1.4  
     1.5  \section{Discussion on architecture}
     1.6  
     1.7 -A program's architecture is maybe the most influencing design decision, and has the greatest impact on the program's future capabilities. %fixme: search quote ... check if good
     1.8 +A program's architecture is probably the most influencing design decision, and has the greatest impact on the program's future capabilities. %fixme: search quote ... check if good
     1.9  
    1.10  \masqmail's current artitecture is monolitic like \sendmail's and \exim's. But more than the other two, is it one block of interweaved code. \sendmail\ provides now, with its \name{milter} interface, standardized connection channels to external modules. \exim\ has a highly structured code with many internal interfaces, like the one for supported authentication ``modules''. \masqmail\ has none of them; it is what \sendmail\ was in the beginning: a single large block.
    1.11  
    1.12 @@ -88,19 +88,139 @@
    1.13  	\label{fig:masqmail-arch}
    1.14  \end{figure}
    1.15  
    1.16 -\sendmail\ improved its old architecture, for example by adding the milter interface. \exim\ was designed and is carefully maintained with a modular-like code structure in mind. \qmail\ started from scratch with a security-first approach, \postfix\ improved on it, and \name{sendmail X}/\name{MeTA1} tries to adopt the best of \qmail\ and \postfix, to completely replace the old \sendmail\ architecture. \person{Hafiz} \cite{hafiz05}. describes this evolution of \mta\ architecture very well.
    1.17 +\sendmail\ improved its old architecture, for example by adding the milter interface. \exim\ was designed and is carefully maintained with a modular-like code structure in mind. \qmail\ started from scratch with a ``security-first'' approach, \postfix\ improved on it, and \name{sendmail X}/\name{MeTA1} tries to adopt the best of \qmail\ and \postfix, to completely replace the old \sendmail\ architecture. \person{Hafiz} \cite{hafiz05}. describes this evolution of \mta\ architecture very well.
    1.18  
    1.19 -Every one of the popular \MTA{}s is more modular, or became more modular, than \masqmail. The logical step is to rewrite \masqmail\ using a modern, modular architecture to get a modern \MTA\ satisfying nowadays needs. But how is the effort of this complete rewrite compared to what is gained afterwards?
    1.20 +Every one of the popular \MTA{}s is more modular, or became more modular over time, than \masqmail\ is. Modern requirements like spam protection and future requirements like the use of new mail transport protocols demand modular designs for keeping the software simple. Simplicity is a key property for security.
    1.21  
    1.22 +\person{Hafiz} agrees:
    1.23 +\begin{quote}
    1.24 +The goal of making software secure can be better achieved by making the design simple and easier to understand and verify. \cite[page64]{hafiz05}
    1.25 +\end{quote}
    1.26 +He identifies the security of \qmail\ to come from it's \name{compartmentalization}, which goes hand in hand with modularity:
    1.27 +\begin{quote}
    1.28 +A perfect example is the contrast between the feature envy early \sendmail\ architecture implemented as one process and the simple, modular architecture of \qmail. The security of \qmail\ comes from its compartmentalized simple processes that perform one task only and are therefor testable for security. \cite[page 64]{hafiz05}
    1.29 +\end{quote}
    1.30  
    1.31 +Modularity is needed for supporting modern \MTA\ requirements, providing a clear interface to add further functionality without increasing the overall complexity much. Modularity is also an enabler for security. Security comes from good design, as \person{Graff} and \person{van Wyk} explain:
    1.32 +\begin{quote}
    1.33 +Good design is the sword and shield of the security-conscious developer. Sound design defends your application from subversion or misuse, protecting your network and the information on it from internal and external attacks alike. It also provides a safe foundation for future extensions and maintainance of the software.
    1.34 +%
    1.35 +%Bad design makes life easier for attackers and harder for the good guys, especially if it contributes to a false sends of security while obscuring pertinent failings.
    1.36 +\cite[page 55]{graff03}
    1.37 +\end{quote}
    1.38  
    1.39 +\person{Hafiz} adds: ``The major idea is that security cannot be retrofitted into an architecture.''\cite[page 64]{hafiz05}
    1.40  
    1.41 -A secure architecture is of need.
    1.42 +All this leads to one logical step: The rewrite of \masqmail\ using a modern, modular architecture, to get a modern \MTA\ satisfying nowadays needs.
    1.43  
    1.44  
    1.45  
    1.46  
    1.47 +\subsection{Modules needed}
    1.48  
    1.49 +This section tries to identify the needed modules for a modern \MTA. They are later the pieces of which the new architecture is built of.
    1.50 +
    1.51 +
    1.52 +\subsubsection*{The simplest MTA}
    1.53 +This view of the problem is taken from \person{Hafiz} \cite[pages 3-5]{hafiz05}.
    1.54 +
    1.55 +The basic job of a \mta\ is to tranport mail from a sender to a recipient. The simplest \MTA\ therefor needs at least a mail receiving facility and a mail sending facility. This basic \MTA---following the definition of an \MTA---is much to abstract. Hence a next step to add some important features is needed, the result is an operational \MTA.
    1.56 +
    1.57 +
    1.58 +
    1.59 +\subsubsection*{Mail queue}
    1.60 +
    1.61 +\person{Hafif} adds a mail queue to make it possible to not deliver at once.
    1.62 +
    1.63 +Mail queues are probably used in all \mta{}s, excluding the simple forwarders. A mail queue is a essential requirement for \masqmail, as it is to be used for non-permanent online connections.
    1.64 +
    1.65 +
    1.66 +\subsubsection*{Incoming channels}
    1.67 +
    1.68 +The second addition \person{Hafiz} made is the split of incoming and outgoing channels into local and remote. The question is, if this is nessesary. It is the way, it was done for a long time, but is this extra complexity needed?
    1.69 +
    1.70 +The common situation is incoming mail on port 25 using \SMTP\ and via the \texttt{sendmail} command. Outgoing mail is either sent using \SMTP, piped into local commands (for example \texttt{uucp}), or delivered locally by appending to a mailbox.
    1.71 +
    1.72 +The \MTA's architecture would be simpler if some of these channels could be merged. The reason is, if various modules do similar jobs, common things might need to be duplicated. On the other side is it better to have more independent modules if each one is simpler then.
    1.73 +
    1.74 +\qmail\ uses \name{qmail-inject} (local message in) and \name{qmail-smtpd} (remote message in), which both handle messages over to \name{qmail-queue} that puts it into the mail queue. \postfix's approach is similar. \name{sendmail X} %fixme: what about meta1 here?
    1.75 +used only \NAME{SMTPS}, which is for receiving mail from remote, to communicate with the queue manager \NAME{QMGR}. Mail from local goes over \NAME{SMTPS}.
    1.76 +
    1.77 +The \name{sendmail X} approach seems to be the simpler one, but does heavily rely on \SMTP\ being the main mail transfer protocol. To \qmail\ and \postfix\ new modules may be added to support other ways of message receival, without any change of other parts of the system.
    1.78 +
    1.79 +
    1.80 +\subsubsection*{Outgoing channels}
    1.81 +
    1.82 +Outgoing channels are similar for \qmail, \postfix, and \name{sendmail X}: All of them have a module to send mail using \SMTP, and one for writing into a local mailbox. Local mail delivery is a job that requires root priveledge to be able to switch to any user in order to write to his mailbox. Modular \MTA{}s do not need \name{setuid root}, but the local delivery process (or its parent) needs to run as root.
    1.83 +
    1.84 +As mail delivery to local users, is \emph{not} included in the basic job of \MTA{}s, why should they care about it? In order to keep the system simple and to have programs do one job well, the local delivery job should be handed over to \NAME{MDA}s. \name{Mail delivery agents} are the tools that are specialized for local delivery. They know about the various mailbox formats and are aware of the problems of concurrent write access and thelike. Hence handling the message and the responsiblity for it over to a mail delivery agent, like \name{procmail} or \name{maildrop}, seems to be the right way to go.
    1.85 +
    1.86 +This means outgoing connections, piping mails into local commands needs to be implemented.
    1.87 +
    1.88 +
    1.89 +\subsubsection*{Mail queue (again)}
    1.90 +
    1.91 +
    1.92 +
    1.93 +
    1.94 +\subsubsection*{Authentication}
    1.95 +
    1.96 +easiest: restricting by static IP addresses (Access control via hosts.allow/hosts.deny)
    1.97 +if dynamic remote hosts need access: some auth is needed
    1.98 +- SASL
    1.99 +- POP/IMAP: pop-before-smtp, DRAC, WHOSON
   1.100 +- TLS (certificates)
   1.101 +
   1.102 +``None of these add-ons is an ideal solution. They require additional code compiled into your existing daemons that may then require special write accesss to system files. They also require additional work for busy system administrators. If you cannot use any of the nonauthenticating alternatives mentioned earlier, or your business requirements demand that all of thyour users' mail pass through your system no matter where they are on the Internet, SASL is probably the solution that offers the most reliable and scalable method to authenticate users.'' (Dent: Postfix, page 44, ch04)
   1.103 +
   1.104 +
   1.105 +\subsubsection*{Encryption}
   1.106 +
   1.107 +
   1.108 +\subsubsection*{Spam prevention}
   1.109 +
   1.110 +
   1.111 +where to filter what
   1.112 +
   1.113 +
   1.114 +postfix: after-queue-content-filter (smtp communication)
   1.115 +exim: content-scan-feature
   1.116 +sendmail: milter (tcp or unix sockets)
   1.117 +
   1.118 +checks while smtp dialog (pre-queue): in MTA implemented (need to be fast)
   1.119 +checks when mail is accepted and queued: external (amavis, spamassassin)
   1.120 +
   1.121 +
   1.122 +AMaViS (amavisd-new): email filter framework to integrate spam and virus scanner
   1.123 +internet -->25 MTA -->10024 amavis -->10025 MTA --> reciptient
   1.124 +                |                            |
   1.125 +                +----------------------------+
   1.126 +mail scanner:
   1.127 +incoming queue --> mail scanner --> outgoing queue
   1.128 +
   1.129 +mimedefang: uses milter interface with sendmail
   1.130 +
   1.131 +
   1.132 +\subsubsection*{Virus checking}
   1.133 +
   1.134 +The same for malicious content (\name{malware}) like viruses, worms, trojan horses. They are related to spam, but affect the \MTA less, as they are in the mail body.
   1.135 +
   1.136 +message body <-> envelope, header
   1.137 +
   1.138 +
   1.139 +anti-virus: clamav
   1.140 +
   1.141 +
   1.142 +
   1.143 +
   1.144 +
   1.145 +\subsubsection*{Archiving}
   1.146 +
   1.147 +
   1.148 +
   1.149 +
   1.150 +
   1.151 +\section{A new architecture}
   1.152  
   1.153  
   1.154  (ssl)
   1.155 @@ -110,13 +230,8 @@
   1.156  -> msg-out (local-delivery by MDA, or remote-protocol-handlers)
   1.157  (ssl)
   1.158  
   1.159 -A design from scratch?
   1.160  
   1.161 -<< what would be needed (effort) >>
   1.162  
   1.163 -<< would one create it at all? >>
   1.164 -
   1.165 -<< should it be done? >>
   1.166  
   1.167  
   1.168  http://fanf.livejournal.com/50917.html %how not to design an mta - the sendmail command
   1.169 @@ -130,56 +245,17 @@
   1.170  http://fanf.livejournal.com/72258.html %how not to design an mta - content scanning
   1.171  
   1.172  
   1.173 -\subsubsection*{local mail delivery}
   1.174 -But for example delivery of mail to local users is \emph{not} what \mta{}s should care about, although most \MTA\ are able to deliver mail, and many do. (\name{mail delivery agents}, like \name{procmail} and \name{maildrop}, are the right programs for this job.)
   1.175  
   1.176  
   1.177  
   1.178  
   1.179  
   1.180 -\subsection{Access and Auth}
   1.181  
   1.182 -easiest: restricting by static IP addresses (Access control via hosts.allow/hosts.deny)
   1.183 -if dynamic remote hosts need access: some auth is needed
   1.184 -- SASL
   1.185 -- POP/IMAP: pop-before-smtp, DRAC, WHOSON
   1.186 -- TLS (certificates)
   1.187  
   1.188 -``None of these add-ons is an ideal solution. They require additional code compiled into your existing daemons that may then require special write accesss to system files. They also require additional work for busy system administrators. If you cannot use any of the nonauthenticating alternatives mentioned earlier, or your business requirements demand that all of thyour users' mail pass through your system no matter where they are on the Internet, SASL is probably the solution that offers the most reliable and scalable method to authenticate users.'' (Dent: Postfix, page 44, ch04)
   1.189  
   1.190  
   1.191  
   1.192 -postfix: after-queue-content-filter (smtp communication)
   1.193 -exim: content-scan-feature
   1.194 -sendmail: milter (tcp or unix sockets)
   1.195  
   1.196 -checks while smtp dialog (pre-queue): in MTA implemented (need to be fast)
   1.197 -checks when mail is accepted and queued: external (amavis, spamassassin)
   1.198 -
   1.199 -anti-virus: clamav
   1.200 -
   1.201 -AMaViS (amavisd-new): email filter framework to integrate spam and virus scanner
   1.202 -internet -->25 MTA -->10024 amavis -->10025 MTA --> reciptient
   1.203 -                |                            |
   1.204 -                +----------------------------+
   1.205 -mail scanner:
   1.206 -incoming queue --> mail scanner --> outgoing queue
   1.207 -
   1.208 -mimedefang: uses milter interface with sendmail
   1.209 -
   1.210 -
   1.211 -
   1.212 -
   1.213 -
   1.214 -
   1.215 -
   1.216 -\subsection{spam and malicious content}
   1.217 -
   1.218 -The same for malicious content (\name{malware}) like viruses, worms, trojan horses. They are related to spam, but affect the \MTA less, as they are in the mail body.
   1.219 -
   1.220 -message body <-> envelope, header
   1.221 -
   1.222 -where to filter what
   1.223  
   1.224  
   1.225  
   1.226 @@ -199,6 +275,16 @@
   1.227  
   1.228  Now how could \masqmail\ be like in, say, five years?
   1.229  
   1.230 +---
   1.231 +
   1.232 +A design from scratch?
   1.233 +<< what would be needed (effort) >>
   1.234 +But how is the effort of this complete rewrite compared to what is gained afterwards?
   1.235 +
   1.236 +<< would one create it at all? >>
   1.237 +
   1.238 +---
   1.239 +
   1.240  << plans to get masqmail more popular again (if that is the goal) >>
   1.241  
   1.242  << More users >>
   1.243 @@ -206,9 +292,14 @@
   1.244  
   1.245  
   1.246  
   1.247 +
   1.248 +
   1.249 +
   1.250  \section{Work to do}
   1.251  
   1.252  << short term goals --- long term goals >>
   1.253  
   1.254 +do it like sendmail: first do the most needed stuff on the old design to make it still usable. Then design a new version from scratch, for the future.
   1.255 +
   1.256  << which parts to take out and do within the thesis >>
   1.257