1.1 --- a/thesis/tex/5-Improvements.tex	Fri Jan 23 12:58:52 2009 +0100
     1.2 +++ b/thesis/tex/5-Improvements.tex	Fri Jan 23 18:40:12 2009 +0100
     1.3 @@ -156,208 +156,157 @@
     1.4  \section{A new design}
     1.5  \label{sec:new-design}
     1.6  
     1.7 -The last chapter identified the requirements for a modern and securt \masqmail. Now the various jobs of an \MTA\ get assigned to modules, of which the new architecture is created. It is inspired by existing \MTA{}s and driven by the identified requirements.
     1.8 +The last chapter identified the requirements for a modern and secure \masqmail. Now the various jobs of an \MTA\ get assigned to modules of which a new architecture is created. It is inspired by existing \MTA{}s and driven by the identified requirements.
     1.9  
    1.10  One wise experience was kept in mind during the design: ``Many times in life, getting off to the right start makes all the difference.'' \cite[page~32]{graff03}.
    1.11  
    1.12  
    1.13 +
    1.14 +\subsection{Design decisions}
    1.15 +
    1.16  Major design ideas of the design were:
    1.17 -\begin{itemize}
    1.18 -	\item free the internal system from in and out channels
    1.19 -	\item arbitrary protocol handlers have to be addable afterwards
    1.20 -	\item a single facility for scanning (all mail goes through it)
    1.21 -	\item concentrate on mail transfer
    1.22 -\end{itemize}
    1.23 +\begin{enumerate}
    1.24 +	\item compartmentalization throughout
    1.25 +	\item free the internal system from the in and out channels
    1.26 +	\item provide interfaces to add arbitrary protocol handlers afterwards
    1.27 +	\item have a single point where all mail goes through for scanning
    1.28 +	\item concentrate on the mail transfer job; use specialized external programs for other jobs
    1.29 +	\item keep it simple, clear, and general
    1.30 +\end{enumerate}
    1.31  
    1.32  
    1.33  
    1.34 -\subsection{Architectural design}
    1.35 -
    1.36  \subsubsection*{Incoming channels}
    1.37  
    1.38 -\sendmail-compatible \mta{}s must support at least two incoming channels: mail submitted using the \sendmail\ command, and mail received via the \SMTP\ daemon. It is therefor common to split the incoming channel into local and remote. This is done by \qmail\ and \postfix. The same way is \person{Hafiz}'s view.
    1.39 +\sendmail-compatible \mta{}s must support at least two incoming channels: mail submitted using the \sendmail\ command, and mail received via the \SMTP\ daemon. It is therefore common to split the incoming channel into local and remote. This is done by \qmail\ and \postfix. The same way is \person{Hafiz}'s view \cite{hafiz05}. %fixme: specify page
    1.40  
    1.41 -In contrast is \name{sendmail X}: Its locally submitted messages go to the \SMTP\ daemon, which is the only connection towards the mail queue. %fixme: is it a smtp dialog? or a second door?
    1.42 -\person{Finch} proposes a similar approach. He wants the \texttt{sendmail} command to be a simple \SMTP\ client that contacts the \SMTP\ daemon of the \MTA\ like it is done by connections from remote. The advantage here is one single module where all \SMTP\ dialog with submitters is done. Hence one single point to accept or refuse incoming mail. Additionally does the module to put mail into the queue not need to be \name{setuid} or \name{setgid} because it is only invoked from the \SMTP\ daemon. The \MTA's architecture would become simpler and common tasks are not duplicated in modules that do similar jobs.
    1.43 +In contrast is \name{sendmail X}: Its locally submitted messages go to the \SMTP\ daemon, which is the only connection towards the mail queue. %fixme: is it a smtp dialog? or a back door?
    1.44 +\person{Finch} proposes a similar approach. He wants the \texttt{sendmail} command to be a simple \SMTP\ client that contacts the \SMTP\ daemon of the \MTA\ like it is done by connections from remote. The advantage here is one single module where all \SMTP\ dialog with submitters is done. Hence one single point to accept or refuse incoming mail. Additionally does the module which puts mail into the queue not need to be \name{setuid} or \name{setgid} because it is only invoked from the \SMTP\ daemon. The \MTA's architecture would become simpler and common tasks are not duplicated in modules that do similar jobs.
    1.45  
    1.46 -But merging the input channels in the \SMTP\ daemon makes the \MTA\ heavily dependent on \SMTP\ being the main mail transfer protocol. To \qmail\ and \postfix\ new modules to support other ways of message receival may be added without change of other parts of the system. Also is it better to have more independent modules if each one is simpler then.
    1.47 +But merging the input channels in the \SMTP\ daemon makes the \MTA\ heavily dependent on \SMTP. To \qmail\ and \postfix\ new modules to support other ways of message receival may be added without change of other parts of the system. Also the \SMTP\ modules can be removed if it is not needed. And it is better to have more independent modules if each one is simpler then---it makes the modules more complicated if each one needs to implement an \SMTP\ client.
    1.48  
    1.49 -With the increasing need for new protocols in mind, it seems better to have single modules for each incoming channel, although this leads to duplicated acceptance checks.
    1.50 +With the increasing need for new protocols in mind, it seems better to have single modules for each incoming channel, although this leads to duplicated acceptance checks. Independent checks in different modules, however, have also the advantage to simply apply different policies. Thus it is possible to run two \SMTP\ modules that listen on different ports; one accessable from the Internet but requires authentication, the other only accessable from the local network but does not require authentication.
    1.51 +
    1.52 +The approach of simple independent modules, one for each incoming channel, should be taken.
    1.53 +
    1.54 +A module which is a \NAME{POP} or \NAME{IMAP} client to import contents of other mail boxes into the system may be added afterwards as it is desired.
    1.55 +
    1.56  
    1.57  
    1.58  \subsubsection*{Outgoing channels}
    1.59  
    1.60 -Outgoing mail is commonly either sent using \SMTP, piped into local commands (for example \texttt{uucp}), or delivered locally by appending to a mailbox.
    1.61 +Outgoing mail is commonly either sent using \SMTP, piped into local commands (for example \path{uucp}), or delivered locally by appending to a mailbox.
    1.62  
    1.63 -Outgoing channels are similar for \qmail, \postfix, and \name{sendmail X}: All of them have a module to send mail using \SMTP, and one for writing into a local mailbox. Local mail delivery is a job that requires root priveledge to be able to switch to any user in order to write to his mailbox. Modular \MTA{}s do not need \name{setuid root}, but the local delivery process (or its parent) needs to run as root.
    1.64 +Outgoing channels are similar for \qmail, \postfix, and \name{sendmail X}: All of them have a module to send mail using \SMTP, and one for writing into a local mailbox. Local mail delivery is a job that requires root privilege to be able to switch to any user in order to write to his mailbox. Modular \MTA{}s do not need \name{setuid root}, but the local delivery process (or its parent) needs to run as root\footnote{root privilege is actually not a mandatory requirement, but any other approach has some disadvantages, so commonly root privilege is used.}.
    1.65  
    1.66 -As mail delivery to local users, is \emph{not} included in the basic job of an \MTA{}, why should it care about it? In order to keep the system simple and to have programs that do one job well, the local delivery job should be handed over to a specialist: the \name{mail delivery agent}. \NAME{MDA}s know about the various mailbox formats and are aware of the problems of concurrent write access and thelike. Hence handling the message and the responsiblity over to a \NAME{MDA}, like \name{procmail} or \name{maildrop}, seems to be the right way to go.
    1.67 +Local mail delivery should not be done by the \MTA, but by an \NAME{MDA}. This decision was discussed in section \ref{sec:functional-requirements}. This means only an outgoing channel that pipes mail into a local command is required for local delivery.
    1.68  
    1.69 -This means an outgoing connection that pipes mail into local commands is required. Other outgoing channels, one for each supportet protocol, may be designed like it was done in other \MTA{}s.
    1.70 +Other outgoing channels, one for each supportet protocol, may be designed like it was done in other \MTA{}s.
    1.71  
    1.72  
    1.73  
    1.74 -\subsubsection*{Mail queue}
    1.75 +\subsubsection*{The mail queue}
    1.76  
    1.77 -Mail queues are probably used in all \mta{}s, excluding the simple forwarders. A mail queue is a essential requirement for \masqmail, as it is to be used for non-permanent online connections. This means, mail must be queued until a online connection is available to send the message.
    1.78 +The mail queue is the central part of an \MTA. This demands especially for robustness and reliability as a failure here can lead to loosing mail.
    1.79  
    1.80 -The mail queue and the module to manage it are the central part of the whole system. This demands especially for robustness and reliability, as a failure here can lead to loosing mail. An \MTA\ takes over responsibility for mail in accepting it, hence loosing mail messages is absolutely to avoid. This covers any kind of crash situation too. The worst thing acceptable to happen is a mail to be sent twice.
    1.81 +%\sendmail, \exim, \qmail, \name{sendmail X}, and \masqmail\ feature one single mail queue. \postfix\ has more of them.
    1.82 +Common \MTA{}s feature one or more mail queues, they sometimes have effectly several queues within one physical representation.
    1.83  
    1.84 -\sendmail, \exim, \qmail, \name{sendmail X}, and \masqmail\ feature one single mail queue. \postfix\ has more of them.
    1.85 +\MTA\ setups that include content scanning tend to require two separate queues. To use \sendmail\ in such setups requires two independent instances with two separate queues. \exim\ can handle it with special \name{router} and \name{transport} rules but the data flow gets complicated. Hence an idea is to use two queues, \name{incoming} and \name{active} in \postfix's terminology, with the content scanning within the move from \name{incoming} to \name{active}.
    1.86  
    1.87 -\MTA\ setups that include content scanning tend to require two separate queues. To use \sendmail\ in such setups requires two independent instances, with two separate queues, running. \exim\ can handle it with special \name{router} and \name{transport} rules, but the data flow gets complicated. Hence an idea is to use two queues, \name{incoming} and \name{active} in \postfix's terminology, with the content scanning within the move from \name{incoming} to \name{active}.
    1.88 +\sendmail, \exim, \qmail, and \masqmail\ all use at least two files to store one message in the queue: one file contains the message body, another the envelope and header information. The one containing the mail body is not modified at all. \postfix\ takes a different approach in storing queued messages in an internal format within one file. \person{Finch} suggest yet another approach: storing the whole queue in one single file with pointers to separating positions \cite{finchFIXME}. %fixme: check, cite, and think about
    1.89  
    1.90 -\sendmail, \exim, \qmail, and \masqmail\ all use at least two files to store one message in the queue: one file contains the message body, another the envelope and header information. The one containing the mail body is not modified at all. \postfix\ takes a different approach in storing queued messages in an internal format within one file. \person{Finch} takes yet another different approach in suggesting to store the whole queue in one single file with pointers to separating positions \cite{finchFIXME}.
    1.91 -%fixme: check, cite, and think about
    1.92 +All of the presented \MTA{}s use the file system to hold the queue; none uses a database to hold it. A database could improve the reliability of the queue through better persistence. This might be a choice for larger \MTA{}s but is none for \masqmail\ which should be kept small and simple. A running database system does likely require much more resources than \masqmail\ itself does. And as the queue's job is more storing data than running data selection queries, a database does not gain so much that it outweighs its costs.
    1.93  
    1.94 -%fixme: discuss: filesystem vs. database
    1.95 -<< \masqmail\ uses the filesytem to store the queue, storing the queue in a databases might improve the reliability through better persistence. >> %fixme
    1.96 +Hence here the choice is having a directory with simple text files in it. This is straight forward, simple, clear, and general \dots\ and thus a good basis for reliability. It is additionally always of advantage if data is stored in the operation system's natural form, which in the case of \unix\ is plain text.
    1.97  
    1.98 -%fixme: what about the ``rule of repair''?
    1.99 +Robustness for the queue is covered in the next section. %fixme: ist this sentence neccessary? Is it still correct.
   1.100  
   1.101  
   1.102 -\subsection{Functional design}
   1.103 +
   1.104 +\subsubsection*{Mail sanitizing}
   1.105 +
   1.106 +Mail coming into the system may be may be malformed, lacking headers, or be an attempt to exploit the system. Care must be taken.
   1.107 +
   1.108 +In \postfix, this is done by the \name{cleanup} module, which invokes \name{rewrite}. The position in the message flow is after the message comes from one of the several incoming channels and before the message is stored into the \name{incoming} queue. \name{cleanup} does a complete check to make the mail header complete and valid.
   1.109 +
   1.110 +\qmail\ has the principle of ``don't parse'' which propagades the avoidance of parsing as possible in the system. The reason is that parsing is a highly complex task which often makes code exploitable.
   1.111 +
   1.112 +Mail should be stored into the queue as it is in \masqmail's new design. A scanning module should then parse the message with high care. It seems best to use a \name{parser generator} for this work. The parsed data should then be modified if needed and written into a second queue. This approach has several advantages. First, the receiving parts of the system do not bother about content, they simply store it into the queue. Second, one single modules does the parsing and generates new messages that contain only valid data. Third, the sending parts of the system will only work on messages that consist of valid data. Of course it must be ensured that each message passes through the \name{scanning} module, but this is required for spam and malware scanning too.
   1.113 +
   1.114 +The mail body will never get modified, except of removing and adding transfer protocol specific requirements like dot stuffing or special line ending characters.
   1.115 +
   1.116 +\person{Jon Postel}'s robustness principle ``Be liberal in what you accept, and conservative in what you send.'', which can be found in this wording in \RFC\,1122 and in different wordings in numerous \RFC{}s, is respected in the \name{scanning} module. It parses the given input in some liberal way and generates clean output. \person{Raymond}'s \name{Rule of Repair} ``Repair what you can -- but when you must fail, fail noisily and as soon as possible.'' can be applied too. But it is important to repair only obvious problems, because repairing functionality is likely a target of attacks.
   1.117 +
   1.118 +
   1.119  
   1.120  
   1.121  \subsubsection*{Aliasing}
   1.122  
   1.123 -Where should aliases get expanded? They appear in different kind. Important are the ones available in the \path{aliases} file. Aliases can be:
   1.124 -\begin{enumerate}
   1.125 -	\item a different local user (e.g.\ ``\texttt{bob: alice}'')
   1.126 -	\item a remote user (e.g.\ ``\texttt{bob: john@example.com}'')
   1.127 -	\item a list of users (e.g.\ ``\texttt{bob: alice, john@example.com}'')
   1.128 -	\item a command (e.g.\ ``\texttt{bob: |foo}'')
   1.129 -\end{enumerate}
   1.130 -Addresses expanding to lists of users lead to more envelopes. Aliases changing the reciptients domain part may make the message unsuitable for a specific online route.
   1.131 +The main question about aliasing is: Where should aliases get expanded?
   1.132  
   1.133 -Aliasing is often handled in expanding the alias and reinjecting the mail into the system. Unfortunately, the mail is processed twice then; additionally does the system have to handle more mail this way. If it is wanted to check the new recipient address for acceptance and do all processing again, then reinjecting it is the best choice.
   1.134 +Two facts are important to consider: Addresses expanding to lists of users lead to more envelopes. And aliases changing the reciptient's domain part may make the message unsuitable for a specific online route.
   1.135 +
   1.136 +Aliasing is often handled in expanding the alias and reinjecting the mail into the system. Unfortunately, the mail is processed twice then; additionally does the system have to handle more mail this way. If it is wanted to check the new recipient address for acceptance and do all processing again, then reinjecting it is the best choice. But already accepted messages may get rejected in the second go, because of an replacement address from within the system. This seems not to be wanted.
   1.137 +
   1.138 +Doing the alias expansion in the scanning module appears to be the best solution. Unfortunately a second alias expansion must be made on delivery, because only at that point in time is clear which route is used for the message. This compromise is accepted.
   1.139  
   1.140  
   1.141  
   1.142  \subsubsection*{Route management}
   1.143  
   1.144 -%fixme: rework!!
   1.145 -One key feature of \masqmail\ is its ability to send mail out in different ways. The decision is based on the current online state and whether a route may be used for a message or not. The online state can be retrieved in tree ways, explained in \ref{sec:fixme}. A route to send is found by checking every available route for being able to transfer the current message, until one matches.
   1.146 +The online state is only important for the sending modules of the system, thus it should be queried in the \name{queue-out} module which selects ready messages from the \name{outgoing} queue and transfers them to the appropriate sending module. Route-based aliasing, which was described in the last section, %fixme: is this still true?
   1.147 +should to be done in the same go.
   1.148  
   1.149 -This functionality should be implemented in the module that is responsible to invoke one of the outgoing channel modules (for example the one for \SMTP\ or the pipe module).
   1.150  
   1.151 -\masqmail\ can rewrite the envelope's from address and the \texttt{From:} header, dependent on the outgoing route to use. This rewrite must be done \emph{after} it is clear which route a mail will take, of course, so this may be not the module where other header editing is done.
   1.152 -%fixme: see hafiz05 page 57: maybe put the rewriting into the sending module (like smx, exim, courier) (problem with archiving of all outgoing mail?)
   1.153  
   1.154  
   1.155 +\subsubsection*{Authentication and Encryption}
   1.156  
   1.157 -\subsubsection*{Authentication}
   1.158 +Both topics were discussed several time throughout this thesis, amoung other places on page \pageref{} and \pageref{}.
   1.159  
   1.160 -One thing to avoid is being an \name{open relay}. Open relays allow to relay mail from everywhere to everywhere. This is a major source of spam. The solution is restricting relay\footnote{Relaying is passing mail, that is not from and not for the own system, through it.} access.
   1.161 +Authentication should be done within the receiving modules. Similar should authentication for outgoing connections be handled by the sending modules.
   1.162  
   1.163 -Several ways to restrict access are available. The most simple one is restrictiction by the \NAME{IP} address. No extra complexity is added this way, but static \NAME{IP} addresses are mandatory. This kind of restriction may be enabled using the operating system's \path{hosts.allow} and \path{hosts.deny} files. To allow only connections to port 25 from localhost or the local network \texttt{192.168.100.0/24} insert the line ``\texttt{25: ALL}'' into \path{hosts.deny} and ``\texttt{25: 127.0.0.1, 192.168.100.}'' into \path{hosts.allow}.
   1.164 +To encryption applies the same as to authentication here. Only receiving and sending modules should come in contact with it.
   1.165  
   1.166 -If static access restriction is not possible, for example if mail from locations with changing \NAME{IP} addresses wants to be accepted, some kind of authentication mechanism is required. Three common kinds exist:
   1.167 -\begin{enumerate}
   1.168 -	\item \SMTP-after-\NAME{POP}: uses authenication on the \NAME{POP} protocol to permit incoming \SMTP\ connections for a limited time afterwards.
   1.169 -	\item \SMTP authentication: is an extension to \SMTP. Authentication can be requested before mail is accepted.
   1.170 -	\item Certificates: confirm the identity of someone.
   1.171 -\end{enumerate}
   1.172 +In order to avoid code duplicates, the actual implementation of both functionalities should be provided by a central source which gets invoked by the various modules.
   1.173  
   1.174  
   1.175  
   1.176 -\subsubsection*{Encryption}
   1.177  
   1.178 -Electronic mail is very weak to sniffing attacks, because all data transfer is unencrypted. This concerns the message's content, as well as the email addresses in header and envelope, but also authentication dialogs that may transfer plain text passwords (\NAME{PLAIN} and \NAME{LOGIN} are examples). Adding encryption is therefor wanted.
   1.179  
   1.180 -The common way to encrypt \SMTP\ dialogs is using \name{Transport Layer Security} (short: \TLS, successor of \NAME{SSL}). \TLS\ encrypts the datagrams of the \name{transport layer}. This means it works below the application protocols and can be used by any of them\citeweb{wikipedia:tls}.
   1.181  
   1.182 -\TLS\ allows to create secure tunnels through which arbitrary programs can communicate. Hence one can add secure communication afterwards to programs without changing them. \name{OpenSSL} for example---a free implementation---allows traffic to be piped into a command; a secure tunnel is created and the traffic is forwarded through it. Or a secure tunnel can be set up between a local and a remote port; this tunnel can then be used by any application.
   1.183 +\subsubsection*{Spam and malware handling}
   1.184  
   1.185 -The \NAME{POP} protocol, for example, is good suited for such tunneling, but \SMTP\ is is not generally. Outgoing \SMTP\ client connections can be tunneled without problem---\masqmail\ already provides a configure option called \texttt{wrapper} to do so. Tunneling incomming connections to a server leads to problems with \SMTP. As data comes encrypted through the tunnel to the receiving host and gets then decrypted and forwarded on local to the port the application listens on. From the \MTA's view, this makes all connections appear to come from localhost, unfortunately. Figure \ref{fig:stunnel} depicts the data flow.
   1.186 -
   1.187 -For incoming connections, \NAME{STARTTLS}---defined in \RFC2487---is what \mta{}s implement.
   1.188 -
   1.189 -\masqmail\ is already able to encrypt outgoing connections, but encryption of incoming connections, using \NAME{STARTTLS} should be implemented. This only affects the \SMTP\ server module.
   1.190 -
   1.191 -
   1.192 -
   1.193 -
   1.194 -
   1.195 -\subsubsection*{Spam prevention}
   1.196 -
   1.197 ----
   1.198 -
   1.199 -Spam is a major threat nowadays and the goal is to reduce it to a bearable level (see section \ref{sec:swot-analysis}). Spam fighting is a war in which the good guys tend to lose. Putting too much effort there will result in few gain. Real success will only be possible with new---better---protocols and abandonning the weak legacy technologies. Hence \masqmail\ should be able to provide state-of-the-art spam protection, but not more.
   1.200 -
   1.201 ----
   1.202 -
   1.203 -Spam is a major threat to email, as described in section \ref{sec:swot-analysis}. The two main problems are forgable sender addresses and that it is cheap to send hundreds of thousands of messages. Hence, spam senders can operate in disguise and have minimal cost.
   1.204 -
   1.205 -As spam is not just a nuisance for end users but also for the infrastructure---the \mta{}s---by increasing the amount of mail messages. Thus \MTA{}s need to protect themself. Two different approaches are used:
   1.206 +The two approaches for spam handling were already presented to the reader in section \ref{}. Here they are described in more detail:
   1.207  
   1.208  \begin{enumerate}
   1.209  \item Refusing spam during the \SMTP\ dialog. This is the way it was meant by the designers of the \SMTP\ protocol. They thought checking the sender and reciptient mail addresses would be enough, but as they are forgable it is not. More and more complex checks need to be done. Checking needs time, but \SMTP\ dialogs time out if it takes too long. Thus only limited time can be used, during the \SMTP\ dialog, for checking if a message seems to be spam. The advantage is that acceptance of bad messages can be simply refused---no responsibility for the message is taken and no further system load is added. See \RFC2505 (especially section 1.5) for detail.
   1.210  
   1.211 -\item
   1.212 -Checking for spam after the mail was accepted and queued. Here more processing time can be invested, so more detailed checks can be done. But, as responsibility for messages was taken by accepting them, it is no choice to simply delete spam mail. Checks for spam do not lead to sure results, they just indicate the possibility the message is unwanted mail. \person{Eisentraut} indicates actions to take after a message is recognized as probably spam \cite[pages 18--20]{eisentraut05}. The only acceptable one, for mail the \MTA\ is responsible for, is adding further or rewriting existent header lines. Thus all further work on the message is the same as for non-spam messages.
   1.213 +\item Checking for spam after the mail was accepted and queued. Here more processing time can be invested, so more detailed checks can be done. But, as responsibility for messages was taken by accepting them, it is no choice to simply delete spam mail. Checks for spam do not lead to sure results, they just indicate the possibility the message is unwanted mail. \person{Eisentraut} indicates actions to take after a message is recognized as probably spam \cite[pages 18--20]{eisentraut05}. The only acceptable one, for mail the \MTA\ is responsible for, is adding further or rewriting existent header lines. Thus all further work on the message is the same as for non-spam messages.
   1.214  \end{enumerate}
   1.215  
   1.216 -Modern \MTA{}s use both techniques in combination. Checks during the \SMTP\ dialog tend to be implemented in the \mta\ to make it fast; checks after the message was queued are often done using external programs (\name{spamassassin} is a well known one). \person{Eisentraut} sees the checks during the \SMTP\ dialog to be essentiell: ``Ganz ohne Analyse w\"ahrend der \SMTP-Phase kommt sowieso kein \MTA\ aus, und es ist eine Frage der Einsch\"atzung, wie weit man diese Phase belasten m\"ochte.''\cite[page 25]{eisentraut05} (translated: ``No \MTA\ can go without analysis during the \SMTP\ phase anyway, but the amount of stress one likes to put on this phase is left to his discretion.'')
   1.217 +Modern \MTA{}s use both techniques in combination. Checks during the \SMTP\ dialog tend to be implemented in the \mta\ to make it fast; checks after the message was queued are often done using external programs (\name{spamassassin} is a well known one). \person{Eisentraut} sees the checks during the \SMTP\ dialog to be essentiell: ``Ganz ohne Analyse w\"ahrend der \SMTP-Phase kommt sowieso kein \MTA\ aus, und es ist eine Frage der Einsch\"atzung, wie weit man diese Phase belasten m\"ochte.'' \cite[page 25]{eisentraut05} (translated: ``No \MTA\ can go without analysis during the \SMTP\ phase anyway, but the amount of stress one likes to put on this phase is left to his discretion.'')
   1.218  
   1.219 -\NAME{DNS} blacklists (short: \NAME{DNSBL}) and \name{greylisting} are checks to be done before accepting the message. Invoking \name{spamassassin}, to add headers containing the estimated spam probability, is best to be invoked after the message is queued.
   1.220 +Checking before a message is accepted, like \NAME{DNS} blacklists and \name{greylisting}, needs to be invoked from within the receiving modules. Like for authentication and encryption, the implementation of the functionality should be provided by a central source.
   1.221  
   1.222 +All checking after the message was queued should be done by pushing the message through external scanners like \name{spamassassin}. The \name{scanning} module is the best place to handle this. Hence this module needs interfaces to external scanners.
   1.223  
   1.224  
   1.225 +Malware scanning is similar like the second type of spam scanning. The \name{amavis} framework is a popular mail scanning framework that includes all kinds of malware and also spam scanners; it communicates by using \SMTP.
   1.226  
   1.227 -\subsubsection*{Virus checking}
   1.228 +Providing \SMTP\ in and out channels from the \name{scanning} module to external scanner applications seems to be a desired goal. Using further instances of the already available \name{smtp} and \name{smtpd} modules therefore appears to be the best solution.
   1.229  
   1.230 -Related to spam is malicous content (short: \name{malware}) like viruses, worms, trojan horses. They, in contrast to spam, do not affect the \MTA\ itself, as they are in the mail body. The same situation in the real world is post offices opening letters to check if they contain something that could harm the recipient. This is not a mail transport concern. Apart of not being the right program to do the job, the \MTA\---the one which is responsible for the recipient---is at a good position to do this work.
   1.231  
   1.232 -In any way should malware checking be done by external programs that may be invoked by the \mta. But using mail deliver and processing agents, like \name{procmail}, seem to be better suited locations to invoke content scanners.
   1.233  
   1.234 -A popular email filter framework is \name{amavis} which integrates various spam and virus scanners. The common setup includes a receiving \MTA\ which sends it to \name{amavis} using \SMTP, \name{amavis} processes the mail and sends it then to a second \MTA\ that does the outgoing transfer. \postfix\ and \exim\ can be configured so that one instance can work as both, the \MTA\ for incoming and outgoing transfer. A setup with \sendmail\ needs two separate instances running. It must be quarateed that all mail flows through the scanner.
   1.235 -
   1.236 -A future \masqmail\ would do good to have a single point, where all traffic flows through, that is able to invoke external programs to do mail processing of any kind.
   1.237 -
   1.238 -
   1.239 -%AMaViS (amavisd-new): email filter framework to integrate spam and virus scanner
   1.240 -%\begin{verbatim}
   1.241 -%internet -->25 MTA -->10024 amavis -->10025 MTA --> reciptient
   1.242 -                %|                            |
   1.243 -                %+----------------------------+
   1.244 -%\end{verbatim}
   1.245 -%
   1.246 -%postfix and exim can habe both mta servises in the same instance, sendmail needs two instances running.
   1.247 -%
   1.248 -%MailScanner:
   1.249 -%incoming queue --> MailScanner --> outgoing queue
   1.250 -%
   1.251 -%postfix: with one instance possible, exim and sendmail need two instances running
   1.252 -
   1.253 -
   1.254 -%message body <-> envelope, header
   1.255 -%
   1.256 -%anti-virus: clamav
   1.257 -%postfix: via amavis
   1.258 -%exim: via content-scanning-feature called from acl
   1.259 -%sendmail: with milter
   1.260 -%procmail
   1.261 -%
   1.262 -%virus scanner work on file level
   1.263 -%amavis receives mail via smtp or pipe, splits it in its parts (MIME) and extracks archives, the come the virus scanners
   1.264 -%if the mail is okay, it goes via smtp to a second mta
   1.265 -
   1.266 -%what amavis recognizes:
   1.267 -%- invalid headers
   1.268 -%- banned files
   1.269 -%- viruses
   1.270 -%- spam (using spam assassin)
   1.271 -%
   1.272 -%mimedefang: uses milter interface with sendmail
   1.273  
   1.274  
   1.275  
   1.276  \subsubsection*{Archiving}
   1.277  
   1.278 -Mail archiving and auditability become more important as electronic mail becomes more important. Ability to archive verbatim copies of every mail coming into and every mail going out of the system, with relation between them, appears to be a goal to achieve.
   1.279 +The best point to archive copies of every incoming mail is the \name{queue-in} module, respectively the \name{queue-out} module for copies outgoing mail. But not respected with this approach are the changes that are made by the receiving modules (adding further headers) and sending modules (address rewrites).
   1.280  
   1.281 -\postfix\ for example has a \texttt{always\_bcc} feature, to send a copy of every mail to a definable reciptient. At least this funtionality should be given, although a more complete approach is preferable.
   1.282 +\qmail\ has the ability to log complete \SMTP\ dialogs. Logging the complete data transaction into and out of the system into a separate log file is a great feature which should be implemented into each receiving and sending module. But as it will produce a huge amount of output, it should be cared to disabled it by default.
   1.283  
   1.284  
   1.285  
   1.286 @@ -365,18 +314,6 @@
   1.287  
   1.288  
   1.289  
   1.290 -\subsection{Security design}
   1.291 -
   1.292 -\subsubsection*{Sanitize mail}
   1.293 -
   1.294 -Mail coming into the system often lacks important header lines. At least the required ones must be added from the \MTA. A good example is the \texttt{Message-Id:} header.
   1.295 -
   1.296 -In \postfix, this is done by the \name{cleanup} module, which invokes \name{rewrite}. The position in the message flow is after coming from one of the several incoming channels and before the message is stored into the \name{incoming} queue. Modules that handle incoming channels may also add headers, for example the \texttt{From:} and \texttt{Date:} headers. \name{cleanup}, however, does a complete check to make the mail header complete and valid.
   1.297 -
   1.298 -Apart from deciding where to sanitize the mail header, is the question where to generate the envelope. The envelope specifies the actual recipient of the mail, no matter what the \texttt{To:}, \texttt{Cc:}, and \texttt{Bcc:} headers tell. Multiple reciptients lead to multiple different envelopes, containing all the same mail message.
   1.299 -
   1.300 -
   1.301 -
   1.302  
   1.303  
   1.304