1.1 --- a/thesis/bib/websites.bib	Sat Jan 24 17:01:34 2009 +0100
     1.2 +++ b/thesis/bib/websites.bib	Sat Jan 24 17:02:19 2009 +0100
     1.3 @@ -311,3 +311,9 @@
     1.4  	howpublished = "{\small\url{http://www.dwheeler.com/sloccount} (2009-01-20)}",
     1.5  }
     1.6  
     1.7 +@misc{qmail:tls-patch,
     1.8 +	author = "Frederick Vermeulen",
     1.9 +	title = "\emph{Qmail-TLS patch archive}",
    1.10 +	howpublished = "{\small\url{http://inoa.net/qmail-tls/} (2009-01-24)}",
    1.11 +}
    1.12 +

     2.1 --- a/thesis/tex/5-Improvements.tex	Sat Jan 24 17:01:34 2009 +0100
     2.2 +++ b/thesis/tex/5-Improvements.tex	Sat Jan 24 17:02:19 2009 +0100
     2.3 @@ -17,36 +17,29 @@
     2.4  
     2.5  \subsection{Encryption}
     2.6  
     2.7 -Encryption should be the first functionality to add to the current code. This requirement was already discussed on page \pageref{requirement-encryption}. As explained there, \NAME{STARTTLS} encryption---as defined in \RFC\,2487---should be added to \masqmail.
     2.8 +Encryption should be the first functionality to add to the current code. This requirement was already discussed on page \pageref{requirement-encryption}. As explained there, \NAME{STARTTLS} encryption---defined in \RFC\,2487---should be added to \masqmail.
     2.9  
    2.10 -Adding encryption requires changes mainly in three source files: \path{smtp_in.c}, \path{smtp_out.c}, and in \path{conf.c}.
    2.11 +This work requires changes mainly in three source files: \path{smtp_in.c}, \path{smtp_out.c}, and in \path{conf.c}.
    2.12  
    2.13  The first file includes the functionality for the \SMTP\ server. It needs to offer \NAME{STARTTLS} support to clients and needs to initiate the encryption when the client requests it. Additionally, the server should be able to insist on encryption before it accepts any message.
    2.14  
    2.15 -The second file includes the functionality for the \SMTP\ client. It should start the encryption by issuing the \NAME{STARTTLS} keyword if the server supports it. It should be possible to send messages only if encryption is possible.
    2.16 +The second file includes the functionality for the \SMTP\ client. It should start the encryption by issuing the \NAME{STARTTLS} keyword if the server supports it. It should be possible to send messages only over encrypted channels.
    2.17  
    2.18 -The third file controls the configuration files. New configuration option need to be added. The encryption policy for incoming connections needs to be defined. Three choices seem necessary: no encryption, offer encryption, insist on encryption. The encryption policy for outgoing connections should be part of each route setup. The options are the same: never encrypt, encrypt if possible, insist on encryption.
    2.19 +The third file controls the configuration files. New configuration options need to be added. The encryption policy for incoming connections needs to be defined. Three choices seem necessary: no encryption, offer encryption, insist on encryption. The encryption policy for outgoing connections should be part of each route setup. The options are the same: never encrypt, encrypt if possible, insist on encryption.
    2.20  
    2.21 -\NAME{STARTTLS} uses \NAME{TLS} encryption which is based on certificates. Thus the \MTA\ needs its own certificate. This should be generated during installation. A third party application like \name{openssl} should be taken for this job. The encryption itself should also be done using an available library. Open\NAME{SSL} or a substitute like Gnu\NAME{TLS} does then become a dependency for \masqmail. Gnu\NAME{TLS} seems to be the better choice because the Open\NAME{SSL} license is incompatible to the \NAME{GPL}, under which \masqmail\ and Gnu\NAME{TLS} are covered.
    2.22 +\subsubsection*{Depencencies}
    2.23 +\NAME{STARTTLS} uses \NAME{TLS} encryption which is based on certificates. Thus the \MTA\ needs its own certificate. This should be generated during installation. A third party application like \name{openssl} should be taken for this job. The encryption itself should also be done using an available library. Open\NAME{SSL} or a substitute like Gnu\NAME{TLS} does then become a dependency for \masqmail. Gnu\NAME{TLS} is the better choice because the Open\NAME{SSL} license is incompatible to the \NAME{GPL}, under which \masqmail\ and Gnu\NAME{TLS} are covered.
    2.24  
    2.25  User definable paths to \masqmail's secret key, \masqmail's certificate, and the public certificates of trusted \name{Certificate Authorities} (short: \NAME{CA}s) are also nice to have.
    2.26  
    2.27  
    2.28 -<< TLS patch of qmail >>
    2.29 +\subsubsection*{Existing code}
    2.30  
    2.31 -%postfix: main.cf
    2.32 -%
    2.33 -%	smtpd_use_tls = yes
    2.34 -%	smtpd_tls_received_header = no (does not log in received headers)
    2.35 -%
    2.36 -%	smtpd_tls_key_file = /etc/postfix/key.pem
    2.37 -%	smtpd_tls_cert_file = /etc/postfix/cert.pem
    2.38 -%	smtpd_tls_CA_file = /etc/postfix/CAcert.pem
    2.39 -%
    2.40 -%	smtp_use_tls = yes  (use TLS for sending)
    2.41 -%	smtp_tls_key_file = /etc/postfix/key.pem
    2.42 -%	smtp_tls_cert_file = /etc/postfix/cert.pem
    2.43 -%	smtp_tls_CA_file = /etc/postfix/CAcert.pem
    2.44 +\person{Frederik Vermeulen} wrote an encryption patch for \qmail\ which adds \NAME{STARTTLS} support \citeweb{qmail:tls-patch}. This patch adds about 500 lines of code for the functionality.
    2.45 +
    2.46 +Adding this code in a similar form to \masqmail\ will be fairly easy. It will save a lot of work as it is not necessary to write the code completely from scratch.
    2.47 +
    2.48 +
    2.49  
    2.50  
    2.51  
    2.52 @@ -57,6 +50,8 @@
    2.53  
    2.54  Static access restriction, based on the \NAME{IP} address is already possible by using \name{TCP Wrappers}. This makes it easy to refuse all connections from outside the local net for example, which is a good prevention of being an open relay. More detailed static restrictions, like splitting between mail for the system and mail to relay, should not be added to the current code. This may be a concern for the new design.
    2.55  
    2.56 +\subsubsection*{One of the dynamic methods}
    2.57 +
    2.58  Of the dynamic, secret based, authentication methods (\SMTP-after-\NAME{POP}, \SMTP\ authentication, and certificates) the first one drops out as it requires a \NAME{POP} server running on the same or a trusted host. \NAME{POP} servers are rare on workstations and home servers do also not regularly include them. Thus it is no option for \masqmail.
    2.59  
    2.60  Authentication based on certificates does suffer from the certificate infrastructure that is required. Although certificates are already used for encryption, its management overhead prevented wide spread usage for authentication.
    2.61 @@ -68,7 +63,9 @@
    2.62  \hfill\cite[page 44]{dent04}
    2.63  \end{quote}
    2.64  
    2.65 -These days is \NAME{SMTP-AUTH}, which is defined in \RFC\,2554, supported by most email clients. If encryption is used then even insecure authentication methods like \NAME{PLAIN} and \NAME{LOGIN} become secure.
    2.66 +These days is \NAME{SMTP-AUTH}---defined in \RFC\,2554---supported by most email clients. If encryption is used then even insecure authentication methods like \NAME{PLAIN} and \NAME{LOGIN} become secure.
    2.67 +
    2.68 +\subsubsection*{SASL}
    2.69  
    2.70  \masqmail\ best uses an available \NAME{SASL} library. \name{Cyrus} \NAME{SASL} is used by \postfix\ and \sendmail. It is a complete framework that makes use of existing authentication concepts like \path{/etc/passwd} or \NAME{PAM}. As advantage it can be included in existing user data bases. \name{gsasl} is an alternative. It comes as a library which helps on deciding for a method and on generating the appropriate dialog data; the actual transmission of the data and the authentication against some database is left open to the programmer. \name{gsasl} is used by \name{msmtp} for example. It seems best to give both concepts a try and decide then which one to use.
    2.71  
    2.72 @@ -81,24 +78,24 @@
    2.73  If the authentication code for outgoing connects shall be changed too, it must be done in \path{smtp_out.c}. The configuration options are already present.
    2.74  
    2.75  
    2.76 +\subsubsection*{Authentication backend}
    2.77 +
    2.78  About the authentication backend. For a small \MTA\ like \masqmail, it seems preferable to store the login data in a text file under \masqmail's control. This is the most simple choice for many usage scenarios. But using a central authentication facility has advantages in larger setups too. \name{Cyrus} \NAME{SASL} supports both, so there is no problem. If \name{gsasl} is chosen, it seems best to start with an authentication file under \masqmail's control.
    2.79  
    2.80  
    2.81 -
    2.82 -
    2.83 -
    2.84 -<< Compare static with dynamic authentication: pros and cons; usecases: when to use what; >>
    2.85 -
    2.86 -<< how could this be covered by architecture (e.g. smtp submission). >>
    2.87 +%fixme: << how could this be covered by architecture (e.g. smtp submission). >>
    2.88  
    2.89  
    2.90  
    2.91  
    2.92  
    2.93  \subsection{Security}
    2.94 +\label{sec:current-code-security}
    2.95  
    2.96  Improvements to \masqmail's security are an important requirement and are the third task to work on. Retrofitting security \emph{into} \masqmail\ is not or hardly possible as it was explained in section \ref{sec:discussion-further-devel}. But adding wrappers and interposition filters can be a large step towards security.
    2.97  
    2.98 +\subsubsection*{Mail security layers}
    2.99 +
   2.100  At first mail security layers like \name{smap} come to mind. The market share analysis in section \ref{sec:market-share} identified such software. This is an interposition filter that stands between the untrusted network and the \MTA. It accepts mail in replacement for the \MTA\ (also called \name{proxy}) in order to separate the \MTA\ from the untrusted network.
   2.101  
   2.102  The work \name{smap} does is described in \cite{cabral01}: \name{smap} accepts messages as proxy for the \MTA\ and puts it into a queue. \name{smapd} a brother program runs as daemon and watches for new messages in the queue which it submits into the \MTA\ then.
   2.103 @@ -344,13 +341,13 @@
   2.104  %fixme: do i need all this ``quesses''??
   2.105  
   2.106  
   2.107 -\subsubsection*{Modules and queues}
   2.108 +\subsubsection*{The modules}
   2.109  
   2.110 -The new architecture consists of several modules and two queues plus a data pool. They are described in more detail now. First the three main modules, then the queues, and afterwards the modules for incoming and outgoing transfer.
   2.111 +The new architecture consists of several modules. They are described in more detail now. First the three main modules, afterwards the modules for incoming and outgoing transfer.
   2.112  
   2.113  
   2.114 -The \name{queue-in} module creates new spool files in the \name{incoming} queue and in the message \name{pool} for incoming messages. It is a process running in background, waiting for connections from one of the receiver modules. When one of them is receiving a new message, it connects to the \name{queue-in} module which creates a spool file in the \name{incoming} queue and a message body file in the \name{pool} and returns success. The receiver module then sends the envelope, the message header, and the message body. The first two get written into the spool file by \name{queue-in}, the latter is stored into the \name{pool}. If all went well another positive result is returned.
   2.115 -%fixme: should be no daemon
   2.116 +The \name{queue-in} module creates new spool files in the \name{incoming} queue and in the message \name{pool} for incoming messages. It is a process in background, waiting for connections from one of the receiver modules. When one of them is receiving a new message, it connects to the \name{queue-in} module which creates a spool file in the \name{incoming} queue and a message body file in the \name{pool} and returns success. The receiver module then sends the envelope, the message header, and the message body. The first two get written into the spool file by \name{queue-in}, the latter is stored into the \name{pool}. If all went well another positive result is returned.
   2.117 +%fixme: daemon or no daemon?
   2.118  
   2.119  
   2.120  The \name{scanning} module is the central part of the system. It reads spooled messages from the \name{incoming} queue, works on the data, and writes new spool files to the \name{outgoing} queue. Then the message is removed from the \name{incoming} queue. The main job of this module is the processing of the message. Headers are fixed and missing ones are added if necessary, aliasing is done, and external processing of any kind is triggered. The \name{scanning} module can run in background and look for new mail in regular intervals or signals may be sent to it by \name{queue-in}. Alternatively it can be called by \name{cron} to do single runs. The \name{scanning} module work on the spool files primary but may read the mail body from the \name{pool} if necessary.
   2.121 @@ -359,13 +356,6 @@
   2.122  The \name{queue-out} module takes messages from the \name{outgoing} queue, queries information about the online state which specifies the route to use, and passes the messages to the correct transport module. Successfully transferred messages are removed from the \name{outgoing} queue. This module handles the \masqmail\ specific task of the route management.
   2.123  
   2.124  
   2.125 -The \name{incoming} queue stores envelope and the message header of messages received via one of the incoming channels. The data is in unprocessed form.
   2.126 -
   2.127 -The \name{outgoing} queue contains processed data. The header and envelope information is complete and in valid form.
   2.128 -
   2.129 -The \name{pool} is the storage of the message bodies of queued messages. This data is not changed within the \MTA, it is written on reception and read on dispatch.
   2.130 -
   2.131 -
   2.132  \name{Receiver modules} are the communication interface between external senders and the \name{queue-in} module. Each protocol needs a corresponding \name{receiver module} to be supported. Most popular are the \name{sendmail} module which is a command to be called from the local host and the \name{smtpd} module which listens on port 25. Other modules to support other protocols may be added as needed. Receiving modules that need to listen on ports should get invoked by \name{inetd} or a more secure replacement like \person{Bernstein}'s \name{ucspi-tcp}. This makes it possible to run them with least privilege.
   2.133  
   2.134  \name{Transport modules}, on the opposite side of the system, are the modules to send outgoing mail. They are the interface between \name{queue-out} and remote hosts or local commands for further processing. The most popular ones are the \name{smtp} module which acts as the \SMTP\ client and the \name{pipe} module to interface gateways to other systems or networks, like fax or \NAME{UUCP}. A module for local delivery is not included, \masqmail\ passes this job to the \NAME{MDA} (see section \ref{sec:functional-requirements} for reasons). Thus a \name{mail delivery agent} (like \name{procmail}) is to be used with the \name{pipe} module.
   2.135 @@ -403,10 +393,13 @@
   2.136  
   2.137  \subsubsection*{The queue}
   2.138  
   2.139 -The queue consists of three directories within the queue path. Two, named \name{incoming} and \name{outgoing}, for storing the spool files; one, called \name{pool}, to store the data files. The files being part of one message share the same unique name. The spool file's internal structure can remain the same as the one of current \masqmail. A queued message is represented by a spool file in \name{incoming} or \name{outgoing} and a data file in the \name{pool}.
   2.140 +The queue is actually two queues and a data pool. The queues store the spool files---unprocessed in \name{incoming} and in complete and valid form in \name{outgoing}. The \name{pool} is the storage of data files, the message bodies of queued messages.
   2.141 +
   2.142 +Three directories within the queue path contain the queue on disk. Two, named \name{incoming} and \name{outgoing}, for storing the spool files; one, called \name{pool}, to store the data files. The files being part of one message share the same unique name. A queued message is represented by a spool file in \name{incoming} or \name{outgoing} and a data file in the \name{pool}.
   2.143  
   2.144  The spool file owner's executable bit shows if the file is ready for further processing: The module that writes the file into the queue sets the bit as last action. Modules that read from the queue can process messages that have the bit set. This approach is derived from \postfix.
   2.145  
   2.146 +The spool file's internal structure can remain the same as the one of current \masqmail.
   2.147  
   2.148  The spool file format is basically the same as the one in current \masqmail: one file for the envelope and message header information (it is called ``spool file'' here), a second file for the message body (called ``data file'').
   2.149  
   2.150 @@ -424,6 +417,10 @@
   2.151  %	\label{fig:queue-data-flow}
   2.152  %\end{figure}
   2.153  
   2.154 +%The \name{incoming} queue stores envelope and the message header of messages received via one of the incoming channels. The data is in unprocessed form. The \name{outgoing} queue contains processed data. The header and envelope information is complete and in valid form. The \name{pool} is the storage of the message bodies of queued messages. This data is not changed within the \MTA, it is written on reception and read on dispatch.
   2.155 +
   2.156 +
   2.157 +
   2.158  
   2.159  A sample spool file. With comments in parenthesis.
   2.160  \codeinput{input/sample-spool-file.txt}
   2.161 @@ -433,7 +430,7 @@
   2.162  
   2.163  \subsubsection*{Rights and permission}
   2.164  
   2.165 -The user set required for \qmail\ seems to be too complex. One special user, like \postfix\ uses, is more appropriate. \name{root} privilege and \name{setuid} permission is avoided as much as possible.
   2.166 +The set of system users that is required for \qmail\ seems to be too complex. One system user, like \postfix\ uses, is more appropriate. \name{root} privilege and \name{setuid} permission needs to be avoided as feasible.
   2.167  
   2.168  Table \ref{tab:new-masqmail-permissions} shows the suggested ownership and permissions of the modules.
   2.169