docs/diploma

annotate thesis/tex/4-MasqmailsFuture.tex @ 170:204683bfb4e7

find changesets by author, revision, files, or words in the commit message
wrote much about modules of an MTA
author meillo@marmaro.de
date Sat, 20 Dec 2008 20:37:09 +0100
parents 74151e26ba6a
children 5c873e6478ef
rev   line source
meillo@109 1 \chapter{\masqmail's present and future}
meillo@93 2
meillo@137 3 \section{Existing code base}
meillo@142 4 Here regarded is version 0.2.21 of \masqmail. This is the last version released by Oliver \person{Kurth}, and the basis for my thesis.
meillo@142 5
meillo@93 6
meillo@137 7 \subsubsection*{Features}
meillo@93 8
meillo@142 9 \masqmail\ accepts mail on the command line and via \SMTP. Mail queueing and alias expansion is supported. \masqmail\ is able to deliver mail to local mailboxes (in \name{mbox} or \name{maildir} format) or pass it to a \name{mail delivery agent} (like \name{procmail}). Mail destinated to remote locations is sent using \SMTP\ or can be piped to commands, being gatesways to \NAME{UUCP} or \NAME{FAX} for example.
meillo@93 10
meillo@142 11 Outgoing \SMTP\ connections feature \SMTP-\NAME{AUTH} and \SMTP-after-\NAME{POP} authentication, but incoming connections do not. Using wrappers for outgoing connections is supported. This offers a two way communication through a wrapper application like \name{openssl}.
meillo@137 12 %todo: what about SSL/TLS encryption?
meillo@93 13
meillo@142 14 \masqmail\ focuses on non-permanent online connections, thus a concept of online routes is used. One may configure any amount of routes to send mail. Each route can have criterias, like matching \texttt{From:} or \texttt{To:} headers, to determine if mail is allowed to be sent using it. Mail to destinations outside the local net gets queued until \masqmail\ is informed about the existance of a online connection.
meillo@142 15
meillo@137 16 The \masqmail\ executable can be called under various names for sendmail-compatibility reasons. This is organized by symbolic links with different names pointing to the \masqmail\ executable. The \sendmail\ names are \path{/usr/lib/sendmail} and \path{/usr/sbin/sendmail} because many programs expect the \mta\ to be located there. Further more \sendmail\ supports calling it with a different name instead of supplying command line arguments. The best known of this shortcuts is \path{mailq}, which is equivilent to calling it with the argument \verb+-bq+. \masqmail\ recognizes the names \path{mailq}, \path{smtpd}, \path{mailrm}, \path{runq}, \path{rmail}, and \path{in.smtpd}. The first two are inspired by \sendmail. Not implemented is the name \path{newaliases} because \masqmail\ does not generate binary representations of the alias file.\footnote{A shell script located named \path{newaliases}, that invokes \texttt{masqmail -bi}, can provide the command to satisfy other software needing it.} \path{hoststat} and \path{purgestat} are missing for sendmail-compatibility.
meillo@109 17 %masqmail: mailq, mailrm, runq, rmail, smtpd/in.smtpd
meillo@109 18 %sendmail: hoststat, mailq, newaliases, purgestat, smtpd
meillo@109 19
meillo@137 20 Additional to the \mta\ job, \masqmail\ also offers mail retrieval services with being a \NAME{POP3} client. It can fetch mail from different remote locations, dependent on the active online route.
meillo@109 21
meillo@137 22
meillo@137 23
meillo@137 24 \subsubsection*{The code}
meillo@137 25
meillo@137 26 \masqmail\ is written in the C programming language. The program, as of version 0.2.21, consists of 34 source code and eight header files, containing about 9,000 lines of code\footnote{Measured with \name{sloccount} by David A.\ Wheeler.}. Additionally, it includes a \name{base64} implementation (about 300 lines) and \name{md5} code (about 150 lines). For systems that do not provide \name{libident}, this library is distributed as well (circa 600 lines); an available shared library however has higher precedence in linking.
meillo@137 27
meillo@137 28 The only mandatory dependency is \name{glib}---a cross-platform software utility library, originated in the \NAME{GTK+} project. It provides safer replacements for many standard library functions. It also offers handy data containers, easy-to-use implementations of data structures, and much more.
meillo@137 29
meillo@109 30
meillo@109 31 With \masqmail\ comes the small tool \path{mservdetect}; it helps setting up a configuration that uses the \name{mserver} system to detect the online state. Two other binaries get compiled for testing purposes: \path{readtest} and \path{smtpsend}. All three programms use \masqmail\ source code; they only add a file with a \verb+main()+ function each.
meillo@109 32
meillo@93 33
meillo@137 34 \masqmail\ does not provide an interface to plug in modules with additional functionality. There exists no add-on or module system. The code is only separated by function to the various source files. Some functional parts can be included or excluded by defining symbols. Adding maildir support at compile time, means giving the option \verb+--enable-maildir+ to the \path{configure} call. This preserves the concerning code to get removed by the preprocessor. Unfortunately the \verb+#ifdef+s are scattered through all the source, leading to a FIXME(holperig) code base.
meillo@93 35
meillo@132 36
meillo@132 37
meillo@132 38
meillo@132 39
meillo@170 40 \section{\masqmail\ next generation}
meillo@146 41
meillo@170 42 \subsection{Requirements}
meillo@146 43
meillo@146 44 Following is a list of current and future requirements to make \masqmail\ ready for the future.
meillo@146 45
meillo@146 46
meillo@146 47 \subsubsection*{Large message handling}
meillo@155 48 Trends in the market for electronic communication go towards consolidated communication, hence email will be used more to transfer voice and video messages. This leads to larger messages. The store-and-forward transport of email is not good suited for large data. Thus new protocols, like \NAME{QMTP} (described in section %\ref{FIXME}
meillo@155 49 ), may become popular.
meillo@146 50
meillo@146 51
meillo@146 52 \subsubsection*{Ressource friendly software}
meillo@149 53 The merge of communication hardware and the move of email services from providers to homes, demands smaller and more resource-friendly software. The amount of mail will be lower, even if much more mail will be sent. More important will be the energy consumption and heat emission. These topics increased in relevance during the past years and they are expected to become more central. \masqmail\ is not a program to be used on large servers, but to be used on small devices. Thus focusing on energy and heat, not on performance, is the direction to go.
meillo@146 54
meillo@146 55
meillo@146 56 \subsubsection*{New mail transfer protocols}
meillo@149 57 Large messages demand more efficient transport through the net. As well is a final solution needed to defeat the spam problem. New mail transport protocols may be the only good solutions for both problems. They also can improve reliability, authentication, and verification issues. \masqmail\ should be able to support new protocols as they appear and are used.
meillo@146 58
meillo@146 59
meillo@149 60 \subsubsection*{Spam handling}
meillo@149 61 Spam is a major threat. According to the \NAME{SWOT} analysis, the goal is to reduce it to a bearable level. Spam fighting is a war are where the good guys tend to lose. Putting too much effort there will result in few gain. Real success will only be possible with new---better---protocols and abandonning the weak legacy technologies. Hence \masqmail\ should be able to provide state-of-the-art spam protection, but not more.
meillo@146 62
meillo@146 63
meillo@161 64 \subsubsection*{Security}
meillo@161 65 \MTA{}s are critical points for computer security, as they are accessable from external networks. They must be secured with high effort. Properties like high priviledge level, work load influenced from extern, work on unsafe data, and demand for reliability, increase the security needed. Unsecure and unreliable \mta{}s are of no value. \masqmail\ needs to b e secure enough for its target field of operation.
meillo@161 66
meillo@161 67
meillo@146 68 \subsubsection*{Easy configuration}
meillo@149 69 Having \mta{}s on many home servers and clients, requires easy and standardized configuration. The common setups should be configurable with single actions by the user. Complex configuration should be possible, but focused must be the most common form of configuration: choosing one of several standard setups.
meillo@146 70
meillo@146 71
meillo@146 72
meillo@146 73
meillo@146 74
meillo@146 75
meillo@170 76 \subsection{Discussion on architecture}
meillo@146 77
meillo@163 78 A program's architecture is probably the most influencing design decision, and has the greatest impact on the program's future capabilities. %fixme: search quote ... check if good
meillo@132 79
meillo@161 80 \masqmail's current artitecture is monolitic like \sendmail's and \exim's. But more than the other two, is it one block of interweaved code. \sendmail\ provides now, with its \name{milter} interface, standardized connection channels to external modules. \exim\ has a highly structured code with many internal interfaces, like the one for supported authentication ``modules''. \masqmail\ has none of them; it is what \sendmail\ was in the beginning: a single large block.
meillo@161 81
meillo@161 82 Figure \ref{fig:masqmail-arch} is an attempt to depict \masqmail's internal structure.
meillo@161 83
meillo@161 84 \begin{figure}
meillo@161 85 \begin{center}
meillo@161 86 \input{input/masqmail-arch.tex}
meillo@161 87 \end{center}
meillo@161 88 \caption{Internal architecture of \masqmail}
meillo@161 89 \label{fig:masqmail-arch}
meillo@161 90 \end{figure}
meillo@161 91
meillo@163 92 \sendmail\ improved its old architecture, for example by adding the milter interface. \exim\ was designed and is carefully maintained with a modular-like code structure in mind. \qmail\ started from scratch with a ``security-first'' approach, \postfix\ improved on it, and \name{sendmail X}/\name{MeTA1} tries to adopt the best of \qmail\ and \postfix, to completely replace the old \sendmail\ architecture. \person{Hafiz} \cite{hafiz05}. describes this evolution of \mta\ architecture very well.
meillo@161 93
meillo@163 94 Every one of the popular \MTA{}s is more modular, or became more modular over time, than \masqmail\ is. Modern requirements like spam protection and future requirements like the use of new mail transport protocols demand modular designs for keeping the software simple. Simplicity is a key property for security.
meillo@161 95
meillo@163 96 \person{Hafiz} agrees:
meillo@163 97 \begin{quote}
meillo@163 98 The goal of making software secure can be better achieved by making the design simple and easier to understand and verify. \cite[page64]{hafiz05}
meillo@163 99 \end{quote}
meillo@163 100 He identifies the security of \qmail\ to come from it's \name{compartmentalization}, which goes hand in hand with modularity:
meillo@163 101 \begin{quote}
meillo@163 102 A perfect example is the contrast between the feature envy early \sendmail\ architecture implemented as one process and the simple, modular architecture of \qmail. The security of \qmail\ comes from its compartmentalized simple processes that perform one task only and are therefor testable for security. \cite[page 64]{hafiz05}
meillo@163 103 \end{quote}
meillo@165 104 As well does \person{Dent}: ``The modular architecture of Postfix forms the basis for much of its security.''\cite[page 7]{dent04}
meillo@161 105
meillo@163 106 Modularity is needed for supporting modern \MTA\ requirements, providing a clear interface to add further functionality without increasing the overall complexity much. Modularity is also an enabler for security. Security comes from good design, as \person{Graff} and \person{van Wyk} explain:
meillo@163 107 \begin{quote}
meillo@163 108 Good design is the sword and shield of the security-conscious developer. Sound design defends your application from subversion or misuse, protecting your network and the information on it from internal and external attacks alike. It also provides a safe foundation for future extensions and maintainance of the software.
meillo@163 109 %
meillo@163 110 %Bad design makes life easier for attackers and harder for the good guys, especially if it contributes to a false sends of security while obscuring pertinent failings.
meillo@163 111 \cite[page 55]{graff03}
meillo@163 112 \end{quote}
meillo@161 113
meillo@163 114 \person{Hafiz} adds: ``The major idea is that security cannot be retrofitted into an architecture.''\cite[page 64]{hafiz05}
meillo@161 115
meillo@163 116 All this leads to one logical step: The rewrite of \masqmail\ using a modern, modular architecture, to get a modern \MTA\ satisfying nowadays needs.
meillo@161 117
meillo@161 118
meillo@161 119
meillo@161 120
meillo@170 121 \subsection{Jobs of an MTA}
meillo@161 122
meillo@163 123 This section tries to identify the needed modules for a modern \MTA. They are later the pieces of which the new architecture is built of.
meillo@163 124
meillo@170 125 The basic job of a \mta\ is to tranport mail from a sender to a recipient. This is the definition of such a program and this is how \person{Dent}\cite[page 19]{dent04} and \person{Hafiz} \cite[pages 3-5]{hafiz05} generally see its design.
meillo@163 126
meillo@170 127 An \MTA\ therefor needs at least a mail receiving facility and a mail sending facility. Additionally probably all \MTA\ developers (excluded the only forwarders), see the need for a mail queue. A mail queue removes the need to deliver at once a message is received. They also provide fail-safe storage of mails until they are delivered.
meillo@163 128
meillo@163 129
meillo@163 130
meillo@163 131 \subsubsection*{Incoming channels}
meillo@163 132
meillo@170 133 \sendmail-compatible \mta{}s must support at least two incoming channels: mail submitted using the \sendmail\ command, and mail received via the \SMTP\ daemon. It is therefor common to split the incoming channel into local and remote. This is done by \qmail\ and \postfix. The same way is \person{Hafiz}'s view.
meillo@163 134
meillo@170 135 In contrast is \name{sendmail X}: Its locally submitted messages go to the \SMTP\ daemon, which is the only connection towards the mail queue. %fixme: is it a smtp dialog? or a second door?
meillo@170 136 \person{fanf} proposes a similar approach. He wants the \texttt{sendmail} command to be a simple \SMTP\ client that contacts the \SMTP\ daemon of the \MTA\ like it is done by connections from remote. The advantage here is one single module where all \SMTP\ dialog with submitters is done. Hence one single point to accept or refuse incoming mail. Additionally does the module to put mail into the queue not need to be \name{setuid} or \name{setgid} because it is only invoked from the \SMTP\ daemon. The \MTA's architecture would become simpler and common tasks are not duplicated in modules that do similar jobs.
meillo@163 137
meillo@170 138 But merging the input channels in the \SMTP\ daemon makes the \MTA\ heavily dependent on \SMTP\ being the main mail transfer protocol. To \qmail\ and \postfix\ new modules to support other ways of message receival may be added without change of other parts of the system. Also is it better to have more independent modules if each one is simpler then.
meillo@163 139
meillo@170 140 With the increasing need for new protocols in mind, it seems better to have single modules for each incoming channel, although this leads to duplicated acceptance checks.
meillo@163 141
meillo@163 142
meillo@163 143 \subsubsection*{Outgoing channels}
meillo@163 144
meillo@170 145 Outgoing mail is commonly either sent using \SMTP, piped into local commands (for example \texttt{uucp}), or delivered locally by appending to a mailbox.
meillo@170 146
meillo@163 147 Outgoing channels are similar for \qmail, \postfix, and \name{sendmail X}: All of them have a module to send mail using \SMTP, and one for writing into a local mailbox. Local mail delivery is a job that requires root priveledge to be able to switch to any user in order to write to his mailbox. Modular \MTA{}s do not need \name{setuid root}, but the local delivery process (or its parent) needs to run as root.
meillo@163 148
meillo@170 149 As mail delivery to local users, is \emph{not} included in the basic job of an \MTA{}, why should it care about it? In order to keep the system simple and to have programs that do one job well, the local delivery job should be handed over to a specialist: the \name{mail delivery agent}. \NAME{MDA}s know about the various mailbox formats and are aware of the problems of concurrent write access and thelike. Hence handling the message and the responsiblity over to a \NAME{MDA}, like \name{procmail} or \name{maildrop}, seems to be the right way to go.
meillo@163 150
meillo@170 151 This means an outgoing connection that pipes mail into local commands is required. Other outgoing channels, one for each supportet protocol, may be designed like it was done in other \MTA{}s.
meillo@170 152
meillo@170 153
meillo@170 154
meillo@170 155 \subsubsection*{Mail queue}
meillo@170 156
meillo@170 157 Mail queues are probably used in all \mta{}s, excluding the simple forwarders. A mail queue is a essential requirement for \masqmail, as it is to be used for non-permanent online connections. This means, mail must be queued until a online connection is available to send the message.
meillo@170 158
meillo@170 159 The mail queue and the module to manage it are the central part of the whole system. This demands especially for robustness and reliability, as a failure here can lead to loosing mail. An \MTA\ takes over responsibility for mail in accepting it, hence loosing mail messages is absolutely to avoid. This covers any kind of crash situation too. The worst thing acceptable to happen is a mail to be sent twice.
meillo@170 160
meillo@170 161 \sendmail, \exim, \qmail, \name{sendmail X}, and \masqmail\ feature one single mail queue. \postfix\ has three of them: \name{incoming}, \name{active}, and \name{deferred}. (The \name{maildrop} queue is excluded, as it is only used for the \texttt{sendmail} command.)
meillo@170 162
meillo@170 163 \MTA\ setups that include content scanning tend to require two separate queues. To use \sendmail\ in such setups requires two independent instances, with two separate queues, running. \exim\ can handle it with special \name{router} and \name{transport} rules, but the data flow gets complicated. Hence an idea is to use two queues, \name{incoming} and \name{active} in \postfix's terminology, with the content scanning within the move from \name{incoming} to \name{active}.
meillo@163 164
meillo@163 165
meillo@165 166
meillo@165 167 \subsubsection*{Sanitize mail}
meillo@165 168
meillo@170 169 Mail coming into the system often lacks important header lines. At least the required ones must be added from the \MTA. A good example is the \texttt{Message-Id:} header.
meillo@165 170
meillo@170 171 In \postfix, this is done by the \name{cleanup} module, which invokes \name{rewrite}. The position in the message flow is after coming from one of the several incoming channels and before the message is stored into the \name{incoming} queue. Modules that handle incoming channels may also add headers, for example the \texttt{From:} and \texttt{Date:} headers. \name{cleanup}, however, does a complete check to make the mail header complete and valid.
meillo@170 172
meillo@170 173 Apart from deciding where to sanitize the mail header, is the question where to generate the envelope. The envelope specifies the actual recipient of the mail, no matter what the \texttt{To:}, \texttt{Cc:}, and \texttt{Bcc:} headers tell. Multiple reciptients lead to multiple different envelopes, containing all the same mail message.
meillo@170 174
meillo@170 175
meillo@170 176
meillo@170 177 \subsubsection*{Choose route to use}
meillo@170 178
meillo@170 179 One key feature of \masqmail\ is its ability to send mail out in different ways. The decision is based on the current online state and whether a route may be used for a message or not. The online state can be retrieved in tree ways, explained in \ref{sec:fixme}. A route to send is found by checking every available route for being able to transfer the current message, until one matches.
meillo@170 180
meillo@170 181 This functionality should be implemented in the module that is responsible to invoke one of the outgoing channel modules (for example the one for \SMTP\ or the pipe module).
meillo@170 182
meillo@170 183 \masqmail\ can rewrite the envelope's from address and the \texttt{From:} header, dependent on the outgoing route to use. This rewrite must be done \emph{after} it is clear which route a mail will take, of course, so this may be not the module where other header editing is done.
meillo@165 184
meillo@165 185
meillo@165 186
meillo@165 187 \subsubsection*{Aliasing}
meillo@165 188
meillo@170 189 Where should aliases get expanded? They appear in different kind. Important are the ones available in the \path{aliases} file. Aliases can be:
meillo@170 190 \begin{itemize}
meillo@170 191 \item a different local user (e.g.\ ``\texttt{bob: alice}'')
meillo@170 192 \item a remote user (e.g.\ ``\texttt{bob: john@example.com}'')
meillo@170 193 \item a list of users (e.g.\ ``\texttt{bob: alice, john@example.com}'')
meillo@170 194 \item a command (e.g.\ ``\texttt{bob: |foo}'')
meillo@170 195 \end{itemize}
meillo@170 196 Aliases can be cascaded like in the following example:
meillo@170 197 \begin{verbatim}
meillo@170 198 team: alice, bob
meillo@170 199 bob: bob@example.com
meillo@170 200 \end{verbatim}
meillo@165 201
meillo@170 202 Addresses expanding to lists of users lead to more envelopes. Aliases changing the reciptients domain part may require a different route to use.
meillo@165 203
meillo@170 204 Aliasing is often handled in expanding the alias and reinjecting the mail into the system. Unfortunately, the mail is processed twice using this approach; also does the system need to handle more mail this way. Is it needed to check the new recipient address for acceptance, or should it be accepted generally? (The aliase actually came from inside the system.) A second point for access control seems to be no choice.
meillo@163 205
meillo@163 206
meillo@163 207
meillo@163 208 \subsubsection*{Authentication}
meillo@163 209
meillo@170 210 One thing to avoid is being an \name{open relay}. Open relays allow to relay mail from everywhere to everywhere. This is a major source of spam. The solution is restricting relay\footnote{Relaying is passing mail, that is not from and not for the own system, through it.} access.
meillo@163 211
meillo@170 212 Several ways to restrict access are available. The most simple one is restrictiction by the \NAME{IP} address. No extra complexity is added this way, but static \NAME{IP} addresses are mandatory. This kind of restriction may be enabled using the operating system's \path{hosts.allow} and \path{hosts.deny} files. To allow only connections to port 25 from localhost or the local network \texttt{192.168.100.0/24} insert the line ``\texttt{25: ALL}'' into \path{hosts.deny} and ``\texttt{25: 127.0.0.1, 192.168.100.}'' into \path{hosts.allow}.
meillo@170 213
meillo@170 214 If static access restriction is not possible, for example if mail from locations with changing \NAME{IP} addresses wants to be accepted, some kind of authentication mechanism is required. Three common kinds exist:
meillo@170 215 \begin{enumerate}
meillo@170 216 \item \SMTP-after-\NAME{POP}: uses authenication on the \NAME{POP} protocol to permit incoming \SMTP\ connections for a limited time afterwards.
meillo@170 217 \item \SMTP authentication: is an extension to \SMTP. Authentication can be requested before mail is accepted.
meillo@170 218 \item Certificates: confirm the identity of someone.
meillo@170 219 \end{enumerate}
meillo@170 220 The first mechanism requires a \NAME{POP} (or \NAME{IMAP}) server running on the same host (or a trusted one), to enable the \SMTP\ server to use the login dates on the \NAME{POP} server. This is a common practice used by mail service providers, but is not adequate for the environments \masqmail\ is designed for.
meillo@170 221
meillo@170 222 Certificate based authentication, like provided by \NAME{TLS}, suffers from the overhead of certificate management. But \NAME{TLS} provides encryption too, so is useful anyway.
meillo@170 223
meillo@170 224 \SMTP\ authentication (also refered to as \NAME{SMTP-AUTH}) suppoert is easiest received by using a \name{Simple Authentication and Security Layer} implementation. \person{Dent} sees in \NAME{SASL} the best solution for authenticating dynamic users:
meillo@165 225 \begin{quote}
meillo@170 226 %None of these add-ons is an ideal solution. They require additional code compiled into your existing daemons that may then require special write accesss to system files. They also require additional work for busy system administrators. If you cannot use any of the nonauthenticating alternatives mentioned earlier, or your business requirements demand that all of your users' mail pass through your system no matter where they are on the Internet, SASL is probably the solution that offers the most reliable and scalable method to authenticate users.
meillo@170 227 None of these [authentication methods] is an ideal solution. They require additional code compiled into your existing daemons that may then require special write accesss to system files. They also require additional work for busy system administrators. If you cannot use any of the nonauthenticating alternatives mentioned earlier, or your business requirements demand that all of your users' mail pass through your system no matter where they are on the Internet, \NAME{SASL} is probably the solution that offers the most reliable and scalable method to authenticate users.
meillo@165 228 \cite[page 44]{dent04}
meillo@165 229 \end{quote}
meillo@163 230
meillo@170 231 %either by
meillo@170 232 %- network/ip address
meillo@170 233 % easiest: restricting by static IP addresses (Access control via hosts.allow/hosts.deny)
meillo@170 234 %or
meillo@170 235 %- some kind of auth (for dynamic remote hosts)
meillo@170 236 % adds complexity
meillo@170 237 % - SASL
meillo@170 238 % - POP/IMAP: pop-before-smtp, DRAC, WHOSON
meillo@170 239 % - TLS (certificates)
meillo@170 240
meillo@170 241
meillo@163 242
meillo@163 243 \subsubsection*{Encryption}
meillo@163 244
meillo@165 245 TLS/SSL prevents attackers to listen on the cable
meillo@165 246 but it does not prevent man-in-the-middle attacks
meillo@165 247 signed certificates help here
meillo@165 248
meillo@165 249
meillo@165 250 ch /usr/share/ssl/misc
meillo@165 251
meillo@165 252 create new CA:
meillo@165 253 \begin{verbatim}
meillo@165 254 CA.pl -newca
meillo@165 255 country: DE
meillo@165 256 state: schwaben
meillo@165 257 city: Ulm
meillo@166 258 company:
meillo@165 259 section:
meillo@165 260 name:
meillo@165 261 emailaddress:
meillo@165 262 \end{verbatim}
meillo@165 263
meillo@165 264 generate ssl key:
meillo@165 265 \begin{verbatim}
meillo@165 266 CA.pl -newreq
meillo@165 267 ... the same questions
meillo@165 268 \end{verbatim}
meillo@165 269
meillo@165 270 sign request with CA:
meillo@165 271 \begin{verbatim}
meillo@165 272 CA.pl -sign
meillo@165 273 \end{verbatim}
meillo@165 274
meillo@165 275 remove passphrase from private key:
meillo@165 276 \begin{verbatim}
meillo@165 277 openssl rsa <newreq.pem >key.pem
meillo@165 278 (to be used by programs automaticly)
meillo@165 279 \end{verbatim}
meillo@165 280
meillo@165 281 secure:
meillo@165 282 \begin{verbatim}
meillo@165 283 chmod 400 *.pem
meillo@165 284 cp newcert.pem /etc/postfix/cert.pem
meillo@165 285 cp key.pem /etc/postfix/key.pem
meillo@165 286 cp demoCA/cacert.pem /etc/postfix/CAcert.pem
meillo@165 287 chmode 400 /etc/postfix/*.pem
meillo@165 288
meillo@165 289 mkdir /etc/stunnel
meillo@165 290 cat newcert.pem key.pem >/etc/stunnel/stunnel.pem
meillo@165 291 chmod 400 /etc/stunnel/stunnel.pem
meillo@165 292 (check /etc/stunnel with `stunnel -V')
meillo@165 293 \end{verbatim}
meillo@165 294
meillo@165 295
meillo@165 296 set up stunnels for POP, etc:
meillo@165 297 \begin{verbatim}
meillo@165 298 nmap localhost
meillo@165 299 stunnel -d pop3s -r localhost:pop3 -p /etc/stunnel/stunnel.pem
meillo@165 300 stunnel -d imaps -r localhost:imap -p /etc/stunnel/stunnel.pem
meillo@165 301 nmap localhost
meillo@165 302 pop3s 995
meillo@165 303 imaps 993
meillo@165 304 \end{verbatim}
meillo@165 305
meillo@165 306 do not use stunnel wit SMTP:
meillo@165 307 because all incoming mail would be from 127.0.0.1 !!
meillo@165 308 use STARTTLS instead
meillo@165 309
meillo@165 310 postfix: main.cf
meillo@165 311 \begin{verbatim}
meillo@165 312 smtpd_use_tls = yes
meillo@165 313 smtpd_tls_received_header = no (does not log in received headers)
meillo@165 314
meillo@165 315 smtpd_tls_key_file = /etc/postfix/key.pem
meillo@165 316 smtpd_tls_cert_file = /etc/postfix/cert.pem
meillo@165 317 smtpd_tls_CA_file = /etc/postfix/CAcert.pem
meillo@165 318
meillo@165 319 smtp_use_tls = yes (use TLS for sending)
meillo@165 320 smtp_tls_key_file = /etc/postfix/key.pem
meillo@165 321 smtp_tls_cert_file = /etc/postfix/cert.pem
meillo@165 322 smtp_tls_CA_file = /etc/postfix/CAcert.pem
meillo@165 323 \end{verbatim}
meillo@165 324
meillo@165 325
meillo@165 326
meillo@163 327
meillo@163 328 \subsubsection*{Spam prevention}
meillo@163 329
meillo@163 330
meillo@163 331 where to filter what
meillo@163 332
meillo@163 333
meillo@165 334 postfix:
meillo@165 335 content-filter: arbitrary programs that talk smtp, can filter, rewrite or delete mail
meillo@165 336 - before-queue-c-f: need to be fast, can prevent system load
meillo@165 337 - after-queue-c-f: need more resources in global, more load
meillo@165 338
meillo@165 339 exim:
meillo@165 340 acls: to filter, what to accept (hook into smtp dialog) (complex)
meillo@165 341 routers: take recipient address and choose a matching transport
meillo@165 342 transports: ways to deliver mail (smtp, local)
meillo@165 343
meillo@165 344
meillo@163 345 postfix: after-queue-content-filter (smtp communication)
meillo@165 346 exim: content-scan-feature (analyses the content: MIME stuff, blacklisted words, virus scanning) (all within smtp dialog)
meillo@163 347 sendmail: milter (tcp or unix sockets)
meillo@163 348
meillo@163 349 checks while smtp dialog (pre-queue): in MTA implemented (need to be fast)
meillo@163 350 checks when mail is accepted and queued: external (amavis, spamassassin)
meillo@163 351
meillo@163 352
meillo@163 353
meillo@165 354
meillo@165 355
meillo@165 356
meillo@165 357 what do do with recognized mail?
meillo@165 358 - reject (only possible if recognized during SMTP dialog)
meillo@165 359 - forward with added header line or changed subject
meillo@165 360 (eisentraut05: page 18--20)
meillo@165 361
meillo@165 362 check incoming and outgoing mail
meillo@165 363 (eisentraut05: page 21)
meillo@165 364
meillo@165 365
meillo@165 366 milter:
meillo@165 367 communication with external daemons via a special protocol
meillo@165 368 at various times in the smtp dialog possible
meillo@165 369 can reject, delete or alter messages
meillo@165 370 http://milter.org
meillo@165 371 (eisentraut05: page 69)
meillo@165 372
meillo@165 373
meillo@165 374 use SA with exim:
meillo@165 375 - with transport: piped into sa
meillo@165 376 - content-scanning-feature: with ACL during smtp dialog
meillo@165 377 - plugin: sa-exim
meillo@165 378 - within amavis
meillo@165 379
meillo@165 380 use SA with sendmail:
meillo@165 381 - with milter
meillo@165 382 - within mimedefang or amavis
meillo@165 383
meillo@165 384 use SA with postfix:
meillo@165 385 - within amavis or mailfilter
meillo@165 386
meillo@165 387
meillo@165 388 (eisentraut05: page 25) ``Ganz ohne Analyse während der SMTP-Phase kommt sowieso kein MTA aus, und es ist eine Frage der Einschätzung, wie weit man diese Phase belasten möchte.''
meillo@165 389
meillo@165 390
meillo@165 391 DNSBL can contain:
meillo@165 392 - open relays
meillo@165 393 - dynamic IP addresses
meillo@165 394 - verified spam sources
meillo@165 395 - open multistage relays
meillo@165 396 - vulnerable CGI scripts
meillo@165 397 - open proxy servers
meillo@165 398 example: NJABL (http://njabl.org)
meillo@165 399
meillo@165 400 DNSBL in smpt dialog is aggressive and can lead to problems (eisentraut05: page 126)
meillo@165 401
meillo@165 402
meillo@165 403 greylisting:
meillo@165 404 if first contact from that address: temp failure and add to list
meillo@165 405 sender will retry, then accept
meillo@165 406
meillo@165 407 ``Das Greylisting zählt derzeit zu den effektivsten Methoden, um gegen unerwünschte E-Mails vorzugehen. Allein durch Greylisting können derzeit rund 70\% des potenziellen Spam-Aufkommens auf einem Mailserver vollständig geblockt werden. Allerdings ist es auch nur eine Frage der Zeit, bis sich die Gemeinde der Spammer und Virenautoren auf diese Methode der Spam-Bekämpfung eingerichtet und entsprechende Queues in ihre Software eingebaut hat.''(eisentraut05: page 138)
meillo@165 408 Probleme: load balancing using multiple servers with different IPs.
meillo@165 409 postfix: with policy server
meillo@165 410 exim: direct in config
meillo@165 411 sendmail: with greylist milter
meillo@165 412
meillo@165 413
meillo@165 414
meillo@165 415 hashcash
meillo@163 416
meillo@163 417
meillo@163 418 \subsubsection*{Virus checking}
meillo@163 419
meillo@163 420 The same for malicious content (\name{malware}) like viruses, worms, trojan horses. They are related to spam, but affect the \MTA less, as they are in the mail body.
meillo@163 421
meillo@163 422 message body <-> envelope, header
meillo@163 423
meillo@163 424
meillo@163 425 anti-virus: clamav
meillo@165 426 postfix: via amavis
meillo@165 427 exim: via content-scanning-feature called from acl
meillo@165 428 sendmail: with milter
meillo@165 429 procmail
meillo@163 430
meillo@163 431
meillo@165 432 virus scanner work on file level
meillo@165 433 amavis receives mail via smtp or pipe, splits it in its parts (MIME) and extracks archives, the come the virus scanners
meillo@165 434 if the mail is okay, it goes via smtp to a second mta
meillo@165 435
meillo@165 436
meillo@165 437 AMaViS (amavisd-new): email filter framework to integrate spam and virus scanner
meillo@165 438 \begin{verbatim}
meillo@165 439 internet -->25 MTA -->10024 amavis -->10025 MTA --> reciptient
meillo@165 440 | |
meillo@165 441 +----------------------------+
meillo@165 442 \end{verbatim}
meillo@165 443
meillo@165 444 postfix and exim can habe both mta servises in the same instance, sendmail needs two instances running.
meillo@165 445
meillo@165 446 what amavis recognizes:
meillo@165 447 - invalid headers
meillo@165 448 - banned files
meillo@165 449 - viruses
meillo@165 450 - spam (using spam assassin)
meillo@165 451
meillo@165 452
meillo@165 453 mimedefang: uses milter interface with sendmail
meillo@165 454
meillo@165 455
meillo@165 456 MailScanner:
meillo@165 457 incoming queue --> MailScanner --> outgoing queue
meillo@165 458
meillo@165 459 postfix: with one instance possible, exim and sendmail need two instances running
meillo@165 460
meillo@163 461
meillo@163 462
meillo@163 463
meillo@163 464 \subsubsection*{Archiving}
meillo@163 465
meillo@163 466
meillo@165 467 \texttt{always\_bcc} feature of postfix
meillo@163 468
meillo@163 469
meillo@163 470
meillo@163 471 \section{A new architecture}
meillo@161 472
meillo@161 473
meillo@161 474 (ssl)
meillo@161 475 -> msg-in (local or remote protocol handlers)
meillo@161 476 -> spam-filter (and more)
meillo@161 477 -> queue
meillo@161 478 -> msg-out (local-delivery by MDA, or remote-protocol-handlers)
meillo@161 479 (ssl)
meillo@161 480
meillo@161 481
meillo@161 482
meillo@161 483
meillo@161 484
meillo@161 485 http://fanf.livejournal.com/50917.html %how not to design an mta - the sendmail command
meillo@161 486 http://fanf.livejournal.com/51349.html %how not to design an mta - partitioning for security
meillo@161 487 http://fanf.livejournal.com/61132.html %how not to design an mta - local delivery
meillo@161 488 http://fanf.livejournal.com/64941.html %how not to design an mta - spool file format
meillo@161 489 http://fanf.livejournal.com/65203.html %how not to design an mta - spool file logistics
meillo@161 490 http://fanf.livejournal.com/65911.html %how not to design an mta - more about log-structured MTA queues
meillo@161 491 http://fanf.livejournal.com/67297.html %how not to design an mta - more log-structured MTA queues
meillo@161 492 http://fanf.livejournal.com/70432.html %how not to design an mta - address verification
meillo@161 493 http://fanf.livejournal.com/72258.html %how not to design an mta - content scanning
meillo@161 494
meillo@161 495
meillo@161 496
meillo@132 497
meillo@132 498
meillo@137 499
meillo@137 500
meillo@149 501
meillo@149 502
meillo@149 503
meillo@149 504
meillo@149 505
meillo@149 506
meillo@149 507
meillo@149 508
meillo@93 509
meillo@93 510
meillo@99 511
meillo@93 512
meillo@93 513
meillo@161 514 \section{Directions to go}
meillo@161 515
meillo@161 516 This section discusses about what shapes \masqmail\ could have---which directions the development could go to.
meillo@161 517
meillo@93 518
meillo@146 519
meillo@146 520
meillo@146 521
meillo@146 522 \subsubsection*{\masqmail\ in five years}
meillo@146 523
meillo@146 524 Now how could \masqmail\ be like in, say, five years?
meillo@146 525
meillo@163 526 ---
meillo@163 527
meillo@163 528 A design from scratch?
meillo@163 529 << what would be needed (effort) >>
meillo@163 530 But how is the effort of this complete rewrite compared to what is gained afterwards?
meillo@163 531
meillo@163 532 << would one create it at all? >>
meillo@163 533
meillo@163 534 ---
meillo@163 535
meillo@146 536 << plans to get masqmail more popular again (if that is the goal) >>
meillo@146 537
meillo@146 538 << More users >>
meillo@146 539
meillo@146 540
meillo@146 541
meillo@146 542
meillo@163 543
meillo@163 544
meillo@163 545
meillo@93 546 \section{Work to do}
meillo@93 547
meillo@146 548 << short term goals --- long term goals >>
meillo@146 549
meillo@163 550 do it like sendmail: first do the most needed stuff on the old design to make it still usable. Then design a new version from scratch, for the future.
meillo@163 551
meillo@140 552 << which parts to take out and do within the thesis >>
meillo@93 553
meillo@167 554