annotate thesis/tex/4-MasqmailsFuture.tex @ 163:5681a18270b5

new content about architecture; some restructuring
author meillo@marmaro.de
date Thu, 18 Dec 2008 13:39:23 +0100
parents 18b7b517e2dd
children a7fd6d974d3c
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
109
de590ff06051 changes suggested by schaeffter
meillo@marmaro.de
parents: 99
diff changeset
1 \chapter{\masqmail's present and future}
93
a6f8a93abd64 new chapter 4 (split ch5); added lots of annotations
meillo@marmaro.de
parents:
diff changeset
2
137
c60b164bfd3c rearranging of ch04
meillo@marmaro.de
parents: 133
diff changeset
3 \section{Existing code base}
142
1b0ba5151d1b person names in small caps
meillo@marmaro.de
parents: 140
diff changeset
4 Here regarded is version 0.2.21 of \masqmail. This is the last version released by Oliver \person{Kurth}, and the basis for my thesis.
1b0ba5151d1b person names in small caps
meillo@marmaro.de
parents: 140
diff changeset
5
132
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
6
137
c60b164bfd3c rearranging of ch04
meillo@marmaro.de
parents: 133
diff changeset
7 \subsubsection*{Features}
c60b164bfd3c rearranging of ch04
meillo@marmaro.de
parents: 133
diff changeset
8
142
1b0ba5151d1b person names in small caps
meillo@marmaro.de
parents: 140
diff changeset
9 \masqmail\ accepts mail on the command line and via \SMTP. Mail queueing and alias expansion is supported. \masqmail\ is able to deliver mail to local mailboxes (in \name{mbox} or \name{maildir} format) or pass it to a \name{mail delivery agent} (like \name{procmail}). Mail destinated to remote locations is sent using \SMTP\ or can be piped to commands, being gatesways to \NAME{UUCP} or \NAME{FAX} for example.
132
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
10
142
1b0ba5151d1b person names in small caps
meillo@marmaro.de
parents: 140
diff changeset
11 Outgoing \SMTP\ connections feature \SMTP-\NAME{AUTH} and \SMTP-after-\NAME{POP} authentication, but incoming connections do not. Using wrappers for outgoing connections is supported. This offers a two way communication through a wrapper application like \name{openssl}.
137
c60b164bfd3c rearranging of ch04
meillo@marmaro.de
parents: 133
diff changeset
12 %todo: what about SSL/TLS encryption?
132
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
13
142
1b0ba5151d1b person names in small caps
meillo@marmaro.de
parents: 140
diff changeset
14 \masqmail\ focuses on non-permanent online connections, thus a concept of online routes is used. One may configure any amount of routes to send mail. Each route can have criterias, like matching \texttt{From:} or \texttt{To:} headers, to determine if mail is allowed to be sent using it. Mail to destinations outside the local net gets queued until \masqmail\ is informed about the existance of a online connection.
1b0ba5151d1b person names in small caps
meillo@marmaro.de
parents: 140
diff changeset
15
137
c60b164bfd3c rearranging of ch04
meillo@marmaro.de
parents: 133
diff changeset
16 The \masqmail\ executable can be called under various names for sendmail-compatibility reasons. This is organized by symbolic links with different names pointing to the \masqmail\ executable. The \sendmail\ names are \path{/usr/lib/sendmail} and \path{/usr/sbin/sendmail} because many programs expect the \mta\ to be located there. Further more \sendmail\ supports calling it with a different name instead of supplying command line arguments. The best known of this shortcuts is \path{mailq}, which is equivilent to calling it with the argument \verb+-bq+. \masqmail\ recognizes the names \path{mailq}, \path{smtpd}, \path{mailrm}, \path{runq}, \path{rmail}, and \path{in.smtpd}. The first two are inspired by \sendmail. Not implemented is the name \path{newaliases} because \masqmail\ does not generate binary representations of the alias file.\footnote{A shell script located named \path{newaliases}, that invokes \texttt{masqmail -bi}, can provide the command to satisfy other software needing it.} \path{hoststat} and \path{purgestat} are missing for sendmail-compatibility.
c60b164bfd3c rearranging of ch04
meillo@marmaro.de
parents: 133
diff changeset
17 %masqmail: mailq, mailrm, runq, rmail, smtpd/in.smtpd
c60b164bfd3c rearranging of ch04
meillo@marmaro.de
parents: 133
diff changeset
18 %sendmail: hoststat, mailq, newaliases, purgestat, smtpd
132
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
19
137
c60b164bfd3c rearranging of ch04
meillo@marmaro.de
parents: 133
diff changeset
20 Additional to the \mta\ job, \masqmail\ also offers mail retrieval services with being a \NAME{POP3} client. It can fetch mail from different remote locations, dependent on the active online route.
132
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
21
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
22
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
23
137
c60b164bfd3c rearranging of ch04
meillo@marmaro.de
parents: 133
diff changeset
24 \subsubsection*{The code}
132
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
25
137
c60b164bfd3c rearranging of ch04
meillo@marmaro.de
parents: 133
diff changeset
26 \masqmail\ is written in the C programming language. The program, as of version 0.2.21, consists of 34 source code and eight header files, containing about 9,000 lines of code\footnote{Measured with \name{sloccount} by David A.\ Wheeler.}. Additionally, it includes a \name{base64} implementation (about 300 lines) and \name{md5} code (about 150 lines). For systems that do not provide \name{libident}, this library is distributed as well (circa 600 lines); an available shared library however has higher precedence in linking.
132
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
27
137
c60b164bfd3c rearranging of ch04
meillo@marmaro.de
parents: 133
diff changeset
28 The only mandatory dependency is \name{glib}---a cross-platform software utility library, originated in the \NAME{GTK+} project. It provides safer replacements for many standard library functions. It also offers handy data containers, easy-to-use implementations of data structures, and much more.
132
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
29
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
30
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
31 With \masqmail\ comes the small tool \path{mservdetect}; it helps setting up a configuration that uses the \name{mserver} system to detect the online state. Two other binaries get compiled for testing purposes: \path{readtest} and \path{smtpsend}. All three programms use \masqmail\ source code; they only add a file with a \verb+main()+ function each.
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
32
137
c60b164bfd3c rearranging of ch04
meillo@marmaro.de
parents: 133
diff changeset
33
c60b164bfd3c rearranging of ch04
meillo@marmaro.de
parents: 133
diff changeset
34 \masqmail\ does not provide an interface to plug in modules with additional functionality. There exists no add-on or module system. The code is only separated by function to the various source files. Some functional parts can be included or excluded by defining symbols. Adding maildir support at compile time, means giving the option \verb+--enable-maildir+ to the \path{configure} call. This preserves the concerning code to get removed by the preprocessor. Unfortunately the \verb+#ifdef+s are scattered through all the source, leading to a FIXME(holperig) code base.
132
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
35
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
36
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
37
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
38
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
39
146
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
40
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
41 \section{Requirements}
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
42
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
43 Following is a list of current and future requirements to make \masqmail\ ready for the future.
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
44
137
c60b164bfd3c rearranging of ch04
meillo@marmaro.de
parents: 133
diff changeset
45
146
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
46 \subsubsection*{Large message handling}
155
0b17f6e5edae new books; websites to books
meillo@marmaro.de
parents: 149
diff changeset
47 Trends in the market for electronic communication go towards consolidated communication, hence email will be used more to transfer voice and video messages. This leads to larger messages. The store-and-forward transport of email is not good suited for large data. Thus new protocols, like \NAME{QMTP} (described in section %\ref{FIXME}
0b17f6e5edae new books; websites to books
meillo@marmaro.de
parents: 149
diff changeset
48 ), may become popular.
146
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
49
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
50
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
51 \subsubsection*{Ressource friendly software}
149
ccf0de1ae337 new content and rework
meillo@marmaro.de
parents: 146
diff changeset
52 The merge of communication hardware and the move of email services from providers to homes, demands smaller and more resource-friendly software. The amount of mail will be lower, even if much more mail will be sent. More important will be the energy consumption and heat emission. These topics increased in relevance during the past years and they are expected to become more central. \masqmail\ is not a program to be used on large servers, but to be used on small devices. Thus focusing on energy and heat, not on performance, is the direction to go.
132
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
53
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
54
146
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
55 \subsubsection*{New mail transfer protocols}
149
ccf0de1ae337 new content and rework
meillo@marmaro.de
parents: 146
diff changeset
56 Large messages demand more efficient transport through the net. As well is a final solution needed to defeat the spam problem. New mail transport protocols may be the only good solutions for both problems. They also can improve reliability, authentication, and verification issues. \masqmail\ should be able to support new protocols as they appear and are used.
137
c60b164bfd3c rearranging of ch04
meillo@marmaro.de
parents: 133
diff changeset
57
c60b164bfd3c rearranging of ch04
meillo@marmaro.de
parents: 133
diff changeset
58
149
ccf0de1ae337 new content and rework
meillo@marmaro.de
parents: 146
diff changeset
59 \subsubsection*{Spam handling}
ccf0de1ae337 new content and rework
meillo@marmaro.de
parents: 146
diff changeset
60 Spam is a major threat. According to the \NAME{SWOT} analysis, the goal is to reduce it to a bearable level. Spam fighting is a war are where the good guys tend to lose. Putting too much effort there will result in few gain. Real success will only be possible with new---better---protocols and abandonning the weak legacy technologies. Hence \masqmail\ should be able to provide state-of-the-art spam protection, but not more.
137
c60b164bfd3c rearranging of ch04
meillo@marmaro.de
parents: 133
diff changeset
61
c60b164bfd3c rearranging of ch04
meillo@marmaro.de
parents: 133
diff changeset
62
161
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
63 \subsubsection*{Security}
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
64 \MTA{}s are critical points for computer security, as they are accessable from external networks. They must be secured with high effort. Properties like high priviledge level, work load influenced from extern, work on unsafe data, and demand for reliability, increase the security needed. Unsecure and unreliable \mta{}s are of no value. \masqmail\ needs to b e secure enough for its target field of operation.
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
65
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
66
146
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
67 \subsubsection*{Easy configuration}
149
ccf0de1ae337 new content and rework
meillo@marmaro.de
parents: 146
diff changeset
68 Having \mta{}s on many home servers and clients, requires easy and standardized configuration. The common setups should be configurable with single actions by the user. Complex configuration should be possible, but focused must be the most common form of configuration: choosing one of several standard setups.
146
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
69
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
70
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
71
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
72
137
c60b164bfd3c rearranging of ch04
meillo@marmaro.de
parents: 133
diff changeset
73
146
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
74
161
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
75 \section{Discussion on architecture}
146
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
76
163
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
77 A program's architecture is probably the most influencing design decision, and has the greatest impact on the program's future capabilities. %fixme: search quote ... check if good
161
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
78
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
79 \masqmail's current artitecture is monolitic like \sendmail's and \exim's. But more than the other two, is it one block of interweaved code. \sendmail\ provides now, with its \name{milter} interface, standardized connection channels to external modules. \exim\ has a highly structured code with many internal interfaces, like the one for supported authentication ``modules''. \masqmail\ has none of them; it is what \sendmail\ was in the beginning: a single large block.
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
80
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
81 Figure \ref{fig:masqmail-arch} is an attempt to depict \masqmail's internal structure.
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
82
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
83 \begin{figure}
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
84 \begin{center}
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
85 \input{input/masqmail-arch.tex}
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
86 \end{center}
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
87 \caption{Internal architecture of \masqmail}
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
88 \label{fig:masqmail-arch}
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
89 \end{figure}
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
90
163
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
91 \sendmail\ improved its old architecture, for example by adding the milter interface. \exim\ was designed and is carefully maintained with a modular-like code structure in mind. \qmail\ started from scratch with a ``security-first'' approach, \postfix\ improved on it, and \name{sendmail X}/\name{MeTA1} tries to adopt the best of \qmail\ and \postfix, to completely replace the old \sendmail\ architecture. \person{Hafiz} \cite{hafiz05}. describes this evolution of \mta\ architecture very well.
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
92
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
93 Every one of the popular \MTA{}s is more modular, or became more modular over time, than \masqmail\ is. Modern requirements like spam protection and future requirements like the use of new mail transport protocols demand modular designs for keeping the software simple. Simplicity is a key property for security.
161
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
94
163
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
95 \person{Hafiz} agrees:
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
96 \begin{quote}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
97 The goal of making software secure can be better achieved by making the design simple and easier to understand and verify. \cite[page64]{hafiz05}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
98 \end{quote}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
99 He identifies the security of \qmail\ to come from it's \name{compartmentalization}, which goes hand in hand with modularity:
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
100 \begin{quote}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
101 A perfect example is the contrast between the feature envy early \sendmail\ architecture implemented as one process and the simple, modular architecture of \qmail. The security of \qmail\ comes from its compartmentalized simple processes that perform one task only and are therefor testable for security. \cite[page 64]{hafiz05}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
102 \end{quote}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
103
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
104 Modularity is needed for supporting modern \MTA\ requirements, providing a clear interface to add further functionality without increasing the overall complexity much. Modularity is also an enabler for security. Security comes from good design, as \person{Graff} and \person{van Wyk} explain:
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
105 \begin{quote}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
106 Good design is the sword and shield of the security-conscious developer. Sound design defends your application from subversion or misuse, protecting your network and the information on it from internal and external attacks alike. It also provides a safe foundation for future extensions and maintainance of the software.
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
107 %
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
108 %Bad design makes life easier for attackers and harder for the good guys, especially if it contributes to a false sends of security while obscuring pertinent failings.
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
109 \cite[page 55]{graff03}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
110 \end{quote}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
111
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
112 \person{Hafiz} adds: ``The major idea is that security cannot be retrofitted into an architecture.''\cite[page 64]{hafiz05}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
113
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
114 All this leads to one logical step: The rewrite of \masqmail\ using a modern, modular architecture, to get a modern \MTA\ satisfying nowadays needs.
161
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
115
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
116
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
117
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
118
163
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
119 \subsection{Modules needed}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
120
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
121 This section tries to identify the needed modules for a modern \MTA. They are later the pieces of which the new architecture is built of.
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
122
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
123
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
124 \subsubsection*{The simplest MTA}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
125 This view of the problem is taken from \person{Hafiz} \cite[pages 3-5]{hafiz05}.
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
126
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
127 The basic job of a \mta\ is to tranport mail from a sender to a recipient. The simplest \MTA\ therefor needs at least a mail receiving facility and a mail sending facility. This basic \MTA---following the definition of an \MTA---is much to abstract. Hence a next step to add some important features is needed, the result is an operational \MTA.
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
128
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
129
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
130
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
131 \subsubsection*{Mail queue}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
132
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
133 \person{Hafif} adds a mail queue to make it possible to not deliver at once.
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
134
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
135 Mail queues are probably used in all \mta{}s, excluding the simple forwarders. A mail queue is a essential requirement for \masqmail, as it is to be used for non-permanent online connections.
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
136
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
137
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
138 \subsubsection*{Incoming channels}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
139
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
140 The second addition \person{Hafiz} made is the split of incoming and outgoing channels into local and remote. The question is, if this is nessesary. It is the way, it was done for a long time, but is this extra complexity needed?
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
141
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
142 The common situation is incoming mail on port 25 using \SMTP\ and via the \texttt{sendmail} command. Outgoing mail is either sent using \SMTP, piped into local commands (for example \texttt{uucp}), or delivered locally by appending to a mailbox.
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
143
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
144 The \MTA's architecture would be simpler if some of these channels could be merged. The reason is, if various modules do similar jobs, common things might need to be duplicated. On the other side is it better to have more independent modules if each one is simpler then.
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
145
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
146 \qmail\ uses \name{qmail-inject} (local message in) and \name{qmail-smtpd} (remote message in), which both handle messages over to \name{qmail-queue} that puts it into the mail queue. \postfix's approach is similar. \name{sendmail X} %fixme: what about meta1 here?
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
147 used only \NAME{SMTPS}, which is for receiving mail from remote, to communicate with the queue manager \NAME{QMGR}. Mail from local goes over \NAME{SMTPS}.
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
148
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
149 The \name{sendmail X} approach seems to be the simpler one, but does heavily rely on \SMTP\ being the main mail transfer protocol. To \qmail\ and \postfix\ new modules may be added to support other ways of message receival, without any change of other parts of the system.
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
150
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
151
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
152 \subsubsection*{Outgoing channels}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
153
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
154 Outgoing channels are similar for \qmail, \postfix, and \name{sendmail X}: All of them have a module to send mail using \SMTP, and one for writing into a local mailbox. Local mail delivery is a job that requires root priveledge to be able to switch to any user in order to write to his mailbox. Modular \MTA{}s do not need \name{setuid root}, but the local delivery process (or its parent) needs to run as root.
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
155
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
156 As mail delivery to local users, is \emph{not} included in the basic job of \MTA{}s, why should they care about it? In order to keep the system simple and to have programs do one job well, the local delivery job should be handed over to \NAME{MDA}s. \name{Mail delivery agents} are the tools that are specialized for local delivery. They know about the various mailbox formats and are aware of the problems of concurrent write access and thelike. Hence handling the message and the responsiblity for it over to a mail delivery agent, like \name{procmail} or \name{maildrop}, seems to be the right way to go.
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
157
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
158 This means outgoing connections, piping mails into local commands needs to be implemented.
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
159
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
160
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
161 \subsubsection*{Mail queue (again)}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
162
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
163
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
164
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
165
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
166 \subsubsection*{Authentication}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
167
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
168 easiest: restricting by static IP addresses (Access control via hosts.allow/hosts.deny)
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
169 if dynamic remote hosts need access: some auth is needed
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
170 - SASL
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
171 - POP/IMAP: pop-before-smtp, DRAC, WHOSON
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
172 - TLS (certificates)
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
173
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
174 ``None of these add-ons is an ideal solution. They require additional code compiled into your existing daemons that may then require special write accesss to system files. They also require additional work for busy system administrators. If you cannot use any of the nonauthenticating alternatives mentioned earlier, or your business requirements demand that all of thyour users' mail pass through your system no matter where they are on the Internet, SASL is probably the solution that offers the most reliable and scalable method to authenticate users.'' (Dent: Postfix, page 44, ch04)
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
175
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
176
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
177 \subsubsection*{Encryption}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
178
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
179
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
180 \subsubsection*{Spam prevention}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
181
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
182
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
183 where to filter what
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
184
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
185
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
186 postfix: after-queue-content-filter (smtp communication)
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
187 exim: content-scan-feature
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
188 sendmail: milter (tcp or unix sockets)
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
189
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
190 checks while smtp dialog (pre-queue): in MTA implemented (need to be fast)
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
191 checks when mail is accepted and queued: external (amavis, spamassassin)
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
192
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
193
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
194 AMaViS (amavisd-new): email filter framework to integrate spam and virus scanner
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
195 internet -->25 MTA -->10024 amavis -->10025 MTA --> reciptient
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
196 | |
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
197 +----------------------------+
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
198 mail scanner:
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
199 incoming queue --> mail scanner --> outgoing queue
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
200
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
201 mimedefang: uses milter interface with sendmail
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
202
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
203
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
204 \subsubsection*{Virus checking}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
205
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
206 The same for malicious content (\name{malware}) like viruses, worms, trojan horses. They are related to spam, but affect the \MTA less, as they are in the mail body.
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
207
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
208 message body <-> envelope, header
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
209
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
210
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
211 anti-virus: clamav
161
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
212
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
213
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
214
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
215
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
216
163
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
217 \subsubsection*{Archiving}
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
218
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
219
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
220
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
221
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
222
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
223 \section{A new architecture}
161
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
224
146
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
225
161
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
226 (ssl)
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
227 -> msg-in (local or remote protocol handlers)
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
228 -> spam-filter (and more)
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
229 -> queue
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
230 -> msg-out (local-delivery by MDA, or remote-protocol-handlers)
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
231 (ssl)
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
232
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
233
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
234
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
235
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
236
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
237 http://fanf.livejournal.com/50917.html %how not to design an mta - the sendmail command
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
238 http://fanf.livejournal.com/51349.html %how not to design an mta - partitioning for security
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
239 http://fanf.livejournal.com/61132.html %how not to design an mta - local delivery
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
240 http://fanf.livejournal.com/64941.html %how not to design an mta - spool file format
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
241 http://fanf.livejournal.com/65203.html %how not to design an mta - spool file logistics
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
242 http://fanf.livejournal.com/65911.html %how not to design an mta - more about log-structured MTA queues
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
243 http://fanf.livejournal.com/67297.html %how not to design an mta - more log-structured MTA queues
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
244 http://fanf.livejournal.com/70432.html %how not to design an mta - address verification
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
245 http://fanf.livejournal.com/72258.html %how not to design an mta - content scanning
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
246
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
247
149
ccf0de1ae337 new content and rework
meillo@marmaro.de
parents: 146
diff changeset
248
93
a6f8a93abd64 new chapter 4 (split ch5); added lots of annotations
meillo@marmaro.de
parents:
diff changeset
249
132
a83a29e10b10 new books
meillo@marmaro.de
parents: 129
diff changeset
250
93
a6f8a93abd64 new chapter 4 (split ch5); added lots of annotations
meillo@marmaro.de
parents:
diff changeset
251
a6f8a93abd64 new chapter 4 (split ch5); added lots of annotations
meillo@marmaro.de
parents:
diff changeset
252
a6f8a93abd64 new chapter 4 (split ch5); added lots of annotations
meillo@marmaro.de
parents:
diff changeset
253
109
de590ff06051 changes suggested by schaeffter
meillo@marmaro.de
parents: 99
diff changeset
254
149
ccf0de1ae337 new content and rework
meillo@marmaro.de
parents: 146
diff changeset
255
ccf0de1ae337 new content and rework
meillo@marmaro.de
parents: 146
diff changeset
256
ccf0de1ae337 new content and rework
meillo@marmaro.de
parents: 146
diff changeset
257
163
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
258
149
ccf0de1ae337 new content and rework
meillo@marmaro.de
parents: 146
diff changeset
259
ccf0de1ae337 new content and rework
meillo@marmaro.de
parents: 146
diff changeset
260
93
a6f8a93abd64 new chapter 4 (split ch5); added lots of annotations
meillo@marmaro.de
parents:
diff changeset
261
a6f8a93abd64 new chapter 4 (split ch5); added lots of annotations
meillo@marmaro.de
parents:
diff changeset
262
a6f8a93abd64 new chapter 4 (split ch5); added lots of annotations
meillo@marmaro.de
parents:
diff changeset
263
a6f8a93abd64 new chapter 4 (split ch5); added lots of annotations
meillo@marmaro.de
parents:
diff changeset
264
99
d24fdd3d5990 added lots of comments and annotations about what to do
meillo@marmaro.de
parents: 93
diff changeset
265
161
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
266 \section{Directions to go}
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
267
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
268 This section discusses about what shapes \masqmail\ could have---which directions the development could go to.
18b7b517e2dd wrote about discussion on architecture
meillo@marmaro.de
parents: 155
diff changeset
269
93
a6f8a93abd64 new chapter 4 (split ch5); added lots of annotations
meillo@marmaro.de
parents:
diff changeset
270
a6f8a93abd64 new chapter 4 (split ch5); added lots of annotations
meillo@marmaro.de
parents:
diff changeset
271
a6f8a93abd64 new chapter 4 (split ch5); added lots of annotations
meillo@marmaro.de
parents:
diff changeset
272
146
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
273
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
274 \subsubsection*{\masqmail\ in five years}
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
275
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
276 Now how could \masqmail\ be like in, say, five years?
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
277
163
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
278 ---
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
279
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
280 A design from scratch?
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
281 << what would be needed (effort) >>
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
282 But how is the effort of this complete rewrite compared to what is gained afterwards?
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
283
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
284 << would one create it at all? >>
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
285
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
286 ---
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
287
146
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
288 << plans to get masqmail more popular again (if that is the goal) >>
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
289
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
290 << More users >>
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
291
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
292
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
293
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
294
163
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
295
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
296
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
297
93
a6f8a93abd64 new chapter 4 (split ch5); added lots of annotations
meillo@marmaro.de
parents:
diff changeset
298 \section{Work to do}
a6f8a93abd64 new chapter 4 (split ch5); added lots of annotations
meillo@marmaro.de
parents:
diff changeset
299
146
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
300 << short term goals --- long term goals >>
2c4673d983c3 wrote about requirements (related to directions to go)
meillo@marmaro.de
parents: 142
diff changeset
301
163
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
302 do it like sendmail: first do the most needed stuff on the old design to make it still usable. Then design a new version from scratch, for the future.
5681a18270b5 new content about architecture; some restructuring
meillo@marmaro.de
parents: 161
diff changeset
303
140
002fd18820cc small changes
meillo@marmaro.de
parents: 137
diff changeset
304 << which parts to take out and do within the thesis >>
93
a6f8a93abd64 new chapter 4 (split ch5); added lots of annotations
meillo@marmaro.de
parents:
diff changeset
305