Mercurial > docs > diploma

comparison thesis/tex/4-MasqmailsFuture.tex @ 287:6cf649e62d42

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
minor renames and commenting
author meillo@marmaro.de
date Fri, 16 Jan 2009 10:35:48 +0100
parents 391793afb4cb
children 8341092a7554
comparison
equal deleted inserted replaced
286:980eb42256ff 287:6cf649e62d42
63 63
64 64
65 65
66 66
67 \paragraph{\RF2: Mail queuing} 67 \paragraph{\RF2: Mail queuing}
68 Mail queuing removes the need to deliver instantly as a message is received. The queue provides fail-safe storage of mails until they are delivered. Mail queues are probably used in all \mta{}s, excluding the simple forwarders. The mail queue is essential for \masqmail, as \masqmail\ is used for non-permanent online connections. This means, mail must be queued until a online connection is available to send the message. This may be after a reboot. Hence the mail queue must provide persistence. 68 Mail queuing removes the need to deliver instantly as a message is received. The queue provides fail-safe storage of mails until they are delivered. Mail queues are probably used in all \mta{}s, even in some simple forwarders. The mail queue is essential for \masqmail, as \masqmail\ is used for non-permanent online connections. This means, mail must be queued until a online connection is available to send the message. This may be after a reboot. Hence the mail queue must provide persistence.
69 69
70 The mail queue and the module(s) to manage it are the central part of the whole system. This demands especially for robustness and reliability, as a failure here can lead to loosing mail. An \MTA\ takes over responsibility for mail in accepting it, hence loosing mail messages is absolutely to avoid. This covers any kind of crash situation too. The worst thing acceptable to happen is an already sent mail to be sent again. 70 The mail queue and the module(s) to manage it are the central part of the whole system. This demands especially for robustness and reliability, as a failure here can lead to loosing mail. An \MTA\ takes over responsibility for mail in accepting it, hence loosing mail messages is absolutely to avoid. This covers any kind of crash situation too. The worst thing acceptable to happen is an already sent mail to be sent again.
71 71
72 72
73 73
84 Email addresses can have aliases, thus they need to be expanded. Aliases can be of different kind: another local user, a remote user, a list containing local and remote users, or a command. Most important are the aliases in the \path{aliases} file, usually located at \path{/etc/aliases}. Addresses expanding to lists of users lead to more envelopes. Aliases changing the recipient's domain part may require a different route to be used. 84 Email addresses can have aliases, thus they need to be expanded. Aliases can be of different kind: another local user, a remote user, a list containing local and remote users, or a command. Most important are the aliases in the \path{aliases} file, usually located at \path{/etc/aliases}. Addresses expanding to lists of users lead to more envelopes. Aliases changing the recipient's domain part may require a different route to be used.
85 85
86 86
87 87
88 88
89 \paragraph{\RF5: Selecting a route} 89 \paragraph{\RF5: Route management}
90 One key feature of \masqmail\ is its ability to send mail out over different routes. The online state defines the active route to be used. A specific route may not be suited for all messages, thus these messages are hold back until a suiting route is active. For more information on this concept see section \ref{sec:masqmail-routes}. 90 One key feature of \masqmail\ is its ability to send mail out over different routes. The online state defines the active route to be used. A specific route may not be suited for all messages, thus these messages are hold back until a suiting route is active. For more information on this concept see section \ref{sec:masqmail-routes}.
91 91
92 92
93 93
94 94
143 143
144 144
145 145
146 146
147 147
148 \paragraph{\RF9: Virus checking} 148 \paragraph{\RF9: Malware handling}
149 Related to spam is malicious content (short: \name{malware}) like viruses, worms, trojan horses. They, in contrast to spam, do not affect the \MTA\ itself, as they are in the mail's body. \MTA{}s searching for malware is equal to real world's post offices opening letters to check if they contain something that could harm the recipient. This is not a mail transport job. But the \MTA\ responsible for the recipient seems to be at a good position to do this work, so it is often done there. 149 Related to spam is malicious content (short: \name{malware}) like viruses, worms, trojan horses. They, in contrast to spam, do not affect the \MTA\ itself, as they are in the mail's body. \MTA{}s searching for malware is equal to real world's post offices opening letters to check if they contain something that could harm the recipient. This is not a mail transport job. But the \MTA\ responsible for the recipient seems to be at a good position to do this work, so it is often done there.
150 150
151 In any way should malware checking be performed by external programs that may be invoked by the \mta. But using mail deliver agents, like \name{procmail}, are better suited locations to invoke content scanners. 151 In any way should malware checking be performed by external programs that may be invoked by the \mta. But using mail deliver agents, like \name{procmail}, are better suited locations to invoke content scanners.
152 152
153 A popular email filter framework is \name{amavis} which integrates various spam and virus scanners. The common setup includes a receiving \MTA\ which sends it to \name{amavis} using \SMTP, \name{amavis} processes the mail and sends it then to a second \MTA\ that does the outgoing transfer. Having interfaces to such scanners is nice to have, though. 153 A popular email filter framework is \name{amavis} which integrates various spam and malware scanners. The common setup includes a receiving \MTA\ which sends it to \name{amavis} using \SMTP, \name{amavis} processes the mail and sends it then to a second \MTA\ that does the outgoing transfer. Having interfaces to such scanners is nice to have, though.
154 154
155 155
156 156
157 \paragraph{\RF10: Archiving} 157 \paragraph{\RF10: Archiving}
158 Mail archiving and auditability become more important as email establishes as technology for serious business communication. The ability to archive verbatim copies of every mail coming into and every mail going out of the system, with relation between them, appears to be a goal to achieve. 158 Mail archiving and auditability become more important as email establishes as technology for serious business communication. The ability to archive verbatim copies of every mail coming into and every mail going out of the system, with relation between them, appears to be a goal to achieve.
174 174
175 175
176 \paragraph{\RG1: Security} 176 \paragraph{\RG1: Security}
177 \MTA{}s are critical points for computer security, as they are accessible from external networks. They must be secured with high effort. Properties like the need for high privilege level, from outside influenced work load, work on unsafe data, and demand for reliability, increase the need for security. This is best done by modularization, also called \name{compartementalization}, as described in section \ref{sec:discussion-mta-arch}. \masqmail\ needs to be secure enough for its target field of operation. \masqmail\ is targeted to workstations and private networks, with explicit warning to not use it on permanent online hosts \citeweb{masqmail:homepage2}. But as non-permanent online connections and trustable environments become rare, \masqmail's security should be so good, that it is usable with permanent online connections and in unsafe environments. For example should mails with bad content not break \masqmail. 177 \MTA{}s are critical points for computer security, as they are accessible from external networks. They must be secured with high effort. Properties like the need for high privilege level, from outside influenced work load, work on unsafe data, and demand for reliability, increase the need for security. This is best done by modularization, also called \name{compartementalization}, as described in section \ref{sec:discussion-mta-arch}. \masqmail\ needs to be secure enough for its target field of operation. \masqmail\ is targeted to workstations and private networks, with explicit warning to not use it on permanent online hosts \citeweb{masqmail:homepage2}. But as non-permanent online connections and trustable environments become rare, \masqmail's security should be so good, that it is usable with permanent online connections and in unsafe environments. For example should mails with bad content not break \masqmail.
178 178
179 << conditional compilation >>
179 180
180 181
181 \paragraph{\RG2: Reliability} 182 \paragraph{\RG2: Reliability}
182 Reliability is the second essential quality property for an \MTA. Mail for which the \MTA\ took responsibility must never get lost while it is within the \MTA{}s responsibility. The \MTA\ must not be \emph{the cause} of any mail loss, no matter what happens. Unreliable \mta{}s are of no value. However, as the mail transport infrastructure are distributed systems, one of the communication partners or the transport medium may crash at any time during mail tranfer. Thus reliability is needed for mail transfer communication too. 183 Reliability is the second essential quality property for an \MTA. Mail for which the \MTA\ took responsibility must never get lost while it is within the \MTA{}s responsibility. The \MTA\ must not be \emph{the cause} of any mail loss, no matter what happens. Unreliable \mta{}s are of no value. However, as the mail transport infrastructure are distributed systems, one of the communication partners or the transport medium may crash at any time during mail tranfer. Thus reliability is needed for mail transfer communication too.
183 184
184 The goal is to transfer exactly one copy of the message. \person{Tanenbaum} evaluates the situation and comes to the conclusion that ``in general, there is no way to arrange this.'' \cite[pages~377--379]{tanenbaum02}. Only strategies where now mail gets lost are acceptable; he identifies three of them, but one generates more duplicates than the others, so two strategies remain. (1) The client always reissues the transfer; the server first sends an acknowledgement, then handles the transfer. (2) The client reissues the transfer only if no acknowledgement was received; the server first handles the transfer and sends the acknowledgement afterwards. The first strategy does not need acknowledgements at all, however, it will lose mail if the second transfer fails too. 185 The goal is to transfer exactly one copy of the message. \person{Tanenbaum} evaluates the situation and comes to the conclusion that ``in general, there is no way to arrange this.'' \cite[pages~377--379]{tanenbaum02}. Only strategies where now mail gets lost are acceptable; he identifies three of them, but one generates more duplicates than the others, so two strategies remain. (1) The client always reissues the transfer; the server first sends an acknowledgement, then handles the transfer. (2) The client reissues the transfer only if no acknowledgement was received; the server first handles the transfer and sends the acknowledgement afterwards. The first strategy does not need acknowledgements at all, however, it will lose mail if the second transfer fails too.
185 186
186 Hence, mail transfer between two processes must use the strategy: The client reissues if it receives no acknowledgement; the server first handles the message and then sends the acknowledgement. This strategy only leads to duplicates if a crash happens in the time between the message is fully transfered to the server and the acknowlegement is received by the client. No mail will get lost. 187 Hence, mail transfer between two processes must use the strategy: The client reissues if it receives no acknowledgement; the server first handles the message and then sends the acknowledgement. This strategy only leads to duplicates if a crash happens in the time between the message is fully transfered to the server and the acknowlegement is received by the client. No mail will get lost.
187 188
189 << DB as queue >>
190
191 << exactly one copy of a message at one time >>
188 192
189 \paragraph{\RG3: Robustness} 193 \paragraph{\RG3: Robustness}
190 Being robust means handling errors properly. Small errors may get corrected, large errors may kill a process. Killed processes should restarted automatically and lead to a clean state again. Log messages should be written in every case. Robust software does not need a special environment, it creates a friendly environment itself. \person{Raymond}'s \name{Rule of Robustness} and his \name{Rule of Repair} are good descriptions \cite[pages~18--21]{raymond03}. 194 Being robust means handling errors properly. Small errors may get corrected, large errors may kill a process. Killed processes should restarted automatically and lead to a clean state again. Log messages should be written in every case. Robust software does not need a special environment, it creates a friendly environment itself. \person{Raymond}'s \name{Rule of Robustness} and his \name{Rule of Repair} are good descriptions \cite[pages~18--21]{raymond03}.
191 195
192 196
194 \masqmail's architecture needs to be extendable, to allow new features to be added afterwards. The reason for this need are changing requirements. New requirements appear, like more efficient mail transfer of large messages or a final solution for spam problem. Extendability is the ability of software to include new function with little work. 198 \masqmail's architecture needs to be extendable, to allow new features to be added afterwards. The reason for this need are changing requirements. New requirements appear, like more efficient mail transfer of large messages or a final solution for spam problem. Extendability is the ability of software to include new function with little work.
195 199
196 200
197 \paragraph{\RG5: Maintainability} 201 \paragraph{\RG5: Maintainability}
198 Maintaining software takes much time and effort. \person{Spinellis} guesses ``40\,\% to 70\,\% of the effort that goes into a software system is expended after the system is written first time.'' \cite[page~1]{spinellis03}. This work is called \emph{maintaining}. Hence making software good to maintain will ease work afterwards. 202 Maintaining software takes much time and effort. \person{Spinellis} guesses ``40\,\% to 70\,\% of the effort that goes into a software system is expended after the system is written first time.'' \cite[page~1]{spinellis03}. This work is called \emph{maintaining}. Hence making software good to maintain will ease work afterwards.
203
204 << conditional compilation >>
199 205
200 206
201 \paragraph{\RG6: Testability} 207 \paragraph{\RG6: Testability}
202 Good testability make maintenance easier too, because functionality is directly verifiable when changes are done, thus removing uncertainty. Modularized software makes testing easier, because parts can be tested without external influences. \person{Spinellis} sees testability as a sub-quality of maintainability. 208 Good testability make maintenance easier too, because functionality is directly verifiable when changes are done, thus removing uncertainty. Modularized software makes testing easier, because parts can be tested without external influences. \person{Spinellis} sees testability as a sub-quality of maintainability.
203 209
236 \begin{center} 242 \begin{center}
237 \vspace*{2ex} 243 \vspace*{2ex}
238 %\includegraphics[scale=0.75]{img/callgraph.eps} 244 %\includegraphics[scale=0.75]{img/callgraph.eps}
239 \includegraphics[scale=0.75]{img/masqmail-3-omitlog5.eps} 245 \includegraphics[scale=0.75]{img/masqmail-3-omitlog5.eps}
240 \end{center} 246 \end{center}
241 \caption{Call graph of \masqmail\ to show its internal structure} 247 \caption{Internal structure of \masqmail, showed by a call graph}
242 \label{fig:masqmail-arch} 248 \label{fig:masqmail-arch}
243 \end{figure} 249 \end{figure}
244 250
245 \sendmail\ improved its old architecture by adding the milter interface, to include further functionality by invoking external programs. \exim\ was designed, and is carefully maintained, with a modular-like code structure in mind. \qmail\ started from scratch with a ``security-first'' approach, \postfix\ improved on it, and \name{sendmail X}/\name{MeTA1} tries to adopt the best of \qmail\ and \postfix\ to completely replace the old \sendmail\ architecture. \person{Hafiz} describes this evolution of \mta\ architecture very well \cite{hafiz05}. 251 \sendmail\ improved its old architecture by adding the milter interface, to include further functionality by invoking external programs. \exim\ was designed, and is carefully maintained, with a modular-like code structure in mind. \qmail\ started from scratch with a ``security-first'' approach, \postfix\ improved on it, and \name{sendmail X}/\name{MeTA1} tries to adopt the best of \qmail\ and \postfix\ to completely replace the old \sendmail\ architecture. \person{Hafiz} describes this evolution of \mta\ architecture very well \cite{hafiz05}.
246 252
270 276
271 277
272 \paragraph{\RF1: In/out channels} 278 \paragraph{\RF1: In/out channels}
273 \masqmail's incoming and outgoing channels are the ones required for an \MTA{}s at the moment. They are depicted in figure \ref{fig:masqmail-in-out} on page \pageref{fig:masqmail-in-out}. This is all what is currently needed. But new protocols and mailing concepts are likely to appear (see section \ref{sec:electronic-mail}). \masqmail\ has no support for adding further protocols. Thus modifications at many places in the source are needed to add them though. Today, support for further protocols is not needed, so \masqmail\ is regarded to fulfill \RF1, but the probable future need should be kept in mind. 279 \masqmail's incoming and outgoing channels are the ones required for an \MTA{}s at the moment. They are depicted in figure \ref{fig:masqmail-in-out} on page \pageref{fig:masqmail-in-out}. This is all what is currently needed. But new protocols and mailing concepts are likely to appear (see section \ref{sec:electronic-mail}). \masqmail\ has no support for adding further protocols. Thus modifications at many places in the source are needed to add them though. Today, support for further protocols is not needed, so \masqmail\ is regarded to fulfill \RF1, but the probable future need should be kept in mind.
274 280
281 << smtp submission >>
282
275 \paragraph{\RF2: Queueing} 283 \paragraph{\RF2: Queueing}
276 One single mail queue is used in \masqmail; it satisfies all current requirements. 284 One single mail queue is used in \masqmail; it satisfies all current requirements.
277 285
286 << persistence: DB >>
287
278 \paragraph{\RF3: Header sanitizing} 288 \paragraph{\RF3: Header sanitizing}
279 The envelope and mail headers are generated when the mail is put into the queue. The requirements are fulfilled. 289 The envelope and mail headers are generated when the mail is put into the queue. The requirements are fulfilled.
280 290
281 \paragraph{\RF4: Aliasing} 291 \paragraph{\RF4: Aliasing}
282 Aliasing is done on delivery. All common kinds of aliases in the global aliases file are supported. \name{.forward} aliasing is not, but this is less common and seldom used. 292 Aliasing is done on delivery. All common kinds of aliases in the global aliases file are supported. \name{.forward} aliasing is not, but this is less common and seldom used.
283 293
284 \paragraph{\RF5: Select route} 294 \paragraph{\RF5: Route management}
285 Setting of the route to use is done on delivery. Headers can get rewritten a second time then. This part does provide all the functionality required. 295 Setting of the route to use is done on delivery. Headers can get rewritten a second time then. This part does provide all the functionality required.
286 296
287 \paragraph{\RF6: Authentication} 297 \paragraph{\RF6: Authentication}
288 Static authentication, based on \NAME{IP} addresses, can be achieved with \person{Venema}'s \NAME{TCP} \name{Wrapper} \cite{venema92}, by editing the \path{hosts.allow} and \path{hosts.deny} files. This is only relevant to authenticate host that try to submit mail into the system. Dynamic (secret-based) \SMTP\ authentication is already supported in form of \NAME{SMTP-AUTH} and \SMTP-after-\NAME{POP}, but only for outgoing connections. For incoming connections, only address-based authentication is supported. 298 Static authentication, based on \NAME{IP} addresses, can be achieved with \person{Venema}'s \NAME{TCP} \name{Wrapper} \cite{venema92}, by editing the \path{hosts.allow} and \path{hosts.deny} files. This is only relevant to authenticate host that try to submit mail into the system. Dynamic (secret-based) \SMTP\ authentication is already supported in form of \NAME{SMTP-AUTH} and \SMTP-after-\NAME{POP}, but only for outgoing connections. For incoming connections, only address-based authentication is supported.
289 299
304 314
305 \paragraph{\RG1: Security} 315 \paragraph{\RG1: Security}
306 \masqmail's current security is bad. However, it seems acceptable for using \masqmail\ on workstations and private networks, if the environment is trustable and \masqmail\ is protected against remote attackers. In environments where untrusted components or persons have access to \masqmail, its security is too low. 316 \masqmail's current security is bad. However, it seems acceptable for using \masqmail\ on workstations and private networks, if the environment is trustable and \masqmail\ is protected against remote attackers. In environments where untrusted components or persons have access to \masqmail, its security is too low.
307 Its author states it ``is not designed to'' such usage \citeweb{masqmail:homepage2}. This is a clear indicator for being careful. Issues like high memory consumption, low performance, and denial-of-service attacks---things not regarded by design---may cause serious problems. In any way, is a security report missing that confirms \masqmail's security level. 317 Its author states it ``is not designed to'' such usage \citeweb{masqmail:homepage2}. This is a clear indicator for being careful. Issues like high memory consumption, low performance, and denial-of-service attacks---things not regarded by design---may cause serious problems. In any way, is a security report missing that confirms \masqmail's security level.
308 318
319 << conditional compilation >>
309 320
310 \paragraph{\RG2: Reliability} 321 \paragraph{\RG2: Reliability}
311 Similar is its reliability not good enough. Situations where only one part of sent message was removed from the queue, and the other part remained as garbage, showed off \citeweb{debian:bug245882}. Problems with large mail and small bandwidth were also reported \citeweb{debian:bug216226}. Fortunately, lost email was no big problem yet, but \person{Kurth} warns: 322 Similar is its reliability not good enough. Situations where only one part of sent message was removed from the queue, and the other part remained as garbage, showed off \citeweb{debian:bug245882}. Problems with large mail and small bandwidth were also reported \citeweb{debian:bug216226}. Fortunately, lost email was no big problem yet, but \person{Kurth} warns:
312 \begin{quote} 323 \begin{quote}
313 There may still be serious bugs in [masqmail], so mail might get lost. But in the nearly two years of its existence so far there was only one time a bug which caused mail retrieved via pop3 to be lost in rare circumstances. 324 There may still be serious bugs in [masqmail], so mail might get lost. But in the nearly two years of its existence so far there was only one time a bug which caused mail retrieved via pop3 to be lost in rare circumstances.
314 \hfill\citeweb{masqmail:homepage2} 325 \hfill\citeweb{masqmail:homepage2}
315 \end{quote} 326 \end{quote}
316 In summary: Current reliability needs to be improved. 327 In summary: Current reliability needs to be improved.
317 %fixme: state machine 328 %fixme: state machine
318 329
330 << persistence: db >>
331
319 \paragraph{\RG3: Robustness} 332 \paragraph{\RG3: Robustness}
320 The logging behavior of \masqmail\ is good, although it does not cover all problem situations. For example, if the queue directory is world writeable by accident (or as action of an intruder), any user can remove messages from the queue or replace them with own ones. \masqmail\ does not even write a debug message in this case. The origin of this problem, however, is \masqmail's trust in its environment. 333 The logging behavior of \masqmail\ is good, although it does not cover all problem situations. For example, if the queue directory is world writeable by accident (or as action of an intruder), any user can remove messages from the queue or replace them with own ones. \masqmail\ does not even write a debug message in this case. The origin of this problem, however, is \masqmail's trust in its environment.
321 334
322 \paragraph{\RG4: Extendability} 335 \paragraph{\RG4: Extendability}
323 \masqmail's extendability is very poor. This is a general problem of monolithic software, but can thus be provided with high effort. \exim\ is an example for good extendability in a monolithic program. 336 \masqmail's extendability is very poor. This is a general problem of monolithic software, but can thus be provided with high effort. \exim\ is an example for good extendability in a monolithic program.
324 337
325 \paragraph{\RG5: Maintainability} 338 \paragraph{\RG5: Maintainability}
326 The maintainability of \masqmail\ is equivalent to other software of similar kind. Missing modularity and therefore more complexity makes the maintainer's work harder. In summary is \masqmail's maintainability bearable, like in average Free Software projects. 339 The maintainability of \masqmail\ is equivalent to other software of similar kind. Missing modularity and therefore more complexity makes the maintainer's work harder. In summary is \masqmail's maintainability bearable, like in average Free Software projects.
327 340
341 << conditional compilation >>
342
328 \paragraph{\RG6: Testability} 343 \paragraph{\RG6: Testability}
329 The testability suffers from missing modularity. Testing program parts is hard to do. Nevertheless, it is done by compiling parts of the source to special test programs. 344 The testability suffers from missing modularity. Testing program parts is hard to do. Nevertheless, it is done by compiling parts of the source to special test programs. %fixme: what are the names? what do they test?
330 345
331 \paragraph{\RG7: Performance} 346 \paragraph{\RG7: Performance}
332 The performance---efficiency---of \masqmail\ is good enough for its target field of operation, where this is a minor goal. 347 The performance---efficiency---of \masqmail\ is good enough for its target field of operation, where this is a minor goal.
333 348
334 \paragraph{\RG8: Availability} 349 \paragraph{\RG8: Availability}