1.1 --- a/thesis/tex/4-MasqmailsFuture.tex	Thu Dec 18 13:47:27 2008 +0100
     1.2 +++ b/thesis/tex/4-MasqmailsFuture.tex	Thu Dec 18 16:41:42 2008 +0100
     1.3 @@ -100,6 +100,7 @@
     1.4  \begin{quote}
     1.5  A perfect example is the contrast between the feature envy early \sendmail\ architecture implemented as one process and the simple, modular architecture of \qmail. The security of \qmail\ comes from its compartmentalized simple processes that perform one task only and are therefor testable for security. \cite[page 64]{hafiz05}
     1.6  \end{quote}
     1.7 +As well does \person{Dent}: ``The modular architecture of Postfix forms the basis for much of its security.''\cite[page 7]{dent04}
     1.8  
     1.9  Modularity is needed for supporting modern \MTA\ requirements, providing a clear interface to add further functionality without increasing the overall complexity much. Modularity is also an enabler for security. Security comes from good design, as \person{Graff} and \person{van Wyk} explain:
    1.10  \begin{quote}
    1.11 @@ -121,18 +122,13 @@
    1.12  This section tries to identify the needed modules for a modern \MTA. They are later the pieces of which the new architecture is built of.
    1.13  
    1.14  
    1.15 -\subsubsection*{The simplest MTA}
    1.16 -This view of the problem is taken from \person{Hafiz} \cite[pages 3-5]{hafiz05}.
    1.17 +\subsubsection*{The simple view}
    1.18  
    1.19 -The basic job of a \mta\ is to tranport mail from a sender to a recipient. The simplest \MTA\ therefor needs at least a mail receiving facility and a mail sending facility. This basic \MTA---following the definition of an \MTA---is much to abstract. Hence a next step to add some important features is needed, the result is an operational \MTA.
    1.20 +The basic job of a \mta\ is to tranport mail from a sender to a recipient. This is the definition of such a program and this is how \person{Dent}\cite[page 19]{dent04} and \person{Hafiz} \cite[pages 3-5]{hafiz05} start on the design.
    1.21  
    1.22 +An \MTA\ therefor needs at least a mail receiving facility and a mail sending facility. But both, and probably all \MTA\ developers (excluded the only forwarders), see the need for a mail queue. A mail queue removed the need to deliver at once. They also provide fail-safe storage of mails until they are delivered.
    1.23  
    1.24  
    1.25 -\subsubsection*{Mail queue}
    1.26 -
    1.27 -\person{Hafif} adds a mail queue to make it possible to not deliver at once.
    1.28 -
    1.29 -Mail queues are probably used in all \mta{}s, excluding the simple forwarders. A mail queue is a essential requirement for \masqmail, as it is to be used for non-permanent online connections.
    1.30  
    1.31  
    1.32  \subsubsection*{Incoming channels}
    1.33 @@ -158,24 +154,134 @@
    1.34  This means outgoing connections, piping mails into local commands needs to be implemented.
    1.35  
    1.36  
    1.37 -\subsubsection*{Mail queue (again)}
    1.38 +
    1.39 +\subsubsection*{Sanitize mail}
    1.40 +generate valid headers: add, rewrite
    1.41 +... better before inserting into the queue
    1.42 +
    1.43 +(determine the method to send at that position?)
    1.44 +
    1.45 +
    1.46 +
    1.47 +
    1.48 +\subsubsection*{Aliasing}
    1.49 +
    1.50 +where to expand aliases?
    1.51 +
    1.52 +
    1.53 +
    1.54 +\subsubsection*{Mail queue}
    1.55 +
    1.56 +Mail queues are probably used in all \mta{}s, excluding the simple forwarders. A mail queue is a essential requirement for \masqmail, as it is to be used for non-permanent online connections.
    1.57 +
    1.58 +
    1.59 +
    1.60  
    1.61  
    1.62  
    1.63  
    1.64  \subsubsection*{Authentication}
    1.65  
    1.66 -easiest: restricting by static IP addresses (Access control via hosts.allow/hosts.deny)
    1.67 -if dynamic remote hosts need access: some auth is needed
    1.68 -- SASL
    1.69 -- POP/IMAP: pop-before-smtp, DRAC, WHOSON
    1.70 -- TLS (certificates)
    1.71 +either by
    1.72 +- network/ip address
    1.73 +	easiest: restricting by static IP addresses (Access control via hosts.allow/hosts.deny)
    1.74 +or
    1.75 +- some kind of auth (for dynamic remote hosts)
    1.76 +	adds complexity
    1.77 +	- SASL
    1.78 +	- POP/IMAP: pop-before-smtp, DRAC, WHOSON
    1.79 +	- TLS (certificates)
    1.80  
    1.81 -``None of these add-ons is an ideal solution. They require additional code compiled into your existing daemons that may then require special write accesss to system files. They also require additional work for busy system administrators. If you cannot use any of the nonauthenticating alternatives mentioned earlier, or your business requirements demand that all of thyour users' mail pass through your system no matter where they are on the Internet, SASL is probably the solution that offers the most reliable and scalable method to authenticate users.'' (Dent: Postfix, page 44, ch04)
    1.82 +\begin{quote}
    1.83 +None of these add-ons is an ideal solution. They require additional code compiled into your existing daemons that may then require special write accesss to system files. They also require additional work for busy system administrators. If you cannot use any of the nonauthenticating alternatives mentioned earlier, or your business requirements demand that all of thyour users' mail pass through your system no matter where they are on the Internet, SASL is probably the solution that offers the most reliable and scalable method to authenticate users.
    1.84 +\cite[page 44]{dent04}
    1.85 +\end{quote}
    1.86  
    1.87  
    1.88  \subsubsection*{Encryption}
    1.89  
    1.90 +TLS/SSL prevents attackers to listen on the cable
    1.91 +but it does not prevent man-in-the-middle attacks
    1.92 +signed certificates help here
    1.93 +
    1.94 +
    1.95 +ch /usr/share/ssl/misc
    1.96 +
    1.97 +create new CA:
    1.98 +\begin{verbatim}
    1.99 +	CA.pl -newca
   1.100 +		country: DE
   1.101 +		state: schwaben
   1.102 +		city: Ulm
   1.103 +		company: 
   1.104 +		section:
   1.105 +		name:
   1.106 +		emailaddress:
   1.107 +\end{verbatim}
   1.108 +
   1.109 +generate ssl key:
   1.110 +\begin{verbatim}
   1.111 +	CA.pl -newreq
   1.112 +		... the same questions
   1.113 +\end{verbatim}
   1.114 +
   1.115 +sign request with CA:
   1.116 +\begin{verbatim}
   1.117 +	CA.pl -sign
   1.118 +\end{verbatim}
   1.119 +
   1.120 +remove passphrase from private key:
   1.121 +\begin{verbatim}
   1.122 +	openssl rsa <newreq.pem >key.pem
   1.123 +	(to be used by programs automaticly)
   1.124 +\end{verbatim}
   1.125 +
   1.126 +secure:
   1.127 +\begin{verbatim}
   1.128 +	chmod 400 *.pem
   1.129 +	cp newcert.pem /etc/postfix/cert.pem
   1.130 +	cp key.pem /etc/postfix/key.pem
   1.131 +	cp demoCA/cacert.pem /etc/postfix/CAcert.pem
   1.132 +	chmode 400 /etc/postfix/*.pem
   1.133 +
   1.134 +	mkdir /etc/stunnel
   1.135 +	cat newcert.pem key.pem >/etc/stunnel/stunnel.pem
   1.136 +	chmod 400 /etc/stunnel/stunnel.pem
   1.137 +	(check /etc/stunnel with `stunnel -V')
   1.138 +\end{verbatim}
   1.139 +
   1.140 +
   1.141 +set up stunnels for POP, etc:
   1.142 +\begin{verbatim}
   1.143 +	nmap localhost
   1.144 +	stunnel -d pop3s -r localhost:pop3 -p /etc/stunnel/stunnel.pem
   1.145 +	stunnel -d imaps -r localhost:imap -p /etc/stunnel/stunnel.pem
   1.146 +	nmap localhost
   1.147 +		pop3s 995
   1.148 +		imaps 993
   1.149 +\end{verbatim}
   1.150 +
   1.151 +do not use stunnel wit SMTP:
   1.152 +because all incoming mail would be from 127.0.0.1 !!
   1.153 +use STARTTLS instead
   1.154 +
   1.155 +postfix: main.cf
   1.156 +\begin{verbatim}
   1.157 +	smtpd_use_tls = yes
   1.158 +	smtpd_tls_received_header = no (does not log in received headers)
   1.159 +
   1.160 +	smtpd_tls_key_file = /etc/postfix/key.pem
   1.161 +	smtpd_tls_cert_file = /etc/postfix/cert.pem
   1.162 +	smtpd_tls_CA_file = /etc/postfix/CAcert.pem
   1.163 +
   1.164 +	smtp_use_tls = yes  (use TLS for sending)
   1.165 +	smtp_tls_key_file = /etc/postfix/key.pem
   1.166 +	smtp_tls_cert_file = /etc/postfix/cert.pem
   1.167 +	smtp_tls_CA_file = /etc/postfix/CAcert.pem
   1.168 +\end{verbatim}
   1.169 +
   1.170 +
   1.171 +
   1.172  
   1.173  \subsubsection*{Spam prevention}
   1.174  
   1.175 @@ -183,22 +289,88 @@
   1.176  where to filter what
   1.177  
   1.178  
   1.179 +postfix:
   1.180 +content-filter: arbitrary programs that talk smtp, can filter, rewrite or delete mail
   1.181 +- before-queue-c-f: need to be fast, can prevent system load
   1.182 +- after-queue-c-f: need more resources in global, more load
   1.183 +
   1.184 +exim:
   1.185 +acls: to filter, what to accept (hook into smtp dialog) (complex)
   1.186 +routers: take recipient address and choose a matching transport
   1.187 +transports: ways to deliver mail (smtp, local)
   1.188 +
   1.189 +
   1.190  postfix: after-queue-content-filter (smtp communication)
   1.191 -exim: content-scan-feature
   1.192 +exim: content-scan-feature (analyses the content: MIME stuff, blacklisted words, virus scanning) (all within smtp dialog)
   1.193  sendmail: milter (tcp or unix sockets)
   1.194  
   1.195  checks while smtp dialog (pre-queue): in MTA implemented (need to be fast)
   1.196  checks when mail is accepted and queued: external (amavis, spamassassin)
   1.197  
   1.198  
   1.199 -AMaViS (amavisd-new): email filter framework to integrate spam and virus scanner
   1.200 -internet -->25 MTA -->10024 amavis -->10025 MTA --> reciptient
   1.201 -                |                            |
   1.202 -                +----------------------------+
   1.203 -mail scanner:
   1.204 -incoming queue --> mail scanner --> outgoing queue
   1.205  
   1.206 -mimedefang: uses milter interface with sendmail
   1.207 +
   1.208 +
   1.209 +
   1.210 +what do do with recognized mail?
   1.211 +- reject (only possible if recognized during SMTP dialog)
   1.212 +- forward with added header line or changed subject
   1.213 +(eisentraut05: page 18--20)
   1.214 +
   1.215 +check incoming and outgoing mail
   1.216 +(eisentraut05: page 21)
   1.217 +
   1.218 +
   1.219 +milter:
   1.220 +communication with external daemons via a special protocol
   1.221 +at various times in the smtp dialog possible
   1.222 +can reject, delete or alter messages
   1.223 +http://milter.org
   1.224 +(eisentraut05: page 69)
   1.225 +
   1.226 +
   1.227 +use SA with exim:
   1.228 +- with transport: piped into sa
   1.229 +- content-scanning-feature: with ACL during smtp dialog
   1.230 +- plugin: sa-exim
   1.231 +- within amavis
   1.232 +
   1.233 +use SA with sendmail:
   1.234 +- with milter
   1.235 +- within mimedefang or amavis
   1.236 +
   1.237 +use SA with postfix:
   1.238 +- within amavis or mailfilter
   1.239 +
   1.240 +
   1.241 +(eisentraut05: page 25) ``Ganz ohne Analyse während der SMTP-Phase kommt sowieso kein MTA aus, und es ist eine Frage der Einschätzung, wie weit man diese Phase belasten möchte.''
   1.242 +
   1.243 +
   1.244 +DNSBL can contain:
   1.245 +- open relays
   1.246 +- dynamic IP addresses
   1.247 +- verified spam sources
   1.248 +- open multistage relays
   1.249 +- vulnerable CGI scripts
   1.250 +- open proxy servers
   1.251 +example: NJABL (http://njabl.org)
   1.252 +
   1.253 +DNSBL in smpt dialog is aggressive and can lead to problems (eisentraut05: page 126)
   1.254 +
   1.255 +
   1.256 +greylisting:
   1.257 +if first contact from that address: temp failure and add to list
   1.258 +sender will retry, then accept
   1.259 +
   1.260 +``Das Greylisting zählt derzeit zu den effektivsten Methoden, um gegen unerwünschte E-Mails vorzugehen. Allein durch Greylisting können derzeit rund 70\% des potenziellen Spam-Aufkommens auf einem Mailserver vollständig geblockt werden. Allerdings ist es auch nur eine Frage der Zeit, bis sich die Gemeinde der Spammer und Virenautoren auf diese Methode der Spam-Bekämpfung eingerichtet und entsprechende Queues in ihre Software eingebaut hat.''(eisentraut05: page 138)
   1.261 +Probleme: load balancing using multiple servers with different IPs.
   1.262 +postfix: with policy server
   1.263 +exim: direct in config
   1.264 +sendmail: with greylist milter
   1.265 +
   1.266 +
   1.267 +
   1.268 +hashcash
   1.269  
   1.270  
   1.271  \subsubsection*{Virus checking}
   1.272 @@ -209,14 +381,48 @@
   1.273  
   1.274  
   1.275  anti-virus: clamav
   1.276 +postfix: via amavis
   1.277 +exim: via content-scanning-feature called from acl
   1.278 +sendmail: with milter
   1.279 +procmail
   1.280  
   1.281  
   1.282 +virus scanner work on file level
   1.283 +amavis receives mail via smtp or pipe, splits it in its parts (MIME) and extracks archives, the come the virus scanners
   1.284 +if the mail is okay, it goes via smtp to a second mta
   1.285 +
   1.286 +
   1.287 +AMaViS (amavisd-new): email filter framework to integrate spam and virus scanner
   1.288 +\begin{verbatim}
   1.289 +internet -->25 MTA -->10024 amavis -->10025 MTA --> reciptient
   1.290 +                |                            |
   1.291 +                +----------------------------+
   1.292 +\end{verbatim}
   1.293 +
   1.294 +postfix and exim can habe both mta servises in the same instance, sendmail needs two instances running.
   1.295 +
   1.296 +what amavis recognizes:
   1.297 +- invalid headers
   1.298 +- banned files
   1.299 +- viruses
   1.300 +- spam (using spam assassin)
   1.301 +
   1.302 +
   1.303 +mimedefang: uses milter interface with sendmail
   1.304 +
   1.305 +
   1.306 +MailScanner:
   1.307 +incoming queue --> MailScanner --> outgoing queue
   1.308 +
   1.309 +postfix: with one instance possible, exim and sendmail need two instances running
   1.310 +
   1.311  
   1.312  
   1.313  
   1.314  \subsubsection*{Archiving}
   1.315  
   1.316  
   1.317 +\texttt{always\_bcc} feature of postfix
   1.318  
   1.319  
   1.320