1.1 --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
     1.2 +++ b/intro.roff	Sat Jun 23 22:12:14 2012 +0200
     1.3 @@ -0,0 +1,394 @@
     1.4 +.RN 1
     1.5 +
     1.6 +.H0 "Introduction
     1.7 +.P
     1.8 +MH is a set of mail handling tools with a common concept, similar to
     1.9 +the Unix tool chest, which is a set of file handling tools with a common
    1.10 +concept. \fInmh\fP is the currently most popular implementation of an
    1.11 +MH-like mail handling system.
    1.12 +This thesis describes an experimental version of nmh, named \fImmh\fP.
    1.13 +.P
    1.14 +This chapter introduces MH, its history, concepts and how it is used.
    1.15 +It describes nmh's code base and community to give the reader
    1.16 +a better understanding of the state of mmh when it started off.
    1.17 +Further more, this chapter outlines the mmh project itself,
    1.18 +describing the motivation for it and its goals.
    1.19 +
    1.20 +
    1.21 +.H1 "MH \(en the Mail Handler
    1.22 +.P
    1.23 +MH is a conceptual email system design and its concrete implementation.
    1.24 +Notably, MH had started as a design proposal at RAND Corporation,
    1.25 +where the first implementation followed later.
    1.26 +In spirit, MH is similar to Unix, which
    1.27 +influenced the world more in being a set of system design concepts
    1.28 +than in being a specific software product.
    1.29 +The ideas behind Unix are summarized in the \fIUnix philosophy\fP.
    1.30 +MH follows this philosophy.
    1.31 +
    1.32 +.U2 "History
    1.33 +.P
    1.34 +In 1977 at RAND Corporation, Norman Shapiro and Stockton Gaines
    1.35 +proposed the design
    1.36 +of a new mail handling system, called ``Mail Handler'' (MH),
    1.37 +to superseed RAND's old monolithic ``Mail System'' (MS).
    1.38 +Two years later, in 1979, Bruce Borden took the proposal and implemented a
    1.39 +prototype of MH.
    1.40 +Before the prototype's existence, the concept was
    1.41 +believed to be practically unusable.
    1.42 +But the prototype proved successful and replaced MS thereafter.
    1.43 +In replacing MS, MH grew to an all-in-one mail system.
    1.44 +.P
    1.45 +In the early eighties,
    1.46 +the University of California at Irvine (UCI) started to use MH.
    1.47 +Marshall T. Rose and John L. Romine then became the driving force.
    1.48 +They took over the development and pushed MH forward.
    1.49 +RAND had put the code into the public domain by then.
    1.50 +MH was developed at UCI at the time when the Internet appeared,
    1.51 +when UCB implemented the TCP/IP stack, and when Allman wrote Sendmail.
    1.52 +MH was extended as emailing became more featured.
    1.53 +The development of MH was closely related to the development of email
    1.54 +RFCs. In the advent of MIME, MH was the first implementation of this new
    1.55 +email standard.
    1.56 +.P
    1.57 +In the nineties, the Internet had become popular and in December 1996,
    1.58 +Richard Coleman initiated the ``New Mail Handler'' (nmh) project.
    1.59 +Nmh is a fork of MH 6.8.3 and bases strongly on the
    1.60 +\fILBL changes\fP by Van Jacobson, Mike Karels and Craig Leres.
    1.61 +Colman intended to modernize MH and improve its portability and
    1.62 +MIME handling capabilities.
    1.63 +This should be done openly within the Internet community.
    1.64 +The development of MH at UCI stopped after the 6.8.4 release in
    1.65 +February 1996, soon after the development of nmh had started.
    1.66 +Today, nmh has almost completely replaced the original MH.
    1.67 +Some systems might still provide old MH, but mainly for historical reasons.
    1.68 +.P
    1.69 +In the last years, the work on nmh was mostly maintenance work.
    1.70 +However, development was revived in December 2011
    1.71 +and stayed busy since then.
    1.72 +
    1.73 +.U2 "Concepts
    1.74 +.P
    1.75 +MH consists of a set of tools, each covering a specific task of
    1.76 +email handling, like composing a message, replying to a message,
    1.77 +refiling a message to a different folder, listing the messages in a folder.
    1.78 +All of the programs operate on a common mail storage.
    1.79 +.P
    1.80 +The mail storage consists of \fImail folders\fP (directories) and
    1.81 +\fPmessages\fP (regular files).
    1.82 +Each message is stored in a separate file in the format it was
    1.83 +received (i.e. transfer format).
    1.84 +The files are named with ascending numbers in each folder.
    1.85 +The specific format of the mail storage characterizes MH in the same way
    1.86 +as the format of the file system characterizes Unix.
    1.87 +.P
    1.88 +MH tools maintain a \fIcontext\fP, which includes the current mail folder.
    1.89 +Processes in Unix have a similar context, containing the current working
    1.90 +directory, for instance. In contrast, the process context is maintained
    1.91 +by the Unix kernel automatically, whereas MH tools need to maintain the MH
    1.92 +context themselves.
    1.93 +The user can have one MH context or multiple ones; he can even share it
    1.94 +with others.
    1.95 +.P
    1.96 +Messages are named by their numeric filename, but they can have symbolic names,
    1.97 +too. These are either automatically updated
    1.98 +position names such as the next or the last message,
    1.99 +or user-settable group names for arbitrary sets of messages.
   1.100 +These names are called sequences.
   1.101 +Sequences can be bound to the containing folder or to the context.
   1.102 +.P
   1.103 +The user's \fIprofile\fP is a file that contains his MH configuration.
   1.104 +Default switches for the individual tools can be specified to
   1.105 +adjust them to the user's personal preferences.
   1.106 +Multiple versions of the same command with different
   1.107 +default values can also be created very easily.
   1.108 +Form templates for new messages or for replies are easily changeable,
   1.109 +and output is adjustable with format files.
   1.110 +Almost every part of the system can be adjusted to personal preference.
   1.111 +.P
   1.112 +The system is well scriptable and extensible.
   1.113 +New MH tools are built out of or on top of existing ones quickly.
   1.114 +Further more, MH encourages the user to tailor, extend and automate the system.
   1.115 +As the MH tool chest was modeled after the Unix tool chest, the
   1.116 +properties of the latter apply to the former as well.
   1.117 +
   1.118 +
   1.119 +.ig \"XXX
   1.120 +
   1.121 +.P
   1.122 +To ease typing, the switches can be abbreviated as much as the remaining
   1.123 +prefix remains unambiguous.
   1.124 +If in our example no other switch would start with the letter `t', then
   1.125 +.Cl "-truncate" ,
   1.126 +.Cl "-trunc" ,
   1.127 +.Cl "-tr" ,
   1.128 +and
   1.129 +.Cl "-t
   1.130 +would all be the same.
   1.131 +As a result, switches can neither be grouped (as in
   1.132 +.Cl "ls -ltr" )
   1.133 +nor can switch arguments be appended directly to the switch (as in
   1.134 +.Cl "sendmail -q30m" ).
   1.135 +.P
   1.136 +Many switches have negating counter-parts, which start with `no'.
   1.137 +For example
   1.138 +.Cl "-notruncate
   1.139 +inverts the
   1.140 +.Cl "-truncate
   1.141 +switch.
   1.142 +They exist to undo the effect of default switches in the profile.
   1.143 +If the user has chosen to change the default behavior of some tool
   1.144 +by adding a default switch to the profile,
   1.145 +he can still undo this change in behavior by specifying the inverse
   1.146 +switch on the command line.
   1.147 +
   1.148 +..
   1.149 +
   1.150 +
   1.151 +.U2 "Using MH
   1.152 +.P
   1.153 +It is strongly recommended to have a look at the MH Book,
   1.154 +which offers a thorough introduction to using MH.
   1.155 +.[ [
   1.156 +peek mh book
   1.157 +.], Part II]
   1.158 +Rose and Romine provide a deeper and more technical
   1.159 +though slightly outdated introduction in only about two dozens pages.
   1.160 +.[
   1.161 +rose romine real work
   1.162 +.]
   1.163 +.P
   1.164 +Following is an example mail handling session.
   1.165 +It uses mmh but is mostly compatible with nmh and old MH.
   1.166 +Details might vary but the look and feel is the same.
   1.167 +
   1.168 +.VF input/mh-session
   1.169 +
   1.170 +
   1.171 +.H1 "nmh: Code and Community
   1.172 +.P
   1.173 +In order to understand the condition, goals and dynamics of a project,
   1.174 +one needs to know the reasons behind them.
   1.175 +This section explains the background.
   1.176 +.P
   1.177 +MH predates the Internet; it comes from times before networking was universal,
   1.178 +it comes from times when emailing was small, short and simple.
   1.179 +Then it grew, spread and adapted to the changes email went through.
   1.180 +Its core-concepts, however, remained the same.
   1.181 +During the eighties, students at UCI actively worked on MH.
   1.182 +They added new features and optimized the code for the then popular systems.
   1.183 +All this still was in times before POSIX and ANSI C.
   1.184 +As large parts of the code stem from this time, today's nmh source code
   1.185 +still contains many ancient parts.
   1.186 +BSD-specific code and constructs tailored for hardware of that time
   1.187 +are frequent.
   1.188 +.P
   1.189 +Nmh started about a decade after the POSIX and ANSI C standards were
   1.190 +established. A more modern coding style entered the code base, but still
   1.191 +a part of the developers came from ``the old days''. The developer
   1.192 +base became more diverse, thus broadening the range of different
   1.193 +coding styles.
   1.194 +Programming practices from different decades merged in the project.
   1.195 +As several peers added code, the system became more a conglomeration
   1.196 +of single tools rather than a homogeneous of-one-cast mail system.
   1.197 +Still, the existing basic concepts held it together.
   1.198 +They were mostly untouched throughout the years.
   1.199 +.P
   1.200 +Despite the separation of the tool chest approach at the surface
   1.201 +\(en a collection of small, separate programs \(en
   1.202 +on the source code level, it is much more interweaved.
   1.203 +Several separate components were compiled into one program
   1.204 +for efficiency reasons.
   1.205 +This led to intricate innards.
   1.206 +While clearly separated on the outside,
   1.207 +the programs turned out to be fairly interweaved inside.
   1.208 +.\" XXX FIXME rewrite...
   1.209 +.\" Unfortunately, the clear separation on the outside turned out to be
   1.210 +.\" fairly interweaved inside.
   1.211 +.P
   1.212 +The advent of MIME raised the complexity of email by a magnitude.
   1.213 +This is visible in nmh. The MIME-related parts are the most complex ones.
   1.214 +It is also visible that MIME support was added on top of the old MH core.
   1.215 +MH's tool chest style made this easily possible and encourages
   1.216 +such approaches, but unfortunately, it led to duplicated functions
   1.217 +and half-hearted implementation of the concepts.
   1.218 +.P
   1.219 +To provide backward-compatibility, it is a common understanding to not
   1.220 +change the default settings.
   1.221 +In consequence, the user needs to activate modern features explicitly
   1.222 +to be able to use them.
   1.223 +This puts a burden on new users, because out-of-the-box nmh remains
   1.224 +in the same ancient style.
   1.225 +If nmh is seen to be a back-end, then this compatibility surely is important.
   1.226 +However, in the same go, new users have difficulties using nmh for modern
   1.227 +emailing.
   1.228 +The small but mature community around nmh needs few change
   1.229 +as they have had their convenient setups for decades.
   1.230 +
   1.231 +
   1.232 +.H1 "mmh
   1.233 +.P
   1.234 +I started to work on my experimental version in October 2011,
   1.235 +at a time when there had been no more than three commits to nmh
   1.236 +since the beginning of the year.
   1.237 +In December, when I announced my work in progress on the
   1.238 +nmh-workers mailing list,
   1.239 +.[
   1.240 +nmh-workers mmh announce December
   1.241 +.]
   1.242 +nmh's community became active, too.
   1.243 +This movement was heavily pushed by Paul Vixie's ``edginess'' comment.
   1.244 +.[
   1.245 +nmh-workers vixie edginess
   1.246 +.]
   1.247 +After long years of stagnation, nmh became actively developed again.
   1.248 +Hence, while I was working on mmh, the community was once more working
   1.249 +on nmh, in parallel.
   1.250 +.P
   1.251 +The name \fImmh\fP may stand for \fImodern mail handler\fP,
   1.252 +because the project tries to modernize nmh.
   1.253 +Personally however, I prefer to call mmh \fImeillo's mail handler\fP,
   1.254 +emphasizing that the project follows my visions and preferences.
   1.255 +(My login name is \fImeillo\fP.)
   1.256 +This project model was inspired by \fIdwm\fP,
   1.257 +which is Anselm Garbe's personal window manager \(en
   1.258 +targeted to satisfy Garbe's personal needs whenever conflicts appear.
   1.259 +Dwm had retained its lean elegance and its focused character, whereas
   1.260 +its community-driven predecessor \fIwmii\fP had grown fat over time.
   1.261 +The development of mmh should remain focused.
   1.262 +
   1.263 +
   1.264 +.U2 "Motivation
   1.265 +.P
   1.266 +MH is the most important of very few command line tool chest email systems.
   1.267 +Tool chests are powerful because they can be perfectly automated and
   1.268 +extended. They allow arbitrary kinds of front-ends to be
   1.269 +implemented on top of them quickly and without internal knowledge.
   1.270 +Additionally, tool chests are easier to maintain than monolithic
   1.271 +programs.
   1.272 +As there are few tool chests for emailing and as MH-like ones are the most
   1.273 +popular among them, they should be developed further.
   1.274 +This keeps their
   1.275 +conceptional elegance and unique scripting qualities available to users.
   1.276 +Mmh creates a modern and convenient entry point to MH-like systems
   1.277 +for new and interested users.
   1.278 +.P
   1.279 +The mmh project is motivated by deficits of nmh and
   1.280 +my wish for general changes, combined
   1.281 +with the nmh community's reluctancy to change.
   1.282 +.P
   1.283 +At that time, nmh had not adjusted to modern emailing needs well enough.
   1.284 +The default setup was completely unusable for modern emailing.
   1.285 +Too much setup work was required.
   1.286 +Several modern features were already available but the community
   1.287 +did not want to have them as default.
   1.288 +Mmh is a way to change this.
   1.289 +.P
   1.290 +In my eyes, MH's concepts could be exploited even better and
   1.291 +the style of the tools could be improved. Both would simplify
   1.292 +and generalize the system, providing cleaner interfaces and more
   1.293 +software leverage at the same time.
   1.294 +Mmh is a way to demonstrate this.
   1.295 +.P
   1.296 +In providing several parts of an email system, nmh can hardly
   1.297 +compete with the large specialized projects that focus
   1.298 +on only one of the components.
   1.299 +The situation can be improved by concentrating the development power
   1.300 +on the most unique part of MH and letting the user pick his preferred
   1.301 +set of other mail components.
   1.302 +Today's pre-packaged software components encourage this model.
   1.303 +Mmh is a way to go for this approach.
   1.304 +.P
   1.305 +It is worthwhile to fork nmh for the development of mmh, because
   1.306 +the two projects focus on different goals and differ in fundamental questions.
   1.307 +The nmh community's reluctance regarding change conflicts
   1.308 +with my strong desire for it.
   1.309 +In developing a separate experimental version new approaches can
   1.310 +easily be tried out without the need to discuss changes beforehand.
   1.311 +In fact, revolutionary changes are hardly possible otherwise.
   1.312 +.P
   1.313 +The mmh project implements and demonstrates the listed ideas
   1.314 +without the need to change nmh or its community.
   1.315 +Of course, the results of the mmh project shall improve nmh, in the end.
   1.316 +
   1.317 +.U2 "Target Field
   1.318 +.P
   1.319 +Any effort needs to be targeted towards a specific goal
   1.320 +in order to be successful.
   1.321 +Following is a description of the imagined typical mmh user.
   1.322 +mmh should satisfy his needs.
   1.323 +.\" XXX  Remove the next sentence?
   1.324 +Actually, as mmh is my personal version of MH, this is a description
   1.325 +of myself.
   1.326 +.P
   1.327 +The target user of mmh likes Unix and its philosophy.
   1.328 +He likes to use programs that are conceptionally appealing.
   1.329 +He's familiar with the command line and enjoys its power.
   1.330 +He is at least capable of shell scripting and wants to improve his
   1.331 +productivity by scripting the mail system.
   1.332 +He naturally uses modern email features, like attachments,
   1.333 +non-ASCII text, and digital cryptography.
   1.334 +He is able to setup email system components besides mmh,
   1.335 +and actually likes the choice to pick the ones he prefers.
   1.336 +He has a reasonably modern system that complies to standards,
   1.337 +like POSIX and ANSI C.
   1.338 +.P
   1.339 +The typical user invokes mmh commands directly in an interactive
   1.340 +shell session, but as well, he uses them to automate mail handling tasks.
   1.341 +Likely, he runs his mail setup on a server machine, to which he connects
   1.342 +via ssh. He might also have local mmh installations on his workstations,
   1.343 +but does rather not rely on graphical front-ends. He definitely wants
   1.344 +to be flexible and thus be able to change his setup to suite his needs.
   1.345 +.P
   1.346 +The typical mmh user is a programmer himself.
   1.347 +He likes to, occasionally, take the opportunity of Free Software to put
   1.348 +hands on and get involved in the software he uses.
   1.349 +Hence, he likes small and clean code bases and he cares for code quality.
   1.350 +In general, he believes that:
   1.351 +.BU
   1.352 +Elegance \(en i.e. simplicity, clarity and generality \(en
   1.353 +is most important.
   1.354 +.BU
   1.355 +Concepts are more important than the concrete implementation.
   1.356 +.BU
   1.357 +Code optimizations for anything but readability should be avoided
   1.358 +if possible.
   1.359 +.BU
   1.360 +Having a lot of choice is bad.
   1.361 +.BU
   1.362 +Removed code is debugged code.
   1.363 +
   1.364 +.U2 "Goals
   1.365 +.P
   1.366 +The general goals for the mmh project are the following:
   1.367 +.IP "Stream-lining
   1.368 +Mmh should be stripped down to its core, which is the user agent (MUA).
   1.369 +The feature set should be distilled to the ones really needed,
   1.370 +effectively removing corner-cases.
   1.371 +Parts that don't add to the main task of being a conceptionally
   1.372 +appealing MUA should be removed.
   1.373 +This includes, the mail submission and mail retrieval facilities.
   1.374 +Choice should be reduced to the main options.
   1.375 +.IP "Modernizing
   1.376 +Mmh's feature set needs to become more modern.
   1.377 +Better support for attachment and digital cryptography needs to be added.
   1.378 +MIME support needs to be integrated deeper and more naturally.
   1.379 +The modern email features need to be readily available, out-of-the-box.
   1.380 +And on the other hand,
   1.381 +bulletin board support and similar obsolete facilities need to be dropped
   1.382 +out.
   1.383 +Likewise, ancient technologies, like hardcopy terminals, should not
   1.384 +be supported any further.
   1.385 +.IP "Code style
   1.386 +Mmh's source code needs to be updated to modern standards.
   1.387 +Standardized library functions should replace non-standard versions
   1.388 +whenever possible.
   1.389 +Code should be separated into distinct modules when possible.
   1.390 +Time and space optimizations should to be replaced by
   1.391 +clear and readable code.
   1.392 +A uniform programming style should prevail.
   1.393 +.IP "Homogeneity
   1.394 +The available concepts need to be expanded as far as possible.
   1.395 +A small set of concepts should prevail thoroughly throughout the system.
   1.396 +The whole system should appear to be of-one-style.
   1.397 +It should feel like being cast as one.