meillo@39: .RN 1
meillo@197: .H0 "Introduction
meillo@197: .Id introduction
meillo@39: 
meillo@0: .P
meillo@53: MH is a set of mail handling tools with a common concept, similar to
meillo@53: the Unix tool chest, which is a set of file handling tools with a common
meillo@53: concept. \fInmh\fP is the currently most popular implementation of an
meillo@53: MH-like mail handling system.
meillo@53: This thesis describes an experimental version of nmh, named \fImmh\fP.
meillo@53: .P
meillo@32: This chapter introduces MH, its history, concepts and how it is used.
meillo@47: It describes nmh's code base and community to give the reader
meillo@106: a better understanding of the state of mmh when it started off.
meillo@181: Furthermore, this chapter outlines the mmh project itself,
meillo@47: describing the motivation for it and its goals.
meillo@8: 
meillo@0: 
meillo@28: .H1 "MH \(en the Mail Handler
meillo@197: .Id mh
meillo@0: .P
meillo@47: MH is a conceptual email system design and its concrete implementation.
meillo@47: Notably, MH had started as a design proposal at RAND Corporation,
meillo@47: where the first implementation followed later.
meillo@47: In spirit, MH is similar to Unix, which
meillo@42: influenced the world more in being a set of system design concepts
meillo@32: than in being a specific software product.
meillo@47: The ideas behind Unix are summarized in the \fIUnix philosophy\fP.
meillo@42: MH follows this philosophy.
meillo@2: 
meillo@11: .U2 "History
meillo@2: .P
meillo@32: In 1977 at RAND Corporation, Norman Shapiro and Stockton Gaines
meillo@106: proposed the design
meillo@181: of a new mail handling system, called \fIMail Handler\fP (MH),
meillo@181: to superseed RAND's old monolithic \fIMail System\fP (MS).
meillo@27: Two years later, in 1979, Bruce Borden took the proposal and implemented a
meillo@32: prototype of MH.
meillo@106: Before the prototype's existence, the concept was
meillo@47: believed to be practically unusable.
meillo@106: But the prototype proved successful and replaced MS thereafter.
meillo@164: In replacing MS, MH grew to provide anything necessary for emailing.
meillo@2: .P
meillo@106: In the early eighties,
meillo@106: the University of California at Irvine (UCI) started to use MH.
meillo@106: Marshall T. Rose and John L. Romine then became the driving force.
meillo@57: They took over the development and pushed MH forward.
meillo@57: RAND had put the code into the public domain by then.
meillo@57: MH was developed at UCI at the time when the Internet appeared,
meillo@57: when UCB implemented the TCP/IP stack, and when Allman wrote Sendmail.
meillo@47: MH was extended as emailing became more featured.
meillo@32: The development of MH was closely related to the development of email
meillo@32: RFCs. In the advent of MIME, MH was the first implementation of this new
meillo@32: email standard.
meillo@2: .P
meillo@117: In the nineties, the Internet became popular and in December 1996,
meillo@181: Richard Coleman initiated the \fINew Mail Handler\fP (nmh) project.
meillo@57: Nmh is a fork of MH 6.8.3 and bases strongly on the
meillo@47: \fILBL changes\fP by Van Jacobson, Mike Karels and Craig Leres.
meillo@32: Colman intended to modernize MH and improve its portability and
meillo@32: MIME handling capabilities.
meillo@32: This should be done openly within the Internet community.
meillo@47: The development of MH at UCI stopped after the 6.8.4 release in
meillo@47: February 1996, soon after the development of nmh had started.
meillo@57: Today, nmh has almost completely replaced the original MH.
meillo@47: Some systems might still provide old MH, but mainly for historical reasons.
meillo@47: .P
meillo@171: In the last years, the changes in nmh were mostly maintenance work.
meillo@117: However, the development was revived in December 2011
meillo@57: and stayed busy since then.
meillo@0: 
meillo@197: 
meillo@11: .U2 "Concepts
meillo@0: .P
meillo@53: MH consists of a set of tools, each covering a specific task of
meillo@53: email handling, like composing a message, replying to a message,
meillo@53: refiling a message to a different folder, listing the messages in a folder.
meillo@47: All of the programs operate on a common mail storage.
meillo@42: .P
meillo@32: The mail storage consists of \fImail folders\fP (directories) and
meillo@32: \fPmessages\fP (regular files).
meillo@106: Each message is stored in a separate file in the format it was
meillo@47: received (i.e. transfer format).
meillo@47: The files are named with ascending numbers in each folder.
meillo@47: The specific format of the mail storage characterizes MH in the same way
meillo@106: as the format of the file system characterizes Unix.
meillo@42: .P
meillo@164: MH tools maintain a \fIcontext\fP, which includes for instance the
meillo@164: current mail folder.
meillo@32: Processes in Unix have a similar context, containing the current working
meillo@32: directory, for instance. In contrast, the process context is maintained
meillo@32: by the Unix kernel automatically, whereas MH tools need to maintain the MH
meillo@32: context themselves.
meillo@106: The user can have one MH context or multiple ones; he can even share it
meillo@106: with others.
meillo@42: .P
meillo@164: Messages are named by their numeric filename,
meillo@164: but they can have symbolic names, too.
meillo@164: These are either automatically updated
meillo@106: position names such as the next or the last message,
meillo@32: or user-settable group names for arbitrary sets of messages.
meillo@32: These names are called sequences.
meillo@47: Sequences can be bound to the containing folder or to the context.
meillo@2: .P
meillo@47: The user's \fIprofile\fP is a file that contains his MH configuration.
meillo@47: Default switches for the individual tools can be specified to
meillo@47: adjust them to the user's personal preferences.
meillo@164: Additionally, a single command can be linked under different names
meillo@164: with different default values easily.
meillo@51: Form templates for new messages or for replies are easily changeable,
meillo@47: and output is adjustable with format files.
meillo@47: Almost every part of the system can be adjusted to personal preference.
meillo@42: .P
meillo@51: The system is well scriptable and extensible.
meillo@47: New MH tools are built out of or on top of existing ones quickly.
meillo@181: Furthermore, MH encourages the user to tailor, extend and automate the system.
meillo@51: As the MH tool chest was modeled after the Unix tool chest, the
meillo@32: properties of the latter apply to the former as well.
meillo@8: 
meillo@102: 
meillo@102: .ig \"XXX
meillo@102: 
meillo@102: .P
meillo@102: To ease typing, the switches can be abbreviated as much as the remaining
meillo@102: prefix remains unambiguous.
meillo@102: If in our example no other switch would start with the letter `t', then
meillo@102: .Cl "-truncate" ,
meillo@102: .Cl "-trunc" ,
meillo@102: .Cl "-tr" ,
meillo@102: and
meillo@102: .Cl "-t
meillo@102: would all be the same.
meillo@102: As a result, switches can neither be grouped (as in
meillo@102: .Cl "ls -ltr" )
meillo@102: nor can switch arguments be appended directly to the switch (as in
meillo@102: .Cl "sendmail -q30m" ).
meillo@102: .P
meillo@102: Many switches have negating counter-parts, which start with `no'.
meillo@102: For example
meillo@102: .Cl "-notruncate
meillo@102: inverts the
meillo@102: .Cl "-truncate
meillo@102: switch.
meillo@102: They exist to undo the effect of default switches in the profile.
meillo@102: If the user has chosen to change the default behavior of some tool
meillo@102: by adding a default switch to the profile,
meillo@102: he can still undo this change in behavior by specifying the inverse
meillo@102: switch on the command line.
meillo@102: 
meillo@102: ..
meillo@102: 
meillo@102: 
meillo@54: .U2 "Using MH
meillo@53: .P
meillo@54: It is strongly recommended to have a look at the MH Book,
meillo@106: which offers a thorough introduction to using MH.
meillo@54: .[ [
meillo@54: peek mh book
meillo@54: .], Part II]
meillo@54: Rose and Romine provide a deeper and more technical
meillo@159: though slightly outdated introduction in only about two dozen pages.
meillo@54: .[
meillo@54: rose romine real work
meillo@54: .]
meillo@27: .P
meillo@53: Following is an example mail handling session.
meillo@53: It uses mmh but is mostly compatible with nmh and old MH.
meillo@106: Details might vary but the look and feel is the same.
meillo@82: 
meillo@202: .so input/mh-session
meillo@27: 
meillo@27: 
meillo@131: .H1 "nmh
meillo@2: .P
meillo@49: In order to understand the condition, goals and dynamics of a project,
meillo@106: one needs to know the reasons behind them.
meillo@53: This section explains the background.
meillo@53: .P
meillo@197: MH predates the Internet;
meillo@197: it comes from times before networking was universal,
meillo@49: it comes from times when emailing was small, short and simple.
meillo@106: Then it grew, spread and adapted to the changes email went through.
meillo@49: Its core-concepts, however, remained the same.
meillo@106: During the eighties, students at UCI actively worked on MH.
meillo@164: They added new features and optimized the code for the systems
meillo@164: popular at that time.
meillo@49: All this still was in times before POSIX and ANSI C.
meillo@49: As large parts of the code stem from this time, today's nmh source code
meillo@49: still contains many ancient parts.
meillo@51: BSD-specific code and constructs tailored for hardware of that time
meillo@49: are frequent.
meillo@2: .P
meillo@106: Nmh started about a decade after the POSIX and ANSI C standards were
meillo@49: established. A more modern coding style entered the code base, but still
meillo@49: a part of the developers came from ``the old days''. The developer
meillo@106: base became more diverse, thus broadening the range of different
meillo@106: coding styles.
meillo@49: Programming practices from different decades merged in the project.
meillo@51: As several peers added code, the system became more a conglomeration
meillo@51: of single tools rather than a homogeneous of-one-cast mail system.
meillo@49: Still, the existing basic concepts held it together.
meillo@8: They were mostly untouched throughout the years.
meillo@8: .P
meillo@106: Despite the separation of the tool chest approach at the surface
meillo@106: \(en a collection of small, separate programs \(en
meillo@171: on the source code level, it is much more interwoven.
meillo@49: Several separate components were compiled into one program
meillo@51: for efficiency reasons.
meillo@106: This led to intricate innards.
meillo@106: While clearly separated on the outside,
meillo@171: the programs turned out to be fairly interwoven inside.
meillo@106: .\" XXX FIXME rewrite...
meillo@171: .\" nicht zweimal ``interwoven''
meillo@106: .\" Unfortunately, the clear separation on the outside turned out to be
meillo@171: .\" fairly interwoven inside.
meillo@8: .P
meillo@106: The advent of MIME raised the complexity of email by a magnitude.
meillo@49: This is visible in nmh. The MIME-related parts are the most complex ones.
meillo@106: It is also visible that MIME support was added on top of the old MH core.
meillo@51: MH's tool chest style made this easily possible and encourages
meillo@106: such approaches, but unfortunately, it led to duplicated functions
meillo@49: and half-hearted implementation of the concepts.
meillo@49: .P
meillo@159: To provide backward-compatibility, it is a common understanding not to
meillo@49: change the default settings.
meillo@51: In consequence, the user needs to activate modern features explicitly
meillo@47: to be able to use them.
meillo@49: This puts a burden on new users, because out-of-the-box nmh remains
meillo@49: in the same ancient style.
meillo@171: If nmh is seen to be a back-end,
meillo@171: then this compatibility surely is important.
meillo@171: However, at the same time, new users have difficulties using nmh for
meillo@171: modern emailing.
meillo@173: The small but mature community around nmh needs little change
meillo@106: as they have had their convenient setups for decades.
meillo@159: .\" XXX Explain more
meillo@8: 
meillo@8: 
meillo@27: .H1 "mmh
meillo@28: .P
meillo@49: I started to work on my experimental version in October 2011,
meillo@197: basing my work on nmh version \fInmh-1.3-dev\fP.
meillo@197: At that time no more than three commits were made to nmh
meillo@197: since the beginning of the year, the latest one being
meillo@197: .Ci a01a41d031c796b526329a4170eb23f0ac93b949
meillo@197: on 2011-04-13.
meillo@53: In December, when I announced my work in progress on the
meillo@53: nmh-workers mailing list,
meillo@42: .[
meillo@51: nmh-workers mmh announce December
meillo@42: .]
meillo@197: nmh's community became active, all of a sudden.
meillo@49: This movement was heavily pushed by Paul Vixie's ``edginess'' comment.
meillo@42: .[
meillo@42: nmh-workers vixie edginess
meillo@42: .]
meillo@53: After long years of stagnation, nmh became actively developed again.
meillo@197: Hence, while I was working on mmh, the community was working on nmh,
meillo@197: in parallel.
meillo@28: .P
meillo@53: The name \fImmh\fP may stand for \fImodern mail handler\fP,
meillo@53: because the project tries to modernize nmh.
meillo@53: Personally however, I prefer to call mmh \fImeillo's mail handler\fP,
meillo@53: emphasizing that the project follows my visions and preferences.
meillo@42: (My login name is \fImeillo\fP.)
meillo@53: This project model was inspired by \fIdwm\fP,
meillo@159: .\" XXX Ref
meillo@42: which is Anselm Garbe's personal window manager \(en
meillo@42: targeted to satisfy Garbe's personal needs whenever conflicts appear.
meillo@53: Dwm had retained its lean elegance and its focused character, whereas
meillo@53: its community-driven predecessor \fIwmii\fP had grown fat over time.
meillo@159: .\" XXX ref
meillo@53: The development of mmh should remain focused.
meillo@27: 
meillo@45: 
meillo@27: .U2 "Motivation
meillo@27: .P
meillo@51: MH is the most important of very few command line tool chest email systems.
meillo@51: Tool chests are powerful because they can be perfectly automated and
meillo@53: extended. They allow arbitrary kinds of front-ends to be
meillo@53: implemented on top of them quickly and without internal knowledge.
meillo@106: Additionally, tool chests are easier to maintain than monolithic
meillo@43: programs.
meillo@53: As there are few tool chests for emailing and as MH-like ones are the most
meillo@106: popular among them, they should be developed further.
meillo@53: This keeps their
meillo@43: conceptional elegance and unique scripting qualities available to users.
meillo@106: Mmh creates a modern and convenient entry point to MH-like systems
meillo@53: for new and interested users.
meillo@43: .P
meillo@51: The mmh project is motivated by deficits of nmh and
meillo@45: my wish for general changes, combined
meillo@45: with the nmh community's reluctancy to change.
meillo@45: .P
meillo@106: At that time, nmh had not adjusted to modern emailing needs well enough.
meillo@45: The default setup was completely unusable for modern emailing.
meillo@45: Too much setup work was required.
meillo@45: Several modern features were already available but the community
meillo@106: did not want to have them as default.
meillo@106: Mmh is a way to change this.
meillo@45: .P
meillo@45: In my eyes, MH's concepts could be exploited even better and
meillo@45: the style of the tools could be improved. Both would simplify
meillo@45: and generalize the system, providing cleaner interfaces and more
meillo@53: software leverage at the same time.
meillo@106: Mmh is a way to demonstrate this.
meillo@45: .P
meillo@45: In providing several parts of an email system, nmh can hardly
meillo@45: compete with the large specialized projects that focus
meillo@45: on only one of the components.
meillo@45: The situation can be improved by concentrating the development power
meillo@51: on the most unique part of MH and letting the user pick his preferred
meillo@45: set of other mail components.
meillo@45: Today's pre-packaged software components encourage this model.
meillo@106: Mmh is a way to go for this approach.
meillo@45: .P
meillo@197: It is worthwhile to fork nmh for the development of mmh,
meillo@197: because the two projects focus on different goals and differ in
meillo@197: fundamental questions.
meillo@106: The nmh community's reluctance regarding change conflicts
meillo@106: with my strong desire for it.
meillo@43: In developing a separate experimental version new approaches can
meillo@43: easily be tried out without the need to discuss changes beforehand.
meillo@43: In fact, revolutionary changes are hardly possible otherwise.
meillo@43: .P
meillo@117: The mmh project provides the basis on which the aforementioned
meillo@117: ideas can be implemented and demonstrated,
meillo@164: without the need to change the nmh project or its community.
meillo@43: Of course, the results of the mmh project shall improve nmh, in the end.
meillo@159: By no means it is my intent to work against the nmh project.
meillo@117: 
meillo@27: 
meillo@27: .U2 "Target Field
meillo@27: .P
meillo@45: Any effort needs to be targeted towards a specific goal
meillo@45: in order to be successful.
meillo@197: Therefore, a description of an imagined typical mmh user follows.
meillo@197: Mmh should satisfy his needs.
meillo@48: Actually, as mmh is my personal version of MH, this is a description
meillo@48: of myself.
meillo@197: Writing software for oneself is a reliable way to produce software
meillo@197: that matches the user's desires.
meillo@45: .P
meillo@197: The target user of mmh likes Unix and its philosophy.
meillo@197: He appreciates to use programs that are conceptionally appealing.
meillo@197: He is familiar with the command line and enjoys its power.
meillo@197: He is capable of shell scripting and wants to improve his
meillo@27: productivity by scripting the mail system.
meillo@197: He uses modern email features, such as attachments,
meillo@169: non-ASCII text, digital signatures and message encryption in a natural way.
meillo@197: He is able to set up mail system components,
meillo@197: and like to have the choice to pick the ones he prefers.
meillo@197: He has a reasonably modern operating system that complies to the
meillo@164: POSIX and ANSI C standards.
meillo@27: .P
meillo@197: The typical user invokes mmh commands directly in an interactive
meillo@197: shell session, but he uses them to automate mail handling tasks as well.
meillo@197: Likely, he runs his mail setup on a server machine,
meillo@197: to which he connects via ssh.
meillo@197: He might also have a local mmh installation on his workstation.
meillo@197: Still, he tend to use mmh directly in the shell instead
meillo@117: of using graphical front-ends.
meillo@197: He definitely wants to be flexible and thus be able to change
meillo@197: his setup to suit his needs.
meillo@8: .P
meillo@197: The typical mmh user is a programmer.
meillo@197: He likes to, occasionally, take the opportunity of free software to put
meillo@197: hands on and get involved in the software he uses.
meillo@197: In consequence, he likes small and clean code bases and cares for
meillo@197: code quality.
meillo@197: In general, he believes that:
meillo@8: .BU
meillo@197: The elegance of source code is most important.
meillo@8: .BU
meillo@197: Concepts are more important than concrete implementations.
meillo@8: .BU
meillo@197: Code optimizations for anything but readability should be avoided.
meillo@8: .BU
meillo@45: Having a lot of choice is bad.
meillo@48: .BU
meillo@48: Removed code is debugged code.
meillo@8: 
meillo@197: 
meillo@48: .U2 "Goals
meillo@45: .P
meillo@45: The general goals for the mmh project are the following:
meillo@128: .IP "Streamlining
meillo@87: Mmh should be stripped down to its core, which is the user agent (MUA).
meillo@117: The feature set should be distilled to the indispensable ones,
meillo@171: effectively removing corner cases.
meillo@173: Parts that do not add to the main task of being a conceptionally
meillo@187: appealing user agent should be removed.
meillo@117: This includes the mail submission and mail retrieval facilities.
meillo@48: Choice should be reduced to the main options.
meillo@131: All tools should be tightly shaped.
meillo@48: .IP "Modernizing
meillo@48: Mmh's feature set needs to become more modern.
meillo@164: Better support for attachments, digital signatures and message encryption
meillo@164: should be added.
meillo@159: MIME support should be integrated deeper and more naturally.
meillo@48: The modern email features need to be readily available, out-of-the-box.
meillo@117: On the other hand,
meillo@117: bulletin board support and similar obsolete facilities can be dropped out.
meillo@131: Likewise, ancient technologies should not be supported any further.
meillo@131: The available concepts need to be expanded as far as possible.
meillo@131: A small set of concepts should recur consistently.
meillo@131: .IP "Styling
meillo@48: Mmh's source code needs to be updated to modern standards.
meillo@48: Standardized library functions should replace non-standard versions
meillo@48: whenever possible.
meillo@117: Code should be separated into distinct modules when feasible.
meillo@48: Time and space optimizations should to be replaced by
meillo@48: clear and readable code.
meillo@48: A uniform programming style should prevail.
meillo@117: The whole system should appear to be of-one-style;
meillo@117: it should feel like being cast as one.