meillo@39: .RN 1 meillo@39: meillo@0: .H0 "Introduction meillo@0: .P meillo@53: MH is a set of mail handling tools with a common concept, similar to meillo@53: the Unix tool chest, which is a set of file handling tools with a common meillo@53: concept. \fInmh\fP is the currently most popular implementation of an meillo@53: MH-like mail handling system. meillo@53: This thesis describes an experimental version of nmh, named \fImmh\fP. meillo@53: .P meillo@32: This chapter introduces MH, its history, concepts and how it is used. meillo@47: It describes nmh's code base and community to give the reader meillo@32: a better understanding of the state from which mmh started off. meillo@47: Further more, this chapter outlines the mmh project itself, meillo@47: describing the motivation for it and its goals. meillo@8: meillo@0: meillo@28: .H1 "MH \(en the Mail Handler meillo@0: .P meillo@47: MH is a conceptual email system design and its concrete implementation. meillo@47: Notably, MH had started as a design proposal at RAND Corporation, meillo@47: where the first implementation followed later. meillo@47: In spirit, MH is similar to Unix, which meillo@42: influenced the world more in being a set of system design concepts meillo@32: than in being a specific software product. meillo@47: The ideas behind Unix are summarized in the \fIUnix philosophy\fP. meillo@42: MH follows this philosophy. meillo@2: meillo@11: .U2 "History meillo@2: .P meillo@32: In 1977 at RAND Corporation, Norman Shapiro and Stockton Gaines meillo@32: had proposed the design meillo@32: of a new mail handling system, called ``Mail Handler'' (MH), meillo@32: to superseed RAND's old monolithic ``Mail System'' (MS). meillo@27: Two years later, in 1979, Bruce Borden took the proposal and implemented a meillo@32: prototype of MH. meillo@32: Before the prototype had been available, the concept was meillo@47: believed to be practically unusable. meillo@47: But the prototype had proven successful and replaced MS thereafter. meillo@47: In replacing MS, MH grew to an all-in-one mail system. meillo@2: .P meillo@47: In the early Eighties, meillo@47: the University of California at Irvine (UCI) had started to use MH. meillo@2: They also took over its development and pushed MH forward. meillo@47: Marshall T. Rose and John L. Romine became the driving force then. meillo@47: This was the time when the Internet appeared, when UCB implemented meillo@47: the TCP/IP stack, and when Allman wrote Sendmail. meillo@47: MH was extended as emailing became more featured. meillo@32: The development of MH was closely related to the development of email meillo@32: RFCs. In the advent of MIME, MH was the first implementation of this new meillo@32: email standard. meillo@2: .P meillo@47: In the Nineties, MH had been moved into the public domain, making it meillo@32: attractive to Free Software developers. meillo@47: The Internet had became popular and in December 1996, meillo@47: Richard Coleman initiated the ``New Mail Handler'' (nmh) project. meillo@47: The project is a fork of MH 6.8.3 and bases strongly on the meillo@47: \fILBL changes\fP by Van Jacobson, Mike Karels and Craig Leres. meillo@32: Colman intended to modernize MH and improve its portability and meillo@32: MIME handling capabilities. meillo@32: This should be done openly within the Internet community. meillo@47: The development of MH at UCI stopped after the 6.8.4 release in meillo@47: February 1996, soon after the development of nmh had started. meillo@53: Today, nmh almost completely replaces the original MH. meillo@47: Some systems might still provide old MH, but mainly for historical reasons. meillo@47: .P meillo@47: In the last years, the work on nmh was mostly maintenance work. meillo@47: However, the development revived in December 2011 and stayed busy since then. meillo@0: meillo@11: .U2 "Concepts meillo@0: .P meillo@53: MH consists of a set of tools, each covering a specific task of meillo@53: email handling, like composing a message, replying to a message, meillo@53: refiling a message to a different folder, listing the messages in a folder. meillo@47: All of the programs operate on a common mail storage. meillo@42: .P meillo@32: The mail storage consists of \fImail folders\fP (directories) and meillo@32: \fPmessages\fP (regular files). meillo@32: Each message is stored in a separate file in the format it had been meillo@47: received (i.e. transfer format). meillo@47: The files are named with ascending numbers in each folder. meillo@47: The specific format of the mail storage characterizes MH in the same way meillo@47: like the format of the file system characterizes Unix. meillo@42: .P meillo@47: MH tools maintain a \fIcontext\fP, which includes the current mail folder. meillo@32: Processes in Unix have a similar context, containing the current working meillo@32: directory, for instance. In contrast, the process context is maintained meillo@32: by the Unix kernel automatically, whereas MH tools need to maintain the MH meillo@32: context themselves. meillo@32: The user can have one MH context or multiple ones, he can even share it meillo@32: with other users. meillo@42: .P meillo@47: Messages are named by their numeric filename, but they can have symbolic names, meillo@47: too. These are either automatically updated meillo@32: position names like being the next or the last message, meillo@32: or user-settable group names for arbitrary sets of messages. meillo@32: These names are called sequences. meillo@47: Sequences can be bound to the containing folder or to the context. meillo@2: .P meillo@47: The user's \fIprofile\fP is a file that contains his MH configuration. meillo@47: Default switches for the individual tools can be specified to meillo@47: adjust them to the user's personal preferences. meillo@47: Multiple versions of the same command with different meillo@47: default values can also be created very easily. meillo@51: Form templates for new messages or for replies are easily changeable, meillo@47: and output is adjustable with format files. meillo@47: Almost every part of the system can be adjusted to personal preference. meillo@42: .P meillo@51: The system is well scriptable and extensible. meillo@47: New MH tools are built out of or on top of existing ones quickly. meillo@51: Further more, MH encourages the user to tailor, extend and automate the system. meillo@51: As the MH tool chest was modeled after the Unix tool chest, the meillo@32: properties of the latter apply to the former as well. meillo@8: meillo@53: .U2 "A Tour through MH meillo@53: .P meillo@53: XXX cf. peek; why UP still matters; real work. meillo@8: meillo@27: .U2 "Example Session meillo@27: .P meillo@53: Following is an example mail handling session. meillo@53: It uses mmh but is mostly compatible with nmh and old MH. meillo@32: Details might vary but the look'n'feel is the same. meillo@42: .DS meillo@42: $ \f(CBinc\fP meillo@42: Incorporating new mail into inbox... meillo@42: meillo@42: 1+ 2012-05-16 11:16 meillo@dream.home Hello meillo@42: 2 2012-05-16 11:17 meillo@dream.home book meillo@42: meillo@42: $ \f(CBshow\fP meillo@42: Date: Wed, 16 May 2012 11:16:00 +0200 meillo@42: To: meillo meillo@42: From: meillo@42: Subject: Hello meillo@42: meillo@42: part text/plain 13 meillo@42: mmh is great meillo@42: meillo@42: $ \f(CBnext\fP meillo@42: Date: Wed, 16 May 2012 11:17:24 +0200 meillo@42: To: meillo meillo@42: From: meillo@42: Subject: book meillo@42: meillo@42: part text/plain 79 meillo@42: Hello meillo, meillo@42: meillo@42: have a look at the ``Daemon book''. You need to read that! meillo@42: meillo@42: foo meillo@42: meillo@42: $ \f(CBrmm 1\fP meillo@42: meillo@42: $ \f(CBscan\fP meillo@42: 2+ 2012-05-16 11:17 meillo@dream.home book meillo@42: meillo@42: $ meillo@42: .DE meillo@27: meillo@27: meillo@28: .H1 "nmh: Code and Community meillo@2: .P meillo@49: In order to understand the condition, goals and dynamics of a project, meillo@49: one needs to know the reasons. meillo@53: This section explains the background. meillo@53: .P meillo@49: MH predates the Internet, it comes from times before networking was universal, meillo@49: it comes from times when emailing was small, short and simple. meillo@42: Then it grew, spread and adopted to the changes email went through. meillo@49: Its core-concepts, however, remained the same. meillo@49: During the Eighties students at UCI actively worked on MH. meillo@49: They added new features and optimized the code for the then popular systems. meillo@49: All this still was in times before POSIX and ANSI C. meillo@49: As large parts of the code stem from this time, today's nmh source code meillo@49: still contains many ancient parts. meillo@51: BSD-specific code and constructs tailored for hardware of that time meillo@49: are frequent. meillo@2: .P meillo@49: Nmh started about a decade after the POSIX and ANSI C standards had been meillo@49: established. A more modern coding style entered the code base, but still meillo@49: a part of the developers came from ``the old days''. The developer meillo@49: base became more diverse and thus resulted in code of different style. meillo@49: Programming practices from different decades merged in the project. meillo@51: As several peers added code, the system became more a conglomeration meillo@51: of single tools rather than a homogeneous of-one-cast mail system. meillo@49: Still, the existing basic concepts held it together. meillo@8: They were mostly untouched throughout the years. meillo@8: .P meillo@51: Despite the tool chest approach at the surface \(en a collection meillo@49: of separate small programs \(en on the source code level meillo@49: it is much more interweaved. meillo@49: Several separate components were compiled into one program meillo@51: for efficiency reasons. meillo@8: This lead to intricate innards. meillo@49: Unfortunately, the clear separation on the outside appeared as being meillo@49: pretty interweaved inside. meillo@8: .P meillo@49: The advent of MIME rose the complexity of email by a magnitude. meillo@49: This is visible in nmh. The MIME-related parts are the most complex ones. meillo@49: It's also visible that MIME support had been added on top of the old MH core. meillo@51: MH's tool chest style made this easily possible and encourages meillo@49: such approaches, but unfortunately, it lead to duplicated functions meillo@49: and half-hearted implementation of the concepts. meillo@49: .P meillo@49: To provide backward-compatibility, it is a common understanding to not meillo@49: change the default settings. meillo@51: In consequence, the user needs to activate modern features explicitly meillo@47: to be able to use them. meillo@49: This puts a burden on new users, because out-of-the-box nmh remains meillo@49: in the same ancient style. meillo@49: If nmh is seen to be a back-end, then this compatibility surely is important. meillo@49: However, in the same go, new users have difficulties to use nmh for modern meillo@49: emailing. meillo@49: The small but matured community around nmh hardly needs much change meillo@49: as they have their convenient setups since decades. meillo@8: meillo@8: meillo@27: .H1 "mmh meillo@28: .P meillo@49: I started to work on my experimental version in October 2011, meillo@53: at a time when there were no more than three commits to nmh meillo@53: since the beginning of the year. meillo@53: In December, when I announced my work in progress on the meillo@53: nmh-workers mailing list, meillo@42: .[ meillo@51: nmh-workers mmh announce December meillo@42: .] meillo@53: nmh's community became active, too. meillo@49: This movement was heavily pushed by Paul Vixie's ``edginess'' comment. meillo@42: .[ meillo@42: nmh-workers vixie edginess meillo@42: .] meillo@53: After long years of stagnation, nmh became actively developed again. meillo@42: Hence, while I was working on mmh, the community was working on nmh, meillo@42: in parallel. meillo@28: .P meillo@53: The name \fImmh\fP may stand for \fImodern mail handler\fP, meillo@53: because the project tries to modernize nmh. meillo@53: Personally however, I prefer to call mmh \fImeillo's mail handler\fP, meillo@53: emphasizing that the project follows my visions and preferences. meillo@42: (My login name is \fImeillo\fP.) meillo@53: This project model was inspired by \fIdwm\fP, meillo@42: which is Anselm Garbe's personal window manager \(en meillo@42: targeted to satisfy Garbe's personal needs whenever conflicts appear. meillo@53: Dwm had retained its lean elegance and its focused character, whereas meillo@53: its community-driven predecessor \fIwmii\fP had grown fat over time. meillo@53: The development of mmh should remain focused. meillo@27: meillo@45: meillo@27: .U2 "Motivation meillo@27: .P meillo@51: MH is the most important of very few command line tool chest email systems. meillo@51: Tool chests are powerful because they can be perfectly automated and meillo@53: extended. They allow arbitrary kinds of front-ends to be meillo@53: implemented on top of them quickly and without internal knowledge. meillo@53: Additionally, tool chests are much better to maintain than monolithic meillo@43: programs. meillo@53: As there are few tool chests for emailing and as MH-like ones are the most meillo@53: popular among them they should be developed further. meillo@53: This keeps their meillo@43: conceptional elegance and unique scripting qualities available to users. meillo@53: Mmh will create a modern and convenient entry point to MH-like systems meillo@53: for new and interested users. meillo@43: .P meillo@51: The mmh project is motivated by deficits of nmh and meillo@45: my wish for general changes, combined meillo@45: with the nmh community's reluctancy to change. meillo@45: .P meillo@45: nmh hadn't adjusted to modern emailing needs well enough. meillo@45: The default setup was completely unusable for modern emailing. meillo@45: Too much setup work was required. meillo@45: Several modern features were already available but the community meillo@45: didn't wanted to have them as default. meillo@45: mmh is a way to change this. meillo@45: .P meillo@45: In my eyes, MH's concepts could be exploited even better and meillo@45: the style of the tools could be improved. Both would simplify meillo@45: and generalize the system, providing cleaner interfaces and more meillo@53: software leverage at the same time. meillo@45: mmh is a way to demonstrate this. meillo@45: .P meillo@45: In providing several parts of an email system, nmh can hardly meillo@45: compete with the large specialized projects that focus meillo@45: on only one of the components. meillo@45: The situation can be improved by concentrating the development power meillo@51: on the most unique part of MH and letting the user pick his preferred meillo@45: set of other mail components. meillo@45: Today's pre-packaged software components encourage this model. meillo@45: mmh is a way to go for this approach. meillo@45: .P meillo@43: It's worthwhile to fork nmh for the development of mmh, because meillo@43: the two projects focus on different goals and differ in fundamental questions. meillo@43: The nmh community's reluctance to change conflicts meillo@43: with my strong will to change. meillo@43: In developing a separate experimental version new approaches can meillo@43: easily be tried out without the need to discuss changes beforehand. meillo@43: In fact, revolutionary changes are hardly possible otherwise. meillo@43: .P meillo@45: The mmh project provides the basis to implemented and demonstrated meillo@45: the listed ideas without the need to change nmh or its community. meillo@43: Of course, the results of the mmh project shall improve nmh, in the end. meillo@27: meillo@27: .U2 "Target Field meillo@27: .P meillo@45: Any effort needs to be targeted towards a specific goal meillo@45: in order to be successful. meillo@45: Following is a description of the imagined typical mmh user. meillo@45: mmh should satisfy his needs. meillo@53: .\" XXX Remove the next sentence? meillo@48: Actually, as mmh is my personal version of MH, this is a description meillo@48: of myself. meillo@45: .P meillo@43: The target user of mmh likes Unix and its philosophy. meillo@27: He likes to use programs that are conceptionally appealing. meillo@27: He's familiar with the command line and enjoys its power. meillo@27: He is at least capable of shell scripting and wants to improve his meillo@27: productivity by scripting the mail system. meillo@43: He naturally uses modern email features, like attachments, meillo@51: non-ASCII text, and digital cryptography. meillo@43: He is able to setup email system components besides mmh, meillo@51: and actually likes the choice to pick the ones he prefers. meillo@43: He has a reasonably modern system that complies to standards, meillo@43: like POSIX and ANSI C. meillo@27: .P meillo@48: The typical user invokes mmh commands directly in an interactive meillo@48: shell session, but as well, he uses them to automate mail handling tasks. meillo@48: Likely, he runs his mail setup on a server machine, to which he connects meillo@48: via ssh. He might also have local mmh installations on his workstations, meillo@51: but does rather not rely on graphical front-ends. He definitely wants meillo@48: to be flexible and thus be able to change his setup to suite his needs. meillo@8: .P meillo@48: The typical mmh user is a programmer himself. meillo@48: He likes to, occasionally, take the opportunity of Free Software to put meillo@48: hands on and get involved in the software he uses. meillo@48: Hence, he likes small and clean code bases and he cares for code quality. meillo@48: In general, he believes that: meillo@8: .BU meillo@48: Elegance \(en i.e. simplicity, clarity and generality \(en meillo@48: is most important. meillo@8: .BU meillo@48: Concepts are more important than the concrete implementation. meillo@8: .BU meillo@48: Code optimizations for anything but readability should be avoided meillo@48: if possible. meillo@8: .BU meillo@45: Having a lot of choice is bad. meillo@48: .BU meillo@48: Removed code is debugged code. meillo@8: meillo@48: .U2 "Goals meillo@45: .P meillo@45: The general goals for the mmh project are the following: meillo@48: .IP "Stream-lining meillo@48: Mmh should be stripped down to its core, which is the MUA part of emailing. meillo@48: The feature set should be distilled to the ones really needed, meillo@48: effectively removing corner-cases. meillo@53: Parts that don't add to the main task of being a conceptionally meillo@53: appealing MUA should be removed. meillo@48: This includes, the MTA and MRA facilities. meillo@48: Choice should be reduced to the main options. meillo@48: .IP "Modernizing meillo@48: Mmh's feature set needs to become more modern. meillo@48: Better support for attachment and digital cryptography needs to be added. meillo@48: MIME support needs to be integrated deeper and more naturally. meillo@48: The modern email features need to be readily available, out-of-the-box. meillo@48: And on the other hand, meillo@48: bulletin board support and similar obsolete facilities need to be dropped meillo@48: out. meillo@48: Likewise, ancient technologies, like hardcopy terminals, should not meillo@48: be supported any further. meillo@48: .IP "Code style meillo@48: Mmh's source code needs to be updated to modern standards. meillo@48: Standardized library functions should replace non-standard versions meillo@48: whenever possible. meillo@48: Code should be separated into distinct modules when possible. meillo@48: Time and space optimizations should to be replaced by meillo@48: clear and readable code. meillo@48: A uniform programming style should prevail. meillo@51: .IP "Homogeneity meillo@48: The available concepts need to be expanded as far as possible. meillo@48: A small set of concepts should prevail thoroughly throughout the system. meillo@48: The whole system should appear to be of-one-style. meillo@48: It should feel like being cast as one.