Mercurial > docs > master
diff discussion.roff @ 107:9f672d3a25f9
Renamed the chapters to speaking names.
author | markus schnalke <meillo@marmaro.de> |
---|---|
date | Sat, 23 Jun 2012 22:12:14 +0200 |
parents | ch03.roff@9ff356d84c57 |
children | dd5620bf8659 |
line wrap: on
line diff
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/discussion.roff Sat Jun 23 22:12:14 2012 +0200 @@ -0,0 +1,2527 @@ +.H0 "Discussion +.P +This main chapter discusses the practical work done in the mmh project. +It is structured along the goals to achieve. +The concrete work done +is described in the examples of how the general goals were achieved. +The discussion compares the current version of mmh with the state of +nmh just before the mmh project started, i.e. Fall 2011. +Current changes of nmh will be mentioned only as side notes. +.\" XXX where do I discuss the parallel development of nmh? + + + +.H1 "Stream-Lining + +.P +MH had been considered an all-in-one system for mail handling. +The community around nmh has a similar understanding. +In fundamental difference, mmh shall be a MUA only. +I believe that the development of all-in-one mail systems is obsolete. +Today, email is too complex to be fully covered by single projects. +Such a project won't be able to excel in all aspects. +Instead, the aspects of email should be covered my multiple projects, +which then can be combined to form a complete system. +Excellent implementations for the various aspects of email exist already. +Just to name three examples: Postfix is a specialized MTA, +Procmail is a specialized MDA, and Fetchmail is a specialized MRA. +I believe that it is best to use such specialized tools instead of +providing the same function again as a side-component in the project. +.P +Doing something well, requires to focus on a small set of specific aspects. +Under the assumption that focused development produces better results +in the particular area, specialized projects will be superior +in their field of focus. +Hence, all-in-one mail system projects \(en no matter if monolithic +or modular \(en will never be the best choice in any of the fields. +Even in providing the best consistent all-in-one system they are likely +to be beaten by projects that focus only on integrating existing mail +components to a homogeneous system. +.P +The limiting resource in Free Software community development +is usually man power. +If the development power is spread over a large development area, +it becomes even more difficult to compete with the specialists in the +various fields. +The concrete situation for MH-based mail systems is even tougher, +given the small and aged community, including both developers and users, +it has. +.P +In consequence, I believe that the available development resources +should focus on the point where MH is most unique. +This is clearly the user interface \(en the MUA. +Peripheral parts should be removed to stream-line mmh for the MUA task. + + +.H2 "Mail Transfer Facilities +.P +In contrast to nmh, which also provides mail submission and mail retrieval +agents, mmh is a MUA only. +This general difference initiated the development of mmh. +Removing the mail transfer facilities had been the first work task +in the mmh project. +.P +Focusing on one mail agent role only is motivated by Eric Allman's +experience with Sendmail. +He identified limiting Sendmail the MTA task had be one reason for +its success: +.[ [ +costales sendmail +.], p. xviii] +.QS +Second, I limited myself to the routing function \(en +I wouldn't write user agents or delivery backends. +This was a departure of the dominant through of the time, +in which routing logic, local delivery, and often the network code +were incorporated directly into the user agents. +.QE +.P +In mmh, the Mail Submission Agent (MSA) is called +\fIMessage Transfer Service\fP (MTS). +This facility, implemented by the +.Pn post +command, established network connections and spoke SMTP to submit +messages for relay to the outside world. +The changes in email demanded changes in this part of nmh too. +Encryption and authentication for network connections +needed to be supported, hence TLS and SASL were introduced into nmh. +This added complexity to nmh without improving it in its core functions. +Also, keeping up with recent developments in the field of +mail transfer requires development power and specialists. +In mmh this whole facility was simply cut off. +.Ci f6aa95b724fd8c791164abe7ee5468bf5c34f226 +.Ci fecd5d34f65597a4dfa16aeabea7d74b191532c3 +.Ci 156d35f6425bea4c1ed3c4c79783dc613379c65b +Instead, mmh depends on an external MSA. +The only outgoing interface available to mmh is the +.Pn sendmail +command, which almost any MSA provides. +If not, a wrapper program can be written. +It must read the message from the standard input, extract the +recipient addresses from the message header, and hand the message +over to the MSA. +For example, a wrapper script for qmail would be: +.VS +#!/bin/sh +# ignore command line arguments +exec qmail-inject +VE +The requirement to parse the recipient addresses out of the message header +is likely to be removed in the future. +Then mmh would give the recipient addresses as command line arguments. +This appears to be the better interface. +.\" XXX implement it +.P +To retrieve mail, the +.Pn inc +command acted as Mail Retrieval Agent (MRA). +It established network connections +and spoke POP3 to retrieve mail from remote servers. +As with mail submission, the network connections required encryption and +authentication, thus TLS and SASL were added. +Support for message retrieval through IMAP will become necessary +to be added soon, too, and likewise for any other changes in mail transfer. +Not so for mmh because it has dropped the support for retrieving mail +from remote locations. +.Ci ab7b48411962d26439f92f35ed084d3d6275459c +Instead, it depends on an external tool to cover this task. +In mmh exist two paths for messages to enter mmh's mail storage: +(1) Mail can be incorporated with +.Pn inc +from the system maildrop, or (2) with +.Pn rcvstore +by reading them, one at a time, from the standard input. +.P +With the removal of the MSA and MRA, mmh converted from an all-in-one +mail system to being a MUA only. +Now, of course, mmh depends on third-party software. +An external MSA is required to transfer mail to the outside world; +an external MRA is required to retrieve mail from remote machines. +There exist excellent implementations of such software, +which do this specific task likely better than the internal +versions had done it. +Also, the best suiting programs can be freely chosen. +.P +As it had already been possible to use an external MSA or MRA, +why not keep the internal version for convenience? +The question whether there is sense in having a fall-back pager in all +the command line tools, for the cases when +.Pn more +or +.Pn less +aren't available, appears to be ridiculous. +Of course, MSAs and MRAs are more complex than text pagers +and not necessarily available but still the concept of orthogonal +design holds: ``Write programs that do one thing and do it well.'' +.[ +mcilroy unix phil +p. 53 +.] +.[ +mcilroy bstj foreword +.] +Here, this part of the Unix philosophy was applied not only +to the programs but to the project itself. +In other words: +``Develop projects that focus on one thing and do it well.'' +Projects grown complex should be split for the same reasons programs grown +complex should be split. +If it is conceptionally more elegant to have the MSA and MRA as +separate projects then they should be separated. +This is the case here, in my opinion. +The RFCs propose this separation by clearly distinguishing the different +mail handling tasks. +.[ +rfc 821 +.] +The small interfaces between the mail agents support the separation. +.P +In the beginning, email had been small and simple. +At that time, +.Pn /bin/mail +had covered anything there was to email and still had been small +and simple. +Later, the essential complexity of email increased. +(Essential complexity is the complexity defined by the problem itself.\0 +.[[ +brooks no silver bullet +.]]) +Email systems reacted to this change: They grew. +RFCs started to introduce the concept of mail agents to separate the +various tasks because they became more extensive and new tasks appeared. +As the mail systems grew even more, parts were split off. +In nmh, for instance, the POP server, which was included in the original +MH, was removed. +Now is the time to go one step further and split the MSA and MRA off, too. +Not only does this decrease the code size of the project, +but, more important, it unburdens mmh of the whole field of +message transfer with all its implications for the project. +There is no more need to concern with changes in network transfer. +This independence is received by depending on an external program +that covers the field. +Today, this is a reasonable exchange. +.P +Functionality can be added in three different ways: +.BU +Implementing the function originally in the project. +.BU +Depending on a library that provides the function. +.BU +Depending on a program that provides the function. +.P +Whereas adding the function originally to the project increases the +code size most and requires most maintenance and development work, +it makes the project most independent of other software. +Using libraries or external programs require less maintenance work +but introduces dependencies on external software. +Programs have the smallest interfaces and provide the best separation +but possibly limit the information exchange. +External libraries are stronger connected than external programs, +thus information can be exchanged more flexible. +Adding code to a project increases maintenance work. +.\" XXX ref +Implementing complex functions originally in the project adds +a lot of code. +This should be avoided if possible. +Hence, the dependencies only change in kind, not in their existence. +In mmh, library dependencies on +.Pn libsasl2 +and +.Pn libcrypto /\c +.Pn libssl +were treated against program dependencies on an MSA and an MRA. +This also meant treating build-time dependencies against run-time +dependencies. +Besides program dependencies providing the stronger separation +and being more flexible, they also allowed +over 6\|000 lines of code to be removed from mmh. +This made mmh's code base about 12\|% smaller. +Reducing the project's code size by such an amount without actually +losing functionality is a convincing argument. +Actually, as external MSAs and MRAs are likely superior to the +project's internal versions, the common user even gains functionality. +.P +Users of MH should not have problems to set up an external MSA and MRA. +Also, the popular MSAs and MRAs have large communities and a lot +of documentation available. +Choices for MSAs range from full-featured MTAs like +.I Postfix +over mid-size MTAs like +.I masqmail +and +.I dma +to small forwarders like +.I ssmtp +and +.I nullmailer . +Choices for MRAs include +.I fetchmail , +.I getmail , +.I mpop +and +.I fdm . + + +.H2 "Non-MUA Tools +.P +One goal of mmh is to remove the tools that are not part of the MUA's task. +Further more, any tools that don't improve the MUA's job significantly +should be removed. +Loosely related and rarely used tools distract from the lean appearance. +They require maintenance work without adding much to the core task. +By removing these tools, the project shall become more stream-lined +and focused. +In mmh the following tools are not available anymore: +.BU +.Pn conflict +was removed +.Ci 8b235097cbd11d728c07b966cf131aa7133ce5a9 +because it is a mail system maintenance tool that is not MUA-related. +It even checked +.Fn /etc/passwd +and +.Fn /etc/group +for consistency, which is completely unrelated to email. +A tool like +.Pn conflict +is surely useful, but it should not be shipped with mmh. +.\" XXX historic reasons? +.BU +.Pn rcvtty +was removed +.Ci 14767c94b3827be7c867196467ed7aea5f6f49b0 +because its use case of writing to the user's terminal +on receiving of mail is obsolete. +If users like to be informed of new mail, the shell's +.Ev MAILPATH +variable or graphical notifications are technically more appealing. +Writing directly to terminals is hardly ever wanted today. +If though one wants to have it this way, the standard tool +.Pn write +can be used in a way similar to: +.VS +scan -file - | write `id -un` +VE +.BU +.Pn viamail +was removed +.Ci eda72d6a7a7c20ff123043fb7f19c509ea01f932 +when the new attachment system was activated, because +.Pn forw +could then cover the task itself. +The program +.Pn sendfiles +was rewritten as a shell script wrapper around +.Pn forw . +.Ci 0e82199cf3c991a173e0ac8aa776efdb3ded61e6 +.BU +.Pn msgchk +was removed +.Ci bb9360ead7eb7a3fedcce2eeedfc660014e41dbe , +because it lost its use case when POP support was removed. +A call to +.Pn msgchk +provided hardly more information than: +.VS +ls -l /var/mail/meillo +VE +It did distinguish between old and new mail, but +this detail information can be retrieved with +.Pn stat (1), +too. +A small shell script could be written to print the information +in a similar way, if truly necessary. +As mmh's +.Pn inc +only incorporates mail from the user's local maildrop, +and thus no data transfers over slow networks are involved, +there's hardly any need to check for new mail before incorporating it. +.BU +.Pn msh +was removed +.Ci 916690191222433a6923a4be54b0d8f6ac01bd02 +because the tool was in conflict with the philosophy of MH. +It provided an interactive shell to access the features of MH, +but it wasn't just a shell, tailored to the needs of mail handling. +Instead it was one large program that had several MH tools built in. +This conflicts with the major feature of MH of being a tool chest. +.Pn msh 's +main use case had been accessing Bulletin Boards, which have seized to +be popular. +.P +Removing +.Pn msh , +together with the truly archaic code relicts +.Pn vmh +and +.Pn wmh , +saved more than 7\|000 lines of C code \(en +about 15\|% of the project's original source code amount. +Having less code \(en with equal readability, of course \(en +for the same functionality is an advantage. +Less code means less bugs and less maintenance work. +As +.Pn rcvtty +and +.Pn msgchk +are assumed to be rarely used and can be implemented in different ways, +why should one keep them? +Removing them stream-lines mmh. +.Pn viamail 's +use case is now partly obsolete and partly covered by +.Pn forw , +hence there's no reason to still maintain it. +.Pn conflict +is not related to the mail client, and +.Pn msh +conflicts with the basic concept of MH. +Theses two tools might still be useful, but they should not be part of mmh. +.P +Finally, there's +.Pn slocal . +.Pn slocal +is an MDA and thus not directly MUA-related. +It should be removed from mmh, because including it conflicts with +the idea that mmh is a MUA only. +.Pn slocal +should rather become a separate project. +However, +.Pn slocal +provides rule-based processing of messages, like filing them into +different folders, which is otherwise not available in mmh. +Although +.Pn slocal +does neither pull in dependencies nor does it include a separate +technical area (cf. Sec. XXX), still, +it accounts for about 1\|000 lines of code that need to be maintained. +As +.Pn slocal +is almost self-standing, it should be split off into a separate project. +This would cut the strong connection between the MUA mmh and the MDA +.Pn slocal . +For anyone not using MH, +.Pn slocal +would become yet another independent MDA, like +.I procmail . +Then +.Pn slocal +could be installed without the complete MH system. +Likewise, mmh users could decide to use +.I procmail +without having a second, unused MDA, +.Pn slocal , +installed. +That appears to be conceptionally the best solution. +Yet, +.Pn slocal +is not split off. +I defer the decision over +.Pn slocal +in need for deeper investigation. +In the meanwhile, it remains part of mmh. +That does not hurt because +.Pn slocal +is unrelated to the rest of the project. + + +.H2 "\fLshow\fP and \fPmhshow\fP +.P +Since the very beginning \(en already in the first concept paper \(en +.Pn show +had been MH's message display program. +.Pn show +mapped message numbers and sequences to files and invoked +.Pn mhl +to have the files formatted. +With MIME, this approach wasn't sufficient anymore. +MIME messages can consist of multiple parts. Some parts are not +directly displayable and text content might be encoded in +foreign charsets. +.Pn show 's +understanding of messages and +.Pn mhl 's +display capabilities couldn't cope with the task any longer. +.P +Instead of extending these tools, additional tools were written from +scratch and added to the MH tool chest. +Doing so is encouraged by the tool chest approach. +Modular design is a great advantage for extending a system, +as new tools can be added without interfering with existing ones. +First, the new MIME features were added in form of the single program +.Pn mhn . +The command +.Cl "mhn -show 42 +would show the MIME message numbered 42. +With the 1.0 release of nmh in February 1999, Richard Coleman finished +the split of +.Pn mhn +into a set of specialized tools, which together covered the +multiple aspects of MIME. +One of them was +.Pn mhshow , +which replaced +.Cl "mhn -show" . +It was capable of displaying MIME messages appropriately. +.P +From then on, two message display tools were part of nmh, +.Pn show +and +.Pn mhshow . +To ease the life of users, +.Pn show +was extended to automatically hand the job over to +.Pn mhshow +if displaying the message would be beyond +.Pn show 's +abilities. +In consequence, the user would simply invoke +.Pn show +(possibly through +.Pn next +or +.Pn prev ) +and get the message printed with either +.Pn show +or +.Pn mhshow , +whatever was more appropriate. +.P +Having two similar tools for essentially the same task is redundant. +Usually, +users wouldn't distinguish between +.Pn show +and +.Pn mhshow +in their daily mail reading. +Having two separate display programs was therefore mainly unnecessary +from a user's point of view. +Besides, the development of both programs needed to be in sync, +to ensure that the programs behaved in a similar way, +because they were used like a single tool. +Different behavior would have surprised the user. +.P +Today, non-MIME messages are rather seen to be a special case of +MIME messages, although it is the other way round. +As +.Pn mhshow +had already be able to display non-MIME messages, it appeared natural +to drop +.Pn show +in favor of using +.Pn mhshow +exclusively. +.Ci 4c1efddfd499300c7e74263e57d8aa137e84c853 +Removing +.Pn show +is no loss in function, because functionally +.Pn mhshow +covers it completely. +The old behavior of +.Pn show +can still be emulated with the simple command line: +.VS +mhl `mhpath c` +VE +.P +For convenience, +.Pn mhshow +was renamed to +.Pn show +after +.Pn show +was gone. +It is clear that such a rename may confuse future developers when +trying to understand the history. +Nevertheless, I consider the convenience on the user's side, +to call +.Pn show +when they want a message to be displayed, to outweigh the inconvenience +on the developer's side when understanding the project history. +.P +To prepare for the transition, +.Pn mhshow +was reworked to behave more like +.Pn show +first. +(cf. Sec. XXX) +Once the tools behaved more alike, the replacing appeared to be +even more natural. +Today, mmh's new +.Pn show +became the one single message display program again, with the difference +that today it handles MIME messages as well as non-MIME messages. +The outcome of the transition is one program less to maintain, +no second display program for users to deal with, +and less system complexity. +.P +Still, removing the old +.Pn show +hurts in one regard: It had been such a simple program. +Its lean elegance is missing to the new +.Pn show . +But there is no chance; +supporting MIME demands for higher essential complexity. + + +.H2 "Configure Options +.P +Customization is a double-edged sword. +It allows better suiting setups, but not for free. +There is the cost of code complexity to be able to customize. +There is the cost of less tested setups, because there are +more possible setups and especially corner-cases. +And, there is the cost of choice itself. +The code complexity directly affects the developers. +Less tested code affects both, users and developers. +The problem of choice affects the users, for once by having to +choose, but also by more complex interfaces that require more documentation. +Whenever options add little advantages, they should be considered for +removal. +I have reduced the number of project-specific configure options from +fifteen to three. + +.U3 "Mail Transfer Facilities +.P +With the removal of the mail transfer facilities five configure +options vanished: +.P +The switches +.Sw --with-tls +and +.Sw --with-cyrus-sasl +had activated the support for transfer encryption and authentication. +This is not needed anymore. +.Ci fecd5d34f65597a4dfa16aeabea7d74b191532c3 +.Ci 156d35f6425bea4c1ed3c4c79783dc613379c65b +.P +The configure switch +.Sw --enable-pop +activated the message retrieval facility. +The code area that would be conditionally compiled in for TLS and SASL +support had been small. +The conditionally compiled code area for POP support had been much larger. +Whereas the code base changes would only slightly change on toggling +TLS or SASL support, it changed much on toggling POP support. +The changes in the code base could hardly be overviewed. +By having POP support togglable a second code base had been created, +one that needed to be tested. +This situation is basically similar for the conditional TLS and SASL +code, but there the changes are minor and can yet be overviewed. +Still, conditional compilation of a code base creates variations +of the original program. +More variations require more testing and maintenance work. +.P +Two other options only specified default configuration values: +.Sw --with-mts +defined the default transport service, either +.Ar smtp +or +.Ar sendmail . +In mmh this fixed to +.Ar sendmail . +.Ci f6aa95b724fd8c791164abe7ee5468bf5c34f226 +With +.Sw --with-smtpservers +default SMTP servers for the +.Ar smtp +transport service could be specified. +.Ci 128545e06224233b7e91fc4c83f8830252fe16c9 +Both of them became irrelevant. + +.U3 "Backup Prefix +.P +The backup prefix is the string that was prepended to message +filenames to tag them as deleted. +By default it had been the comma character `\f(CW,\fP'. +In July 2000, Kimmo Suominen introduced +the configure option +.Sw --with-hash-backup +to change the default to the hash symbol `\f(CW#\fP'. +The choice was probably personal preference, because first, the +option was named +.Sw --with-backup-prefix. +and had the prefix symbol as argument. +But giving the hash symbol as argument caused too many problems +for Autoconf, +thus the option was limited to use the hash symbol as the default prefix. +This supports the assumption, that the choice for the hash was +personal preference only. +Being related or not, words that start with the hash symbol +introduce a comment in the Unix shell. +Thus, the command line +.Cl "rm #13 #15 +calls +.Pn rm +without arguments because the first hash symbol starts the comment +that reaches until the end of the line. +To delete the backup files, +.Cl "rm ./#13 ./#15" +needs to be used. +Using the hash as backup prefix can be seen as a precaution against +data loss. +.P +I removed the configure option but added the profile entry +.Pe backup-prefix , +which allows to specify an arbitrary string as backup prefix. +.Ci 6c40d481d661d532dd527eaf34cebb6d3f8ed086 +Profile entries are the common method to change mmh's behavior. +This change did not remove the choice but moved it to a location where +it suited better. +.P +Eventually, however, the new trash folder concept +.Cf "Sec. XXX +obsoleted the concept of the backup prefix completely. +.Ci 8edc5aaf86f9f77124664f6801bc6c6cdf258173 +.\" (Well, there still are corner-cases to remove until the backup +.\" prefix can be laid to rest, eventually.) +.\" FIXME: Do this work in the code! + +.U3 "Editor and Pager +.P +The two configure options +.CW --with-editor=EDITOR +.CW --with-pager=PAGER +were used to specify the default editor and pager at configure time. +Doing so at configure time made sense in the Eighties, +when the set of available editors and pagers varied much across +different systems. +Today, the situation is more homogeneous. +The programs +.Pn vi +and +.Pn more +can be expected to be available on every Unix system, +as they are specified by POSIX since two decades. +(The specifications for +.Pn vi +and +.Pn more +appeared in +.[ +posix 1987 +.] +and, +.[ +posix 1992 +.] +respectively.) +As a first step, these two tools were hard-coded as defaults. +.Ci 5d43a99db70c12a673028c7758c20cbe3e13ef5f +Not changed were the +.Pe editor +and +.Pe moreproc +profile entries, which allowed the user to override the system defaults. +Later, the concept was reworked to respect the standard environment +variables +.Ev VISUAL +and +.Ev PAGER +if they are set. +Today, mmh determines the editor to use in the following order, +taking the first available and non-empty item: +.IP (1) +Environment variable +.Ev MMHEDITOR +.IP (2) +Profile entry +.Pe Editor +.IP (3) +Environment variable +.Ev VISUAL +.IP (4) +Environment variable +.Ev EDITOR +.IP (5) +Command +.Pn vi . +.P +.Ci f85f4b7ae62e3d05a945dcd46ead51f0a2a89a9b +.P +The pager to use is determined in a similar order, +also taking the first available and non-empty item: +.IP (1) +Environment variable +.Ev MMHPAGER +.IP (2) +Profile entry +.Pe Pager +(replaces +.Pe moreproc ) +.IP (3) +Environment variable +.Ev PAGER +.IP (4) +Command +.Pn more . +.P +.Ci 0c4214ea2aec6497d0d67b436bbee9bc1d225f1e +.P +By respecting the +.Ev VISUAL /\c +.Ev EDITOR +and +.Ev PAGER +environment variables, +the new behavior confirms better to the common style on Unix systems. +Additionally, the new approach is more uniform and clearer to users. + + +.U3 "ndbm +.P +.Pn slocal +used to depend on +.I ndbm , +a database library. +The database is used to store the `\fLMessage-ID\fP's of all +messages delivered. +This enables +.Pn slocal +to suppress delivering the same message to the same user twice. +(This features was enabled by the +.Sw -suppressdup +switch.) +.P +A variety of versions of the database library exist. +.[ +wolter unix incompat notes dbm +.] +Complicated autoconf code was needed to detect them correctly. +Further more, the configure switches +.Sw --with-ndbm=ARG +and +.Sw --with-ndbmheader=ARG +were added to help with difficult setups that would +not be detected automatically or correctly. +.P +By removing the suppress duplicates feature of +.Pn slocal , +the dependency on +.I ndbm +vanished and 120 lines of complex autoconf code could be saved. +.Ci ecd6d6a20cb7a1507e3a20d6c4cb3a1cf14c6bbf +The change removed functionality too, but that is minor to the +improvement by dropping the dependency and the complex autoconf code. + +.U3 "mh-e Support +.P +The configure option +.Sw --disable-mhe +was removed when the mh-e support was reworked. +Mh-e is the Emacs front-end to MH. +It requires MH to provide minor additional functions. +The +.Sw --disable-mhe +configure option could switch these extensions off. +After removing the support for old versions of mh-e, +only the +.Sw -build +switches of +.Pn forw +and +.Pn repl +are left to be mh-e extensions. +They are now always built in because they add little code and complexity. +In consequence, the +.Sw --disable-mhe +configure option was removed +.Ci a7ce7b4a580d77b6c2c4d980812beb589aa4c643 +Removing the option removed a second code setup that would have +needed to be tested. +This change was first done in nmh and thereafter merged into mmh. +.P +The interface changes in mmh require mh-e to be adjusted in order +to be able to use mmh as back-end. +This will require minor changes to mh-e, but removing the +.Sw -build +switches would require more rework. + +.U3 "Masquerading +.P +The configure option +.Sw --enable-masquerade +could take up to three arguments: +`draft_from', `mmailid', and `username_extension'. +They activated different types of address masquerading. +All of them were implemented in the SMTP-speaking +.Pn post +command, which provided an MSA. +Address masquerading is an MTA's task and mmh does not cover +this field anymore. +Hence, true masquerading needs to be implemented in the external MTA. +.P +The +.I mmailid +masquerading type is the oldest one of the three and the only one +available in the original MH. +It provided a +.I username +to +.I fakeusername +mapping, based on the password file's GECOS field. +The man page +.Mp mh-tailor(5) +described the use case as being the following: +.QS +This is useful if you want the messages you send to always +appear to come from the name of an MTA alias rather than your +actual account name. For instance, many organizations set up +`First.Last' sendmail aliases for all users. If this is +the case, the GECOS field for each user should look like: +``First [Middle] Last <First.Last>'' +.QE +.P +As mmh sends outgoing mail via the local MTA only, +the best location to do such global rewrites is there. +Besides, the MTA is conceptionally the right location because it +does the reverse mapping for incoming mail (aliasing), too. +Further more, masquerading set up there is readily available for all +mail software on the system. +Hence, mmailid masquerading was removed. +.Ci 0836c8000ccb34b59410ef1c15b1b7feac70ce5f +.P +The +.I username_extension +masquerading type did not replace the username but would append a suffix, +specified by the +.Ev USERNAME_EXTENSION +environment variable, to it. +This provided support for the +.I user-extension +feature of qmail and the similar +.I "plussed user +processing of sendmail. +The decision to remove this username_extension masquerading was +motivated by the fact that +.Pn spost +hadn't supported it already. +.Ci 2abae0bfd0ad5bf898461e50aa4b466d641f23d9 +Username extensions are possible in mmh, but less convenient to use. +.\" XXX format file %(getenv USERNAME_EXTENSION) +.P +The +.I draft_from +masquerading type instructed +.Pn post +to use the value of the +.Hd From +header field as SMTP envelope sender. +Sender addresses could be replaced completely. +.Ci b14ea6073f77b4359aaf3fddd0e105989db9 +Mmh offers a kind of masquerading similar in effect, but +with technical differences. +As mmh does not transfer messages itself, the local MTA has final control +over the sender's address. Any masquerading mmh introduces may be reverted +by the MTA. +In times of pedantic spam checking, an MTA will take care to use +sensible envelope sender addresses to keep its own reputation up. +Nonetheless, the MUA can set the +.Hd From +header field and thereby propose +a sender address to the MTA. +The MTA may then decide to take that one or generate the canonical sender +address for use as envelope sender address. +.P +In mmh, the MTA will always extract the recipient and sender from the +message header (\c +.Pn sendmail 's +.Sw -t +switch). +The +.Hd From +header field of the draft may be set arbitrary by the user. +If it is missing, the canonical sender address will be generated by the MTA. + +.U3 "Remaining Options +.P +Two configure options remain in mmh. +One is the locking method to use: +.Sw --with-locking=[dot|fcntl|flock|lockf] . +The idea of removing all methods except the portable dot locking +and having that one as the default is appealing, but this change +requires deeper technical investigation into the topic. +The other option, +.Sw --enable-debug , +compiles the programs with debugging symbols and does not strip them. +This option is likely to stay. + + + + +.H2 "Command Line Switches +.P +The command line switches of MH tools follow the X Window style. +They are words, introduced by a single dash. +For example: +.Cl "-truncate" . +Every program in mmh has two generic switches: +.Sw -help , +to print a short message on how to use the program, and +.Sw -Version , +to tell what version of mmh the program belongs to. +.P +Switches change the behavior of programs. +Programs that do one thing in one way require no switches. +In most cases, doing something in exactly one way is too limiting. +If there is basically one task to accomplish, but it should be done +in various ways, switches are a good approach to alter the behavior +of a program. +Changing the behavior of programs provides flexibility and customization +to users, but at the same time it complicates the code, documentation and +usage of the program. +.\" XXX: Ref +Therefore, the number of switches should be kept small. +A small set of well-chosen switches does no harm. +But usually, the number of switches increases over time. +Already in 1985, Rose and Romine have identified this as a major +problem of MH: +.[ [ +rose romine real work +.], p. 12] +.QS +A complaint often heard about systems which undergo substantial development +by many people over a number of years, is that more and more options are +introduced which add little to the functionality but greatly increase the +amount of information a user needs to know in order to get useful work done. +This is usually referred to as creeping featurism. +.QP +Unfortunately MH, having undergone six years of off-and-on development by +ten or so well-meaning programmers (the present authors included), +suffers mightily from this. +.QE +.P +Being reluctant to adding new switches \(en or `options', +as Rose and Romine call them \(en is one part of a counter-action, +the other part is removing hardly used switches. +Nmh's tools had lots of switches already implemented, +hence, cleaning up by removing some of them was the more important part +of the counter-action. +Removing existing functionality is always difficult because it +breaks programs that use these functions. +Also, for every obsolete feature, there'll always be someone who still +uses it and thus opposes its removal. +This puts the developer into the position, +where sensible improvements to style are regarded as destructive acts. +Yet, living with the featurism is far worse, in my eyes, because +future needs will demand adding further features, +worsening the situation more and more. +Rose and Romine added in a footnote, +``[...] +.Pn send +will no doubt acquire an endless number of switches in the years to come.'' +Although clearly humorous, the comment points to the nature of the problem. +Refusing to add any new switches would encounter the problem at its root, +but this is not practical. +New needs will require new switches and it would be unwise to block +them strictly. +Nevertheless, removing obsolete switches still is an effective approach +to deal with the problem. +Working on an experimental branch without an established user base, +eased my work because I did not offend users when I removed existing +funtions. +.P +Rose and Romine counted 24 visible and 9 more hidden switches for +.Pn send . +In nmh, they increased up to 32 visible and 12 hidden ones. +At the time of writing, no more than 7 visible switches and 1 hidden switch +have remained in mmh's +.Pn send . +(These numbers include two generic switches, help and version.) +.P +Fig. XXX +.\" XXX Ref +displays the number of switches for each of the tools that is available +in both, nmh and mmh. +The tools are sorted by the number of switches they had in nmh. +Visible and hidden switches were counted, +but not the generic help and version switches. +Whereas in the beginning of the project, the average tool had 11 switches, +now it has no more than 5 \(en only half as many. +If the `no' switches and similar inverse variant are folded onto +their counter-parts, the average tool had 8 switches in pre-mmh times and +has 4 now. +The total number of functional switches in mmh dropped from 465 +to 234. + +.KS +.in 1c +.so input/switches.grap +.KE + +.P +A part of the switches vanished after functions were removed. +This was the case for network mail transfer, for instance. +Sometimes, however, the work flow was the other way: +I looked through the +.Mp mh-chart (7) +man page to identify the tools with apparently too many switches. +Then considering the value of each of the switches by examining +the tool's man page and source code, aided by recherche and testing. +This way, the removal of functions was suggested by the aim to reduce +the number of switches per command. + + +.U3 "Draft Folder Facility +.P +A change early in the project was the complete transition from +the single draft message to the draft folder facility. +.Ci 337338b404931f06f0db2119c9e145e8ca5a9860 +The draft folder facility was introduced in the mid-Eighties, when +Rose and Romine called it a ``relatively new feature''. +.[ +rose romine real work +.] +Since then, the facility had existed but was deactivated by default. +The default activation and the related rework of the tools made it +possible to remove the +.Sw -[no]draftfolder , +and +.Sw -draftmessage +switches from +.Pn comp , +.Pn repl , +.Pn forw , +.Pn dist , +.Pn whatnow , +and +.Pn send . +.Ci 337338b404931f06f0db2119c9e145e8ca5a9860 +The only flexibility removed with this change is having multiple +draft folders within one profile. +I consider this a theoretical problem only. +In the same go, the +.Sw -draft +switch of +.Pn anno , +.Pn refile , +and +.Pn send +was removed. +The special-casing of `the' draft message became irrelevant after +the rework of the draft system. +(See Sec. XXX.) +Equally, +.Pn comp +lost its +.Sw -file +switch. +The draft folder facility, together with the +.Sw -form +switch, are sufficient. + + +.U3 "In Place Editing +.P +.Pn anno +had the switches +.Sw -[no]inplace +to either annotate the message in place and thus preserve hard links, +or annotate a copy to replace the original message, breaking hard links. +Following the assumption that linked messages should truly be the +same message, and annotating it should not break the link, the +.Sw -[no]inplace +switches were removed and the previous default +.Sw -inplace +was made the only behavior. +.Ci c8195849d2e366c569271abb0f5f60f4ebf0b4d0 +The +.Sw -[no]inplace +switches of +.Pn repl , +.Pn forw , +and +.Pn dist +could be removed, too, as they were simply passed through to +.Pn anno . +.P +.Pn burst +also had +.Sw -[no]inplace +switches, but with different meaning. +With +.Sw -inplace , +the digest had been replaced by the table of contents (i.e. the +introduction text) and the bursted messages were placed right +after this message, renumbering all following messages. +Also, any trailing text of the digest was lost, though, +in practice, it usually consists of an end-of-digest marker only. +Nontheless, this behavior appeared less elegant than the +.Sw -noinplace +behavior, which already had been the default. +Nmh's +.Mp burst (1) +man page reads: +.sp \n(PDu +.QS +If -noinplace is given, each digest is preserved, no table +of contents is produced, and the messages contained within +the digest are placed at the end of the folder. Other messages +are not tampered with in any way. +.QE +.LP +The decision to drop the +.Sw -inplace +behavior was supported by the code complexity and the possible data loss +it caused. +.Sw -noinplace +was chosen to be the definitive behavior. +.Ci 68a686adeb39223a5e1ad35e4a24890ec053679d + + +.U3 "Forms and Format Strings +.P +Historically, the tools that had +.Sw -form +switches to supply a form file had +.Sw -format +switches as well to supply the contents of a form file as a string +on the command line directly. +In consequence, the following two lines equaled: +.VS +scan -form scan.mailx +scan -format "`cat .../scan.mailx`" +VE +The +.Sw -format +switches were dropped in favor for extending the +.Sw -form +switches. +.Ci f51956be123db66b00138f80464d06f030dbb88d +If their argument starts with an equal sign (`='), +then the rest of the argument is taken as a format string, +otherwise the arguments is treated as the name of a format file. +Thus, now the following two lines equal: +.VS +scan -form scan.mailx +scan -form "=`cat .../scan.mailx`" +VE +This rework removed the prefix collision between +.Sw -form +and +.Sw -format . +Now, typing +.Sw -fo +suffices to specify form or format string. +.P +The different meaning of +.Sw -format +for +.Pn repl +and +.Pn forw +was removed in mmh. +.Pn forw +was completely switched to MIME-type forwarding, thus removing the +.Sw -[no]format . +.Ci 6e271608b7b9c23771523f88d23a4d3593010cf1 +For +.Pn repl , +the +.Sw -[no]format +switches were reworked to +.Sw -[no]filter +switches. +.Ci 67411b1f95d6ec987b4c732459e1ba8a8ac192c6 +The +.Sw -format +switches of +.Pn send +and +.Pn post , +which had a third meaning, +were removed likewise. +.Ci f3cb7cde0e6f10451b6848678d95860d512224b9 +Eventually, the ambiguity of the +.Sw -format +switches was resolved by not anymore having any such switch in mmh. + + +.U3 "MIME Tools +.P +The MIME tools, which were once part of +.Pn mhn +[sic!], +had several switches that added little practical value to the programs. +The +.Sw -[no]realsize +switches of +.Pn mhbuild +and +.Pn mhlist +were removed, doing real size calculations always now +.Ci 8d8f1c3abc586c005c904e52c4adbfe694d2201c , +as +``This provides an accurate count at the expense of a small delay.'' +This small delay is not noticable on modern systems. +.P +The +.Sw -[no]check +switches were removed together with the support for +.Hd Content-MD5 +header fields. +.[ +rfc 1864 +.] +.Ci 31dc797eb5178970d68962ca8939da3fd9a8efda +(See Sec. XXX) +.P +The +.Sw -[no]ebcdicsafe +and +.Sw -[no]rfc934mode +switches of +.Pn mhbuild +were removed because they are considered obsolete. +.Ci 01a3480928da485b4d6109d36d751dfa71799d58 +.Ci 3363e2624dce0eb8164cf8b3f1ab385c8ff72e88 +.P +Content caching of external MIME parts, activated with the +.Sw -rcache +and +.Sw -wcache +switches was completely removed. +.Ci d1fefd9f614e4dc3cda16da6c69133c1b2005269 +External MIME parts are rare today, having a caching facility +for them is appears to be unnecessary. +.P +In pre-MIME times, +.Pn mhl +had covered many tasks that are part of MIME handling today. +Therefore, +.Pn mhl +could be simplified to a large extend, reducing the number of its +switches from 21 to 6. +.Ci 350ad6d3542a07639213cf2a4fe524e829c1e7b6 +.Ci 0e46503be3c855bddaeae3843e1b659279c35d70 + + +.U3 "Mail Transfer Switches +.P +With the removal of the mail transfer facilities, a lot of switches +vanished automatically. +.Pn inc +lost 9 switches, namely +.Sw -host , +.Sw -port , +.Sw -user , +.Sw -proxy , +.Sw -snoop , +.Sw -[no]pack , +as well as +.Sw -sasl +and +.Sw -saslmech . +.Pn send +and +.Pn post +lost 11 switches each, namely +.Sw -server , +.Sw -port , +.Sw -client , +.Sw -user , +.Sw -mail , +.Sw -saml , +.Sw -send , +.Sw -soml , +.Sw -snoop , +as well as +.Sw -sasl , +.Sw -saslmech , +and +.Sw -tls . +.Pn send +had the switches only to pass them further to +.Pn post , +because the user would invoke +.Pn post +not directly, but through +.Pn send . +All these switches, except +.Sw -snoop +were usually defined as default switches in the user's profile, +but hardly given in interactive usage. +.P +Of course, those switches did not really ``vanish'', but the configuration +they did was handed over to external MSAs and MRAs. +Instead of setting up the mail transfer in mmh, it is set up in +external tools. +Yet, this simplifies mmh. +Specialized external tools will likely have simple configuration files. +Hence, instead of having one complicated central configuration file, +the configuration of each domain is separate. +Although the user needs to learn to configure each of the tools, +each configuration is likely much simpler. + + +.U3 "Maildrop Formats +.P +With the removal of MMDF maildrop format support, +.Pn packf +and +.Pn rcvpack +no longer needed their +.Sw -mbox +and +.Sw -mmdf +switches. +.Sw -mbox +is the sole behavior now. +.Ci 3916ab66ad5d183705ac12357621ea8661afd3c0 +In the same go, +.Pn packf +and +.Pn rcvpack +were reworked (see Sec. XXX) and their +.Sw -file +switch became unnecessary. +.Ci ca1023716d4c2ab890696f3e41fa0d94267a940e + + +.U3 "Terminal Magic +.P +Mmh's tools will no longer clear the screen (\c +.Pn scan 's +and +.Pn mhl 's +.Sw -[no]clear +switches +.Ci e57b17343dcb3ff373ef4dd089fbe778f0c7c270 +.Ci 943765e7ac5693ae177fd8d2b5a2440e53ce816e ). +Neither will +.Pn mhl +ring the bell (\c +.Sw -[no]bell +.Ci e11983f44e59d8de236affa5b0d0d3067c192e24 ) +nor page the output itself (\c +.Sw -length +.Ci 5b9d883db0318ed2b84bb82dee880d7381f99188 ). +.P +Generally, the pager to use is no longer specified with the +.Sw -[no]moreproc +command line switches for +.Pn mhl +and +.Pn show /\c +.Pn mhshow . +.Ci 39e87a75b5c2d3572ec72e717720b44af291e88a +.P +.Pn prompter +lost its +.Sw -erase +and +.Sw -kill +switches because today the terminal cares for the line editing keys. + + +.U3 "Header Printing +.P +.Pn folder 's +data output is self-explaining enough that +displaying the header line makes few sense. +Hence, the +.Sw -[no]header +switch was removed and headers are never printed. +.Ci 601cc73d1fa05ce96faa728f036d6c51b91701c7 +.P +In +.Pn mhlist , +the +.Sw -[no]header +switches were removed, too. +.Ci b24f96523aaf60e44e04a3ffb1d22e69a13a602f +But in this case headers are always printed, +because the output is not self-explaining. +.P +.Pn scan +also had +.Sw -[no]header +switches. +Printing the header had been sensible until the introduction of +format strings made it impossible to display the column headings. +Only the folder name and the current date remained to be printed. +As this information can be perfectly retrieved by +.Pn folder +and +.Pn date , +consequently, the switches were removed. +.Ci c477dc5d1d03fa6d9a8ab3dd3508c63cbddc044e +.P +By removing all +.Sw -header +switches, the collision with +.Sw -help +on the first two letters was resolved. +Currently, +.Sw -h +evaluates to +.Sw -help +for all tools of mmh. + + +.U3 "Suppressing Edits or the WhatNow Shell +.P +The +.Sw -noedit +switch of +.Pn comp , +.Pn repl , +.Pn forw , +.Pn dist , +and +.Pn whatnow +was removed, but it can now be replaced by specifying +.Sw -editor +with an empty argument. +.Ci 75fca31a5b9d5c1a99c74ab14c94438d8852fba9 +(Specifying +.Cl "-editor true +is nearly the same, only differing by the previous editor being set.) +.P +The more important change is the removal of the +.Sw -nowhatnowproc +switch. +.Ci ee4f43cf2ef0084ec698e4e87159a94c01940622 +This switch had introduced an awkward behavior, as explained in nmh's +man page for +.Mp comp (1): +.QS +The \-editor editor switch indicates the editor to use for +the initial edit. Upon exiting from the editor, comp will +invoke the whatnow program. See whatnow(1) for a discussion +of available options. The invocation of this program can be +inhibited by using the \-nowhatnowproc switch. (In truth of +fact, it is the whatnow program which starts the initial +edit. Hence, \-nowhatnowproc will prevent any edit from +occurring.) +.QE +.P +Effectively, the +.Sw -nowhatnowproc +switch creates only a draft message. +As +.Cl "-whatnowproc true +causes the same behavior, the +.Sw -nowhatnowproc +switch was removed for being redundant. +Likely, the +.Sw -nowhatnowproc +switch was intended to be used by front-ends. + + +.U3 "Compatibility Switches +.BU +The hidden +.Sw -[no]total +switches of +.Pn flist . +They were simply the inverse of the visible +.Sw -[no]fast +switches: +.Sw -total +was +.Sw -nofast +and +.Sw -nototal +was +.Sw -fast . +I removed the +.Sw -[no]total +legacy. +.Ci ea21fe2c4bd23c639bef251398fae809875732ec +.BU +The +.Sw -subject +switch of +.Pn sortm +existed for compatibility only. +It can be fully replaced by +.Cl "-textfield subject +thus it was removed. +.Ci 00140a3c86e9def69d98ba2ffd4d6e50ef6326ea + + +.U3 "Various +.BU +In order to avoid prefix collisions among switch names, the +.Sw -version +switch was renamed to +.Sw -Version +(with capital `V'). +.Ci 32b2354dbaf4bf934936eb5b102a4a3d2fdd209a +Every program has the +.Sw -version +switch but its first three letters collided with the +.Sw -verbose +switch, present in many programs. +The rename solved this problem once for all. +Although this rename breaks a basic interface, having the +.Sw -V +abbreviation to display the version information, isn't all too bad. +.BU +.Sw -[no]preserve +of +.Pn refile +was removed because what use was it anyway? +.QS +Normally when a message is refiled, for each destination +folder it is assigned the number which is one above the current +highest message number in that folder. Use of the +\-preserv [sic!] switch will override this message renaming, and try +to preserve the number of the message. If a conflict for a +particular folder occurs when using the \-preserve switch, +then refile will use the next available message number which +is above the message number you wish to preserve. +.QE +.BU +The removal of the +.Sw -[no]reverse +switches of +.Pn scan +.Ci 8edc5aaf86f9f77124664f6801bc6c6cdf258173 +is a bug fix, supported by the comments +``\-[no]reverse under #ifdef BERK (I really HATE this)'' +by Rose and +``Lists messages in reverse order with the `\-reverse' switch. +This should be considered a bug.'' by Romine in the documentation. +The question remains why neither Rose and Romine had fixed this +bug in the Eighties when they wrote these comments nor has anyone +thereafter. + + +.ig + +forw: [no]dashstuffing(mhl) + +mhshow: [no]pause [no]serialonly + +mhmail: resent queued +inc: snoop, (pop) + +mhl: [no]faceproc folder sleep + [no]dashstuffing(forw) digest list volume number issue number + +prompter: [no]doteof + +refile: [no]preserve [no]unlink [no]rmmproc + +send: [no]forward [no]mime [no]msgid + [no]push split [no]unique (sasl) width snoop [no]dashstuffing + attach attachformat +whatnow: (noedit) attach + +slocal: [no]suppressdups + +spost: [no]filter [no]backup width [no]push idanno + [no]check(whom) whom(whom) + +whom: ??? + +.. + + +.ig + +.P +In the best case, all switches are unambiguous on the first character, +or on the three-letter prefix for the `no' variants. +Reducing switch prefix collisions, shortens the necessary prefix length +the user must type. +Having less switches helps best. + +.. + + +.\" XXX: whatnow prompt commands + + + + +.H1 "Modernizing +.P +The code base of mmh originates from the late Seventies. +Through the Eighties, extensive work had been done on it. +In the Nineties, it had been partly reorganized and extended. +Relicts from each decade have gathered in the code base. +My goal was to modernize the code base. + +.P +FIXME functional aspect only here +.P +FIXME ref to `code style' for non-functional aspects. + + +.H2 "Code Relicts +.P +My position to drop obsolete functionality of mmh to remove old code +is much more revolutional than the nmh community likes to have it. +Working on an experimental version, I was able to quickly drop +functionality I considered ancient. +The need for consensus with peers would have slowed this process down. +Without the need to justify my decisions, I was able to rush forward. +In Dezember 2011, Paul Vixie motivated the nmh developers to just +do the work: +.[ +paul vixie edginess nmh-workers +.] +.QS +let's stop walking on egg shells with this code base. there's no need to +discuss whether to keep using vfork, just note in [sic!] passing, [...] +we don't need a separate branch for removing vmh +or ridding ourselves of #ifdef's or removing posix replacement functions +or depending on pure ansi/posix "libc". +.QP +these things should each be a day or two of work and the "main branch" +should just be modern. [...] +let's push forward, aggressively. +.QE +.LP +I did so already in the months before. +I pushed forward. +I simply dropped the cruft. +.P +The decision to drop a feature was based on literature research and +careful thinking, but whether having had contact to this particular +feature within my own computer life served as a rule of thumb. +My reasons are always made clean in the commit message for the +version control system. +Hence, others can comprehend my view and argue for undoing the change +if I have missed an important aspect. + + +.U3 "Forking +.P +In being a tool chest, MH creates many processes. +In earlier times +.Fu fork() +had been an expensive system call, because the process's image needed +to be duplicated completely at once. +This was especially painfull in the common case when the image gets +replaced by a call to +.Fu exec() +right after having forked the child process. +The +.Fu vfork() +system call was invented to speed up this particular case. +It completely omits the duplication of the image. +On old systems this resulted in significant speed ups. +Therefore MH used +.Fu vfork() +whenever possible. +.P +Modern memory management units support copy-on-write semantics, which make +.Fu fork() +almost as fast as +.Fu vfork() . +The man page of +.Mp vfork (2) +in FreeBSD 8.0 states: +.QS +This system call will be eliminated when proper system sharing mechanisms +are implemented. Users should not depend on the memory sharing semantics +of vfork() as it will, in that case, be made synonymous to fork(2). +.QE +.LP +Vixie supports the removal with the note that ``the last +system on which fork was so slow that an mh user would notice it, was +Eunice. that was 1987''. +.[ +nmh-workers vixie edginess +.] +I replaced all calls to +.Fu vfork() +with calls to +.Fu fork() . +.P +Related to the costs of +.Fu fork() +is the probability of its success. +In the Eighties on heavy loaded systems, calls to +.Fu fork() +were prone to failure. +Hence, many of the +.Fu fork() +calls in the code were wrapped into loops to retry the +.Fu fork() +several times, for higher changes to succeed, eventually. +On modern systems, failing calls to +.Fu fork() +are unusual. +Hence, in the rare case when +.Fu fork() +fails, mmh programs simply abort. + + +.U3 "Obsolete Header Fields +.BU +The +.Hd Encrypted +header field was introduced by RFC\|822, +but already marked legacy in RFC\|2822. +OpenPGP provides the basis for standardized exchange of encrypted +messages [RFC\|4880, RFC\|3156]. +The support for +.Hd Encrypted +header fields is removed in mmh. +.BU +Native support for +.Hd Face +header fields has been removed, as well. +This feature is similar to the +.Hd X-Face +header field in its intent, +but takes a different approach to store the image. +Instead of encoding the image data directly into the header field, +the it contains the hostname and UDP port where the image +date could be retrieved. +There is even a third system, invented in 2005. +Although it re-uses the +.Hd Face +header field, it is the successor of +.Hd X-Face +with support for colored PNG images. +None of the Face systems described here is popular today. +Hence, mmh has no direct support for them. +.BU +The +.Hd Content-MD5 +header field was introduced by RFC\|1864. +It provides detection of data corruption during the transfer. +But it can not ensure verbatim end-to-end delivery of the contents +[RFC\|1864]. +The proper approach to verify content integrity in an +end-to-end relationship is the use of digital cryptography. +.\" XXX (RFCs FIXME). +On the other hand, transfer protocols should detect corruption during +each transmission. The TCP includes a checksum field therefore. +These two approaches in combinations render the +.Hd Content-MD5 +header field superfluous. +The nmh-workers mailing list archive contains about 4\|200 messages, +ranging from 1992 until today. +Not a single one had a +.Hd Content-MD5 +header field. +Neither did any of the 60\|000 messages in my personal mail storage. +Removing the support for this header field, +removed the last place where MD5 computation was needed. +Hence, the MD5 code could be removed as well. +Over 500 lines of code vanished by this one change. + + +.U3 "MMDF maildrop support +.P +This type of format is conceptionally similar to the mbox format, +but uses a different message delimiter (`\fL^A^A^A^A\fP' instead of +`\fLFrom\0\fP'). +Mbox is the de-facto standard maildrop format on Unix, +whereas the MMDF maildrop format is hardly still known today. +I did drop MMDF maildrop format support. +.P +The simplifications within the code were only moderate. +Switches could be removed from +.L packf +and +.L rcvpack , +which generate packed mailboxes. +Only one packed mailbox format remained: mbox. +The more important changes affected the equally named mail parsing +routine in +.Fn sbr/m_getfld.c . +The MMDF code had been removed there, but as now only one packed mailbox +format is left, further code structure simplifications may be possible. +I have not worked on them yet because +.Fu m_getfld() +is heavily optimized and thus dangerous to touch. +The risk of damaging the intricate workings of the optimized code is +too high. +.\" XXX: move somewhere else +This problem is know to the developers of nmh, too. +They also avoid touching this minefield if possible. + + +.U3 "Prompter's Control Keys +.P +The program +.Pn prompter +queries the user to fill in a message form. +When used by +.Pn comp +as +.Cl "comp -editor prompter" , +the resulting behavior is similar to +.Pn mailx . +Apparently, +.Pn prompter +hadn't been touched lately. +Otherwise it's hardly explainable why it +still offered the switches +.Sw -erase +.Ar chr +and +.Sw -kill +.Ar chr +to name the characters for command line editing. +The times when this had been necessary are long time gone. +Today these things work out-of-the-box, and if not, are configured +with the standard tool +.Pn stty . +The switches are removed now +.Ci 0bd9750710cdbab80cfb4036dd87af20afe1552f . + + +.U3 "Hardcopy terminal support +.P +More of a funny anecdote is a check for printing to a +hardcopy terminal that remained in the code until Spring 2012, +when I finally removed it +.Ci b7764c4a6b71d37918a97594d866258f154017ca . +I surely would be very happy to see such a terminal in action, +maybe actually being able to work on it, but I fear my chances are null. +.P +The check only prevented a pager to be placed between the outputting +program (\c +.Pn mhl ) +and the terminal. +In nmh, this could have been ensured with the +.Sw -nomoreproc +at the command line statically, too. +In mmh, set the profile entry +.Pe Pager +or the environment variable +.Ev PAGER +to +.Pn cat . + + + + +.H2 "Attachments +.P +The mind model of email attachments is unrelated to MIME. +Although the MIME RFCs (2045 through 2049) define the technical +requirements for having attachments, they do not mention the the word +``attachment''. +Instead of attachments, MIME talks about ``multi-part message bodies'' +[RFC\|2045], a more general concept. +Multi-part messages are messages +``in which one or more different +sets of data are combined in a single body'' +[RFC\|2046]. +MIME keeps its descriptions generic; +it does not imply specific usage models. +In email one usage model became prevalent: attachments. +The idea is having a main text document with files of arbitrary kind +attached to it. +In MIME terms, this is a multi-part message having a text part first +and parts of arbitray type following. +.P +MH's MIME support is a direct implementation of the RFCs. +The perception of the topic described in the RFCs is clearly visible +in MH's implementation. +Thus, MH had all the MIME features but no idea of attachments. +Today, however, users don't need all the MIME features but they want +convenient attachment handling. + +.U3 "Composing MIME Messages +.P +In order to improve the situation on the message composing side, +Jon Steinhart had added an attachment system to nmh in 2002. +.Ci 7480dbc14bc90f2d872d434205c0784704213252 +In the file +.Fn docs/README-ATTACHMENTS , +he described his motivation to do so as such: +.QS +Although nmh contains the necessary functionality for MIME message handing, +the interface to this functionality is pretty obtuse. +There's no way that I'm ever going to convince my partner to write +.Pn mhbuild +composition files! +.QE +.LP +With this change, the mind model of attachments entered nmh. +In the same document: +.QS +These changes simplify the task of managing attachments on draft files. +They allow attachments to be added, listed, and deleted. +MIME messages are automatically created when drafts with attachments +are sent. +.QE +.LP +Unfortunately, the attachment system, +like any new facilities in nmh, +was deactive by default. +.P +During my work in Argentina, I tried to improve the attachment system. +But, because of great opposition in the nmh community, +my patch died as a proposal on the mailing list, after long discussions. +.[ +nmh-workers attachment proposal +.] +In Januar 2012, I extended the patch and applied it to mmh. +.Ci 8ff284ff9167eff8f5349481529332d59ed913b1 +In mmh, the attachment system is active by default. +Instead of command line switches, the +.Pe Attachment-Header +profile entry is used to specify +the name of the attachment header field. +It is pre-defined to +.Hd Attach . +.P +To add an attachment to a draft, simply add an attachment header: +.VS +To: bob +Subject: The file you wanted +Attach: /path/to/the/file-bob-wanted +-------- +Here it is. +VE +The header field can be added to the draft manually in the editor, +or by using the `attach' command at the WhatNow prompt, or +non-interactively with +.Pn anno : +.VS +anno -append -nodate -component Attach -text /path/to/attachment +VE +Drafts with attachment headers are converted to MIME automatically by +.Pn send . +The conversion to MIME is invisible to the user. +The draft stored in the draft folder is always in source form, with +attachment headers. +If the MIMEification fails, for instance because the file to attach +is not accessible, the original draft is not changed. +.P +The attachment system handles the forwarding of messages, too. +If the attachment header value starts with a plus character (`+'), +like in +.Cl "Attach: +bob 30 42" , +The given messages in the specified folder will be attached. +This allowed to simplify +.Pn forw . +.Ci f41f04cf4ceca7355232cf7413e59afafccc9550 +.P +Closely related to attachments is non-ASCII text content, +because it requires MIME too. +In nmh, the user needed to call `mime' at the WhatNow prompt +to have the draft converted to MIME. +This was necessary whenever the draft contained non-ASCII characters. +If the user did not call `mime', a broken message would be sent. +Therefore, the +.Pe automimeproc +profile entry could be specified to have the `mime' command invoked +automatically each time. +Unfortunately, this approach conflicted with with attachment system +because the draft would already be in MIME format at the time +when the attachment system wanted to MIMEify it. +To use nmh's attachment system, `mime' must not be called at the +WhatNow prompt and +.Pe automimeproc +must not be set in the profile. +But then the case of non-ASCII text without attachment headers was +not caught. +All in all, the solution was complex and irritating. +My patch from December 2010 would have simplified the situation. +.P +Mmh's current solution is even more elaborate. +Any necessary MIMEification is done automatically. +There is no `mime' command at the WhatNow prompt anymore. +The draft will be converted automatically to MIME when either an +attachment header or non-ASCII text is present. +Further more, the special meaning of the hash character (`#') +at line beginnings in the draft message is removed. +Users need not at all deal with the whole topic. +.P +Although the new approach does not anymore support arbitrary MIME +compositions directly, the full power of +.Pn mhbuild +can still be accessed. +Given no attachment headers are included, the user can create +.Pn mhbuild +composition drafts like in nmh. +Then, at the WhatNow prompt, he needs to invoke +.Cl "edit mhbuild +to convert it to MIME. +Because the resulting draft does neither contain non-aASCII characters +nor has it attachment headers, the attachment system will not touch it. +.P +The approach taken in mmh is taylored towards todays most common case: +a text part with possibly attachments. +This case is simplified a lot for users. + +.U3 "MIME Type Guessing +.P +The use of +.Pn mhbuild +composition drafts had one notable advantage over attachment headers +from the programmer's point of view: The user provides the appropriate +MIME types for files to include. +The attachment system needs to find out the correct MIME type itself. +This is a difficult task, yet it spares the user irritating work. +Determining the correct MIME type of content is partly mechanical, +partly intelligent work. +Forcing the user to find out the correct MIME type, +forces him to do partly mechanical work. +Letting the computer do the work, can lead to bad choices for difficult +content. +For mmh, the latter option was chosen. +.P +Determining the MIME type by the suffix of the file name is a dumb +approach, yet it is simple to implement and provides good results +for the common cases. +Mmh implements this approach in the +.Pn print-mimetype +script. +Using it is the default choice. +.P +A far better but less portable approach is the use of +.Pn file . +This standard tool tries to determine the type of files. +Unfortunately, its capabilities and accuracy varies from system to system. +Additionally, its output was only intended for human beings, +but not to be used by programs. +It varies much. +Nevertheless, modern versions of GNU +.Pn file , +which is prevalent on the popular GNU/Linux systems, +provides MIME type output in machine-readable form. +Although this solution is highly system-dependent, +it solves the difficult problem well. +On systems where GNU +.Pn file , +version 5.04 or higher, is available it should be used. +One needs to specify the following profile entry to do so: +.VS +Mime-Type-Query: file -b --mime +VE +.LP +Other versions of +.Pn file +might possibly be usable with wrapper scripts to reformat the output. +The diversity among +.Pn file +implementations is great; one needs to check the local variant. +.P +If no MIME type can be determined, text content gets sent as +`text/plain' and anything else under the generic fall-back type +`application/octet-stream'. +It is not possible in mmh to override the automatic MIME type guessing +for a specific file. +To do so, the user would need to know in advance for which file +the automatic guessing does fail, or the system would require interaction. +I consider both cases impractical. +The existing solution should be sufficient. +If not, the user may always fall back to +.Pn mhbuild +composition drafts and ignore the attachment system. + + +.U3 "Storing Attachments +.P +FIXME + + +.U3 "Showing MIME Messages +.P +FIXME + + + +.H2 "Digital Cryptography +.P +Signing and encryption. + + + +.H2 "Modern Defaults +.P +Just to give one example, for me it took one year of using nmh +before I became aware of the existence of the attachment system. +One could argue that this fact disqualifies my reading of the +documentation. +If I would have installed nmh from source back then, I could agree. +Yet I had used a prepackaged version and had expected that it would +just work. + + + +.H1 "Code Style +.P +foo + + +.H2 "Standard Code +.P +POSIX + +.U3 "Converting to Standard Code +.P +One part of this task was converting obsolete code constructs +to standard constructs. +As I'm not even thirty years old and have no more than seven years of +Unix experience, I needed to learn about the history in retrospective. +Older people likely have used those ancient constructs themselves +and have suffered from their incompatibilities and have longed for +standardization. +Unfortunately, I have only read that others had done so. +This put me in a much more difficult positions when working on the old +code. +I needed to recherche what other would have known by heart from +experience. +All my programming experience comes from a time past ANSI C +and past POSIX. +Although I knew about the times before, I took the +current state implicitly for granted most of the time. +.P +Being aware of +these facts, I rather let people with more historic experience solve the +task of converting the ancient code constructs to standardized ones. +Luckily, Lyndon Nerenberg focused on this task at the nmh project. +He converted large parts of the code to POSIX constructs, removing +the conditionals compilation for now standardized features. +I'm thankful for this task being solved. +I only pulled the changes into +mmh. + + + + +.H2 "Separation + +.U2 "MH Directory Split +.P +In MH and nmh, a personal setup had consisted of two parts: +The MH profile, named +.Fn \&.mh_profile +and being located directly in the user's home directory. +And the MH directory, where all his mail messages and also his personal +forms, scan formats, other configuration files are stored. +The location +of this directory could be user-chosen. +The default was to name it +.Fn Mail +and have it directly in the home directory. +.P +I've never liked the data storage and the configuration to be intermixed. +They are different kinds of data. +One part, are the messages, +which are the data to operate on. +The other part, are the personal +configuration files, which are able to change the behavior of the operations. +The actual operations are defined in the profile, however. +.P +When storing data, one should try to group data by its type. +There's sense in the Unix file system hierarchy, where configuration +file are stored separate (\c +.Fn /etc ) +to the programs (\c +.Fn /bin +and +.Fn /usr/bin ) +to their sources (\c +.Fn /usr/src ). +Such separation eases the backup management, for instance. +.P +In mmh, I've reorganized the file locations. +Still there are two places: +There's the mail storage directory, which, like in MH, contains all the +messages, but, unlike in MH, nothing else. +Its location still is user-chosen, with the default name +.Fn Mail , +in the user's home directory. +This is much similar to the case in nmh. +The configuration files, however, are grouped together in the new directory +.Fn \&.mmh +in the user's home directory. +The user's profile now is a file, named +.Fn profile , +in this mmh directory. +Consistently, the context file and all the personal forms, scan formats, +and the like, are also there. +.P +The naming changed with the relocation. +The directory where everything, except the profile, had been stored (\c +.Fn $HOME/Mail ), +used to be called \fIMH directory\fP. +Now, this directory is called the +user's \fImail storage\fP. +The name \fImmh directory\fP is now given to +the new directory +(\c +.Fn $HOME/.mmh ), +containing all the personal configuration files. +.P +The separation of the files by type of content is logical and convenient. +There are no functional differences as any possible setup known to me +can be implemented with both approaches, although likely a bit easier +with the new approach. +The main goal of the change had been to provide +sensible storage locations for any type of personal mmh file. +.P +In order for one user to have multiple MH setups, he can use the +environment variable +.Ev MH +the point to a different profile file. +The MH directory (mail storage plus personal configuration files) is +defined by the +.Pe Path +profile entry. +The context file could be defined by the +.Pe context +profile entry or by the +.Ev MHCONTEXT +environment variable. +The latter is useful to have a distinct context (e.g. current folders) +in each terminal window, for instance. +In mmh, there are three environment variables now. +.Ev MMH +may be used to change the location of the mmh directory. +.Ev MMHP +and +.Ev MMHC +change the profile and context files, respectively. +Besides providing a more consistent feel (which simply is the result +of being designed anew), the set of personal configuration files can +be chosen independently from the profile (including mail storage location) +and context, now. +Being it relevant for practical use or not, it +de-facto is an improvement. +However, the main achievement is the +split between mail storage and personal configuration files. + + +.H2 "Modularization +.P +whatnowproc +.P +The \fIMH library\fP +.Fn libmh.a +collects a bunch of standard functions that many of the MH tools need, +like reading the profile or context files. +This doesn't hurt the separation. + + +.H2 "Style +.P +Code layout, goto, ... + +.P +anno rework + + + + +.H1 "Concept Exploitation/Homogeneity + + +.H2 "Draft Folder +.P +Historically, MH provided exactly one draft message, named +.Fn draft +and +being located in the MH directory. +When starting to compose another message +before the former one was sent, the user had been questioned whether to use, +refile or replace the old draft. +Working on multiple drafts at the same time +was impossible. +One could only work on them in alteration by refiling the +previous one to some directory and fetching some other one for reediting. +This manual draft management needed to be done each time the user wanted +to switch between editing one draft to editing another. +.P +To allow true parallel editing of drafts, in a straight forward way, the +draft folder facility exists. +It had been introduced already in July 1984 +by Marshall T. Rose. +The facility was deactivated by default. +Even in nmh, the draft folder facility remained deactivated by default. +At least, Richard Coleman added the man page +.Mp mh-draft(5) +to document +the feature well. +.P +The only advantage of not using the draft folder facility is the static +name of the draft file. +This could be an issue for MH front-ends like mh-e. +But as they likely want to provide working on multiple drafts in parallel, +the issue is only concerning compatibility. +The aim of nmh to stay compatible +prevented the default activation of the draft folder facility. +.P +On the other hand, a draft folder is the much more natural concept than +a draft message. +MH's mail storage consists of folders and messages, +the messages named with ascending numbers. +A draft message breaks with this +concept by introducing a message in a file named +.Fn draft . +This draft +message is special. +It can not be simply listed with the available tools, +but instead requires special switches. +I.e. corner-cases were +introduced. +A draft folder, in contrast, does not introduce such +corner-cases. +The available tools can operate on the messages within that +folder like on any messages within any mail folders. +The only difference +is the fact that the default folder for +.Pn send +is the draft folder, +instead of the current folder, like for all other tools. +.P +The trivial part of the change was activating the draft folder facility +by default and setting a default name for this folder. +Obviously, I chose +the name +.Fn +drafts . +This made the +.Sw -draftfolder +and +.Sw -draftmessage +switches useless, and I could remove them. +The more difficult but also the part that showed the real improvement, +was updating the tools to the new concept. +.Sw -draft +switches could +be dropped, as operating on a draft message became indistinguishable to +operating on any other message for the tools. +.Pn comp +still has its +.Sw -use +switch for switching between its two modes: (1) Compose a new +draft, possibly by taking some existing message as a form. +(2) Modify +an existing draft. +In either case, the behavior of +.Pn comp is +deterministic. +There is no more need to query the user. +I consider this +a major improvement. +By making +.Pn send +simply operate on the current +message in the draft folder by default, with message and folder both +overridable by specifying them on the command line, it is now possible +to send a draft anywhere within the storage by simply specifying its folder +and name. +.P +All theses changes converted special cases to regular cases, thus +simplifying the tools and increasing the flexibility. + + +.H2 "Trash Folder +.P +Similar to the situation for drafts is the situation for removed messages. +Historically, a message was deleted by renaming. +A specific +\fIbackup prefix\fP, often comma (\c +.Fn , ) +or hash (\c +.Fn # ), +being prepended to the file name. +Thus, MH wouldn't recognize the file +as a message anymore, as only files whose name consists of digits only +are treated as messages. +The removed messages remained as files in the +same directory and needed some maintenance job to truly delete them after +some grace time. +Usually, by running a command similar to +.VS +find /home/user/Mail -ctime +7 -name ',*' | xargs rm +VE +in a cron job. +Within the grace time interval +the original message could be restored by stripping the +the backup prefix from the file name. +If however, the last message of +a folder is been removed \(en say message +.Fn 6 +becomes file +.Fn ,6 +\(en and a new message enters the same folder, thus the same +numbered being given again \(en in our case +.Fn 6 +\(en, if that one +is removed too, then the backup of the former message gets overwritten. +Thus, the ability to restore removed messages does not only depend on +the ``sweeping cron job'' but also on the removing of further messages. +This is undesirable, because the real mechanism is hidden from the user +and the consequences of further removals are not always obvious. +Further more, the backup files are scattered within the whole mail +storage, instead of being collected at one place. +.P +To improve the situation, the profile entry +.Pe rmmproc +(previously named +.Pe Delete-Prog ) +was introduced, very early. +It could be set to any command, which would care for the mail removal +instead of taking the default action, described above. +Refiling the to-be-removed files to some garbage folder was a common +example. +Nmh's man page +.Mp rmm(1) +proposes +.Cl "refile +d +to move messages to the garbage folder and +.Cl "rm `mhpath +d all` +the empty the garbage folder. +Managing the message removal this way is a sane approach. +It keeps +the removed messages in one place, makes it easy to remove the backup +files, and, most important, enables the user to use the tools of MH +itself to operate on the removed messages. +One can +.Pn scan +them, +.Pn show +them, and restore them with +.Pn refile . +There's no more +need to use +.Pn mhpath +to switch over from MH tools to Unix tools \(en MH can do it all itself. +.P +This approach matches perfect with the concepts of MH, thus making +it powerful. +Hence, I made it the default. +And even more, I also +removed the old backup prefix approach, as it is clearly less powerful. +Keeping unused alternative in the code is a bad choice as they likely +gather bugs, by not being constantly tested. +Also, the increased code +size and more conditions crease the maintenance costs. +By strictly +converting to the trash folder approach, I simplified the code base. +.Pn rmm +calls +.Pn refile +internally to move the to-be-removed +message to the trash folder (\c +.Fn +trash +by default). +Messages +there can be operated on like on any other message in the storage. +The sweep clean, one can use +.Cl "rmm -unlink +trash a" , +where the +.Sw -unlink +switch causes the files to be truly unliked instead +of moved to the trash folder. + + +.H2 "Path Notations +.P +foo + + +.H2 "MIME Integration +.P +user-visible access to whole messages and MIME parts are inherently +different + + +.H2 "Of One Cast +.P