# HG changeset patch # User markus schnalke # Date 1339517095 -7200 # Node ID 7d5b180de5425356b58982c1d4a6b91d20fb8d59 # Parent 1aa541b2ed9688c8d41fddd804b4e64b5cb15670 All kinds of rework plus new refs. diff -r 1aa541b2ed96 -r 7d5b180de542 bib --- a/bib Tue Jun 12 09:37:01 2012 +0200 +++ b/bib Tue Jun 12 18:04:55 2012 +0200 @@ -109,6 +109,7 @@ %D 1994 %I Addison-Wesley %O \s-1ISBN\s0: 0-201-54777-5 +%K mcilroy unix philosophy (p. 53) %A Ken Thompson %A Dennis M. Ritchie @@ -245,3 +246,24 @@ %B Unix Incompatibility Notes %D 2000\(en2004 %O \f(CW\s-1http://www.unixpapa.com/incnote/dbm.html\s0 + +%L RFC\|821 +%A Jonathan B. Postel +%T Simple Mail Transfer Protocol +%S Request for Comments +%V 821 +%I IETF +%D August 1982 +%O \f(CW\s-1http://www.ietf.org/rfc/rfc821.txt\s0 + +%A M. D. McIlroy +%A E. N. Pinson +%A B. A. Tague +%T UNIX Time-Sharing System: Foreword +%J The Bell System Technical Journal +%I Bell Laboratories +%D 1978 +%V 57 +%N 6 +%P 1902 +%K bstj diff -r 1aa541b2ed96 -r 7d5b180de542 ch01.roff --- a/ch01.roff Tue Jun 12 09:37:01 2012 +0200 +++ b/ch01.roff Tue Jun 12 18:04:55 2012 +0200 @@ -325,12 +325,12 @@ .P The general goals for the mmh project are the following: .IP "Stream-lining -Mmh should be stripped down to its core, which is the MUA part of emailing. +Mmh should be stripped down to its core, which is the user agent (MUA). The feature set should be distilled to the ones really needed, effectively removing corner-cases. Parts that don't add to the main task of being a conceptionally appealing MUA should be removed. -This includes, the MTA and MRA facilities. +This includes, the mail submission and mail retrieval facilities. Choice should be reduced to the main options. .IP "Modernizing Mmh's feature set needs to become more modern. diff -r 1aa541b2ed96 -r 7d5b180de542 ch03.roff --- a/ch03.roff Tue Jun 12 09:37:01 2012 +0200 +++ b/ch03.roff Tue Jun 12 18:04:55 2012 +0200 @@ -3,7 +3,10 @@ This main chapter discusses the practical work done in the mmh project. It is structured along the goals to achieve. The concrete work done is described in the examples of how the general goals were achieved. - +The discussion compares the current version of mmh with the state of +nmh just before the mmh project started, i.e. Fall 2011. +Current changes of nmh will be mentioned only as side notes. +.\" XXX where do I discuss the parallel development of nmh? @@ -12,32 +15,47 @@ .P MH had been considered an all-in-one system for mail handling. The community around nmh has a similar understanding. -In fundamental difference, should be a MUA only. -I believe that all-in-one mail systems are obsolete. -There are excellent specialized MTAs, like Postfix; -there are specialized MDAs, like Procmail; there are specialized -MRAs, like Fetchmail. I believe it's best to use them instead of -providing the same function ourselves. Doing something well, requires to -focus on a small set of aspects. The more -it is possible to focus, the better the result in this particular -area will be. Usually, the limiting resource in Free Software -community development is man power. -If the development power is even spread over a large -development area, it becomes more difficult to -compete with the specialists in the various fields. This is even -increased, given the small community \(en developers and users \(en -that MH-based mail systems have. In consequence, I believe that the -available resources should be focused to the point where MH is -most unique. This is clearly the MUA part. +In fundamental difference, mmh shall be a MUA only. +I believe that the development of all-in-one mail systems is obsolete. +Today, email is too complex to be fully covered by single projects. +Such a project won't be able to excel in all aspects. +Instead, the aspects of email should be covered my multiple projects, +which then can be combined to form a complete system. +Excellent implementations for the various aspects of email exist already. +Just to name three examples: Postfix is a specialized MTA, +Procmail is a specialized MDA, and Fetchmail is a specialized MRA. +I believe that it is best to use such specilized tools instead of +providing the same function again as a side-component in the project. .P -The goal for mmh was to remove peripheral parts and stream-line -it for the MUA task. +Doing something well, requires to focus on a small set of specific aspects. +Under the assumption that focused development produces better results +in the particular area, specialized projects will likely be superior +in their field of focus. +Hence, all-in-one mail system projects \(en no matter if monolithic +or modular \(en will never be the best choice in any of the fields. +Even in providing the best consistent all-in-one system they are likely +to be beaten by projects that focus only on integrating existing mail +components to a homogenious system. +.P +The limiting resource in Free Software community development +is usually man power. +If the development power is spread over a large development area, +it becomes even more difficult to compete with the specialists in the +various fields. +The concrete situation for MH-based mail systems is even tougher, +given the small and aged community, including both developers and users, +it has. +.P +In consequence, I believe that the available development resources +should be focused on the point where MH is most unique. +This is clearly the user interface \(en the MUA. +Peripheral parts should be removed to stream-line mmh for the MUA task. -.H2 "Removal of Mail Transfer Facilities +.H2 "Removal of the Mail Transfer Facilities .P In contrast to nmh, which also provides mail submission and mail retrieval -facilities, mmh is a MUA only. +agents, mmh is a MUA only. This general difference in the view on the character of nmh initiated the development of mmh. Removing the mail transfer facilities had been the first work task @@ -49,58 +67,67 @@ This part was implemented by the .Pn post command. -The changes in emailing -demanded changes in this part of nmh in the last years. +The changes in emailing in the last years +demanded changes in this part of nmh too. Encryption and authetication for network connections -needed to be supported, hence TLS and SASL were introduced -into nmh. This added complexity to the nmh without improving it in -its core functions. Also, keeping up with recent developments in -this field requires development power and specialists. -For mmh this whole facility was cut off. +needed to be supported, hence TLS and SASL were introduced into nmh. +This added complexity to nmh without improving it in its core functions. +Also, keeping up with recent developments in the field of +mail transfer requires development power and specialists. +In mmh this whole facility was simply cut off. .Ci f6aa95b724fd8c791164abe7ee5468bf5c34f226 .Ci fecd5d34f65597a4dfa16aeabea7d74b191532c3 .Ci 156d35f6425bea4c1ed3c4c79783dc613379c65b -Instead, mmh depends on an external MTA. +Instead, mmh depends on an external MSA. The only outgoing interface available to mmh is the .Pn sendmail -command. -Almost any MTA provides a -.Pn sendmail -command. -If not, any program can be substituted if it reads the -message from the standard input, extracts the recipient addresses -from the message header and does not conflict -with sendmail-specific command line options. +command, which almost any MSA provides. +If not, a wrapper program can be written. +It must read the message from the standard input, extract the +recipient addresses from the message header, and hand the message +over to the MSA. +For example, a wrapper script for qmail would be: +.VS +#!/bin/sh +# ignore command line arguments +exec qmail-inject +VE +The requirement to parse the recipient addresses out of the message header +is likely to be removed in the future. +Then mmh would give the recipient addresses as command line arguments. +This is clearly the better interface, but mmh does not provide it yet. +.\" XXX implement it .P To retrieve mail, the .Pn inc command established network connections and spoke POP3 to retrieve mail from remote servers. As with mail submission, the network connections required encryption and -authentication, thus TLS and SASL was added. -As POP3 becomes more and more superseded by IMAP, support for message -retrieval through IMAP will become necessary to be added soon, too. +authentication, thus TLS and SASL were added. +Support for message retrieval through IMAP will become necessary +to be added soon, too, and so on for any changes in mail transfer. Mmh has dropped the support for retrieving mail from remote locations. .Ci ab7b48411962d26439f92f35ed084d3d6275459c Instead, it depends on an external tool to cover this task. -There exist two paths for messages to enter mmh's mail storage: -They can be incorporate with +In mmh there exist two paths for messages to enter mmh's mail storage: +(1) Mail can be incorporate with .Pn inc -from the system maildrop, or +from the system maildrop, or (2) with .Pn rcvstore -reads them from the standard input. +by reading them, one at a time, from the standard input. .P With the removal of the MSA and MRA, mmh converted from an all-in-one -mail system to being only a MUA. -Following the Unix philosophy, it focuses on one job and +mail system to being a MUA only. +Following the Unix philosophy, it now focuses on one job and tries to do that one well. Not only the programs follow that tenet but also the project itself does so. Now, of course, mmh depends on third-party software. -An external MTA/MSA is required to transfer mail to the outside world; +An external MSA is required to transfer mail to the outside world; an external MRA is required to retrieve mail from remote machines. There exist excellent implementations of such software, which do this specific task likely better than the internal -versions had done it. Also, the best suiting programs can be freely chosen. +versions had done it. +Also, the best suiting programs can be freely chosen. .P As it had already been possible to use an external MSA or MRA, why not keep the internal version for convenience? @@ -110,50 +137,79 @@ or .Pn less aren't available, appears to be ridiculous. -Now, an MSA or MRA is clearly more complex than a text pager, -and not necessarily available but still the concept holds: -design the system orthogonally. -If it is conceptionally more elegant to have the MTA to be a separate tool -\(en as the RFCs propose this split, this is likely the case \(en -then separate. +Now, an MSA or MRA is more complex than a text pager +and not necessarily available but still the concept of orthogonal +design holds: ``Write programs that do one thing and do it well.'' +.[ +mcilroy unix phil +p. 53 +.] +.[ +mcilroy bstj foreword +.] +Here, this part of the Unix philosophy was applied not only +to the programs but to the project itself. +In other words: +``Develop projects that focus on one thing and do it well.'' +Projects grown complex should be split for the same reasons programs grown +complex should be split. +If it is conceptionally more elegant to have the MSA and MRA +separate projects then they should be separated. +This is the case here, in my opinion. +The RFCs propose this separation by clearly distinguishing the different +mail handling tasks. +.[ +rfc 821 +.] +The small interfaces between the mail agents support the separation. .P -Further more, if programs become complex, they should be split; -and if projects become complex, they should be split, too. -Essential complexity is defined by the problem. -Decades ago, emailing had been small and simple. +In the beginning, email had been small and simple. (\c .Pn /bin/mail had once covered anything there was to email and still had been small and simple.) Then the essential complexity of email increased. -Email tools needed to react. +(Essential complexity is the complexity defined by the problem itself.\0 +.[[ +brooks no silver bullet +.]]) +Email systems reacted to this change: They grew. +RFCs started to introduce mail agents and separated the various tasks +because the existing tasks became more extensive and new tasks appeared. +Again, email systems grew, or they split parts off. In nmh, for instance, the POP server, which the original MH had included, was removed. -Now is the time to go one step further and remove the MSA and MRA. -Not only does it decrease the code amount of the project, -but more important, it removes the whole field of message transfer -with all its implications for the project. -It removes the need to adjust to any changes concerning network transfer. +Now is the time to go one step further and remove the MSA and MRA, too. +Not only does this decrease the code size of the project, +but, more important, it unburdens mmh of the whole field of +message transfer with all its implications for the project. +There's no more need to concern with changes in network transfer. This independence is received by depending on an external program that covers the field. Today, this is a reasonable exchange. .P -To add some kind of function, there's always the choice -among implementing the function in the project directly, -depending on a library that provides the function, or depending on -a program that provides the function. -Whereas adding the function directly to the project increases the +Function can be added in three different ways: +.BU +Implementing the function originally in the project. +.BU +Depending on a library that provides the function. +.BU +Depending on a program that provides the function. +.P +Whereas adding the function originally to the project increases the code size most and requires most maintenance and development work, -it makes the project most independent. -Using libraries or external programs require less -maintenance work. -Programs have the smallest interfaces, providing the most separation -but possibly limiting the information exchange. -External libraries are stronger connected than external programs but -allow better information exchange. -Adding more code to a project does always increase maintenance work. -Implementing complex functions directly in the project will add -a lot of code. This should be avoided if possible. +it makes the project most independent of other software. +Using libraries or external programs require less maintenance work +but introduces dependencies on external software. +Programs have the smallest interfaces and provide the best separation +but possibly limit the information exchange. +External libraries are stronger connected than external programs, +thus information can be exchanged more flexible. +Adding code to a project increases maintenance work. +.\" XXX ref +Implementing complex functions originally in the project will add +a lot of code. +This should be avoided if possible. Hence, the dependencies only change in kind, not in their existence. In mmh, library dependencies on .Pn libsasl2 @@ -161,21 +217,23 @@ .Pn libcrypto /\c .Pn libssl were treated against program dependencies on an MSA and an MRA. +This also meant treating build-time dependencies against run-time +dependencies. Besides program dependencies providing the stronger separation and being more flexible, they also allowed over 6\|000 lines of code to be removed from mmh. This made mmh's code base about 12\|% smaller. -Reducing the projects code size by such an amount without actually -losing function is a convincing argument. -Actually, as external MSAs and MRAs are likely better -than the project's internal version, the user even gains functionality. +Reducing the project's code size by such an amount without actually +losing functionality is a convincing argument. +Actually, as external MSAs and MRAs are likely superior to the +project's internal versions, the common user even gains functionality. .P Users of MH should not have problems to set up an external MSA and MRA. Also, the popular MSAs and MRAs have large communities and a lot of documentation available. -Choices for MSAs range from the full-featured +Choices for MSAs range from full-featured MTAs like .I Postfix -over mid-size solutions like +over mid-size MTAs like .I masqmail and .I dma @@ -193,32 +251,37 @@ .H2 "Removal of non-MUA Tools .P -Some MH tools were removed because they didn't add to the MUA's job. -It is a design goal of mmh to remove the parts that are rarely used. -The project shall become more stream-lined and focused. -Rarely used and loosely related tools distract from the lean appearance. -They require maintenance work without adding to the core task. +One goal of mmh is to remove the tools that are not part of the MUA's task. +Further more, any tools that don't improve the MUA's job significently +should be removed. +Loosely related and rarely used tools distract from the lean appearance. +They require maintenance work without adding much to the core task. +On removing these tools, the project shall become more stream-lined +and focused. In mmh the following tools are not available anymore: .BU .Pn conflict -was removed because it is a mail system maintenance tool. +was removed .Ci 8b235097cbd11d728c07b966cf131aa7133ce5a9 -Besides, it even checks +because it is a mail system maintenance tool that is not MUA-related. +It even checked .Fn /etc/passwd and .Fn /etc/group -for consistency, which has nothing at all to do with emailing. -The tool might be useful, but it should not be shipped with mmh. +for consistency, which is completely unrelated to email. +A tool like +.Pn conflict +is surely useful, but it should not be shipped with mmh. .\" XXX historic reasons? .BU .Pn rcvtty -was removed because its usecase of writing to the user's terminal +was removed +.Ci 14767c94b3827be7c867196467ed7aea5f6f49b0 +because its usecase of writing to the user's terminal on receiving of mail is obsolete. -.Ci 14767c94b3827be7c867196467ed7aea5f6f49b0 -If users like to be -informed of new mail, the shell's +If users like to be informed of new mail, the shell's .Ev MAILPATH -variable or graphical notifications are more appealing. +variable or graphical notifications are technically more appealing. Writing directly to a terminals is hardly ever wanted today. If though one wants to have it this way, the standard tool .Pn write @@ -228,10 +291,11 @@ VE .BU .Pn viamail -was removed when the new attachment system was activated, because +was removed +.Ci eda72d6a7a7c20ff123043fb7f19c509ea01f932 +when the new attachment system was activated, because .Pn forw could then cover the task itself. -.Ci eda72d6a7a7c20ff123043fb7f19c509ea01f932 The program .Pn sendfiles was rewritten as a shell script wrapper around @@ -239,29 +303,31 @@ .Ci 0e82199cf3c991a173e0ac8aa776efdb3ded61e6 .BU .Pn msgchk -was removed, because it lost its use case when POP support was removed. -.Ci bb9360ead7eb7a3fedcce2eeedfc660014e41dbe +was removed +.Ci bb9360ead7eb7a3fedcce2eeedfc660014e41dbe , +because it lost its use case when POP support was removed. A call to .Pn msgchk -provided hardly more information than +provided hardly more information than: .VS ls -l /var/mail/meillo VE -though it distinguished between old and new mail. -This detail information and can be retrieved with +It did distinguished between old and new mail, but +this detail information and can be retrieved with .Pn stat (1), too. A very small shell script could be written to output the information in a similar way, if truly necessary. As mmh's .Pn inc -only incorporates mail from the user's local maildrop +only incorporates mail from the user's local maildrop, and thus no data transfers over slow networks are involved, there's hardly any need to check for new mail before incorporating it. .BU .Pn msh -was removed because the tool was in conflict with the philosophy of MH. +was removed .Ci 916690191222433a6923a4be54b0d8f6ac01bd02 +because the tool was in conflict with the philosophy of MH. It provided an interactive shell to access the features of MH, but it wasn't just a shell, tailored to the needs of mail handling. Instead it was one large program that had several MH tools built in. @@ -286,8 +352,8 @@ .Pn rcvtty and .Pn msgchk -are rarely used and can be implemented in different ways, -then why should one keep them? +are assumed to be rarely used and can be implemented in different ways, +why should one keep them? Removing them stream-lines mmh. .Pn viamail 's use case is now partly obsolete and partly covered by @@ -306,42 +372,42 @@ It should be removed, because including it is a violation of the idea that mmh is a MUA only. It should become a separate project. -But +However, .Pn slocal provides rule-based processing of messages, like filing them into different folders, which is otherwise not available in mmh. -Further more, +Although .Pn slocal -does neither pull dependencies nor a whole new technical area -into the project. -(See section XXX for the removing of the ndbm dependency.) -Still, -.Pn slocal -accounts for about 1\|000 lines of code that need to be maintained. +does neither pull in dependencies nor does it include a separate +technical area (cf. Sec. XXX), +still it accounts for about 1\|000 lines of code that need to be maintained. As .Pn slocal is almost self-standing, it should be split off into a separate project. This would cut the strong connection between the MUA mmh and the MDA .Pn slocal . -The MDA would become an alternative to -.I procmail , -as it would no longer be the need to install a complete MH system, too. +For anyone not using MH, +.Pn slocal +would become yet another independent MDA, like +.I procmail . +The need to install a complete MH system to have +.Pn slocal +would be gone. Likewise, mmh users could decide to use .I procmail -without having a second, unused MDA (\c -.Pn slocal ) +without having a second, unused MDA, +.Pn slocal , installed. That's conceptionally the best solution. Yet, .Pn slocal -was not removed. -I feel unsure with the removal. -Hence, the decision over +is not split off. +I feel unsure with removing it from mmh. +Hence, I defer the decision over +.Pn slocal . +In the meanwhile .Pn slocal -is deferred. -This does not hurt because -.Pn slocal -is completely unrelated to the rest of mmh. +does not hurt because it is unrelated to the rest of mmh. .H2 "\fLshow\fP and \fPmhshow\fP diff -r 1aa541b2ed96 -r 7d5b180de542 preface.roff --- a/preface.roff Tue Jun 12 09:37:01 2012 +0200 +++ b/preface.roff Tue Jun 12 18:04:55 2012 +0200 @@ -1,25 +1,24 @@ .H0 "Preface" no .P -I have discovered the mail client \fInmh\fP in September 2009. -At that time I used to use \fImutt\fP, as many advanced Unix users do. -When I read about nmh, its concepts had convinced me at once. -The transition from mutt to nmh was similar to -managing files in the Unix shell when being used to the +I have discovered the mail client \fInmh\fP in Fall 2009. +At that time I used \fImutt\fP, as many advanced Unix users do. +When I read about nmh, its concepts convinced me at once. +The transition from mutt to nmh was similar to beginning with +file management in the Unix shell when being used to the \fImidnight commander\fP, -or like editing with vi when being used to modeless editors. -Such a change is not trivial, but in being convinced by the +or like starting with vi when being used to modeless editors. +Such a change is not trivial, but, in being convinced by the concepts and by having done similar transitions for file management and editing already, it was not too difficult. In contrast, setting up nmh to a convenient state became a tedious task that took several months. Once having nmh arranged to a convenient state, I enjoyed using it because of its conceptional elegance and its scripting capabilities. -On the other hand, nevertheless, it still was -inconvenient for handling attachments, non-ASCII character encodings, -and similar features of modern emailing. +Nevertheless, it still was inconvenient for handling attachments, +non-ASCII character encodings, and similar features of modern emailing. My setup demanded more and more additional configuration and helper scripts -to get nmh behave the way I wanted, although my +to have nmh behave the way I wanted; yet my expectations were rather common for modern emailing. In being a computer scientist and programmer, I wanted to improve the situation. @@ -27,8 +26,7 @@ In Spring 2010, I asked on the \fInmh-workers\fP mailing list for the possibility to offer a Google Summer of Code project for me. Participating in the development of nmh this way appeared attractive to me, -because I would have been able to work full time on nmh as the project -could have been part of my official studies at university. +because I would have been able to work full time on nmh. Although the nmh community had been generally positive on the suggestion, the administrative work for a GSoC project had been to much to have it realized. @@ -39,29 +37,30 @@ .[ nmh-workers thread mta mua .] -In this central point, my opinion differed from the opinion of most others. -I argued for the MTA facility of nmh to be removed. -Besides the discussions, hardly any real work was done. -Being unable to work on nmh in a way that would be -accepted as part of my official studies, I needed to choose another project. +I argued for the MTA of nmh to be removed. +In this fundamental question, +my opinion differed from the opinion of most others. +Sadly, besides the discussions, hardly any real work was done. +Being unable to work on nmh in a way that would be accepted at university +as part of my studies, I needed to choose another project. .P Half a year later, starting in August 2010, I took one semester off to travel through Latin America. -During my time in Argentina, I planned to work on Free Software. +During my time in Argentina, I wanted to work on Free Software. This brought me back to nmh. Richard Sandelman, an active nmh user, cared for the official basis. Juan Granda, an argentine Free Software developer, -provided a computer with Internet connection for my work. +provided a computer with Internet connection. Thanks to them, I was able to work on nmh during my three-month -stay in Santiago del Estero in Argentina. -Quickly it became obvious that I wouldn't succeed with my main goal: -improving the character encoding handling within the project. -One of its ramifications is the -missing transfer decoding of quoted text in replies. +stay in Santiago del Estero, Argentina. +Quickly it became obvious that I wouldn't succeed with my main goal, +to improve the character encoding handling. +(One of its ramifications is the +missing transfer decoding of quoted text in replies.) As this is one of the most intricate parts of the system, the goal was simply set too high. Instead, I improved the code base as I read through it. -I found minor bugs for which I proposed fixes to the community. +I found minor bugs for which I proposed fixes. In the same go, I improved the documentation in minor ways. When I started with larger code changes, I had to discover that the community was reluctant to change. @@ -70,16 +69,17 @@ This led to long discussions, again. I came to understand their point of view, but it was different to mine. At the end of my three-month project, I had become familiar with -nmh's code base and community. +nmh's code base and community, I had improved the project in minor ways, -and I still was convinced that I wanted to go on to do so. +and I still was convinced that I wanted to continue to do so. .P Another half year later, the end of my studies came within reach. I needed a topic for my master's thesis. No question, I wanted to work on nmh. But well, not exactly on nmh, because I had accepted that the nmh community has different goals than I have. -This would result in much discussion and thus little progress. +Working on nmh would result in much discussion and, in consequence, +little progress. After careful thought, I decided to start an experimental version of nmh. I wanted to implement my own ideas of how an MH-like system should look like. I wanted to create a usable alternative version to be compared with