docs/master
view ch03.roff @ 76:2e61e0004a8f
Rework of existing text.
| author | markus schnalke <meillo@marmaro.de> |
|---|---|
| date | Tue, 05 Jun 2012 19:00:15 +0200 |
| parents | cefaa856d431 |
| children | 64f177ca2af1 |
line source
1 .H0 "Discussion
2 .P
3 This main chapter discusses the practical work done in the mmh project.
4 It is structured along the goals to achieve. The concrete work done
5 is described in the examples of how the general goals were achieved.
10 .H1 "Stream-lining
12 .P
13 MH had been considered an all-in-one system for mail handling.
14 The community around nmh has a similar understanding.
15 In fundamental difference, should be a MUA only.
16 I believe that all-in-one mail systems are obsolete.
17 There are excellent specialized MTAs, like Postfix;
18 there are specialized MDAs, like Procmail; there are specialized
19 MRAs, like Fetchmail. I believe it's best to use them instead of
20 providing the same function ourselves. Doing something well, requires to
21 focus on a small set of aspects. The more
22 it is possible to focus, the better the result in this particular
23 area will be. Usually, the limiting resource in Free Software
24 community development is man power.
25 If the development power is even spread over a large
26 development area, it becomes more difficult to
27 compete with the specialists in the various fields. This is even
28 increased, given the small community \(en developers and users \(en
29 that MH-based mail systems have. In consequence, I believe that the
30 available resources should be focused to the point where MH is
31 most unique. This is clearly the MUA part.
32 .P
33 The goal for mmh was to remove peripheral parts and stream-line
34 it for the MUA task.
37 .H2 "Removal of Mail Transfer Facilities
38 .P
39 In contrast to nmh, which also provides mail submission and mail retrieval
40 facilities, mmh is a MUA only.
41 This general difference in the view on the character of nmh
42 initiated the development of mmh.
43 Removing the mail transfer facilities had been the first work task
44 in the mmh project.
45 .P
46 The MSA is called \fIMessage Transfer Service\fP (MTS) in nmh.
47 The facility established network connections and spoke SMTP to submit
48 messages for relay to the outside world.
49 This part was implemented by the
50 .Pn post
51 command.
52 The changes in emailing
53 demanded changes in this part of nmh in the last years.
54 Encryption and authetication for network connections
55 needed to be supported, hence TLS and SASL were introduced
56 into nmh. This added complexity to the nmh without improving it in
57 its core functions. Also, keeping up with recent developments in
58 this field requires development power and specialists.
59 For mmh this whole facility was cut off.
60 .Ci f6aa95b724fd8c791164abe7ee5468bf5c34f226
61 .Ci fecd5d34f65597a4dfa16aeabea7d74b191532c3
62 .Ci 156d35f6425bea4c1ed3c4c79783dc613379c65b
63 Instead, mmh depends on an external MTA.
64 The only outgoing interface available to mmh is the
65 .Pn sendmail
66 command.
67 Almost any MTA provides a
68 .Pn sendmail
69 command.
70 If not, any program can be substituted if it reads the
71 message from the standard input, extracts the recipient addresses
72 from the message header and does not conflict
73 with sendmail-specific command line options.
74 .P
75 To retrieve mail, the
76 .Pn inc
77 command established network connections
78 and spoke POP3 to retrieve mail from remote servers.
79 As with mail submission, the network connections required encryption and
80 authentication, thus TLS and SASL was added.
81 As POP3 becomes more and more superseded by IMAP, support for message
82 retrieval through IMAP will become necessary to be added soon, too.
83 Mmh has dropped the support for retrieving mail from remote locations.
84 .Ci ab7b48411962d26439f92f35ed084d3d6275459c
85 Instead, it depends on an external tool to cover this task.
86 There exist two paths for messages to enter mmh's mail storage:
87 They can be incorporate with
88 .Pn inc
89 from the system maildrop, or
90 .Pn rcvstore
91 reads them from the standard input.
92 .P
93 With the removal of the MSA and MRA, mmh converted from an all-in-one
94 mail system to being only a MUA.
95 Following the Unix philosophy, it focuses on one job and
96 tries to do that one well.
97 Not only the programs follow that tenet but also the project itself does so.
98 Now, of course, mmh depends on third-party software.
99 An external MTA/MSA is required to transfer mail to the outside world;
100 an external MRA is required to retrieve mail from remote machines.
101 There exist excellent implementations of such software,
102 which do this specific task likely better than the internal
103 versions had done it. Also, the best suiting programs can be freely chosen.
104 .P
105 As it had already been possible to use an external MSA or MRA,
106 why not keep the internal version for convenience?
107 The question whether there is sense in having a fall-back pager in all
108 the command line tools, for the cases when
109 .Pn more
110 or
111 .Pn less
112 aren't available, appears to be ridiculous.
113 Now, an MSA or MRA is clearly more complex than a text pager,
114 and not necessarily available but still the concept holds:
115 design the system orthogonally.
116 If it is conceptionally more elegant to have the MTA to be a separate tool
117 \(en as the RFCs propose this split, this is likely the case \(en
118 then separate.
119 .P
120 Further more, if programs become complex, they should be split;
121 and if projects become complex, they should be split, too.
122 Essential complexity is defined by the problem.
123 Decades ago, emailing had been small and simple.
124 (\c
125 .Pn /bin/mail
126 had once covered anything there was to email and still had been small
127 and simple.)
128 Then the essential complexity of email increased.
129 Email tools needed to react.
130 In nmh, for instance, the POP server, which the original MH had included,
131 was removed.
132 Now is the time to go one step further and remove the MSA and MRA.
133 Not only does it decrease the code amount of the project,
134 but more important, it removes the whole field of message transfer
135 with all its implications for the project.
136 It removes the need to adjust to any changes concerning network transfer.
137 This independence is received by depending on an external program
138 that covers the field.
139 Today, this is a reasonable exchange.
140 .P
141 To add some kind of function, there's always the choice
142 among implementing the function in the project directly,
143 depending on a library that provides the function, or depending on
144 a program that provides the function.
145 Whereas adding the function directly to the project increases the
146 code size most and requires most maintenance and development work,
147 it makes the project most independent.
148 Using libraries or external programs require less
149 maintenance work.
150 Programs have the smallest interfaces, providing the most separation
151 but possibly limiting the information exchange.
152 External libraries are stronger connected than external programs but
153 allow better information exchange.
154 Adding more code to a project does always increase maintenance work.
155 Implementing complex functions directly in the project will add
156 a lot of code. This should be avoided if possible.
157 Hence, the dependencies only change in kind, not in their existence.
158 In mmh, library dependencies on
159 .Pn libsasl2
160 and
161 .Pn libcrypto /\c
162 .Pn libssl
163 were treated against program dependencies on an MSA and an MRA.
164 Besides program dependencies providing the stronger separation
165 and being more flexible, they also allowed
166 over 6\|000 lines of code to be removed from mmh.
167 This made mmh's code base about 12\|% smaller.
168 Reducing the projects code size by such an amount without actually
169 losing function is a convincing argument.
170 Actually, as external MSAs and MRAs are likely better
171 than the project's internal version, the user even gains functionality.
172 .P
173 Users of MH should not have problems to set up an external MSA and MRA.
174 Also, the popular MSAs and MRAs have large communities and a lot
175 of documentation available.
176 Choices for MSAs range from the full-featured
177 .I Postfix
178 over mid-size solutions like
179 .I masqmail
180 and
181 .I dma
182 to small forwarders like
183 .I ssmtp
184 and
185 .I nullmailer .
186 Choices for MRAs include
187 .I fetchmail ,
188 .I getmail ,
189 .I mpop
190 and
191 .I fdm .
194 .H2 "Removal of non-MUA Tools
195 .P
196 Some MH tools were removed because they didn't add to the MUA's job.
197 It is a design goal of mmh to remove the parts that are rarely used.
198 The project shall become more stream-lined and focused.
199 Rarely used and loosely related tools distract from the lean appearance.
200 They require maintenance work without adding to the core task.
201 In mmh the following tools are not available anymore:
202 .BU
203 .Pn conflict
204 was removed because it is a mail system maintenance tool.
205 .Ci 8b235097cbd11d728c07b966cf131aa7133ce5a9
206 Besides, it even checks
207 .Fn /etc/passwd
208 and
209 .Fn /etc/group
210 for consistency, which has nothing at all to do with emailing.
211 The tool might be useful, but it should not be shipped with mmh.
212 .\" XXX historic reasons?
213 .BU
214 .Pn rcvtty
215 was removed because its usecase of writing to the user's terminal
216 on receiving of mail is obsolete.
217 .Ci 14767c94b3827be7c867196467ed7aea5f6f49b0
218 If users like to be
219 informed of new mail, the shell's
220 .Ev MAILPATH
221 variable or graphical notifications are more appealing.
222 Writing directly to a terminals is hardly ever wanted today.
223 If though one wants to have it this way, the standard tool
224 .Pn write
225 can be used in a way similar to:
226 .DS
227 scan -file - | write `id -un`
228 .DE
229 .BU
230 .Pn viamail
231 was removed when the new attachment system was activated, because
232 .Pn forw
233 could then cover the task itself.
234 .Ci eda72d6a7a7c20ff123043fb7f19c509ea01f932
235 The program
236 .Pn sendfiles
237 was rewritten as a shell script wrapper around
238 .Pn forw .
239 .Ci 0e82199cf3c991a173e0ac8aa776efdb3ded61e6
240 .BU
241 .Pn msgchk
242 was removed, because it lost its use case when POP support was removed.
243 .Ci bb9360ead7eb7a3fedcce2eeedfc660014e41dbe
244 A call to
245 .Pn msgchk
246 provided hardly more information than
247 .DS
248 ls -l /var/mail/meillo
249 .DE
250 though it distinguished between old and new mail.
251 This detail information and can be retrieved with
252 .Pn stat (1),
253 too.
254 A very small shell script could be written to output the information
255 in a similar way, if truly necessary.
256 As mmh's
257 .Pn inc
258 only incorporates mail from the user's local maildrop
259 and thus no data transfers over slow networks are involved,
260 there's hardly any need to check for new mail before incorporating it.
261 .BU
262 .Pn msh
263 was removed because the tool was in conflict with the philosophy of MH.
264 .Ci 916690191222433a6923a4be54b0d8f6ac01bd02
265 It provided an interactive shell to access the features of MH,
266 but it wasn't just a shell, tailored to the needs of mail handling.
267 Instead it was one large program that had several MH tools built in.
268 This conflicts with the major feature of MH of being a tool chest.
269 .Pn msh 's
270 main use case had been accessing Bulletin Boards, which have seized to
271 be popular.
272 .P
273 Removing
274 .Pn msh ,
275 together with the truly archaic code relicts
276 .Pn vmh
277 and
278 .Pn wmh ,
279 saved more than 7\|000 lines of C code \(en
280 about 15\|% of the project's original source code amount.
281 .P
282 Having less code (with equal readability, of course)
283 for the same functionality is an advantage.
284 Less code means less bugs and less maintenance work.
285 As
286 .Pn rcvtty
287 and
288 .Pn msgchk
289 are rarely used and can be implemented in different ways,
290 then why should one keep them?
291 Removing them stream-lines mmh.
292 .Pn viamail 's
293 use case is now partly obsolete and partly covered by
294 .Pn forw ,
295 hence there's no reason to still maintain it.
296 .Pn conflict
297 is not related to the mail client, and
298 .Pn msh
299 conflicts with the basic concept of MH.
300 Theses two tools might still be useful, but they should not be part of mmh.
301 .P
302 Finally, there's
303 .Pn slocal .
304 .Pn slocal
305 is an MDA and thus not directly MUA-related.
306 Conceptionally, it should be removed.
307 But
308 .Pn slocal
309 provides rule-based processing of messages, like filing them into
310 different folders, which is otherwise not available in mmh.
311 Further more,
312 .Pn slocal
313 does neither pull dependencies nor a whole new technical area
314 into the project.
315 (See section XXX for the removing of the ndbm dependency.)
316 Still,
317 .Pn slocal
318 accounts for about 1\|000 lines of code that need to be maintained.
319 As
320 .Pn slocal
321 is almost self-standing, it should be split off into a separate project.
322 This would cut the strong connection between the MUA mmh and the MDA
323 .Pn slocal .
324 The MDA would become an alternative to
325 .I procmail ,
326 as it would no longer be the need to install a complete MH system, too.
327 Likewise, mmh users could decide to use
328 .I procmail
329 without having a second, unused MDA (\c
330 .Pn slocal )
331 installed.
332 That's conceptionally the best solution.
333 Yet,
334 .Pn slocal
335 is not removed,
336 but as its existence does not affect the rest of mmh,
337 it can be removed at any time.
340 .H2 "\fLshow\fP and \fPmhshow\fP
341 .P
342 Since the very beginning \(en already in the first concept paper \(en
343 .Pn show
344 had been MH's message display program.
345 .Pn show
346 mapped message numbers and sequences to files and invoked
347 .Pn mhl
348 to have the files formated.
349 For MIME, this approach wasn't sufficient anymore.
350 MIME messages can consist of multiple parts, some of which aren't
351 directly displayable, and text content might be encoded in
352 foreign charsets.
353 .Pn show 's
354 understanding of messages and
355 .Pn mhl 's
356 limited display facilities couldn't cope with the task any longer.
357 .P
358 Instead of extending these tools, additional tools were written from scratch
359 and then added to the MH tool chest. Doing so is encouraged by the
360 tool chest approach. The new tools could be added without interfering
361 with the existing ones. This is an advantage. The ease of adding new tools
362 made MH the first MUA to implement MIME.
363 .\" XXX ref
364 .P
365 First, the new MIME features were added in form of the single program
366 .Pn mhn .
367 The command
368 .Cl "mhn \-show 42
369 would show the MIME message numbered 42.
370 With the 1.0 release of nmh in February 1999, Richard Coleman finished
371 the split of
372 .Pn mhn
373 into a set of specialized programs, which together covered the
374 multiple aspects of MIME. One of these resulting tools was
375 .Pn mhshow ,
376 which replaced the
377 .Cl "mhn \-show
378 call.
379 It was capable to display a MIME message appropriately.
380 .P
381 From then on, two message display tools were part of nmh:
382 .Pn show
383 and
384 .Pn mhshow .
385 Because the user should not need to invoke the right tool
386 whether the message uses MIME or not,
387 .Pn show
388 was extended to automatically hand the job over to
389 .Pn mhshow
390 if displaying the message would be beyond
391 .Pn show 's
392 abilities.
393 In consequence, the user would invoke
394 .Pn show
395 (possibly through
396 .Pn next
397 or
398 .Pn prev )
399 and get the message printed with either
400 .Pn show
401 or
402 .Pn mhshow ,
403 whatever was more appropriate.
404 (There was also a switch for
405 .Pn show
406 to never invoke
407 .Pn mhshow .
408 .Pn show
409 was able to display MIME messages if they contained only a single text
410 part.)
411 .P
412 Having two similar tools for essentially the same task is redundant.
413 The development of both programs needed to be in sync,
414 to ensure that the programs behaved in a similar way,
415 because they were used like a single tool.
416 Different behavior would have surprised the user.
417 .P
418 Today, non-MIME messages are rather seen to be a special case of
419 MIME messages, than MIME messages are seen to be an extension to
420 original email.
421 As
422 .Pn mhshow
423 had already be able to display non-MIME messages, it was natural
424 to drop
425 .Pn show
426 in favor of using
427 .Pn mhshow
428 exclusively.
429 This decision followed the idea of orthogonal design.
430 For convenience,
431 .Pn mhshow
432 was then renamed to
433 .Pn show .
434 .Ci 4c1efddfd499300c7e74263e57d8aa137e84c853
435 .P
436 To prepare for this transition,
437 .Pn mhshow
438 was reworked to behave more like
439 .Pn show
440 first.
441 (Section XXX describes this rework from a different perspective.)
442 Once the tools behaved similar, the replacing became a natural decision.
443 In mmh,
444 .Pn show
445 is the one single message display program again, but it handles
446 MIME messages as well as non-MIME messages.
447 Now, there's only one program to maintain, and users don't need to deal
448 with the existance of two display programs.
449 .P
450 There's one reason why removing the old
451 .Pn show
452 hurts: It had been such a simple program.
453 Its lean elegance is missing to
454 .Pn mhshow ,
455 i.e. the new
456 .Pn show .
457 But there is no chance, because supporting MIME causes essentially
458 higher complexity.
461 .H2 "Removal of Configure Options
462 .P
463 Customization is a double-edged sword.
464 It allows better suiting setups, but not for free.
465 There is the cost of code complexity to be able to customize.
466 There is the cost of less tested setups, because there are
467 more possible setups and especially corner-cases.
468 And, there is the cost of choice itself.
469 The code complexity directly affects the developers.
470 Less tested code affects both, users and developers.
471 The problem of choice affects the users, for once by having to
472 choose, but also by complexer interfaces that require more documentation.
473 Whenever options add little advantages, they should be considered for
474 removal.
475 I have reduced the number of project-specific configure options from
476 fifteen to three.
478 .U3 "Mail Transfer Facilities
479 .P
480 With the removal of the mail transfer facilities five option vanished:
481 .IP \f(CW--with-mts=[smtp|sendmail]\fP
482 Specified the default mail transport service, which now is sendmail always.
483 .IP \f(CW--with-smtpservers=[server1...]\fP
484 Specified the default SMTP servers for the smtp mail transfer service.
485 .Ci 128545e06224233b7e91fc4c83f8830252fe16c9
486 .IP \f(CW--with-cyrus-sasl\fP
487 Enabled SASL support for mail transfer.
488 .IP \f(CW--with-tls\fP
489 Enabled TLS support for mail transfer.
490 .IP \f(CW--enable-pop\fP
491 Enabled the message retrieval facility.
493 .U3 "Backup Prefix
494 .P
495 The backup prefix is the string that was prepended to message
496 filenames to tag them as deleted.
497 By default it had been the comma character `\f(CW,\fP'.
498 There was a configure option to change the default to the hash symbol
499 `\f(CW#\fP':
500 .CW --with-hash-backup .
501 The implication of the hash symbol is that it introduces a comment
502 in the Unix shell.
503 Thus, the command line
504 .Cl "rm #13 #15
505 calls
506 .Pn rm
507 without arguments because the first hash symbol starts the comment
508 that reaches until the end of the line.
509 To delete the backup files,
510 .Cl "rm ./#13 ./#15"
511 needs to be used.
512 .\" XXX check historical background
513 Besides this effect, the choice was personal preference.
514 I removed the configure option but added the profile entry
515 .Pe backup-prefix ,
516 which allows to specify an arbitrary string as backup prefix.
517 .Ci 6c40d481d661d532dd527eaf34cebb6d3f8ed086
518 Profile entries are the common method to change mmh's behavior.
519 This change did not remove the choice but moved it to a location where
520 it suited better.
521 .P
522 Eventually, however, the new trash folder obsoleted the concept of the
523 backup prefix completely.
524 (Well, there still are corner-cases to remove until the backup
525 prefix can be layed to rest, eventually.)
526 .\" FIXME: Do this work in the code!
528 .U3 "Editor and Pager
529 .P
530 The two configure options
531 .CW --with-editor=EDITOR
532 .CW --with-pager=PAGER
533 were used to specify the default editor and pager at configure time.
534 Doing so at configure time made sense in the Eighties,
535 when the set of available editors and pagers varied much across
536 different systems.
537 Today, the situation is more homegeneic.
538 The programs
539 .Pn vi
540 and
541 .Pn more
542 can be expected to be available on every Unix system,
543 as they are specified by POSIX since two decades.
544 (The specifications for
545 .Pn vi
546 and
547 .Pn more
548 appeared in
549 .[
550 posix 1987
551 .]
552 and,
553 .[
554 posix 1992
555 .]
556 respectively.)
557 As a first step, these two tools were hard-coded as defaults.
558 .Ci 5d43a99db70c12a673028c7758c20cbe3e13ef5f
559 Not changed were the
560 .Pe editor
561 and
562 .Pe moreproc
563 profile entries, which allowed the user to override the system defaults.
564 Later, the concept was reworked to respect the standard environment
565 variables
566 .Ev VISUAL
567 and
568 .Ev PAGER
569 if they are set.
570 Today, mmh determines the editor to use in the following order,
571 taking the first available and non-empty item:
572 .IP (1)
573 Environment variable
574 .Ev MMHEDITOR
575 .IP (2)
576 Profile entry
577 .Pe Editor
578 .IP (3)
579 Environment variable
580 .Ev VISUAL
581 .IP (4)
582 Environment variable
583 .Ev EDITOR
584 .IP (5)
585 Command
586 .Pn vi .
587 .P
588 .Ci f85f4b7ae62e3d05a945dcd46ead51f0a2a89a9b
589 .P
590 The pager to use is deteminded in a similar order,
591 also taking the first available and non-empty item:
592 .IP (1)
593 Environment variable
594 .Ev MMHPAGER
595 .IP (2)
596 Profile entry
597 .Pe Pager
598 (replaces
599 .Pe moreproc )
600 .IP (3)
601 Environment variable
602 .Ev PAGER
603 .IP (4)
604 Command
605 .Pn more .
606 .P
607 .Ci 0c4214ea2aec6497d0d67b436bbee9bc1d225f1e
608 .P
609 By respecting the
610 .Ev VISUAL /\c
611 .Ev EDITOR
612 and
613 .Ev PAGER
614 environment variables,
615 the new behavior confirms better to the common style on Unix systems.
616 Additionally, the new approach is more uniform and clearer to users.
618 .U3 "Locale
619 .P
620 The configure option
621 .Sw --disable-locale
622 was removed because today there's hardly any need to disable locale
623 support.
624 .Ci ccf4f175ef4c4e7522f9510a4a1149c15d810dd9
626 .U3 "ndbm
627 .P
628 .Pn slocal
629 is an MDA included in mmh.
630 This is a violation of the idea that mmh is a MUA only.
631 .Pn slocal
632 should become a separate project.
633 Nonetheless, ouf of convenience and due to lack of convincement,
634 yet it remains being part of mmh.
635 This is likely to change in the future.
636 Already,
637 .Pn slocal
638 was stripped down.
639 It used to depend on
640 .I ndbm ,
641 a database library.
642 The database is used to store the `\fLMessage-ID\fP's of all
643 messages delivered.
644 This enables
645 .Pn slocal
646 to suppress delivering the same message to the same user twice.
647 (This features was enabled by the
648 .Sw -suppressdup
649 switch.)
650 .P
651 A variety of version of the database library exist.
652 Complicated autoconf code was needed to detect them correctly.
653 Further more, the configure switches
654 .Sw --with-ndbm=ARG
655 and
656 .Sw --with-ndbmheader=ARG
657 were added to help with difficult setups that would
658 not be detected automatically.
659 .P
660 By removing the suppress duplicates feature of
661 .Pn slocal ,
662 .Ci ecd6d6a20cb7a1507e3a20d6c4cb3a1cf14c6bbf
663 the dependency on
664 .I ndbm
665 was removed and 120 lines of complex autoconf could be saved.
666 .Ci ecd6d6a20cb7a1507e3a20d6c4cb3a1cf14c6bbf
667 The change removed funtionality too, but the value it would have added
668 is minor to the weight loss by dropping the dependency and
669 the complex autoconf code.
671 .U3 "mh-e Support
672 .P
673 The configure option
674 .Sw --disable-mhe
675 was removed when the mh-e support was reworked.
676 Mh-e is the Emacs front-end to MH.
677 It requires MH to provide minor additional functions.
678 The
679 .Sw --disable-mhe
680 configure option could switch these extensions off.
681 After removing the support for old versions of mh-e,
682 only the
683 .Sw -build
684 switches of
685 .Pn forw
686 and
687 .Pn repl
688 are left to be mh-e extensions.
689 They are now always built in because they add little code and complexity.
690 In consequence, the
691 .Sw --disable-mhe
692 configure option was removed
693 .Ci a7ce7b4a580d77b6c2c4d980812beb589aa4c643
694 Removing the option removed a second code setup that would have
695 needed to be tested.
696 This change was first done in nmh and thereafter merged into mmh.
697 .P
698 The interface changes in mmh require mh-e to be adjusted in order
699 to be able to use mmh as back-end.
700 This will require minor changes to mh-e, but removing the
701 .Sw -build
702 switches would require more rework.
704 .U3 "Masquerading
705 .P
706 The configure option
707 .Sw --enable-masquerade
708 could take up to three arguments:
709 `draft_from', `mmailid', and `username_extension'.
710 They activated different types of address masquerading.
711 All of them were implemented in the SMTP-speaking
712 .Pn post
713 command, which provided an MSA.
714 Address masquerading is an MTA's task and mmh does not cover
715 this field anymore.
716 Hence, true masquerading needs to be implemented in the external MTA.
717 .P
718 The
719 .I mmailid
720 masquerading type is the oldest one of the three and the only one
721 available in the original MH.
722 It provided a
723 .I username
724 to
725 .I fakeusername
726 mapping, based on the password file's GECOS field.
727 The man page
728 .Mp mh-tailor(5)
729 described the use case as being the following:
730 .QP
731 This is useful if you want the messages you send to always
732 appear to come from the name of an MTA alias rather than your
733 actual account name. For instance, many organizations set up
734 `First.Last' sendmail aliases for all users. If this is
735 the case, the GECOS field for each user should look like:
736 ``First [Middle] Last <First.Last>''
737 .P
738 As mmh sends outgoing mail via the local MTA only,
739 the best location to do such global rewrites is there.
740 Besides, the MTA is conceptionally the right location because it
741 does the reverse mapping for incoming mail (aliasing), too.
742 Further more, masquerading set up there is readily available for all
743 mail software on the system.
744 Hence, mmailid masquerading was removed.
745 .Ci 0836c8000ccb34b59410ef1c15b1b7feac70ce5f
746 .P
747 The
748 .I username_extension
749 masquerading type did not replace the username but would append a suffix,
750 specified by the
751 .Ev USERNAME_EXTENSION
752 environment variable, to it.
753 This provided support for the
754 .I user-extension
755 feature of qmail and the similar
756 .I "plussed user
757 processing of sendmail.
758 The decision to remove this username_extension masquerading was
759 motivated by the fact that
760 .Pn spost
761 hadn't supported it already.
762 .Ci 2abae0bfd0ad5bf898461e50aa4b466d641f23d9
763 Username extensions are possible in mmh, but less convenient to use.
764 .\" XXX format file %(getenv USERNAME_EXTENSION)
765 .P
766 The
767 .I draft_from
768 masquerading type instructed
769 .Pn post
770 to use the value of the `From:' header as SMTP envelope sender.
771 Sender addresses could be replaced completely.
772 .Ci b14ea6073f77b4359aaf3fddd0e105989db9
773 Mmh offers a kind of masquerading similar in effect, but
774 with technical differences.
775 As mmh does not transfer messages itself, the local MTA has final control
776 over the sender's address. Any masquerading mmh introduces may be reverted
777 by the MTA.
778 In times of pedantic spam checking, an MTA will take care to use
779 sensible envelope sender addresses to keep its own reputation up.
780 Nonetheless, the MUA can set the `From:' header and thereby propose
781 a sender address to the MTA.
782 The MTA may then decide to take that one or generate the canonical sender
783 address for use as envelope sender address.
784 .P
785 In mmh, the MTA will always extract the recipient and sender from the
786 message headers (\c
787 .Pn sendmail 's
788 .Sw -t
789 switch).
790 The `From:' header of the draft may be set arbitrary by the user.
791 If it is missing, the canonical sender address will be generated by the MTA.
793 .U3 "Remaining Options
794 .P
795 Two configure options remain in mmh.
796 One is the locking method to use:
797 .Sw --with-locking=[dot|fcntl|flock|lockf] .
798 The idea of removing all methods except the portable dot locking
799 and having that one as the default is appealing, but this change
800 requires deeper technical investigation into the topic.
801 The other option,
802 .Sw --enable-debug ,
803 compiles the programs with debugging symbols and does not strip them.
804 This option is likely to stay.
809 .H2 "Removal of switches
810 .P
815 .H1 "Modernizing
818 .H2 "Removal of Code Relicts
819 .P
820 The code base of mmh originates from the late Seventies,
821 had been extensively
822 worked on in the mid Eighties, and had been partly reorganized and extended
823 in the Nineties. Relicts of all those times had gathered in the code base.
824 My goal was to remove any ancient code parts. One part of the task was
825 converting obsolete code constructs to standard constructs, the other part
826 was dropping obsolete functions.
827 .P
828 As I'm not even thirty years old and have no more than seven years of
829 Unix experience, I needed to learn about the history in retrospective.
830 Older people likely have used those ancient constructs themselves
831 and have suffered from their incompatibilities and have longed for
832 standardization. Unfortunately, I have only read that others had done so.
833 This put me in a much more difficult positions when working on the old
834 code. I needed to recherche what other would have known by heart from
835 experience. All my programming experience comes from a time past ANSI C
836 and past POSIX. Although I knew about the times before, I took the
837 current state implicitly for granted most of the time.
838 .P
839 Being aware of
840 these facts, I rather let people with more historic experience solve the
841 task of converting the ancient code constructs to standardized ones.
842 Luckily, Lyndon Nerenberg focused on this task at the nmh project.
843 He converted large parts of the code to POSIX constructs, removing
844 the conditionals compilation for now standardized features.
845 I'm thankful for this task being solved. I only pulled the changes into
846 mmh.
847 .P
848 The other task \(en dropping ancient functionality to remove old code \(en
849 I did myself, though. My position to strip mmh to the bare minimum of
850 frequently used features is much more revolutional than the nmh community
851 likes it. Without the need to justify my decisions, I was able to quickly
852 remove functionality I considered ancient.
853 The need to discuss my decisions with
854 peers likely would have slowed this process down. Of course, I researched
855 if a particular feature really should be dropped. Having not had any
856 contact to this feature within my computer life was a first indicator to
857 drop it, but I also asked others and searched the literature for modern
858 usage of the feature. If it appeared to be truly ancient, I dropped it.
859 The reason for dropping is always part of the commit message in the
860 version control system. Thus, it is easy for others to check their
861 view on the topic with mine and possibly to argue for reinclusion.
863 .U2 "MMDF maildrop support
864 .P
865 I did drop any support for the MMDF maildrop format. This type of format
866 is conceptionally similar to the mbox format, but uses four bytes with
867 value 1 (\fL^A^A^A^A\fP) as message delimiter,
868 instead of the string ``\fLFrom\ \fP''.
869 Due to the similarity and mbox being the de-facto standard maildrop
870 format on Unix, but also due to the larger influence of Sendmail than MMDF,
871 the MMDF maildrop format had vanished.
872 .P
873 The simplifications within the code were only moderate. Switches could
874 be removed from tools like
875 .L packf ,
876 which generate packed mailboxes. Only one packed mailbox format remained:
877 mbox.
878 The most important changes affect the equally named mail parsing routine in
879 .L sbr/m_getfld.c .
880 The direct MMDF code had been removed, but as now only one packed mailbox
881 format is left, code structure simplifications are likely possible.
882 The reason why they are still outstanding is the heavily optimized code
883 of
884 .Fu m_getfld() .
885 Changes beyond a small local scope \(en
886 which restructuring in its core is \(en cause a high risk of damaging
887 the intricate workings of the optimized code. This problem is know
888 to the developers of nmh, too. They also avoid touching this minefield
889 if possible.
891 .U2 "UUCP Bang Paths
892 .P
893 More questionably than the former topic is the removal of support for the
894 UUCP bang path address style. However, the user may translate the bang
895 paths on retrieval to Internet addresses and the other way on posting
896 messages. The former can be done my an MDA like procmail; the latter
897 by a sendmail wrapper. This would ensure that any address handling would
898 work as expected. However, it might just work well without any
899 such modifications, as mmh does not touch addresses much, in general.
900 But I can't ensure as I have never used an environment with bang paths.
901 Also, the behavior might break at any point in further development.
903 .U2 "Hardcopy terminal support
904 .P
905 More of a funny anecdote is the remaining of a check for printing to a
906 hardcopy terminal until Spring 2012, when I finally removed it.
907 I surely would be very happy to see such a terminal in action, maybe
908 actually being able to work on it, but I fear my chances are null.
909 .P
910 The check only prevented a pager to be placed between the outputting
911 program (\c
912 .Pn mhl )
913 and the terminal. This could have been ensured with
914 the
915 .Sw \-nomoreproc
916 at the command line statically, too.
918 .U2 "Removed support for header fields
919 .P
920 The `Encrypted' header had been introduced by RFC\^822, but already
921 marked legacy in RFC 2822. It was superseded by FIXME.
922 Mmh does no more support this header.
923 .P
924 Native support for `Face' headers
925 had been removed, as well.
926 The feature is similar to the `X-Face' header in its intent,
927 but takes a different approach to store the image.
928 Instead of encoding the image data directly into the header,
929 the the header contains the hostname and UDP port where the image
930 date could be retrieved.
931 Neither `X-Face' nor the here described `Face' system
932 \**
933 .FS
934 There is also a newer but different system, invented 2005,
935 using `Face' headers.
936 It is the successor of `X-Face' providing colored PNG images.
937 .FE
938 became well used in the large scale.
939 It's still possible to use a Face systems,
940 although mmh does not provide support for any of the different systems
941 anymore. It's fairly easy to write a small shell script to
942 extract the embedded or fetch the external Face data and display the image.
943 Own Face headers can be added into the draft template files.
944 .P
945 `Content-MD5' headers were introduced by RFC\^1864. They provide only
946 a verification of data corruption during the transfer. By no means can
947 they ensure verbatim end-to-end delivery of the contents. This is clearly
948 stated in the RFC. The proper approach to provide verificationability
949 of content in an end-to-end relationship is the use of digital cryptography
950 (RFCs FIXME). On the other hand, transfer protocols should ensure the
951 integrity of the transmission. In combinations these two approaches
952 make the `Content-MD5' header field useless. In consequence, I removed
953 the support for it. By this removal, MD5 computation is not needed
954 anywhere in mmh. Hence, over 500 lines of code were removed by this one
955 change. Even if the `Content-MD5' header field is useful sometimes,
956 I value its usefulnes less than the improvement in maintainability, caused
957 by the removal.
959 .U2 "Prompter's Control Keys
960 .P
961 The program
962 .Pn prompter
963 queries the user to fill in a message form. When used by
964 .Pn comp
965 as:
966 .DS
967 comp \-editor prompter
968 .DE
969 the resulting behavior is similar to
970 .Pn mailx .
971 Apparently,
972 .Pn prompter
973 hadn't been touched lately. Otherwise it's hardly explainable why it
974 still offered the switches
975 .Sn \-erase \fUchr\fP
976 and
977 .Sn \-kill \fUchr\fP
978 to name the characters for command line editing.
979 The times when this had been necessary are long time gone.
980 Today these things work out-of-the-box, and if not, are configured
981 with the standard tool
982 .Pn stty .
984 .U2 "Vfork and Retry Loops
985 .P
986 MH creates many processes, which is a consequence of the tool chest approach.
987 In earlier times
988 .Fu fork()
989 had been an expensive system call, as the process's whole image needed
990 to be duplicated. One common case is replacing the image with
991 .Fu exec()
992 right after having forked the child process.
993 To speed up this case, the
994 .Fu vfork()
995 system call was invented at Berkeley. It completely omits copying the
996 image. If the image gets replaced right afterwards then unnecessary
997 work is omited. On old systems this results in large speed ups.
998 MH uses
999 .Fu vfork()
1000 whenever possible.
1001 .P
1002 Memory management units that support copy-on-write semantics make
1003 .Fu fork()
1004 almost as fast as
1005 .Fu vfork()
1006 in the cases when they can be exchanged.
1007 With
1008 .Fu vfork()
1009 being more error-prone and hardly faster, it's preferable to simply
1010 use
1011 .Fu fork()
1012 instead.
1013 .P
1014 Related to the costs of
1015 .Fu fork()
1016 is the probability of its success.
1017 Today on modern systems, the system call will succeed almost always.
1018 In the Eighties on heavy loaded systems, as they were common at
1019 universities, this had been different. Thus, many of the
1020 .Fu fork()
1021 calls were wrapped into loops to retry to fork several times in
1022 short intervals, in case of previous failure.
1023 In mmh, the program aborts at once if the fork failed.
1024 The user can reexecute the command then. This is expected to be a
1025 very rare case on modern systems, especially personal ones, which are
1026 common today.
1029 .H2 "Attachments
1030 .P
1031 MIME
1034 .H2 "Digital Cryptography
1035 .P
1036 Signing and encryption.
1039 .H2 "Good Defaults
1040 .P
1041 foo
1046 .H1 "Code style
1047 .P
1048 foo
1051 .H2 "Standard Code
1052 .P
1053 POSIX
1056 .H2 "Separation
1058 .U2 "MH Directory Split
1059 .P
1060 In MH and nmh, a personal setup had consisted of two parts:
1061 The MH profile, named
1062 .Fn \&.mh_profile
1063 and being located directly in the user's home directory.
1064 And the MH directory, where all his mail messages and also his personal
1065 forms, scan formats, other configuration files are stored. The location
1066 of this directory could be user-chosen. The default was to name it
1067 .Fn Mail
1068 and have it directly in the home directory.
1069 .P
1070 I've never liked the data storage and the configuration to be intermixed.
1071 They are different kinds of data. One part, are the messages,
1072 which are the data to operate on. The other part, are the personal
1073 configuration files, which are able to change the behavior of the operations.
1074 The actual operations are defined in the profile, however.
1075 .P
1076 When storing data, one should try to group data by its type.
1077 There's sense in the Unix file system hierarchy, where configuration
1078 file are stored separate (\c
1079 .Fn /etc )
1080 to the programs (\c
1081 .Fn /bin
1082 and
1083 .Fn /usr/bin )
1084 to their sources (\c
1085 .Fn /usr/src ).
1086 Such separation eases the backup management, for instance.
1087 .P
1088 In mmh, I've reorganized the file locations.
1089 Still there are two places:
1090 There's the mail storage directory, which, like in MH, contains all the
1091 messages, but, unlike in MH, nothing else.
1092 Its location still is user-chosen, with the default name
1093 .Fn Mail ,
1094 in the user's home directory. This is much similar to the case in nmh.
1095 The configuration files, however, are grouped together in the new directory
1096 .Fn \&.mmh
1097 in the user's home directory.
1098 The user's profile now is a file, named
1099 .Fn profile ,
1100 in this mmh directory.
1101 Consistently, the context file and all the personal forms, scan formats,
1102 and the like, are also there.
1103 .P
1104 The naming changed with the relocation.
1105 The directory where everything, except the profile, had been stored (\c
1106 .Fn $HOME/Mail ),
1107 used to be called \fIMH directory\fP. Now, this directory is called the
1108 user's \fImail storage\fP. The name \fImmh directory\fP is now given to
1109 the new directory
1110 (\c
1111 .Fn $HOME/.mmh ),
1112 containing all the personal configuration files.
1113 .P
1114 The separation of the files by type of content is logical and convenient.
1115 There are no functional differences as any possible setup known to me
1116 can be implemented with both approaches, although likely a bit easier
1117 with the new approach. The main goal of the change had been to provide
1118 sensible storage locations for any type of personal mmh file.
1119 .P
1120 In order for one user to have multiple MH setups, he can use the
1121 environment variable
1122 .Ev MH
1123 the point to a different profile file.
1124 The MH directory (mail storage plus personal configuration files) is
1125 defined by the
1126 .Pe Path
1127 profile entry.
1128 The context file could be defined by the
1129 .Pe context
1130 profile entry or by the
1131 .Ev MHCONTEXT
1132 environment variable.
1133 The latter is useful to have a distinct context (e.g. current folders)
1134 in each terminal window, for instance.
1135 In mmh, there are three environment variables now.
1136 .Ev MMH
1137 may be used to change the location of the mmh directory.
1138 .Ev MMHP
1139 and
1140 .Ev MMHC
1141 change the profile and context files, respectively.
1142 Besides providing a more consistent feel (which simply is the result
1143 of being designed anew), the set of personal configuration files can
1144 be chosen independently from the profile (including mail storage location)
1145 and context, now. Being it relevant for practical use or not, it
1146 de-facto is an improvement. However, the main achievement is the
1147 split between mail storage and personal configuration files.
1150 .H2 "Modularization
1151 .P
1152 whatnowproc
1153 .P
1154 The \fIMH library\fP
1155 .Fn libmh.a
1156 collects a bunch of standard functions that many of the MH tools need,
1157 like reading the profile or context files.
1158 This doesn't hurt the separation.
1161 .H2 "Style
1162 .P
1163 Code layout, goto, ...
1168 .H1 "Concept Exploitation/Homogeniety
1171 .H2 "Draft Folder
1172 .P
1173 Historically, MH provided exactly one draft message, named
1174 .Fn draft
1175 and
1176 being located in the MH directory. When starting to compose another message
1177 before the former one was sent, the user had been questioned whether to use,
1178 refile or replace the old draft. Working on multiple drafts at the same time
1179 was impossible. One could only work on them in alteration by refiling the
1180 previous one to some directory and fetching some other one for reediting.
1181 This manual draft management needed to be done each time the user wanted
1182 to switch between editing one draft to editing another.
1183 .P
1184 To allow true parallel editing of drafts, in a straight forward way, the
1185 draft folder facility exists. It had been introduced already in July 1984
1186 by Marshall T. Rose. The facility was deactivated by default.
1187 Even in nmh, the draft folder facility remained deactivated by default.
1188 At least, Richard Coleman added the man page
1189 .Mp mh-draft(5)
1190 to document
1191 the feature well.
1192 .P
1193 The only advantage of not using the draft folder facility is the static
1194 name of the draft file. This could be an issue for MH frontends like mh-e.
1195 But as they likely want to provide working on multiple drafts in parallel,
1196 the issue is only concerning compatibility. The aim of nmh to stay compatible
1197 prevented the default activation of the draft folder facility.
1198 .P
1199 On the other hand, a draft folder is the much more natural concept than
1200 a draft message. MH's mail storage consists of folders and messages,
1201 the messages named with ascending numbers. A draft message breaks with this
1202 concept by introducing a message in a file named
1203 .Fn draft .
1204 This draft
1205 message is special. It can not be simply listed with the available tools,
1206 but instead requires special switches. I.e. corner-cases were
1207 introduced. A draft folder, in contrast, does not introduce such
1208 corner-cases. The available tools can operate on the messages within that
1209 folder like on any messages within any mail folders. The only difference
1210 is the fact that the default folder for
1211 .Pn send
1212 is the draft folder,
1213 instead of the current folder, like for all other tools.
1214 .P
1215 The trivial part of the change was activating the draft folder facility
1216 by default and setting a default name for this folder. Obviously, I chose
1217 the name
1218 .Fn +drafts .
1219 This made the
1220 .Sw \-draftfolder
1221 and
1222 .Sw \-draftmessage
1223 switches useless, and I could remove them.
1224 The more difficult but also the part that showed the real improvement,
1225 was updating the tools to the new concept.
1226 .Sw \-draft
1227 switches could
1228 be dropped, as operating on a draft message became indistinguishable to
1229 operating on any other message for the tools.
1230 .Pn comp
1231 still has its
1232 .Sw \-use
1233 switch for switching between its two modes: (1) Compose a new
1234 draft, possibly by taking some existing message as a form. (2) Modify
1235 an existing draft. In either case, the behavior of
1236 .Pn comp is
1237 deterministic. There is no more need to query the user. I consider this
1238 a major improvement. By making
1239 .Pn send
1240 simply operate on the current
1241 message in the draft folder by default, with message and folder both
1242 overridable by specifying them on the command line, it is now possible
1243 to send a draft anywhere within the storage by simply specifying its folder
1244 and name.
1245 .P
1246 All theses changes converted special cases to regular cases, thus
1247 simplifying the tools and increasing the flexibility.
1250 .H2 "Trash Folder
1251 .P
1252 Similar to the situation for drafts is the situation for removed messages.
1253 Historically, a message was deleted by renaming. A specific
1254 \fIbackup prefix\fP, often comma (\c
1255 .Fn , )
1256 or hash (\c
1257 .Fn # ),
1258 being prepended to the file name. Thus, MH wouldn't recognize the file
1259 as a message anymore, as only files whose name consists of digits only
1260 are treated as messages. The removed messages remained as files in the
1261 same directory and needed some maintenance job to truly delete them after
1262 some grace time. Usually, by running a command similar to
1263 .DS
1264 find /home/user/Mail \-ctime +7 \-name ',*' | xargs rm
1265 .DE
1266 in a cron job. Within the grace time interval
1267 the original message could be restored by stripping the
1268 the backup prefix from the file name. If however, the last message of
1269 a folder is been removed \(en say message
1270 .Fn 6
1271 becomes file
1272 .Fn ,6
1273 \(en and a new message enters the same folder, thus the same
1274 numbered being given again \(en in our case
1275 .Fn 6
1276 \(en, if that one
1277 is removed too, then the backup of the former message gets overwritten.
1278 Thus, the ability to restore removed messages does not only depend on
1279 the ``sweeping cron job'' but also on the removing of further messages.
1280 This is undesirable, because the real mechanism is hidden from the user
1281 and the consequences of further removals are not always obvious.
1282 Further more, the backup files are scattered within the whole mail
1283 storage, instead of being collected at one place.
1284 .P
1285 To improve the situation, the profile entry
1286 .Pe rmmproc
1287 (previously named
1288 .Pe Delete-Prog )
1289 was introduced, very early.
1290 It could be set to any command, which would care for the mail removal
1291 instead of taking the default action, described above.
1292 Refiling the to-be-removed files to some garbage folder was a common
1293 example. Nmh's man page
1294 .Mp rmm(1)
1295 proposes
1296 .Cl "refile +d
1297 to move messages to the garbage folder and
1298 .Cl "rm `mhpath +d all`
1299 the empty the garbage folder.
1300 Managing the message removal this way is a sane approach. It keeps
1301 the removed messages in one place, makes it easy to remove the backup
1302 files, and, most important, enables the user to use the tools of MH
1303 itself to operate on the removed messages. One can
1304 .Pn scan
1305 them,
1306 .Pn show
1307 them, and restore them with
1308 .Pn refile .
1309 There's no more
1310 need to use
1311 .Pn mhpath
1312 to switch over from MH tools to Unix tools \(en MH can do it all itself.
1313 .P
1314 This approach matches perfect with the concepts of MH, thus making
1315 it powerful. Hence, I made it the default. And even more, I also
1316 removed the old backup prefix approach, as it is clearly less powerful.
1317 Keeping unused alternative in the code is a bad choice as they likely
1318 gather bugs, by not being constantly tested. Also, the increased code
1319 size and more conditions crease the maintenance costs. By strictly
1320 converting to the trash folder approach, I simplified the code base.
1321 .Pn rmm
1322 calls
1323 .Pn refile
1324 internally to move the to-be-removed
1325 message to the trash folder (\c
1326 .Fn +trash
1327 by default). Messages
1328 there can be operated on like on any other message in the storage.
1329 The sweep clean, one can use
1330 .Cl "rmm \-unlink +trash a" ,
1331 where the
1332 .Sw \-unlink
1333 switch causes the files to be truly unliked instead
1334 of moved to the trash folder.
1337 .H2 "Path Notations
1338 .P
1339 foo
1342 .H2 "MIME Integration
1343 .P
1344 user-visible access to whole messages and MIME parts are inherently
1345 different
1348 .H2 "Of One Cast
1349 .P