docs/master
view ch03.roff @ 74:cefaa856d431
A lot of new text about configure switches.
| author | markus schnalke <meillo@marmaro.de> |
|---|---|
| date | Mon, 04 Jun 2012 22:51:53 +0200 |
| parents | bae9273b5802 |
| children | 2e61e0004a8f |
line source
1 .H0 "Discussion
2 .P
3 This main chapter discusses the practical work done in the mmh project.
4 It is structured along the goals to achieve. The concrete work done
5 is described in the examples of how the general goals were achieved.
10 .H1 "Stream-lining
12 .P
13 MH had been considered an all-in-one system for mail handling.
14 The community around nmh has a similar understanding.
15 In fundamental difference, I believe that mmh should be a MUA but
16 nothing more. I believe that all-in-one mail systems are not the way
17 to go. There are excellent specialized MTAs, like Postfix;
18 there are specialized MDAs, like Procmail; there are specialized
19 MRAs, like Fetchmail. I believe it's best to use them instead of
20 providing the same function ourselves. Doing something well requires to
21 focus on this particular aspect or a small set of aspects. The more
22 it is possible to focus, the better the result in this particular
23 area will be. The limiting resource in Free Software community development
24 usually is human power. If the low development power is even parted
25 into multiple development areas, it will hardly be possible to
26 compete with the specialists in the various fields. This is even
27 increased, given the small community \(en developers and users \(en
28 that MH-based mail systems have. In consequence, I believe that the
29 available resources should be concentrated at the point where MH is
30 most unique. This is clearly the MUA part.
31 .P
32 The goal for mmh was to remove peripheral parts and stream-line
33 it for the MUA task.
36 .H2 "Removal of Mail Transfer Facilities
37 .P
38 In contrast to nmh, which also provides mail submission and mail retrieval
39 facilities, mmh is a MUA only.
40 This general difference in the view on the character of nmh
41 strongly supported the development of mmh.
42 Removing the mail transfer facilities had been the first work task
43 for the mmh project.
44 .P
45 The MSA is called \fIMessage Transfer Service\fP (MTS) in nmh.
46 The facility establishes TCP/IP connections and speaks SMTP to submit
47 messages for relay to the outside world.
48 This part is implemented in the
49 .Pn post
50 command.
51 Demanded by the changes in
52 emailing, this part of nmh required changes in the last years.
53 Encrypted connections needed to be supported, hence SASL was introduced
54 into nmh. This added complexity to the nmh without improving it in
55 its core functions. Also, keeping up with recent developments in
56 this field needs requires development power and specialists.
57 Mmh cuts this whole facility off and depends on an external MTA instead.
58 The only outgoing interface available to mmh is the
59 .Pn sendmail
60 command.
61 Almost any MTA provides a
62 .Pn sendmail
63 command.
64 It not, any program can be substituted if it reads the
65 message from the standard input, extracts the recipient addresses
66 from the message header and does not conflict
67 with sendmail-specific command line arguments.
68 .P
69 To retrieve mail, the
70 .Pn inc
71 command in nmh has the ability to establish TCP/IP connections
72 and speaks POP3 to retrieve mail from remote servers.
73 As with mail submission, here encrypted connections are required
74 today, thus SASL support was added.
75 As POP3 is superseded by IMAP more and more, support for message
76 retrieval through IMAP will become necessary to be added soon.
77 Mmh has no support for retrieving mail from remote locations.
78 It depends on an external tool to cover this task.
79 There are two ways for messages to enter mmh's mail storage:
80 Incorporate them with
81 .Pn inc
82 from the system maildrop, or with
83 .Pn rcvstore
84 from the standard input.
85 In consequence, mmh has not any longer networking code
86 and thus does no more need to do transfer encryption and authentication.
87 Two large functional units are removed.
88 .P
89 With the removal of the MSA and MRA, mmh converted from an all-in-one
90 mail system to being only a MUA.
91 Following the Unix philosophy, it focuses on one job and to do that well.
92 Now, of course, mmh depends on third-party software.
93 An external MTA/MSA is required to transfer mail to the outside world;
94 an external MRA is required to retrieve mail from remote machines.
95 There exist excellent implementations of such software,
96 which do this specific task likely much better than the internal
97 versions of nmh do it. Also, this provides the choice for the best
98 suiting one of the available implementation.
99 .P
100 As it had already been possible to use an external MSA or MRA,
101 why not keep the internal version for convenience?
102 If this would question the sense in having a fall-back pager in all
103 the command line tools, in case
104 .Pn more
105 or
106 .Pn less
107 wouldn't be available, the answer is intuitively seen.
108 Now, an MSA or MRA is clearly more complex than a text pager, but
109 still the concept holds: If programs become complex, split them;
110 if projects become complex, split them.
111 Complexity is demanded by the problem to solve. Decades ago,
112 emailing had been small and simple.
113 (Remember,
114 .Pn /bin/mail
115 had once covered anything there was to email and still had been small.)
116 As the complexity in emailing increased, MH remainded mostly unchanged.
117 Nontheless, in nmh the POP server, which the original MH had included,
118 was removed. Now is the time to take one step further and remove
119 the MSA and MRA.
120 Not only does it decrease the code amount of the project,
121 but more important, it removes the whole field of message transfer
122 with all its implications from the project.
123 .P
124 If a project needs some kind of function, there's always the choice
125 between implementing the the function in the project directly or
126 depending on a library that provides the function or depending on
127 a program that provides the function.
128 Whereas adding the function directly to the project increases the
129 code size most, it makes the project most independent.
130 On the other end, interfacing external programs keeps the interface
131 smallest, but the depencency highest.
132 Using a library is in the middle.
133 Adding the function directly to the project is a bad choice for
134 any function of higher complexity, unless it's not available in other ways.
135 Hence, the dependencies only change in kind, not in their existence.
136 In mmh, library dependencies on
137 .Pn libsasl2
138 and
139 .Pn libcrypto /\c
140 .Pn libssl
141 were treated against program dependencies on an MSA and an MRA.
142 Besides program dependencies providing the stronger separation
143 and being more flexible, they also allowed
144 over 6\|000 lines of code to be removed from mmh.
145 This made mmh's code base about 12\|% smaller.
146 Reducing the projects code size by such an amount without actually
147 losing function is a convincing argument.
148 .P
149 Users of MH should have not problem to set up an external MSA and MRA.
150 Also, the popular MSAs and MRAs have large communities and a lot
151 of documentation available.
152 .P
153 Choices for MSAs range from the full-featured
154 .I Postfix
155 over mid-size solutions like
156 .I masqmail
157 and
158 .I dma
159 to small forwarders like
160 .I ssmtp
161 and
162 .I nullmailer .
163 Choices for MRAs include
164 .I fetchmail ,
165 .I getmail ,
166 .I mpop
167 and
168 .I fdm .
171 .H2 "Removal of non-MUA Tools
172 .P
173 Some of nmh's tools were removed from mmh because they didn't
174 match the main focus of adding to the MUA's task.
175 .BU
176 .Pn conflict
177 was removed because it is a mail system maintenance tool.
178 Besides, it even checks
179 .Fn /etc/passwd
180 and
181 .Fn /etc/group
182 for consistency, which has nothing at all to do with emailing.
183 The tool might be useful, but it should not be shipped with mmh.
184 .BU
185 .Pn rcvtty
186 was removed because its usecase of writing to the user's terminal
187 on receiving of mail is hardly wanted today. If users like to be
188 informed of new mail, then using the shell's
189 .Ev MAILPATH
190 variable or graphical notifications are likely more
191 appealing.
192 Writing directly to a terminals is hardly ever wanted today.
193 If though one wants to have it this way, the standard tool
194 .Pn write
195 can be used in a way similar to:
196 .DS
197 scan -file - | write `id -un`
198 .DE
199 .BU
200 .Pn viamail
201 was removed when the new attachment system was introduced, because
202 .Pn forw
203 could can now the task itself.
204 The program
205 .Pn sendfiles
206 was rewritten as a shell script wrapper around
207 .Pn forw .
208 .BU
209 .Pn msgchk
210 was removed, because it lost its usefulness when POP support was removed.
211 .Pn msgchk
212 provides hardly more information than:
213 .DS
214 ls -l /var/mail/meillo
215 .DE
216 It does separate between old and new mail, but that's merely a detail
217 and can be done with
218 .Pn stat (1)
219 too.
220 A very small shell script could be written to output the information
221 in a convenient way, if truly necessary.
222 As mmh's inc only incorporates mail from the user's local maildrop
223 and thus no data transfers over slow networks are involved,
224 there's hardly need to check for new mail before incorporating it.
225 .BU
226 .Pn msh
227 was removed because the tool was in conflict with the
228 philosophy of MH. It provided an interactive shell to access the
229 features of MH. One major feature of MH is being a tool chest.
230 .Pn msh
231 wouldn't be just another shell, tailored to the needs of mail
232 handling, but one large program to have the MH tools built in.
233 It's main use was for accessing Bulletin Boards, which have seized to
234 be popular.
235 .P
236 Removing
237 .Pn msh ,
238 together with the truly obsolete code relicts
239 .Pn vmh
240 and
241 .Pn wmh ,
242 saved more than 7\|000 lines of C code \(en
243 about 15\|% of the project's original source code amount.
244 Having the same functionality in less code (with equal readability,
245 of course) is an advantage.
246 Less code means less bugs and less maintenance work.
247 If
248 .Pn rcvtty
249 and
250 .Pn msgchk
251 are rarely used and can be implemented in different ways,
252 then why should one keep them?
253 .Pn viamail 's
254 use case is now partly obsolete and partly covered by
255 .Pn forw ,
256 hence there's no reason to still have
257 .Pn viamail
258 around.
259 .Pn conflict
260 is not related with the mail client, and
261 .Pn msh
262 conflicts with the basic concept of MH.
263 Both tools could still be useful, but not as part of mmh.
264 .P
265 It is a design goal of mmh to remove those parts that are rarely used.
266 The project shall become more stream-lined.
267 Rarely used and loosely related tools distract from the lean appearance.
268 They require maintenance cost without adding to the core task.
269 Therefore they were removed.
272 .H2 "Merge of \f(CWshow\fP and \f(CWmhshow\fP
273 .P
274 Since the very beginning \(en already in the first concept paper \(en
275 .Pn show
276 had been MH's message display program.
277 .Pn show
278 found out which pathnames the relevant messages had and invoked
279 .Pn mhl
280 then to have the content formated.
281 With the advent of MIME, this approach wasn't sufficient anymore.
282 MIME messages can consist of multiple parts, some of which aren't
283 directly displayable, and text content might be encoded in
284 foreign charsets.
285 .Pn show 's
286 simple approach and
287 .Pn mhl 's
288 limited display facilities couldn't cope with the task any longer.
289 .P
290 Instead of extending these tools, additional ones were written from scratch
291 and then added to the MH tool chest. Doing so is encouraged by the
292 tool chest approach. The new tools could be added without interfering
293 with the existing ones. This is great. The ease of adding new tools
294 even made MH the first MUA to implement MIME.
295 .P
296 First, the new MIME features were added in form of the single program
297 .Pn mhn .
298 The command
299 .Cl "mhn \-show 42
300 would show the MIME message numbered 42.
301 With the 1.0 release of nmh in February 1999, Richard Coleman finished
302 the split of
303 .Pn mhn
304 into a set of specialized programs, which together covered the
305 multiple aspects of MIME. One of these resulting tools was
306 .Pn mhshow ,
307 which replaced the
308 .Cl "mhn \-show
309 call.
310 .P
311 From then on, two message display tools were part of nmh.
312 Because it should not require user actions to invoke the right tool
313 whether the message uses MIME or not,
314 .Pn show
315 was extended to automatically hand the job over to
316 .Pn mhshow
317 if displaying the message would be beyond
318 .Pn show 's
319 abilities.
320 For convenience,
321 .Pn show
322 would still display MIME messages if they contained only a single text
323 part.
324 In consequence, the user would invoke
325 .Pn show
326 (possibly through
327 .Pn next
328 or
329 .Pn prev )
330 and get the message printed with either
331 .Pn show
332 or
333 .Pn mhshow ,
334 whatever was more appropriate.
335 (There was also a switch for
336 .Pn show
337 to never invoke
338 .Pn mhshow .)
339 .P
340 Having two similar tools for essentially the same task is redundant.
341 Both programs needed to be developed syncronously as they were
342 used as a single tool by the user. Thus they needed to act in a
343 similar way to not distract the user.
344 .P
345 Today, non-MIME messages are rather seen to be a special case of
346 MIME messages, than MIME messages are seen to be an extension to
347 original mail.
348 As
349 .Pn mhshow
350 had already be able to display non-MIME messages, it was natural
351 to drop
352 .Pn show
353 in favor of using
354 .Pn mhshow
355 exclusively.
356 .Ci 4c1efddfd499300c7e74263e57d8aa137e84c853
357 This decision follows the idea of orthogonal design.
358 .P
359 To allow this replacement,
360 .Pn mhshow
361 was reworked to behave more like
362 .Pn show
363 first.
364 Section XXX describes this rework from a different perspective.
365 Once the tools behaved similar, the replacing became a natural decision.
366 In mmh,
367 .Pn show
368 is the one single message display program again, but it handles
369 MIME messages as well as non-MIME messages.
370 There's only one program to maintain and users don't need to deal
371 with the existance of two display programs.
372 .P
373 Though, there's one reason why removing the old
374 .Pn show
375 hurts: It had been such a simple program.
376 Its lean elegance is missing to
377 .Pn mhshow ,
378 i.e. the new
379 .Pn show .
380 But there is no chance, because supporting MIME causes essentially
381 higher complexity.
384 .H2 "Removal of Configure Options
385 .P
386 Choice is a double-edged sword.
387 It allows customization and thus better suiting solutions,
388 but that comes with costs.
389 First, there is the cost of code complexity to have choice.
390 Second, there is the cost of less tested setups, because there are
391 more possible setups and especially corner-cases.
392 Third, there is the cost of choice itself.
393 The code complexity affects the developers.
394 Less tested code affects both, users and developers.
395 The problem of choice affects the users, for once simply by having to
396 choose but also by complexer interfaces that require more documentation.
397 Whenever options add little advantages, they should be considered for
398 removal.
399 .P
400 I have reduced the number of project-specific configure options from
401 fifteen to three.
403 .U3 "Mail Transfer Facility Options
404 .P
405 With the removal of the mail transfer facilities five option vanished:
406 .IP \f(CW--with-mts=[smtp|sendmail]\fP
407 Specified the default mail transport service, which now is sendmail always.
408 .IP \f(CW--with-smtpservers=[server1...]\fP
409 Specified the default SMTP servers for the smtp mail transfer service.
410 .Ci 128545e06224233b7e91fc4c83f8830252fe16c9
411 .IP \f(CW--with-cyrus-sasl\fP
412 Enabled SASL support for mail transfer.
413 .IP \f(CW--with-tls\fP
414 Enabled TLS support for mail transfer.
415 .IP \f(CW--enable-pop\fP
416 Enabled the message retrieval facility.
418 .U3 "Backup Prefix
419 .P
420 The default backup prefix, i.e. the string that was prepended to message
421 filenames to tag them as deleted, had been the comma `\f(CW,\fP'.
422 There was a configure option to change the default to the hash symbol
423 `\f(CW#\fP':
424 .CW --with-hash-backup .
425 The implication of the hash symbol is that it introduces a comment
426 in the Unix shell.
427 Thus, the command line
428 .Cl "rm #13 #15
429 calls
430 .Pn rm
431 without arguments because the first hash symbol starts the comment
432 that reaches until the end of the line.
433 To delete the backup files,
434 .Cl "rm ./#13 ./#15"
435 needs to be used.
436 .\" XXX check historical background
437 Besides this effect, the choice was personal preference.
438 I removed the configure option but added the profile entry
439 .Pe backup-prefix ,
440 which allows to specify an arbitrary string as backup prefix.
441 .Ci 6c40d481d661d532dd527eaf34cebb6d3f8ed086
442 This did not remove the choice but moved it to a location where
443 it suited better.
444 Profile entries are the common method to change mmh's behavior.
445 The name of the
446 .Fn .mh-sequences ,
447 for instance, is specified there, too.
448 Moving the specification of the backup prefix there, appears to be right.
449 Eventually, the new trash folder obsoleted the concept of the
450 backup prefix completely.
451 (Well, there still are corner-cases to remove until the backup
452 prefix can be layed to rest, eventually.)
453 .\" FIXME: Do this work in the code!
454 .P
455 The two configure options
456 .CW --with-editor=EDITOR
457 .CW --with-pager=PAGER
458 were used to specify the default editor and pager at configure time.
459 Doing so at configure time made sense in the Eighties,
460 when the available editors and pagers varied more across different systems.
461 Today, the situation is much more homegenic.
462 The programs
463 .Pn vi
464 and
465 .Pn more
466 can be expected to be available anywhere on every Unix system,
467 as they are specified by POSIX since two decades.
468 (The specifications for
469 .Pn vi
470 and
471 .Pn more
472 appeared in
473 .[
474 posix 1987
475 .]
476 and,
477 .[
478 posix 1992
479 .]
480 respectively.)
481 As a first step, these two tools were hard-coded as defaults.
482 .Ci 5d43a99db70c12a673028c7758c20cbe3e13ef5f
483 Not changed were the
484 .Pe editor
485 and
486 .Pe moreproc
487 profile entries, which allowed the user to change the default
488 by personal preference.
489 Later, the concept was reworked to respect the standard environment
490 variables
491 .Ev VISUAL
492 and
493 .Ev PAGER
494 if they were set.
495 Today, mmh determines the editor to use in the following order,
496 taking the first available and non-empty item:
497 .IP (1)
498 Environment variable
499 .Ev MMHEDITOR
500 .IP (2)
501 Profile entry
502 .Pe Editor
503 .IP (3)
504 Environment variable
505 .Ev VISUAL
506 .IP (4)
507 Environment variable
508 .Ev EDITOR
509 .IP (5)
510 Command
511 .Pn vi .
512 .P
513 The pager to use is deteminded in the following order,
514 also taking the first available and non-empty item:
515 .IP (1)
516 Environment variable
517 .Ev MMHPAGER
518 .IP (2)
519 Profile entry
520 .Pe Pager
521 (replaces
522 .Pe moreproc )
523 .IP (3)
524 Environment variable
525 .Ev PAGER
526 .IP (4)
527 Command
528 .Pn more .
529 .P
530 .Ci f85f4b7ae62e3d05a945dcd46ead51f0a2a89a9b
531 .Ci 0c4214ea2aec6497d0d67b436bbee9bc1d225f1e
532 .P
533 The new behavior confirms better to the common behavior on Unix
534 systems, as
535 .Ev VISUAL /\c
536 .Ev EDITOR
537 and
538 .Ev PAGER
539 are respected.
540 Additionally, the new approach is more uniform and
541 without surprise for users.
543 .U3 "Locale
544 .P
545 The configure option
546 .Sw --disable-locale
547 was removed because today there's hardly any need to disable locale
548 support.
549 .Ci ccf4f175ef4c4e7522f9510a4a1149c15d810dd9
551 .U3 "\fLslocal\fP Supress Duplicates
552 .P
553 .Pn slocal
554 is an MDA included in mmh.
555 This is a violation of the idea that mmh is a MUA only.
556 .Pn slocal
557 should become a separate project.
558 Nonetheless, ouf of convenience and due to lack of convincement,
559 yet it remains being part of mmh.
560 This is likely to change in the future.
561 Already,
562 .Pn slocal was stripped down.
563 It used to depend on
564 .I ndbm ,
565 a database library.
566 The database is used to store the message ids of all messages delivered.
567 This enables
568 .Pn slocal
569 to suppress delivering the same message to the same user twice.
570 (This features was enabled by the
571 .Sw -suppressdup
572 switch.)
573 .P
574 A variety of version of the database library exist.
575 Complicated autoconf code was needed to detect them correctly.
576 Further more, the configure switches
577 .Sw --with-ndbm=ARG
578 and
579 .Sw --with-ndbmheader=ARG
580 were added to help with difficult setups that would
581 not be detected automatically.
582 .P
583 By removing the suppress duplicates feature of
584 .Pn slocal ,
585 the dependency on
586 .I ndbm
587 was removed and 120 lines of complex autoconf could be saved.
588 .Ci ecd6d6a20cb7a1507e3a20d6c4cb3a1cf14c6bbf
589 The change removed funtionality too, but the value it would have added
590 is minor to the weight loss by dropping the dependency and
591 the complex autoconf code.
593 .U3 "mh-e Support
594 .P
595 The configure option
596 .Sw --disable-mhe
597 was removed when the mh-e support was reworked.
598 Mh-e is the Emacs front-end to MH.
599 It requires MH to act different in some minor ways.
600 The configure option could switch the extension off.
601 After removing support for old versions of mh-e,
602 only the
603 .Sw -build
604 switches for
605 .Pn forw
606 and
607 .Pn repl
608 are left to be mh-e-specific.
609 They are now always available because they add little code and complexity.
610 This change was first done in nmh and thereafter merged into mmh.
611 The interface changes in mmh require mh-e to be adjusted to use mmh
612 as the back-end. This requires minor changes to mh-e, though removing
613 the
614 .Sw -build
615 switches would require larger adjustments.
616 The
617 .Sw --disable-mhe
618 configure option was removed and the remaining support for mh-e is always
619 built in.
620 .Ci a7ce7b4a580d77b6c2c4d980812beb589aa4c643
621 Removing the option removed a second code setup that would have
622 needed to be tested.
624 .U3 "Masquerading
625 .P
626 The configure option
627 .Sw --enable-masquerade
628 could take up to three items: draft_from, mmailid, username_extension.
629 They activated different types of address masquerading.
630 All of them were implemented in the SMTP-speaking
631 .Pn post
632 command.
633 Mmh no longer speaks SMTP and the replacing
634 .Pn spost
635 command no longer does MTA jobs like this one.
636 Because address masquerading is an MTA's task and mmh does not cover
637 this field anymore, the funtion needs to be implemented in the
638 external MTA used.
639 .P
640 The
641 .I mmailid
642 masquerading type is the oldest one of the three and the only one
643 available in the original MH.
644 It provided a
645 .I username
646 to
647 .I fakeusername
648 mapping, based on the value of the password file's GECOS field.
649 The man page
650 .Mp mh-tailor(5)
651 described the use case as being the following:
652 .QP
653 This is useful if you want the messages you send to always
654 appear to come from the name of an MTA alias rather than your
655 actual account name. For instance, many organizations set up
656 `First.Last' sendmail aliases for all users. If this is
657 the case, the GECOS field for each user should look like:
658 ``First [Middle] Last <First.Last>''
659 .P
660 As mmh sends outgoing mail via the local MTA only,
661 it is the best location to do such global rewrites.
662 Besides, the MTA is conceptionally the right location because it
663 does the reverse mapping for incoming mail (aliasing), too.
664 The masquerading set up there is set up once for all
665 mail software on the system.
666 .Ci 0836c8000ccb34b59410ef1c15b1b7feac70ce5f
667 .P
668 The
669 .I username_extension
670 masquerading type did not replace the username but could append a suffix
671 to it.
672 The suffix needed to be specified by the
673 .Ev USERNAME_EXTENSION
674 environment variable.
675 It provided support for the
676 .I user-extension
677 feature of qmail and the similar
678 .I "plussed user
679 processing of sendmail.
680 The decision to remove this username_extension masquerading was
681 motivated by the fact that
682 .Pn spost
683 hadn't supported it.
684 .Ci 2abae0bfd0ad5bf898461e50aa4b466d641f23d9_username_extension
685 Mmh now provides a more general, though in this case less convenient,
686 kind of masquerading.
687 .P
688 The
689 .I draft_from
690 masquerading type instructed
691 .Pn post
692 to use the value of the `From:' header as SMTP envelope sender.
693 This allowes to replace the sender address completely.
694 .Ci b14ea6073f77b4359aaf3fddd0e105989db9
695 Mmh now offers a kind of masquerading similar in effect, but
696 with technical differences.
697 As mmh does not transfer messages itself, the local MTA has full control
698 over the sending address. Any masquerading mmh introduces may be reverted
699 by the MTA. In times of pedantic spam checking, an MTA will likely do so
700 to keep its own reputation up.
701 Nonetheless, the MUA can set the `From:' header and thus propose
702 a sender address to be used to the MTA.
703 The MTA may then decide to take that one or generate the canonical sender
704 address for use as envelope sender address.
705 .P
706 In mmh, the MTA will always extract the recipient and sender from the
707 headers (\c
708 .Pn sendmail 's
709 .Sw -t
710 switch).
711 The `From:' header of the draft may be set arbitrary by the user.
712 If it is missing, the canonical sender address will be generated by the MTA.
714 .U3 "Remaining Options
715 .P
716 Two configure options remain in mmh.
717 One is the locking method to use:
718 .Sw --with-locking=[dot|fcntl|flock|lockf] .
719 Removing all other methods except the portable dot locking and having
720 that as default is appealing, but requires deeper investigation into the
721 topic.
722 The other,
723 .Sw --enable-debug ,
724 compiles the programs with debugging symbols and does not strip them.
725 This option is likely to stay.
730 .H2 "Removal of switches
731 .P
736 .H1 "Modernizing
739 .H2 "Removal of Code Relicts
740 .P
741 The code base of mmh originates from the late Seventies,
742 had been extensively
743 worked on in the mid Eighties, and had been partly reorganized and extended
744 in the Nineties. Relicts of all those times had gathered in the code base.
745 My goal was to remove any ancient code parts. One part of the task was
746 converting obsolete code constructs to standard constructs, the other part
747 was dropping obsolete functions.
748 .P
749 As I'm not even thirty years old and have no more than seven years of
750 Unix experience, I needed to learn about the history in retrospective.
751 Older people likely have used those ancient constructs themselves
752 and have suffered from their incompatibilities and have longed for
753 standardization. Unfortunately, I have only read that others had done so.
754 This put me in a much more difficult positions when working on the old
755 code. I needed to recherche what other would have known by heart from
756 experience. All my programming experience comes from a time past ANSI C
757 and past POSIX. Although I knew about the times before, I took the
758 current state implicitly for granted most of the time.
759 .P
760 Being aware of
761 these facts, I rather let people with more historic experience solve the
762 task of converting the ancient code constructs to standardized ones.
763 Luckily, Lyndon Nerenberg focused on this task at the nmh project.
764 He converted large parts of the code to POSIX constructs, removing
765 the conditionals compilation for now standardized features.
766 I'm thankful for this task being solved. I only pulled the changes into
767 mmh.
768 .P
769 The other task \(en dropping ancient functionality to remove old code \(en
770 I did myself, though. My position to strip mmh to the bare minimum of
771 frequently used features is much more revolutional than the nmh community
772 likes it. Without the need to justify my decisions, I was able to quickly
773 remove functionality I considered ancient.
774 The need to discuss my decisions with
775 peers likely would have slowed this process down. Of course, I researched
776 if a particular feature really should be dropped. Having not had any
777 contact to this feature within my computer life was a first indicator to
778 drop it, but I also asked others and searched the literature for modern
779 usage of the feature. If it appeared to be truly ancient, I dropped it.
780 The reason for dropping is always part of the commit message in the
781 version control system. Thus, it is easy for others to check their
782 view on the topic with mine and possibly to argue for reinclusion.
784 .U2 "MMDF maildrop support
785 .P
786 I did drop any support for the MMDF maildrop format. This type of format
787 is conceptionally similar to the mbox format, but uses four bytes with
788 value 1 (\fL^A^A^A^A\fP) as message delimiter,
789 instead of the string ``\fLFrom\ \fP''.
790 Due to the similarity and mbox being the de-facto standard maildrop
791 format on Unix, but also due to the larger influence of Sendmail than MMDF,
792 the MMDF maildrop format had vanished.
793 .P
794 The simplifications within the code were only moderate. Switches could
795 be removed from tools like
796 .L packf ,
797 which generate packed mailboxes. Only one packed mailbox format remained:
798 mbox.
799 The most important changes affect the equally named mail parsing routine in
800 .L sbr/m_getfld.c .
801 The direct MMDF code had been removed, but as now only one packed mailbox
802 format is left, code structure simplifications are likely possible.
803 The reason why they are still outstanding is the heavily optimized code
804 of
805 .Fu m_getfld() .
806 Changes beyond a small local scope \(en
807 which restructuring in its core is \(en cause a high risk of damaging
808 the intricate workings of the optimized code. This problem is know
809 to the developers of nmh, too. They also avoid touching this minefield
810 if possible.
812 .U2 "UUCP Bang Paths
813 .P
814 More questionably than the former topic is the removal of support for the
815 UUCP bang path address style. However, the user may translate the bang
816 paths on retrieval to Internet addresses and the other way on posting
817 messages. The former can be done my an MDA like procmail; the latter
818 by a sendmail wrapper. This would ensure that any address handling would
819 work as expected. However, it might just work well without any
820 such modifications, as mmh does not touch addresses much, in general.
821 But I can't ensure as I have never used an environment with bang paths.
822 Also, the behavior might break at any point in further development.
824 .U2 "Hardcopy terminal support
825 .P
826 More of a funny anecdote is the remaining of a check for printing to a
827 hardcopy terminal until Spring 2012, when I finally removed it.
828 I surely would be very happy to see such a terminal in action, maybe
829 actually being able to work on it, but I fear my chances are null.
830 .P
831 The check only prevented a pager to be placed between the outputting
832 program (\c
833 .Pn mhl )
834 and the terminal. This could have been ensured with
835 the
836 .Sw \-nomoreproc
837 at the command line statically, too.
839 .U2 "Removed support for header fields
840 .P
841 The `Encrypted' header had been introduced by RFC\^822, but already
842 marked legacy in RFC 2822. It was superseded by FIXME.
843 Mmh does no more support this header.
844 .P
845 Native support for `Face' headers
846 had been removed, as well.
847 The feature is similar to the `X-Face' header in its intent,
848 but takes a different approach to store the image.
849 Instead of encoding the image data directly into the header,
850 the the header contains the hostname and UDP port where the image
851 date could be retrieved.
852 Neither `X-Face' nor the here described `Face' system
853 \**
854 .FS
855 There is also a newer but different system, invented 2005,
856 using `Face' headers.
857 It is the successor of `X-Face' providing colored PNG images.
858 .FE
859 became well used in the large scale.
860 It's still possible to use a Face systems,
861 although mmh does not provide support for any of the different systems
862 anymore. It's fairly easy to write a small shell script to
863 extract the embedded or fetch the external Face data and display the image.
864 Own Face headers can be added into the draft template files.
865 .P
866 `Content-MD5' headers were introduced by RFC\^1864. They provide only
867 a verification of data corruption during the transfer. By no means can
868 they ensure verbatim end-to-end delivery of the contents. This is clearly
869 stated in the RFC. The proper approach to provide verificationability
870 of content in an end-to-end relationship is the use of digital cryptography
871 (RFCs FIXME). On the other hand, transfer protocols should ensure the
872 integrity of the transmission. In combinations these two approaches
873 make the `Content-MD5' header field useless. In consequence, I removed
874 the support for it. By this removal, MD5 computation is not needed
875 anywhere in mmh. Hence, over 500 lines of code were removed by this one
876 change. Even if the `Content-MD5' header field is useful sometimes,
877 I value its usefulnes less than the improvement in maintainability, caused
878 by the removal.
880 .U2 "Prompter's Control Keys
881 .P
882 The program
883 .Pn prompter
884 queries the user to fill in a message form. When used by
885 .Pn comp
886 as:
887 .DS
888 comp \-editor prompter
889 .DE
890 the resulting behavior is similar to
891 .Pn mailx .
892 Apparently,
893 .Pn prompter
894 hadn't been touched lately. Otherwise it's hardly explainable why it
895 still offered the switches
896 .Sn \-erase \fUchr\fP
897 and
898 .Sn \-kill \fUchr\fP
899 to name the characters for command line editing.
900 The times when this had been necessary are long time gone.
901 Today these things work out-of-the-box, and if not, are configured
902 with the standard tool
903 .Pn stty .
905 .U2 "Vfork and Retry Loops
906 .P
907 MH creates many processes, which is a consequence of the tool chest approach.
908 In earlier times
909 .Fu fork()
910 had been an expensive system call, as the process's whole image needed
911 to be duplicated. One common case is replacing the image with
912 .Fu exec()
913 right after having forked the child process.
914 To speed up this case, the
915 .Fu vfork()
916 system call was invented at Berkeley. It completely omits copying the
917 image. If the image gets replaced right afterwards then unnecessary
918 work is omited. On old systems this results in large speed ups.
919 MH uses
920 .Fu vfork()
921 whenever possible.
922 .P
923 Memory management units that support copy-on-write semantics make
924 .Fu fork()
925 almost as fast as
926 .Fu vfork()
927 in the cases when they can be exchanged.
928 With
929 .Fu vfork()
930 being more error-prone and hardly faster, it's preferable to simply
931 use
932 .Fu fork()
933 instead.
934 .P
935 Related to the costs of
936 .Fu fork()
937 is the probability of its success.
938 Today on modern systems, the system call will succeed almost always.
939 In the Eighties on heavy loaded systems, as they were common at
940 universities, this had been different. Thus, many of the
941 .Fu fork()
942 calls were wrapped into loops to retry to fork several times in
943 short intervals, in case of previous failure.
944 In mmh, the program aborts at once if the fork failed.
945 The user can reexecute the command then. This is expected to be a
946 very rare case on modern systems, especially personal ones, which are
947 common today.
950 .H2 "Attachments
951 .P
952 MIME
955 .H2 "Digital Cryptography
956 .P
957 Signing and encryption.
960 .H2 "Good Defaults
961 .P
962 foo
967 .H1 "Code style
968 .P
969 foo
972 .H2 "Standard Code
973 .P
974 POSIX
977 .H2 "Separation
979 .U2 "MH Directory Split
980 .P
981 In MH and nmh, a personal setup had consisted of two parts:
982 The MH profile, named
983 .Fn \&.mh_profile
984 and being located directly in the user's home directory.
985 And the MH directory, where all his mail messages and also his personal
986 forms, scan formats, other configuration files are stored. The location
987 of this directory could be user-chosen. The default was to name it
988 .Fn Mail
989 and have it directly in the home directory.
990 .P
991 I've never liked the data storage and the configuration to be intermixed.
992 They are different kinds of data. One part, are the messages,
993 which are the data to operate on. The other part, are the personal
994 configuration files, which are able to change the behavior of the operations.
995 The actual operations are defined in the profile, however.
996 .P
997 When storing data, one should try to group data by its type.
998 There's sense in the Unix file system hierarchy, where configuration
999 file are stored separate (\c
1000 .Fn /etc )
1001 to the programs (\c
1002 .Fn /bin
1003 and
1004 .Fn /usr/bin )
1005 to their sources (\c
1006 .Fn /usr/src ).
1007 Such separation eases the backup management, for instance.
1008 .P
1009 In mmh, I've reorganized the file locations.
1010 Still there are two places:
1011 There's the mail storage directory, which, like in MH, contains all the
1012 messages, but, unlike in MH, nothing else.
1013 Its location still is user-chosen, with the default name
1014 .Fn Mail ,
1015 in the user's home directory. This is much similar to the case in nmh.
1016 The configuration files, however, are grouped together in the new directory
1017 .Fn \&.mmh
1018 in the user's home directory.
1019 The user's profile now is a file, named
1020 .Fn profile ,
1021 in this mmh directory.
1022 Consistently, the context file and all the personal forms, scan formats,
1023 and the like, are also there.
1024 .P
1025 The naming changed with the relocation.
1026 The directory where everything, except the profile, had been stored (\c
1027 .Fn $HOME/Mail ),
1028 used to be called \fIMH directory\fP. Now, this directory is called the
1029 user's \fImail storage\fP. The name \fImmh directory\fP is now given to
1030 the new directory
1031 (\c
1032 .Fn $HOME/.mmh ),
1033 containing all the personal configuration files.
1034 .P
1035 The separation of the files by type of content is logical and convenient.
1036 There are no functional differences as any possible setup known to me
1037 can be implemented with both approaches, although likely a bit easier
1038 with the new approach. The main goal of the change had been to provide
1039 sensible storage locations for any type of personal mmh file.
1040 .P
1041 In order for one user to have multiple MH setups, he can use the
1042 environment variable
1043 .Ev MH
1044 the point to a different profile file.
1045 The MH directory (mail storage plus personal configuration files) is
1046 defined by the
1047 .Pe Path
1048 profile entry.
1049 The context file could be defined by the
1050 .Pe context
1051 profile entry or by the
1052 .Ev MHCONTEXT
1053 environment variable.
1054 The latter is useful to have a distinct context (e.g. current folders)
1055 in each terminal window, for instance.
1056 In mmh, there are three environment variables now.
1057 .Ev MMH
1058 may be used to change the location of the mmh directory.
1059 .Ev MMHP
1060 and
1061 .Ev MMHC
1062 change the profile and context files, respectively.
1063 Besides providing a more consistent feel (which simply is the result
1064 of being designed anew), the set of personal configuration files can
1065 be chosen independently from the profile (including mail storage location)
1066 and context, now. Being it relevant for practical use or not, it
1067 de-facto is an improvement. However, the main achievement is the
1068 split between mail storage and personal configuration files.
1071 .H2 "Modularization
1072 .P
1073 whatnowproc
1074 .P
1075 The \fIMH library\fP
1076 .Fn libmh.a
1077 collects a bunch of standard functions that many of the MH tools need,
1078 like reading the profile or context files.
1079 This doesn't hurt the separation.
1082 .H2 "Style
1083 .P
1084 Code layout, goto, ...
1089 .H1 "Concept Exploitation/Homogeniety
1092 .H2 "Draft Folder
1093 .P
1094 Historically, MH provided exactly one draft message, named
1095 .Fn draft
1096 and
1097 being located in the MH directory. When starting to compose another message
1098 before the former one was sent, the user had been questioned whether to use,
1099 refile or replace the old draft. Working on multiple drafts at the same time
1100 was impossible. One could only work on them in alteration by refiling the
1101 previous one to some directory and fetching some other one for reediting.
1102 This manual draft management needed to be done each time the user wanted
1103 to switch between editing one draft to editing another.
1104 .P
1105 To allow true parallel editing of drafts, in a straight forward way, the
1106 draft folder facility exists. It had been introduced already in July 1984
1107 by Marshall T. Rose. The facility was deactivated by default.
1108 Even in nmh, the draft folder facility remained deactivated by default.
1109 At least, Richard Coleman added the man page
1110 .Mp mh-draft(5)
1111 to document
1112 the feature well.
1113 .P
1114 The only advantage of not using the draft folder facility is the static
1115 name of the draft file. This could be an issue for MH frontends like mh-e.
1116 But as they likely want to provide working on multiple drafts in parallel,
1117 the issue is only concerning compatibility. The aim of nmh to stay compatible
1118 prevented the default activation of the draft folder facility.
1119 .P
1120 On the other hand, a draft folder is the much more natural concept than
1121 a draft message. MH's mail storage consists of folders and messages,
1122 the messages named with ascending numbers. A draft message breaks with this
1123 concept by introducing a message in a file named
1124 .Fn draft .
1125 This draft
1126 message is special. It can not be simply listed with the available tools,
1127 but instead requires special switches. I.e. corner-cases were
1128 introduced. A draft folder, in contrast, does not introduce such
1129 corner-cases. The available tools can operate on the messages within that
1130 folder like on any messages within any mail folders. The only difference
1131 is the fact that the default folder for
1132 .Pn send
1133 is the draft folder,
1134 instead of the current folder, like for all other tools.
1135 .P
1136 The trivial part of the change was activating the draft folder facility
1137 by default and setting a default name for this folder. Obviously, I chose
1138 the name
1139 .Fn +drafts .
1140 This made the
1141 .Sw \-draftfolder
1142 and
1143 .Sw \-draftmessage
1144 switches useless, and I could remove them.
1145 The more difficult but also the part that showed the real improvement,
1146 was updating the tools to the new concept.
1147 .Sw \-draft
1148 switches could
1149 be dropped, as operating on a draft message became indistinguishable to
1150 operating on any other message for the tools.
1151 .Pn comp
1152 still has its
1153 .Sw \-use
1154 switch for switching between its two modes: (1) Compose a new
1155 draft, possibly by taking some existing message as a form. (2) Modify
1156 an existing draft. In either case, the behavior of
1157 .Pn comp is
1158 deterministic. There is no more need to query the user. I consider this
1159 a major improvement. By making
1160 .Pn send
1161 simply operate on the current
1162 message in the draft folder by default, with message and folder both
1163 overridable by specifying them on the command line, it is now possible
1164 to send a draft anywhere within the storage by simply specifying its folder
1165 and name.
1166 .P
1167 All theses changes converted special cases to regular cases, thus
1168 simplifying the tools and increasing the flexibility.
1171 .H2 "Trash Folder
1172 .P
1173 Similar to the situation for drafts is the situation for removed messages.
1174 Historically, a message was deleted by renaming. A specific
1175 \fIbackup prefix\fP, often comma (\c
1176 .Fn , )
1177 or hash (\c
1178 .Fn # ),
1179 being prepended to the file name. Thus, MH wouldn't recognize the file
1180 as a message anymore, as only files whose name consists of digits only
1181 are treated as messages. The removed messages remained as files in the
1182 same directory and needed some maintenance job to truly delete them after
1183 some grace time. Usually, by running a command similar to
1184 .DS
1185 find /home/user/Mail \-ctime +7 \-name ',*' | xargs rm
1186 .DE
1187 in a cron job. Within the grace time interval
1188 the original message could be restored by stripping the
1189 the backup prefix from the file name. If however, the last message of
1190 a folder is been removed \(en say message
1191 .Fn 6
1192 becomes file
1193 .Fn ,6
1194 \(en and a new message enters the same folder, thus the same
1195 numbered being given again \(en in our case
1196 .Fn 6
1197 \(en, if that one
1198 is removed too, then the backup of the former message gets overwritten.
1199 Thus, the ability to restore removed messages does not only depend on
1200 the ``sweeping cron job'' but also on the removing of further messages.
1201 This is undesirable, because the real mechanism is hidden from the user
1202 and the consequences of further removals are not always obvious.
1203 Further more, the backup files are scattered within the whole mail
1204 storage, instead of being collected at one place.
1205 .P
1206 To improve the situation, the profile entry
1207 .Pe rmmproc
1208 (previously named
1209 .Pe Delete-Prog )
1210 was introduced, very early.
1211 It could be set to any command, which would care for the mail removal
1212 instead of taking the default action, described above.
1213 Refiling the to-be-removed files to some garbage folder was a common
1214 example. Nmh's man page
1215 .Mp rmm(1)
1216 proposes
1217 .Cl "refile +d
1218 to move messages to the garbage folder and
1219 .Cl "rm `mhpath +d all`
1220 the empty the garbage folder.
1221 Managing the message removal this way is a sane approach. It keeps
1222 the removed messages in one place, makes it easy to remove the backup
1223 files, and, most important, enables the user to use the tools of MH
1224 itself to operate on the removed messages. One can
1225 .Pn scan
1226 them,
1227 .Pn show
1228 them, and restore them with
1229 .Pn refile .
1230 There's no more
1231 need to use
1232 .Pn mhpath
1233 to switch over from MH tools to Unix tools \(en MH can do it all itself.
1234 .P
1235 This approach matches perfect with the concepts of MH, thus making
1236 it powerful. Hence, I made it the default. And even more, I also
1237 removed the old backup prefix approach, as it is clearly less powerful.
1238 Keeping unused alternative in the code is a bad choice as they likely
1239 gather bugs, by not being constantly tested. Also, the increased code
1240 size and more conditions crease the maintenance costs. By strictly
1241 converting to the trash folder approach, I simplified the code base.
1242 .Pn rmm
1243 calls
1244 .Pn refile
1245 internally to move the to-be-removed
1246 message to the trash folder (\c
1247 .Fn +trash
1248 by default). Messages
1249 there can be operated on like on any other message in the storage.
1250 The sweep clean, one can use
1251 .Cl "rmm \-unlink +trash a" ,
1252 where the
1253 .Sw \-unlink
1254 switch causes the files to be truly unliked instead
1255 of moved to the trash folder.
1258 .H2 "Path Notations
1259 .P
1260 foo
1263 .H2 "MIME Integration
1264 .P
1265 user-visible access to whole messages and MIME parts are inherently
1266 different
1269 .H2 "Of One Cast
1270 .P