docs/master
view ch03.roff @ 85:fb0d1b4c8fb1
Extended the text about mail transfer configure options.
| author | markus schnalke <meillo@marmaro.de> |
|---|---|
| date | Wed, 06 Jun 2012 20:12:37 +0200 |
| parents | c35ec2b7de5d |
| children | 7d5b180de542 |
line source
1 .H0 "Discussion
2 .P
3 This main chapter discusses the practical work done in the mmh project.
4 It is structured along the goals to achieve. The concrete work done
5 is described in the examples of how the general goals were achieved.
10 .H1 "Stream-lining
12 .P
13 MH had been considered an all-in-one system for mail handling.
14 The community around nmh has a similar understanding.
15 In fundamental difference, should be a MUA only.
16 I believe that all-in-one mail systems are obsolete.
17 There are excellent specialized MTAs, like Postfix;
18 there are specialized MDAs, like Procmail; there are specialized
19 MRAs, like Fetchmail. I believe it's best to use them instead of
20 providing the same function ourselves. Doing something well, requires to
21 focus on a small set of aspects. The more
22 it is possible to focus, the better the result in this particular
23 area will be. Usually, the limiting resource in Free Software
24 community development is man power.
25 If the development power is even spread over a large
26 development area, it becomes more difficult to
27 compete with the specialists in the various fields. This is even
28 increased, given the small community \(en developers and users \(en
29 that MH-based mail systems have. In consequence, I believe that the
30 available resources should be focused to the point where MH is
31 most unique. This is clearly the MUA part.
32 .P
33 The goal for mmh was to remove peripheral parts and stream-line
34 it for the MUA task.
37 .H2 "Removal of Mail Transfer Facilities
38 .P
39 In contrast to nmh, which also provides mail submission and mail retrieval
40 facilities, mmh is a MUA only.
41 This general difference in the view on the character of nmh
42 initiated the development of mmh.
43 Removing the mail transfer facilities had been the first work task
44 in the mmh project.
45 .P
46 The MSA is called \fIMessage Transfer Service\fP (MTS) in nmh.
47 The facility established network connections and spoke SMTP to submit
48 messages for relay to the outside world.
49 This part was implemented by the
50 .Pn post
51 command.
52 The changes in emailing
53 demanded changes in this part of nmh in the last years.
54 Encryption and authetication for network connections
55 needed to be supported, hence TLS and SASL were introduced
56 into nmh. This added complexity to the nmh without improving it in
57 its core functions. Also, keeping up with recent developments in
58 this field requires development power and specialists.
59 For mmh this whole facility was cut off.
60 .Ci f6aa95b724fd8c791164abe7ee5468bf5c34f226
61 .Ci fecd5d34f65597a4dfa16aeabea7d74b191532c3
62 .Ci 156d35f6425bea4c1ed3c4c79783dc613379c65b
63 Instead, mmh depends on an external MTA.
64 The only outgoing interface available to mmh is the
65 .Pn sendmail
66 command.
67 Almost any MTA provides a
68 .Pn sendmail
69 command.
70 If not, any program can be substituted if it reads the
71 message from the standard input, extracts the recipient addresses
72 from the message header and does not conflict
73 with sendmail-specific command line options.
74 .P
75 To retrieve mail, the
76 .Pn inc
77 command established network connections
78 and spoke POP3 to retrieve mail from remote servers.
79 As with mail submission, the network connections required encryption and
80 authentication, thus TLS and SASL was added.
81 As POP3 becomes more and more superseded by IMAP, support for message
82 retrieval through IMAP will become necessary to be added soon, too.
83 Mmh has dropped the support for retrieving mail from remote locations.
84 .Ci ab7b48411962d26439f92f35ed084d3d6275459c
85 Instead, it depends on an external tool to cover this task.
86 There exist two paths for messages to enter mmh's mail storage:
87 They can be incorporate with
88 .Pn inc
89 from the system maildrop, or
90 .Pn rcvstore
91 reads them from the standard input.
92 .P
93 With the removal of the MSA and MRA, mmh converted from an all-in-one
94 mail system to being only a MUA.
95 Following the Unix philosophy, it focuses on one job and
96 tries to do that one well.
97 Not only the programs follow that tenet but also the project itself does so.
98 Now, of course, mmh depends on third-party software.
99 An external MTA/MSA is required to transfer mail to the outside world;
100 an external MRA is required to retrieve mail from remote machines.
101 There exist excellent implementations of such software,
102 which do this specific task likely better than the internal
103 versions had done it. Also, the best suiting programs can be freely chosen.
104 .P
105 As it had already been possible to use an external MSA or MRA,
106 why not keep the internal version for convenience?
107 The question whether there is sense in having a fall-back pager in all
108 the command line tools, for the cases when
109 .Pn more
110 or
111 .Pn less
112 aren't available, appears to be ridiculous.
113 Now, an MSA or MRA is clearly more complex than a text pager,
114 and not necessarily available but still the concept holds:
115 design the system orthogonally.
116 If it is conceptionally more elegant to have the MTA to be a separate tool
117 \(en as the RFCs propose this split, this is likely the case \(en
118 then separate.
119 .P
120 Further more, if programs become complex, they should be split;
121 and if projects become complex, they should be split, too.
122 Essential complexity is defined by the problem.
123 Decades ago, emailing had been small and simple.
124 (\c
125 .Pn /bin/mail
126 had once covered anything there was to email and still had been small
127 and simple.)
128 Then the essential complexity of email increased.
129 Email tools needed to react.
130 In nmh, for instance, the POP server, which the original MH had included,
131 was removed.
132 Now is the time to go one step further and remove the MSA and MRA.
133 Not only does it decrease the code amount of the project,
134 but more important, it removes the whole field of message transfer
135 with all its implications for the project.
136 It removes the need to adjust to any changes concerning network transfer.
137 This independence is received by depending on an external program
138 that covers the field.
139 Today, this is a reasonable exchange.
140 .P
141 To add some kind of function, there's always the choice
142 among implementing the function in the project directly,
143 depending on a library that provides the function, or depending on
144 a program that provides the function.
145 Whereas adding the function directly to the project increases the
146 code size most and requires most maintenance and development work,
147 it makes the project most independent.
148 Using libraries or external programs require less
149 maintenance work.
150 Programs have the smallest interfaces, providing the most separation
151 but possibly limiting the information exchange.
152 External libraries are stronger connected than external programs but
153 allow better information exchange.
154 Adding more code to a project does always increase maintenance work.
155 Implementing complex functions directly in the project will add
156 a lot of code. This should be avoided if possible.
157 Hence, the dependencies only change in kind, not in their existence.
158 In mmh, library dependencies on
159 .Pn libsasl2
160 and
161 .Pn libcrypto /\c
162 .Pn libssl
163 were treated against program dependencies on an MSA and an MRA.
164 Besides program dependencies providing the stronger separation
165 and being more flexible, they also allowed
166 over 6\|000 lines of code to be removed from mmh.
167 This made mmh's code base about 12\|% smaller.
168 Reducing the projects code size by such an amount without actually
169 losing function is a convincing argument.
170 Actually, as external MSAs and MRAs are likely better
171 than the project's internal version, the user even gains functionality.
172 .P
173 Users of MH should not have problems to set up an external MSA and MRA.
174 Also, the popular MSAs and MRAs have large communities and a lot
175 of documentation available.
176 Choices for MSAs range from the full-featured
177 .I Postfix
178 over mid-size solutions like
179 .I masqmail
180 and
181 .I dma
182 to small forwarders like
183 .I ssmtp
184 and
185 .I nullmailer .
186 Choices for MRAs include
187 .I fetchmail ,
188 .I getmail ,
189 .I mpop
190 and
191 .I fdm .
194 .H2 "Removal of non-MUA Tools
195 .P
196 Some MH tools were removed because they didn't add to the MUA's job.
197 It is a design goal of mmh to remove the parts that are rarely used.
198 The project shall become more stream-lined and focused.
199 Rarely used and loosely related tools distract from the lean appearance.
200 They require maintenance work without adding to the core task.
201 In mmh the following tools are not available anymore:
202 .BU
203 .Pn conflict
204 was removed because it is a mail system maintenance tool.
205 .Ci 8b235097cbd11d728c07b966cf131aa7133ce5a9
206 Besides, it even checks
207 .Fn /etc/passwd
208 and
209 .Fn /etc/group
210 for consistency, which has nothing at all to do with emailing.
211 The tool might be useful, but it should not be shipped with mmh.
212 .\" XXX historic reasons?
213 .BU
214 .Pn rcvtty
215 was removed because its usecase of writing to the user's terminal
216 on receiving of mail is obsolete.
217 .Ci 14767c94b3827be7c867196467ed7aea5f6f49b0
218 If users like to be
219 informed of new mail, the shell's
220 .Ev MAILPATH
221 variable or graphical notifications are more appealing.
222 Writing directly to a terminals is hardly ever wanted today.
223 If though one wants to have it this way, the standard tool
224 .Pn write
225 can be used in a way similar to:
226 .VS
227 scan -file - | write `id -un`
228 VE
229 .BU
230 .Pn viamail
231 was removed when the new attachment system was activated, because
232 .Pn forw
233 could then cover the task itself.
234 .Ci eda72d6a7a7c20ff123043fb7f19c509ea01f932
235 The program
236 .Pn sendfiles
237 was rewritten as a shell script wrapper around
238 .Pn forw .
239 .Ci 0e82199cf3c991a173e0ac8aa776efdb3ded61e6
240 .BU
241 .Pn msgchk
242 was removed, because it lost its use case when POP support was removed.
243 .Ci bb9360ead7eb7a3fedcce2eeedfc660014e41dbe
244 A call to
245 .Pn msgchk
246 provided hardly more information than
247 .VS
248 ls -l /var/mail/meillo
249 VE
250 though it distinguished between old and new mail.
251 This detail information and can be retrieved with
252 .Pn stat (1),
253 too.
254 A very small shell script could be written to output the information
255 in a similar way, if truly necessary.
256 As mmh's
257 .Pn inc
258 only incorporates mail from the user's local maildrop
259 and thus no data transfers over slow networks are involved,
260 there's hardly any need to check for new mail before incorporating it.
261 .BU
262 .Pn msh
263 was removed because the tool was in conflict with the philosophy of MH.
264 .Ci 916690191222433a6923a4be54b0d8f6ac01bd02
265 It provided an interactive shell to access the features of MH,
266 but it wasn't just a shell, tailored to the needs of mail handling.
267 Instead it was one large program that had several MH tools built in.
268 This conflicts with the major feature of MH of being a tool chest.
269 .Pn msh 's
270 main use case had been accessing Bulletin Boards, which have seized to
271 be popular.
272 .P
273 Removing
274 .Pn msh ,
275 together with the truly archaic code relicts
276 .Pn vmh
277 and
278 .Pn wmh ,
279 saved more than 7\|000 lines of C code \(en
280 about 15\|% of the project's original source code amount.
281 .P
282 Having less code (with equal readability, of course)
283 for the same functionality is an advantage.
284 Less code means less bugs and less maintenance work.
285 As
286 .Pn rcvtty
287 and
288 .Pn msgchk
289 are rarely used and can be implemented in different ways,
290 then why should one keep them?
291 Removing them stream-lines mmh.
292 .Pn viamail 's
293 use case is now partly obsolete and partly covered by
294 .Pn forw ,
295 hence there's no reason to still maintain it.
296 .Pn conflict
297 is not related to the mail client, and
298 .Pn msh
299 conflicts with the basic concept of MH.
300 Theses two tools might still be useful, but they should not be part of mmh.
301 .P
302 Finally, there's
303 .Pn slocal .
304 .Pn slocal
305 is an MDA and thus not directly MUA-related.
306 It should be removed, because including it is a violation
307 of the idea that mmh is a MUA only.
308 It should become a separate project.
309 But
310 .Pn slocal
311 provides rule-based processing of messages, like filing them into
312 different folders, which is otherwise not available in mmh.
313 Further more,
314 .Pn slocal
315 does neither pull dependencies nor a whole new technical area
316 into the project.
317 (See section XXX for the removing of the ndbm dependency.)
318 Still,
319 .Pn slocal
320 accounts for about 1\|000 lines of code that need to be maintained.
321 As
322 .Pn slocal
323 is almost self-standing, it should be split off into a separate project.
324 This would cut the strong connection between the MUA mmh and the MDA
325 .Pn slocal .
326 The MDA would become an alternative to
327 .I procmail ,
328 as it would no longer be the need to install a complete MH system, too.
329 Likewise, mmh users could decide to use
330 .I procmail
331 without having a second, unused MDA (\c
332 .Pn slocal )
333 installed.
334 That's conceptionally the best solution.
335 Yet,
336 .Pn slocal
337 was not removed.
338 I feel unsure with the removal.
339 Hence, the decision over
340 .Pn slocal
341 is deferred.
342 This does not hurt because
343 .Pn slocal
344 is completely unrelated to the rest of mmh.
347 .H2 "\fLshow\fP and \fPmhshow\fP
348 .P
349 Since the very beginning \(en already in the first concept paper \(en
350 .Pn show
351 had been MH's message display program.
352 .Pn show
353 mapped message numbers and sequences to files and invoked
354 .Pn mhl
355 to have the files formated.
356 For MIME, this approach wasn't sufficient anymore.
357 MIME messages can consist of multiple parts, some of which aren't
358 directly displayable, and text content might be encoded in
359 foreign charsets.
360 .Pn show 's
361 understanding of messages and
362 .Pn mhl 's
363 limited display facilities couldn't cope with the task any longer.
364 .P
365 Instead of extending these tools, additional tools were written from scratch
366 and then added to the MH tool chest. Doing so is encouraged by the
367 tool chest approach. The new tools could be added without interfering
368 with the existing ones.
369 Modular design is a great advantage for extending a system.
370 First, the new MIME features were added in form of the single program
371 .Pn mhn .
372 The command
373 .Cl "mhn -show 42
374 would show the MIME message numbered 42.
375 With the 1.0 release of nmh in February 1999, Richard Coleman finished
376 the split of
377 .Pn mhn
378 into a set of specialized programs, which together covered the
379 multiple aspects of MIME. One of these resulting tools was
380 .Pn mhshow ,
381 which replaced the
382 .Cl "mhn -show
383 call.
384 It was capable to display a MIME message appropriately.
385 .P
386 From then on, two message display tools were part of nmh:
387 .Pn show
388 and
389 .Pn mhshow .
390 Because the user should not need to invoke the right tool
391 whether the message uses MIME or not,
392 .Pn show
393 was extended to automatically hand the job over to
394 .Pn mhshow
395 if displaying the message would be beyond
396 .Pn show 's
397 abilities.
398 In consequence, the user would invoke
399 .Pn show
400 (possibly through
401 .Pn next
402 or
403 .Pn prev )
404 and get the message printed with either
405 .Pn show
406 or
407 .Pn mhshow ,
408 whatever was more appropriate.
409 (There was also a switch for
410 .Pn show
411 to never invoke
412 .Pn mhshow .
413 .Pn show
414 was able to display MIME messages if they contained only a single text
415 part.)
416 .P
417 Having two similar tools for essentially the same task is redundant.
418 The development of both programs needed to be in sync,
419 to ensure that the programs behaved in a similar way,
420 because they were used like a single tool.
421 Different behavior would have surprised the user.
422 .P
423 Today, non-MIME messages are rather seen to be a special case of
424 MIME messages, than MIME messages are seen to be an extension to
425 original email.
426 As
427 .Pn mhshow
428 had already be able to display non-MIME messages, it was natural
429 to drop
430 .Pn show
431 in favor of using
432 .Pn mhshow
433 exclusively.
434 This decision followed the idea of orthogonal design.
435 For convenience,
436 .Pn mhshow
437 was then renamed to
438 .Pn show .
439 .Ci 4c1efddfd499300c7e74263e57d8aa137e84c853
440 .P
441 To prepare for this transition,
442 .Pn mhshow
443 was reworked to behave more like
444 .Pn show
445 first.
446 (Section XXX describes this rework from a different perspective.)
447 Once the tools behaved similar, the replacing became a natural decision.
448 In mmh,
449 .Pn show
450 is the one single message display program again, but it handles
451 MIME messages as well as non-MIME messages.
452 Now, there's only one program to maintain, and users don't need to deal
453 with the existance of two display programs.
454 .P
455 There's one reason why removing the old
456 .Pn show
457 hurts: It had been such a simple program.
458 Its lean elegance is missing to
459 .Pn mhshow ,
460 i.e. the new
461 .Pn show .
462 But there is no chance, because supporting MIME causes essentially
463 higher complexity.
466 .H2 "Removal of Configure Options
467 .P
468 Customization is a double-edged sword.
469 It allows better suiting setups, but not for free.
470 There is the cost of code complexity to be able to customize.
471 There is the cost of less tested setups, because there are
472 more possible setups and especially corner-cases.
473 And, there is the cost of choice itself.
474 The code complexity directly affects the developers.
475 Less tested code affects both, users and developers.
476 The problem of choice affects the users, for once by having to
477 choose, but also by complexer interfaces that require more documentation.
478 Whenever options add little advantages, they should be considered for
479 removal.
480 I have reduced the number of project-specific configure options from
481 fifteen to three.
483 .U3 "Mail Transfer Facilities
484 .P
485 With the removal of the mail transfer facilities five configure
486 options vanished:
487 .P
488 The switches
489 .Sw --with-tls
490 and
491 .Sw --with-cyrus-sasl
492 had activated the support for transfer encryption and authetication.
493 This is not needed anymore.
494 .Ci fecd5d34f65597a4dfa16aeabea7d74b191532c3
495 .Ci 156d35f6425bea4c1ed3c4c79783dc613379c65b
496 .P
497 The configure switch
498 .Sw --enable-pop
499 activated the message retrieval facility.
500 The code area that would be conditionally compiled in for TLS and SASL
501 support had been small.
502 The conditionally compiled code area for POP support had been much larger.
503 Whereas the code base changes would only slightly change on toggling
504 TLS or SASL support, it changed much on toggling POP support.
505 The changes in the code base could hardly be overviewed.
506 By having POP support togglable a second code base had been created,
507 one that needed to be tested.
508 This situation is basically similar for the conditional TLS and SASL
509 code, but there the changes are minor and can yet be overviewed.
510 Still, conditional compilation of a code base creates variations
511 of the original program.
512 More variations require more testing and maintenance work.
513 .P
514 Two other options only specified default configuration values:
515 .Sw --with-mts=[smtp|sendmail]
516 defined the default transport service.
517 In mmh this fixed to
518 .Ar sendmail .
519 .Ci f6aa95b724fd8c791164abe7ee5468bf5c34f226
520 With
521 .Sw --with-smtpservers=[server1...]
522 default SMTP servers for the
523 .Ar smtp
524 transport service could be specified.
525 .Ci 128545e06224233b7e91fc4c83f8830252fe16c9
526 Both of them became irrelevant.
528 .U3 "Backup Prefix
529 .P
530 The backup prefix is the string that was prepended to message
531 filenames to tag them as deleted.
532 By default it had been the comma character `\f(CW,\fP'.
533 In July 2000, Kimmo Suominen introduced
534 the configure option
535 .Sw --with-hash-backup
536 to change the default to the hash symbol `\f(CW#\fP'.
537 The choice was probably personal preference, because first, the
538 option was named
539 .Sw --with-backup-prefix.
540 and had the prefix symbol as argument.
541 Because giving the hash symbol as argument caused to many problems
542 for configure,
543 the option was limited to use the hash symbol as the default prefix.
544 This makes me believe, that the choice for the hash was personal preference.
545 Being it related or not, words that start with the hash symbol
546 introduce a comment in the Unix shell.
547 Thus, the command line
548 .Cl "rm #13 #15
549 calls
550 .Pn rm
551 without arguments because the first hash symbol starts the comment
552 that reaches until the end of the line.
553 To delete the backup files,
554 .Cl "rm ./#13 ./#15"
555 needs to be used.
556 Using the hash as backup prefix can be seen as a precaution agains
557 data loss.
558 .P
559 I removed the configure option but added the profile entry
560 .Pe backup-prefix ,
561 which allows to specify an arbitrary string as backup prefix.
562 .Ci 6c40d481d661d532dd527eaf34cebb6d3f8ed086
563 Profile entries are the common method to change mmh's behavior.
564 This change did not remove the choice but moved it to a location where
565 it suited better.
566 .P
567 Eventually, however, the new trash folder concept
568 .Cf "Sec. XXX
569 obsoleted the concept of the backup prefix completely.
570 .Ci 8edc5aaf86f9f77124664f6801bc6c6cdf258173
571 (Well, there still are corner-cases to remove until the backup
572 prefix can be layed to rest, eventually.)
573 .\" FIXME: Do this work in the code!
575 .U3 "Editor and Pager
576 .P
577 The two configure options
578 .CW --with-editor=EDITOR
579 .CW --with-pager=PAGER
580 were used to specify the default editor and pager at configure time.
581 Doing so at configure time made sense in the Eighties,
582 when the set of available editors and pagers varied much across
583 different systems.
584 Today, the situation is more homegeneic.
585 The programs
586 .Pn vi
587 and
588 .Pn more
589 can be expected to be available on every Unix system,
590 as they are specified by POSIX since two decades.
591 (The specifications for
592 .Pn vi
593 and
594 .Pn more
595 appeared in
596 .[
597 posix 1987
598 .]
599 and,
600 .[
601 posix 1992
602 .]
603 respectively.)
604 As a first step, these two tools were hard-coded as defaults.
605 .Ci 5d43a99db70c12a673028c7758c20cbe3e13ef5f
606 Not changed were the
607 .Pe editor
608 and
609 .Pe moreproc
610 profile entries, which allowed the user to override the system defaults.
611 Later, the concept was reworked to respect the standard environment
612 variables
613 .Ev VISUAL
614 and
615 .Ev PAGER
616 if they are set.
617 Today, mmh determines the editor to use in the following order,
618 taking the first available and non-empty item:
619 .IP (1)
620 Environment variable
621 .Ev MMHEDITOR
622 .IP (2)
623 Profile entry
624 .Pe Editor
625 .IP (3)
626 Environment variable
627 .Ev VISUAL
628 .IP (4)
629 Environment variable
630 .Ev EDITOR
631 .IP (5)
632 Command
633 .Pn vi .
634 .P
635 .Ci f85f4b7ae62e3d05a945dcd46ead51f0a2a89a9b
636 .P
637 The pager to use is deteminded in a similar order,
638 also taking the first available and non-empty item:
639 .IP (1)
640 Environment variable
641 .Ev MMHPAGER
642 .IP (2)
643 Profile entry
644 .Pe Pager
645 (replaces
646 .Pe moreproc )
647 .IP (3)
648 Environment variable
649 .Ev PAGER
650 .IP (4)
651 Command
652 .Pn more .
653 .P
654 .Ci 0c4214ea2aec6497d0d67b436bbee9bc1d225f1e
655 .P
656 By respecting the
657 .Ev VISUAL /\c
658 .Ev EDITOR
659 and
660 .Ev PAGER
661 environment variables,
662 the new behavior confirms better to the common style on Unix systems.
663 Additionally, the new approach is more uniform and clearer to users.
665 .U3 "Locale
666 .P
667 The configure option
668 .Sw --disable-locale
669 was removed because POSIX provides locale support and there's
670 hardly any need to disable locale support.
671 .Ci ccf4f175ef4c4e7522f9510a4a1149c15d810dd9
673 .U3 "ndbm
674 .P
675 .Pn slocal
676 used to depend on
677 .I ndbm ,
678 a database library.
679 The database is used to store the `\fLMessage-ID\fP's of all
680 messages delivered.
681 This enables
682 .Pn slocal
683 to suppress delivering the same message to the same user twice.
684 (This features was enabled by the
685 .Sw -suppressdup
686 switch.)
687 .P
688 A variety of version of the database library exist.
689 .[
690 wolter unix incompat notes dbm
691 .]
692 Complicated autoconf code was needed to detect them correctly.
693 Further more, the configure switches
694 .Sw --with-ndbm=ARG
695 and
696 .Sw --with-ndbmheader=ARG
697 were added to help with difficult setups that would
698 not be detected automatically or correctly.
699 .P
700 By removing the suppress duplicates feature of
701 .Pn slocal ,
702 the dependency on
703 .I ndbm
704 vanished and 120 lines of complex autoconf code could be saved.
705 .Ci ecd6d6a20cb7a1507e3a20d6c4cb3a1cf14c6bbf
706 The change removed funtionality too, but that is minor to the
707 improvement by dropping the dependency and the complex autoconf code.
709 .U3 "mh-e Support
710 .P
711 The configure option
712 .Sw --disable-mhe
713 was removed when the mh-e support was reworked.
714 Mh-e is the Emacs front-end to MH.
715 It requires MH to provide minor additional functions.
716 The
717 .Sw --disable-mhe
718 configure option could switch these extensions off.
719 After removing the support for old versions of mh-e,
720 only the
721 .Sw -build
722 switches of
723 .Pn forw
724 and
725 .Pn repl
726 are left to be mh-e extensions.
727 They are now always built in because they add little code and complexity.
728 In consequence, the
729 .Sw --disable-mhe
730 configure option was removed
731 .Ci a7ce7b4a580d77b6c2c4d980812beb589aa4c643
732 Removing the option removed a second code setup that would have
733 needed to be tested.
734 This change was first done in nmh and thereafter merged into mmh.
735 .P
736 The interface changes in mmh require mh-e to be adjusted in order
737 to be able to use mmh as back-end.
738 This will require minor changes to mh-e, but removing the
739 .Sw -build
740 switches would require more rework.
742 .U3 "Masquerading
743 .P
744 The configure option
745 .Sw --enable-masquerade
746 could take up to three arguments:
747 `draft_from', `mmailid', and `username_extension'.
748 They activated different types of address masquerading.
749 All of them were implemented in the SMTP-speaking
750 .Pn post
751 command, which provided an MSA.
752 Address masquerading is an MTA's task and mmh does not cover
753 this field anymore.
754 Hence, true masquerading needs to be implemented in the external MTA.
755 .P
756 The
757 .I mmailid
758 masquerading type is the oldest one of the three and the only one
759 available in the original MH.
760 It provided a
761 .I username
762 to
763 .I fakeusername
764 mapping, based on the password file's GECOS field.
765 The man page
766 .Mp mh-tailor(5)
767 described the use case as being the following:
768 .QP
769 This is useful if you want the messages you send to always
770 appear to come from the name of an MTA alias rather than your
771 actual account name. For instance, many organizations set up
772 `First.Last' sendmail aliases for all users. If this is
773 the case, the GECOS field for each user should look like:
774 ``First [Middle] Last <First.Last>''
775 .P
776 As mmh sends outgoing mail via the local MTA only,
777 the best location to do such global rewrites is there.
778 Besides, the MTA is conceptionally the right location because it
779 does the reverse mapping for incoming mail (aliasing), too.
780 Further more, masquerading set up there is readily available for all
781 mail software on the system.
782 Hence, mmailid masquerading was removed.
783 .Ci 0836c8000ccb34b59410ef1c15b1b7feac70ce5f
784 .P
785 The
786 .I username_extension
787 masquerading type did not replace the username but would append a suffix,
788 specified by the
789 .Ev USERNAME_EXTENSION
790 environment variable, to it.
791 This provided support for the
792 .I user-extension
793 feature of qmail and the similar
794 .I "plussed user
795 processing of sendmail.
796 The decision to remove this username_extension masquerading was
797 motivated by the fact that
798 .Pn spost
799 hadn't supported it already.
800 .Ci 2abae0bfd0ad5bf898461e50aa4b466d641f23d9
801 Username extensions are possible in mmh, but less convenient to use.
802 .\" XXX format file %(getenv USERNAME_EXTENSION)
803 .P
804 The
805 .I draft_from
806 masquerading type instructed
807 .Pn post
808 to use the value of the
809 .Hd From
810 header field as SMTP envelope sender.
811 Sender addresses could be replaced completely.
812 .Ci b14ea6073f77b4359aaf3fddd0e105989db9
813 Mmh offers a kind of masquerading similar in effect, but
814 with technical differences.
815 As mmh does not transfer messages itself, the local MTA has final control
816 over the sender's address. Any masquerading mmh introduces may be reverted
817 by the MTA.
818 In times of pedantic spam checking, an MTA will take care to use
819 sensible envelope sender addresses to keep its own reputation up.
820 Nonetheless, the MUA can set the
821 .Hd From
822 header field and thereby propose
823 a sender address to the MTA.
824 The MTA may then decide to take that one or generate the canonical sender
825 address for use as envelope sender address.
826 .P
827 In mmh, the MTA will always extract the recipient and sender from the
828 message header (\c
829 .Pn sendmail 's
830 .Sw -t
831 switch).
832 The
833 .Hd From
834 header field of the draft may be set arbitrary by the user.
835 If it is missing, the canonical sender address will be generated by the MTA.
837 .U3 "Remaining Options
838 .P
839 Two configure options remain in mmh.
840 One is the locking method to use:
841 .Sw --with-locking=[dot|fcntl|flock|lockf] .
842 The idea of removing all methods except the portable dot locking
843 and having that one as the default is appealing, but this change
844 requires deeper technical investigation into the topic.
845 The other option,
846 .Sw --enable-debug ,
847 compiles the programs with debugging symbols and does not strip them.
848 This option is likely to stay.
853 .H2 "Removal of switches
854 .P
859 .H1 "Modernizing
862 .H2 "Removal of Code Relicts
863 .P
864 The code base of mmh originates from the late Seventies,
865 had been extensively
866 worked on in the mid Eighties, and had been partly reorganized and extended
867 in the Nineties. Relicts of all those times had gathered in the code base.
868 My goal was to remove any ancient code parts. One part of the task was
869 converting obsolete code constructs to standard constructs, the other part
870 was dropping obsolete functions.
871 .P
872 As I'm not even thirty years old and have no more than seven years of
873 Unix experience, I needed to learn about the history in retrospective.
874 Older people likely have used those ancient constructs themselves
875 and have suffered from their incompatibilities and have longed for
876 standardization. Unfortunately, I have only read that others had done so.
877 This put me in a much more difficult positions when working on the old
878 code. I needed to recherche what other would have known by heart from
879 experience. All my programming experience comes from a time past ANSI C
880 and past POSIX. Although I knew about the times before, I took the
881 current state implicitly for granted most of the time.
882 .P
883 Being aware of
884 these facts, I rather let people with more historic experience solve the
885 task of converting the ancient code constructs to standardized ones.
886 Luckily, Lyndon Nerenberg focused on this task at the nmh project.
887 He converted large parts of the code to POSIX constructs, removing
888 the conditionals compilation for now standardized features.
889 I'm thankful for this task being solved. I only pulled the changes into
890 mmh.
891 .P
892 The other task \(en dropping ancient functionality to remove old code \(en
893 I did myself, though. My position to strip mmh to the bare minimum of
894 frequently used features is much more revolutional than the nmh community
895 likes it. Without the need to justify my decisions, I was able to quickly
896 remove functionality I considered ancient.
897 The need to discuss my decisions with
898 peers likely would have slowed this process down. Of course, I researched
899 if a particular feature really should be dropped. Having not had any
900 contact to this feature within my computer life was a first indicator to
901 drop it, but I also asked others and searched the literature for modern
902 usage of the feature. If it appeared to be truly ancient, I dropped it.
903 The reason for dropping is always part of the commit message in the
904 version control system. Thus, it is easy for others to check their
905 view on the topic with mine and possibly to argue for reinclusion.
907 .U2 "MMDF maildrop support
908 .P
909 I did drop any support for the MMDF maildrop format. This type of format
910 is conceptionally similar to the mbox format, but uses four bytes with
911 value 1 (\fL^A^A^A^A\fP) as message delimiter,
912 instead of the string ``\fLFrom\ \fP''.
913 Due to the similarity and mbox being the de-facto standard maildrop
914 format on Unix, but also due to the larger influence of Sendmail than MMDF,
915 the MMDF maildrop format had vanished.
916 .P
917 The simplifications within the code were only moderate. Switches could
918 be removed from tools like
919 .L packf ,
920 which generate packed mailboxes. Only one packed mailbox format remained:
921 mbox.
922 The most important changes affect the equally named mail parsing routine in
923 .L sbr/m_getfld.c .
924 The direct MMDF code had been removed, but as now only one packed mailbox
925 format is left, code structure simplifications are likely possible.
926 The reason why they are still outstanding is the heavily optimized code
927 of
928 .Fu m_getfld() .
929 Changes beyond a small local scope \(en
930 which restructuring in its core is \(en cause a high risk of damaging
931 the intricate workings of the optimized code. This problem is know
932 to the developers of nmh, too. They also avoid touching this minefield
933 if possible.
935 .U2 "UUCP Bang Paths
936 .P
937 More questionably than the former topic is the removal of support for the
938 UUCP bang path address style. However, the user may translate the bang
939 paths on retrieval to Internet addresses and the other way on posting
940 messages. The former can be done my an MDA like procmail; the latter
941 by a sendmail wrapper. This would ensure that any address handling would
942 work as expected. However, it might just work well without any
943 such modifications, as mmh does not touch addresses much, in general.
944 But I can't ensure as I have never used an environment with bang paths.
945 Also, the behavior might break at any point in further development.
947 .U2 "Hardcopy terminal support
948 .P
949 More of a funny anecdote is the remaining of a check for printing to a
950 hardcopy terminal until Spring 2012, when I finally removed it.
951 I surely would be very happy to see such a terminal in action, maybe
952 actually being able to work on it, but I fear my chances are null.
953 .P
954 The check only prevented a pager to be placed between the outputting
955 program (\c
956 .Pn mhl )
957 and the terminal. This could have been ensured with
958 the
959 .Sw -nomoreproc
960 at the command line statically, too.
962 .U2 "Removed support for header fields
963 .P
964 The
965 .Hd Encrypted
966 header field had been introduced by RFC\^822, but already
967 marked legacy in RFC 2822. It was superseded by FIXME.
968 Mmh does no more support this header field.
969 .P
970 Native support for
971 .Hd Face
972 header fields had been removed, as well.
973 The feature is similar to the
974 .Hd X-Face
975 header field in its intent,
976 but takes a different approach to store the image.
977 Instead of encoding the image data directly into the header field,
978 the it contains the hostname and UDP port where the image
979 date could be retrieved.
980 Neither
981 .Hd X-Face
982 nor the here described
983 .Hd Face
984 system
985 \**
986 .FS
987 There is also a newer but different system, invented 2005,
988 using
989 .Hd Face
990 headers.
991 It is the successor of
992 .Hd X-Face
993 providing colored PNG images.
994 .FE
995 became well used in the large scale.
996 It's still possible to use a Face systems,
997 although mmh does not provide support for any of the different systems
998 anymore. It's fairly easy to write a small shell script to
999 extract the embedded or fetch the external Face data and display the image.
1000 Own
1001 .Hd Face
1002 header field can be added into the draft template files.
1003 .P
1004 .Hd Content-MD5
1005 header fields were introduced by RFC\^1864. They provide only
1006 a verification of data corruption during the transfer. By no means can
1007 they ensure verbatim end-to-end delivery of the contents. This is clearly
1008 stated in the RFC. The proper approach to provide verificationability
1009 of content in an end-to-end relationship is the use of digital cryptography
1010 (RFCs FIXME). On the other hand, transfer protocols should ensure the
1011 integrity of the transmission. In combinations these two approaches
1012 make the
1013 .Hd Content-MD5
1014 header field useless. In consequence, I removed
1015 the support for it. By this removal, MD5 computation is not needed
1016 anywhere in mmh. Hence, over 500 lines of code were removed by this one
1017 change. Even if the
1018 .Hd Content-MD5
1019 header field is useful sometimes,
1020 I value its usefulnes less than the improvement in maintainability, caused
1021 by the removal.
1023 .U2 "Prompter's Control Keys
1024 .P
1025 The program
1026 .Pn prompter
1027 queries the user to fill in a message form. When used by
1028 .Pn comp
1029 as:
1030 .VS
1031 comp -editor prompter
1032 VE
1033 the resulting behavior is similar to
1034 .Pn mailx .
1035 Apparently,
1036 .Pn prompter
1037 hadn't been touched lately. Otherwise it's hardly explainable why it
1038 still offered the switches
1039 .Sw -erase
1040 .Ar chr
1041 and
1042 .Sw -kill
1043 .Ar chr
1044 to name the characters for command line editing.
1045 The times when this had been necessary are long time gone.
1046 Today these things work out-of-the-box, and if not, are configured
1047 with the standard tool
1048 .Pn stty .
1050 .U2 "Vfork and Retry Loops
1051 .P
1052 MH creates many processes, which is a consequence of the tool chest approach.
1053 In earlier times
1054 .Fu fork()
1055 had been an expensive system call, as the process's whole image needed
1056 to be duplicated. One common case is replacing the image with
1057 .Fu exec()
1058 right after having forked the child process.
1059 To speed up this case, the
1060 .Fu vfork()
1061 system call was invented at Berkeley. It completely omits copying the
1062 image. If the image gets replaced right afterwards then unnecessary
1063 work is omited. On old systems this results in large speed ups.
1064 MH uses
1065 .Fu vfork()
1066 whenever possible.
1067 .P
1068 Memory management units that support copy-on-write semantics make
1069 .Fu fork()
1070 almost as fast as
1071 .Fu vfork()
1072 in the cases when they can be exchanged.
1073 With
1074 .Fu vfork()
1075 being more error-prone and hardly faster, it's preferable to simply
1076 use
1077 .Fu fork()
1078 instead.
1079 .P
1080 Related to the costs of
1081 .Fu fork()
1082 is the probability of its success.
1083 Today on modern systems, the system call will succeed almost always.
1084 In the Eighties on heavy loaded systems, as they were common at
1085 universities, this had been different. Thus, many of the
1086 .Fu fork()
1087 calls were wrapped into loops to retry to fork several times in
1088 short intervals, in case of previous failure.
1089 In mmh, the program aborts at once if the fork failed.
1090 The user can reexecute the command then. This is expected to be a
1091 very rare case on modern systems, especially personal ones, which are
1092 common today.
1095 .H2 "Attachments
1096 .P
1097 MIME
1100 .H2 "Digital Cryptography
1101 .P
1102 Signing and encryption.
1105 .H2 "Good Defaults
1106 .P
1107 foo
1112 .H1 "Code style
1113 .P
1114 foo
1117 .H2 "Standard Code
1118 .P
1119 POSIX
1122 .H2 "Separation
1124 .U2 "MH Directory Split
1125 .P
1126 In MH and nmh, a personal setup had consisted of two parts:
1127 The MH profile, named
1128 .Fn \&.mh_profile
1129 and being located directly in the user's home directory.
1130 And the MH directory, where all his mail messages and also his personal
1131 forms, scan formats, other configuration files are stored. The location
1132 of this directory could be user-chosen. The default was to name it
1133 .Fn Mail
1134 and have it directly in the home directory.
1135 .P
1136 I've never liked the data storage and the configuration to be intermixed.
1137 They are different kinds of data. One part, are the messages,
1138 which are the data to operate on. The other part, are the personal
1139 configuration files, which are able to change the behavior of the operations.
1140 The actual operations are defined in the profile, however.
1141 .P
1142 When storing data, one should try to group data by its type.
1143 There's sense in the Unix file system hierarchy, where configuration
1144 file are stored separate (\c
1145 .Fn /etc )
1146 to the programs (\c
1147 .Fn /bin
1148 and
1149 .Fn /usr/bin )
1150 to their sources (\c
1151 .Fn /usr/src ).
1152 Such separation eases the backup management, for instance.
1153 .P
1154 In mmh, I've reorganized the file locations.
1155 Still there are two places:
1156 There's the mail storage directory, which, like in MH, contains all the
1157 messages, but, unlike in MH, nothing else.
1158 Its location still is user-chosen, with the default name
1159 .Fn Mail ,
1160 in the user's home directory. This is much similar to the case in nmh.
1161 The configuration files, however, are grouped together in the new directory
1162 .Fn \&.mmh
1163 in the user's home directory.
1164 The user's profile now is a file, named
1165 .Fn profile ,
1166 in this mmh directory.
1167 Consistently, the context file and all the personal forms, scan formats,
1168 and the like, are also there.
1169 .P
1170 The naming changed with the relocation.
1171 The directory where everything, except the profile, had been stored (\c
1172 .Fn $HOME/Mail ),
1173 used to be called \fIMH directory\fP. Now, this directory is called the
1174 user's \fImail storage\fP. The name \fImmh directory\fP is now given to
1175 the new directory
1176 (\c
1177 .Fn $HOME/.mmh ),
1178 containing all the personal configuration files.
1179 .P
1180 The separation of the files by type of content is logical and convenient.
1181 There are no functional differences as any possible setup known to me
1182 can be implemented with both approaches, although likely a bit easier
1183 with the new approach. The main goal of the change had been to provide
1184 sensible storage locations for any type of personal mmh file.
1185 .P
1186 In order for one user to have multiple MH setups, he can use the
1187 environment variable
1188 .Ev MH
1189 the point to a different profile file.
1190 The MH directory (mail storage plus personal configuration files) is
1191 defined by the
1192 .Pe Path
1193 profile entry.
1194 The context file could be defined by the
1195 .Pe context
1196 profile entry or by the
1197 .Ev MHCONTEXT
1198 environment variable.
1199 The latter is useful to have a distinct context (e.g. current folders)
1200 in each terminal window, for instance.
1201 In mmh, there are three environment variables now.
1202 .Ev MMH
1203 may be used to change the location of the mmh directory.
1204 .Ev MMHP
1205 and
1206 .Ev MMHC
1207 change the profile and context files, respectively.
1208 Besides providing a more consistent feel (which simply is the result
1209 of being designed anew), the set of personal configuration files can
1210 be chosen independently from the profile (including mail storage location)
1211 and context, now. Being it relevant for practical use or not, it
1212 de-facto is an improvement. However, the main achievement is the
1213 split between mail storage and personal configuration files.
1216 .H2 "Modularization
1217 .P
1218 whatnowproc
1219 .P
1220 The \fIMH library\fP
1221 .Fn libmh.a
1222 collects a bunch of standard functions that many of the MH tools need,
1223 like reading the profile or context files.
1224 This doesn't hurt the separation.
1227 .H2 "Style
1228 .P
1229 Code layout, goto, ...
1234 .H1 "Concept Exploitation/Homogeniety
1237 .H2 "Draft Folder
1238 .P
1239 Historically, MH provided exactly one draft message, named
1240 .Fn draft
1241 and
1242 being located in the MH directory. When starting to compose another message
1243 before the former one was sent, the user had been questioned whether to use,
1244 refile or replace the old draft. Working on multiple drafts at the same time
1245 was impossible. One could only work on them in alteration by refiling the
1246 previous one to some directory and fetching some other one for reediting.
1247 This manual draft management needed to be done each time the user wanted
1248 to switch between editing one draft to editing another.
1249 .P
1250 To allow true parallel editing of drafts, in a straight forward way, the
1251 draft folder facility exists. It had been introduced already in July 1984
1252 by Marshall T. Rose. The facility was deactivated by default.
1253 Even in nmh, the draft folder facility remained deactivated by default.
1254 At least, Richard Coleman added the man page
1255 .Mp mh-draft(5)
1256 to document
1257 the feature well.
1258 .P
1259 The only advantage of not using the draft folder facility is the static
1260 name of the draft file. This could be an issue for MH frontends like mh-e.
1261 But as they likely want to provide working on multiple drafts in parallel,
1262 the issue is only concerning compatibility. The aim of nmh to stay compatible
1263 prevented the default activation of the draft folder facility.
1264 .P
1265 On the other hand, a draft folder is the much more natural concept than
1266 a draft message. MH's mail storage consists of folders and messages,
1267 the messages named with ascending numbers. A draft message breaks with this
1268 concept by introducing a message in a file named
1269 .Fn draft .
1270 This draft
1271 message is special. It can not be simply listed with the available tools,
1272 but instead requires special switches. I.e. corner-cases were
1273 introduced. A draft folder, in contrast, does not introduce such
1274 corner-cases. The available tools can operate on the messages within that
1275 folder like on any messages within any mail folders. The only difference
1276 is the fact that the default folder for
1277 .Pn send
1278 is the draft folder,
1279 instead of the current folder, like for all other tools.
1280 .P
1281 The trivial part of the change was activating the draft folder facility
1282 by default and setting a default name for this folder. Obviously, I chose
1283 the name
1284 .Fn +drafts .
1285 This made the
1286 .Sw -draftfolder
1287 and
1288 .Sw -draftmessage
1289 switches useless, and I could remove them.
1290 The more difficult but also the part that showed the real improvement,
1291 was updating the tools to the new concept.
1292 .Sw -draft
1293 switches could
1294 be dropped, as operating on a draft message became indistinguishable to
1295 operating on any other message for the tools.
1296 .Pn comp
1297 still has its
1298 .Sw -use
1299 switch for switching between its two modes: (1) Compose a new
1300 draft, possibly by taking some existing message as a form. (2) Modify
1301 an existing draft. In either case, the behavior of
1302 .Pn comp is
1303 deterministic. There is no more need to query the user. I consider this
1304 a major improvement. By making
1305 .Pn send
1306 simply operate on the current
1307 message in the draft folder by default, with message and folder both
1308 overridable by specifying them on the command line, it is now possible
1309 to send a draft anywhere within the storage by simply specifying its folder
1310 and name.
1311 .P
1312 All theses changes converted special cases to regular cases, thus
1313 simplifying the tools and increasing the flexibility.
1316 .H2 "Trash Folder
1317 .P
1318 Similar to the situation for drafts is the situation for removed messages.
1319 Historically, a message was deleted by renaming. A specific
1320 \fIbackup prefix\fP, often comma (\c
1321 .Fn , )
1322 or hash (\c
1323 .Fn # ),
1324 being prepended to the file name. Thus, MH wouldn't recognize the file
1325 as a message anymore, as only files whose name consists of digits only
1326 are treated as messages. The removed messages remained as files in the
1327 same directory and needed some maintenance job to truly delete them after
1328 some grace time. Usually, by running a command similar to
1329 .VS
1330 find /home/user/Mail -ctime +7 -name ',*' | xargs rm
1331 VE
1332 in a cron job. Within the grace time interval
1333 the original message could be restored by stripping the
1334 the backup prefix from the file name. If however, the last message of
1335 a folder is been removed \(en say message
1336 .Fn 6
1337 becomes file
1338 .Fn ,6
1339 \(en and a new message enters the same folder, thus the same
1340 numbered being given again \(en in our case
1341 .Fn 6
1342 \(en, if that one
1343 is removed too, then the backup of the former message gets overwritten.
1344 Thus, the ability to restore removed messages does not only depend on
1345 the ``sweeping cron job'' but also on the removing of further messages.
1346 This is undesirable, because the real mechanism is hidden from the user
1347 and the consequences of further removals are not always obvious.
1348 Further more, the backup files are scattered within the whole mail
1349 storage, instead of being collected at one place.
1350 .P
1351 To improve the situation, the profile entry
1352 .Pe rmmproc
1353 (previously named
1354 .Pe Delete-Prog )
1355 was introduced, very early.
1356 It could be set to any command, which would care for the mail removal
1357 instead of taking the default action, described above.
1358 Refiling the to-be-removed files to some garbage folder was a common
1359 example. Nmh's man page
1360 .Mp rmm(1)
1361 proposes
1362 .Cl "refile +d
1363 to move messages to the garbage folder and
1364 .Cl "rm `mhpath +d all`
1365 the empty the garbage folder.
1366 Managing the message removal this way is a sane approach. It keeps
1367 the removed messages in one place, makes it easy to remove the backup
1368 files, and, most important, enables the user to use the tools of MH
1369 itself to operate on the removed messages. One can
1370 .Pn scan
1371 them,
1372 .Pn show
1373 them, and restore them with
1374 .Pn refile .
1375 There's no more
1376 need to use
1377 .Pn mhpath
1378 to switch over from MH tools to Unix tools \(en MH can do it all itself.
1379 .P
1380 This approach matches perfect with the concepts of MH, thus making
1381 it powerful. Hence, I made it the default. And even more, I also
1382 removed the old backup prefix approach, as it is clearly less powerful.
1383 Keeping unused alternative in the code is a bad choice as they likely
1384 gather bugs, by not being constantly tested. Also, the increased code
1385 size and more conditions crease the maintenance costs. By strictly
1386 converting to the trash folder approach, I simplified the code base.
1387 .Pn rmm
1388 calls
1389 .Pn refile
1390 internally to move the to-be-removed
1391 message to the trash folder (\c
1392 .Fn +trash
1393 by default). Messages
1394 there can be operated on like on any other message in the storage.
1395 The sweep clean, one can use
1396 .Cl "rmm -unlink +trash a" ,
1397 where the
1398 .Sw -unlink
1399 switch causes the files to be truly unliked instead
1400 of moved to the trash folder.
1403 .H2 "Path Notations
1404 .P
1405 foo
1408 .H2 "MIME Integration
1409 .P
1410 user-visible access to whole messages and MIME parts are inherently
1411 different
1414 .H2 "Of One Cast
1415 .P