docs/master
view ch03.roff @ 83:99ab58dc891f
New macro: .VF to read verbatim text from file. Outsourced the MH session.
| author | markus schnalke <meillo@marmaro.de> |
|---|---|
| date | Wed, 06 Jun 2012 18:11:35 +0200 |
| parents | 64f177ca2af1 |
| children | c35ec2b7de5d |
line source
1 .H0 "Discussion
2 .P
3 This main chapter discusses the practical work done in the mmh project.
4 It is structured along the goals to achieve. The concrete work done
5 is described in the examples of how the general goals were achieved.
10 .H1 "Stream-lining
12 .P
13 MH had been considered an all-in-one system for mail handling.
14 The community around nmh has a similar understanding.
15 In fundamental difference, should be a MUA only.
16 I believe that all-in-one mail systems are obsolete.
17 There are excellent specialized MTAs, like Postfix;
18 there are specialized MDAs, like Procmail; there are specialized
19 MRAs, like Fetchmail. I believe it's best to use them instead of
20 providing the same function ourselves. Doing something well, requires to
21 focus on a small set of aspects. The more
22 it is possible to focus, the better the result in this particular
23 area will be. Usually, the limiting resource in Free Software
24 community development is man power.
25 If the development power is even spread over a large
26 development area, it becomes more difficult to
27 compete with the specialists in the various fields. This is even
28 increased, given the small community \(en developers and users \(en
29 that MH-based mail systems have. In consequence, I believe that the
30 available resources should be focused to the point where MH is
31 most unique. This is clearly the MUA part.
32 .P
33 The goal for mmh was to remove peripheral parts and stream-line
34 it for the MUA task.
37 .H2 "Removal of Mail Transfer Facilities
38 .P
39 In contrast to nmh, which also provides mail submission and mail retrieval
40 facilities, mmh is a MUA only.
41 This general difference in the view on the character of nmh
42 initiated the development of mmh.
43 Removing the mail transfer facilities had been the first work task
44 in the mmh project.
45 .P
46 The MSA is called \fIMessage Transfer Service\fP (MTS) in nmh.
47 The facility established network connections and spoke SMTP to submit
48 messages for relay to the outside world.
49 This part was implemented by the
50 .Pn post
51 command.
52 The changes in emailing
53 demanded changes in this part of nmh in the last years.
54 Encryption and authetication for network connections
55 needed to be supported, hence TLS and SASL were introduced
56 into nmh. This added complexity to the nmh without improving it in
57 its core functions. Also, keeping up with recent developments in
58 this field requires development power and specialists.
59 For mmh this whole facility was cut off.
60 .Ci f6aa95b724fd8c791164abe7ee5468bf5c34f226
61 .Ci fecd5d34f65597a4dfa16aeabea7d74b191532c3
62 .Ci 156d35f6425bea4c1ed3c4c79783dc613379c65b
63 Instead, mmh depends on an external MTA.
64 The only outgoing interface available to mmh is the
65 .Pn sendmail
66 command.
67 Almost any MTA provides a
68 .Pn sendmail
69 command.
70 If not, any program can be substituted if it reads the
71 message from the standard input, extracts the recipient addresses
72 from the message header and does not conflict
73 with sendmail-specific command line options.
74 .P
75 To retrieve mail, the
76 .Pn inc
77 command established network connections
78 and spoke POP3 to retrieve mail from remote servers.
79 As with mail submission, the network connections required encryption and
80 authentication, thus TLS and SASL was added.
81 As POP3 becomes more and more superseded by IMAP, support for message
82 retrieval through IMAP will become necessary to be added soon, too.
83 Mmh has dropped the support for retrieving mail from remote locations.
84 .Ci ab7b48411962d26439f92f35ed084d3d6275459c
85 Instead, it depends on an external tool to cover this task.
86 There exist two paths for messages to enter mmh's mail storage:
87 They can be incorporate with
88 .Pn inc
89 from the system maildrop, or
90 .Pn rcvstore
91 reads them from the standard input.
92 .P
93 With the removal of the MSA and MRA, mmh converted from an all-in-one
94 mail system to being only a MUA.
95 Following the Unix philosophy, it focuses on one job and
96 tries to do that one well.
97 Not only the programs follow that tenet but also the project itself does so.
98 Now, of course, mmh depends on third-party software.
99 An external MTA/MSA is required to transfer mail to the outside world;
100 an external MRA is required to retrieve mail from remote machines.
101 There exist excellent implementations of such software,
102 which do this specific task likely better than the internal
103 versions had done it. Also, the best suiting programs can be freely chosen.
104 .P
105 As it had already been possible to use an external MSA or MRA,
106 why not keep the internal version for convenience?
107 The question whether there is sense in having a fall-back pager in all
108 the command line tools, for the cases when
109 .Pn more
110 or
111 .Pn less
112 aren't available, appears to be ridiculous.
113 Now, an MSA or MRA is clearly more complex than a text pager,
114 and not necessarily available but still the concept holds:
115 design the system orthogonally.
116 If it is conceptionally more elegant to have the MTA to be a separate tool
117 \(en as the RFCs propose this split, this is likely the case \(en
118 then separate.
119 .P
120 Further more, if programs become complex, they should be split;
121 and if projects become complex, they should be split, too.
122 Essential complexity is defined by the problem.
123 Decades ago, emailing had been small and simple.
124 (\c
125 .Pn /bin/mail
126 had once covered anything there was to email and still had been small
127 and simple.)
128 Then the essential complexity of email increased.
129 Email tools needed to react.
130 In nmh, for instance, the POP server, which the original MH had included,
131 was removed.
132 Now is the time to go one step further and remove the MSA and MRA.
133 Not only does it decrease the code amount of the project,
134 but more important, it removes the whole field of message transfer
135 with all its implications for the project.
136 It removes the need to adjust to any changes concerning network transfer.
137 This independence is received by depending on an external program
138 that covers the field.
139 Today, this is a reasonable exchange.
140 .P
141 To add some kind of function, there's always the choice
142 among implementing the function in the project directly,
143 depending on a library that provides the function, or depending on
144 a program that provides the function.
145 Whereas adding the function directly to the project increases the
146 code size most and requires most maintenance and development work,
147 it makes the project most independent.
148 Using libraries or external programs require less
149 maintenance work.
150 Programs have the smallest interfaces, providing the most separation
151 but possibly limiting the information exchange.
152 External libraries are stronger connected than external programs but
153 allow better information exchange.
154 Adding more code to a project does always increase maintenance work.
155 Implementing complex functions directly in the project will add
156 a lot of code. This should be avoided if possible.
157 Hence, the dependencies only change in kind, not in their existence.
158 In mmh, library dependencies on
159 .Pn libsasl2
160 and
161 .Pn libcrypto /\c
162 .Pn libssl
163 were treated against program dependencies on an MSA and an MRA.
164 Besides program dependencies providing the stronger separation
165 and being more flexible, they also allowed
166 over 6\|000 lines of code to be removed from mmh.
167 This made mmh's code base about 12\|% smaller.
168 Reducing the projects code size by such an amount without actually
169 losing function is a convincing argument.
170 Actually, as external MSAs and MRAs are likely better
171 than the project's internal version, the user even gains functionality.
172 .P
173 Users of MH should not have problems to set up an external MSA and MRA.
174 Also, the popular MSAs and MRAs have large communities and a lot
175 of documentation available.
176 Choices for MSAs range from the full-featured
177 .I Postfix
178 over mid-size solutions like
179 .I masqmail
180 and
181 .I dma
182 to small forwarders like
183 .I ssmtp
184 and
185 .I nullmailer .
186 Choices for MRAs include
187 .I fetchmail ,
188 .I getmail ,
189 .I mpop
190 and
191 .I fdm .
194 .H2 "Removal of non-MUA Tools
195 .P
196 Some MH tools were removed because they didn't add to the MUA's job.
197 It is a design goal of mmh to remove the parts that are rarely used.
198 The project shall become more stream-lined and focused.
199 Rarely used and loosely related tools distract from the lean appearance.
200 They require maintenance work without adding to the core task.
201 In mmh the following tools are not available anymore:
202 .BU
203 .Pn conflict
204 was removed because it is a mail system maintenance tool.
205 .Ci 8b235097cbd11d728c07b966cf131aa7133ce5a9
206 Besides, it even checks
207 .Fn /etc/passwd
208 and
209 .Fn /etc/group
210 for consistency, which has nothing at all to do with emailing.
211 The tool might be useful, but it should not be shipped with mmh.
212 .\" XXX historic reasons?
213 .BU
214 .Pn rcvtty
215 was removed because its usecase of writing to the user's terminal
216 on receiving of mail is obsolete.
217 .Ci 14767c94b3827be7c867196467ed7aea5f6f49b0
218 If users like to be
219 informed of new mail, the shell's
220 .Ev MAILPATH
221 variable or graphical notifications are more appealing.
222 Writing directly to a terminals is hardly ever wanted today.
223 If though one wants to have it this way, the standard tool
224 .Pn write
225 can be used in a way similar to:
226 .VS
227 scan -file - | write `id -un`
228 VE
229 .BU
230 .Pn viamail
231 was removed when the new attachment system was activated, because
232 .Pn forw
233 could then cover the task itself.
234 .Ci eda72d6a7a7c20ff123043fb7f19c509ea01f932
235 The program
236 .Pn sendfiles
237 was rewritten as a shell script wrapper around
238 .Pn forw .
239 .Ci 0e82199cf3c991a173e0ac8aa776efdb3ded61e6
240 .BU
241 .Pn msgchk
242 was removed, because it lost its use case when POP support was removed.
243 .Ci bb9360ead7eb7a3fedcce2eeedfc660014e41dbe
244 A call to
245 .Pn msgchk
246 provided hardly more information than
247 .VS
248 ls -l /var/mail/meillo
249 VE
250 though it distinguished between old and new mail.
251 This detail information and can be retrieved with
252 .Pn stat (1),
253 too.
254 A very small shell script could be written to output the information
255 in a similar way, if truly necessary.
256 As mmh's
257 .Pn inc
258 only incorporates mail from the user's local maildrop
259 and thus no data transfers over slow networks are involved,
260 there's hardly any need to check for new mail before incorporating it.
261 .BU
262 .Pn msh
263 was removed because the tool was in conflict with the philosophy of MH.
264 .Ci 916690191222433a6923a4be54b0d8f6ac01bd02
265 It provided an interactive shell to access the features of MH,
266 but it wasn't just a shell, tailored to the needs of mail handling.
267 Instead it was one large program that had several MH tools built in.
268 This conflicts with the major feature of MH of being a tool chest.
269 .Pn msh 's
270 main use case had been accessing Bulletin Boards, which have seized to
271 be popular.
272 .P
273 Removing
274 .Pn msh ,
275 together with the truly archaic code relicts
276 .Pn vmh
277 and
278 .Pn wmh ,
279 saved more than 7\|000 lines of C code \(en
280 about 15\|% of the project's original source code amount.
281 .P
282 Having less code (with equal readability, of course)
283 for the same functionality is an advantage.
284 Less code means less bugs and less maintenance work.
285 As
286 .Pn rcvtty
287 and
288 .Pn msgchk
289 are rarely used and can be implemented in different ways,
290 then why should one keep them?
291 Removing them stream-lines mmh.
292 .Pn viamail 's
293 use case is now partly obsolete and partly covered by
294 .Pn forw ,
295 hence there's no reason to still maintain it.
296 .Pn conflict
297 is not related to the mail client, and
298 .Pn msh
299 conflicts with the basic concept of MH.
300 Theses two tools might still be useful, but they should not be part of mmh.
301 .P
302 Finally, there's
303 .Pn slocal .
304 .Pn slocal
305 is an MDA and thus not directly MUA-related.
306 It should be removed, because including it is a violation
307 of the idea that mmh is a MUA only.
308 It should become a separate project.
309 But
310 .Pn slocal
311 provides rule-based processing of messages, like filing them into
312 different folders, which is otherwise not available in mmh.
313 Further more,
314 .Pn slocal
315 does neither pull dependencies nor a whole new technical area
316 into the project.
317 (See section XXX for the removing of the ndbm dependency.)
318 Still,
319 .Pn slocal
320 accounts for about 1\|000 lines of code that need to be maintained.
321 As
322 .Pn slocal
323 is almost self-standing, it should be split off into a separate project.
324 This would cut the strong connection between the MUA mmh and the MDA
325 .Pn slocal .
326 The MDA would become an alternative to
327 .I procmail ,
328 as it would no longer be the need to install a complete MH system, too.
329 Likewise, mmh users could decide to use
330 .I procmail
331 without having a second, unused MDA (\c
332 .Pn slocal )
333 installed.
334 That's conceptionally the best solution.
335 Yet,
336 .Pn slocal
337 was not removed.
338 I feel unsure with the removal.
339 Hence, the decision over
340 .Pn slocal
341 is deferred.
342 This does not hurt because
343 .Pn slocal
344 is completely unrelated to the rest of mmh.
347 .H2 "\fLshow\fP and \fPmhshow\fP
348 .P
349 Since the very beginning \(en already in the first concept paper \(en
350 .Pn show
351 had been MH's message display program.
352 .Pn show
353 mapped message numbers and sequences to files and invoked
354 .Pn mhl
355 to have the files formated.
356 For MIME, this approach wasn't sufficient anymore.
357 MIME messages can consist of multiple parts, some of which aren't
358 directly displayable, and text content might be encoded in
359 foreign charsets.
360 .Pn show 's
361 understanding of messages and
362 .Pn mhl 's
363 limited display facilities couldn't cope with the task any longer.
364 .P
365 Instead of extending these tools, additional tools were written from scratch
366 and then added to the MH tool chest. Doing so is encouraged by the
367 tool chest approach. The new tools could be added without interfering
368 with the existing ones.
369 Modular design is a great advantage for extending a system.
370 First, the new MIME features were added in form of the single program
371 .Pn mhn .
372 The command
373 .Cl "mhn -show 42
374 would show the MIME message numbered 42.
375 With the 1.0 release of nmh in February 1999, Richard Coleman finished
376 the split of
377 .Pn mhn
378 into a set of specialized programs, which together covered the
379 multiple aspects of MIME. One of these resulting tools was
380 .Pn mhshow ,
381 which replaced the
382 .Cl "mhn -show
383 call.
384 It was capable to display a MIME message appropriately.
385 .P
386 From then on, two message display tools were part of nmh:
387 .Pn show
388 and
389 .Pn mhshow .
390 Because the user should not need to invoke the right tool
391 whether the message uses MIME or not,
392 .Pn show
393 was extended to automatically hand the job over to
394 .Pn mhshow
395 if displaying the message would be beyond
396 .Pn show 's
397 abilities.
398 In consequence, the user would invoke
399 .Pn show
400 (possibly through
401 .Pn next
402 or
403 .Pn prev )
404 and get the message printed with either
405 .Pn show
406 or
407 .Pn mhshow ,
408 whatever was more appropriate.
409 (There was also a switch for
410 .Pn show
411 to never invoke
412 .Pn mhshow .
413 .Pn show
414 was able to display MIME messages if they contained only a single text
415 part.)
416 .P
417 Having two similar tools for essentially the same task is redundant.
418 The development of both programs needed to be in sync,
419 to ensure that the programs behaved in a similar way,
420 because they were used like a single tool.
421 Different behavior would have surprised the user.
422 .P
423 Today, non-MIME messages are rather seen to be a special case of
424 MIME messages, than MIME messages are seen to be an extension to
425 original email.
426 As
427 .Pn mhshow
428 had already be able to display non-MIME messages, it was natural
429 to drop
430 .Pn show
431 in favor of using
432 .Pn mhshow
433 exclusively.
434 This decision followed the idea of orthogonal design.
435 For convenience,
436 .Pn mhshow
437 was then renamed to
438 .Pn show .
439 .Ci 4c1efddfd499300c7e74263e57d8aa137e84c853
440 .P
441 To prepare for this transition,
442 .Pn mhshow
443 was reworked to behave more like
444 .Pn show
445 first.
446 (Section XXX describes this rework from a different perspective.)
447 Once the tools behaved similar, the replacing became a natural decision.
448 In mmh,
449 .Pn show
450 is the one single message display program again, but it handles
451 MIME messages as well as non-MIME messages.
452 Now, there's only one program to maintain, and users don't need to deal
453 with the existance of two display programs.
454 .P
455 There's one reason why removing the old
456 .Pn show
457 hurts: It had been such a simple program.
458 Its lean elegance is missing to
459 .Pn mhshow ,
460 i.e. the new
461 .Pn show .
462 But there is no chance, because supporting MIME causes essentially
463 higher complexity.
466 .H2 "Removal of Configure Options
467 .P
468 Customization is a double-edged sword.
469 It allows better suiting setups, but not for free.
470 There is the cost of code complexity to be able to customize.
471 There is the cost of less tested setups, because there are
472 more possible setups and especially corner-cases.
473 And, there is the cost of choice itself.
474 The code complexity directly affects the developers.
475 Less tested code affects both, users and developers.
476 The problem of choice affects the users, for once by having to
477 choose, but also by complexer interfaces that require more documentation.
478 Whenever options add little advantages, they should be considered for
479 removal.
480 I have reduced the number of project-specific configure options from
481 fifteen to three.
483 .U3 "Mail Transfer Facilities
484 .P
485 With the removal of the mail transfer facilities five option vanished:
486 .IP \f(CW--with-mts=[smtp|sendmail]\fP
487 Specified the default mail transport service, which now is sendmail always.
488 .IP \f(CW--with-smtpservers=[server1...]\fP
489 Specified the default SMTP servers for the smtp mail transfer service.
490 .Ci 128545e06224233b7e91fc4c83f8830252fe16c9
491 .IP \f(CW--with-cyrus-sasl\fP
492 Enabled SASL support for mail transfer.
493 .IP \f(CW--with-tls\fP
494 Enabled TLS support for mail transfer.
495 .IP \f(CW--enable-pop\fP
496 Enabled the message retrieval facility.
498 .U3 "Backup Prefix
499 .P
500 The backup prefix is the string that was prepended to message
501 filenames to tag them as deleted.
502 By default it had been the comma character `\f(CW,\fP'.
503 In July 2000, Kimmo Suominen introduced
504 the configure option
505 .Sw --with-hash-backup
506 to change the default to the hash symbol `\f(CW#\fP'.
507 The choice was probably personal preference, because first, the
508 option was named
509 .Sw --with-backup-prefix.
510 and had the prefix symbol as argument.
511 Because giving the hash symbol as argument caused to many problems
512 for configure,
513 the option was limited to use the hash symbol as the default prefix.
514 This makes me believe, that the choice for the hash was personal preference.
515 Being it related or not, words that start with the hash symbol
516 introduce a comment in the Unix shell.
517 Thus, the command line
518 .Cl "rm #13 #15
519 calls
520 .Pn rm
521 without arguments because the first hash symbol starts the comment
522 that reaches until the end of the line.
523 To delete the backup files,
524 .Cl "rm ./#13 ./#15"
525 needs to be used.
526 Using the hash as backup prefix can be seen as a precaution agains
527 data loss.
528 .P
529 I removed the configure option but added the profile entry
530 .Pe backup-prefix ,
531 which allows to specify an arbitrary string as backup prefix.
532 .Ci 6c40d481d661d532dd527eaf34cebb6d3f8ed086
533 Profile entries are the common method to change mmh's behavior.
534 This change did not remove the choice but moved it to a location where
535 it suited better.
536 .P
537 Eventually, however, the new trash folder concept
538 .Cf "Sec. XXX
539 obsoleted the concept of the backup prefix completely.
540 .Ci 8edc5aaf86f9f77124664f6801bc6c6cdf258173
541 (Well, there still are corner-cases to remove until the backup
542 prefix can be layed to rest, eventually.)
543 .\" FIXME: Do this work in the code!
545 .U3 "Editor and Pager
546 .P
547 The two configure options
548 .CW --with-editor=EDITOR
549 .CW --with-pager=PAGER
550 were used to specify the default editor and pager at configure time.
551 Doing so at configure time made sense in the Eighties,
552 when the set of available editors and pagers varied much across
553 different systems.
554 Today, the situation is more homegeneic.
555 The programs
556 .Pn vi
557 and
558 .Pn more
559 can be expected to be available on every Unix system,
560 as they are specified by POSIX since two decades.
561 (The specifications for
562 .Pn vi
563 and
564 .Pn more
565 appeared in
566 .[
567 posix 1987
568 .]
569 and,
570 .[
571 posix 1992
572 .]
573 respectively.)
574 As a first step, these two tools were hard-coded as defaults.
575 .Ci 5d43a99db70c12a673028c7758c20cbe3e13ef5f
576 Not changed were the
577 .Pe editor
578 and
579 .Pe moreproc
580 profile entries, which allowed the user to override the system defaults.
581 Later, the concept was reworked to respect the standard environment
582 variables
583 .Ev VISUAL
584 and
585 .Ev PAGER
586 if they are set.
587 Today, mmh determines the editor to use in the following order,
588 taking the first available and non-empty item:
589 .IP (1)
590 Environment variable
591 .Ev MMHEDITOR
592 .IP (2)
593 Profile entry
594 .Pe Editor
595 .IP (3)
596 Environment variable
597 .Ev VISUAL
598 .IP (4)
599 Environment variable
600 .Ev EDITOR
601 .IP (5)
602 Command
603 .Pn vi .
604 .P
605 .Ci f85f4b7ae62e3d05a945dcd46ead51f0a2a89a9b
606 .P
607 The pager to use is deteminded in a similar order,
608 also taking the first available and non-empty item:
609 .IP (1)
610 Environment variable
611 .Ev MMHPAGER
612 .IP (2)
613 Profile entry
614 .Pe Pager
615 (replaces
616 .Pe moreproc )
617 .IP (3)
618 Environment variable
619 .Ev PAGER
620 .IP (4)
621 Command
622 .Pn more .
623 .P
624 .Ci 0c4214ea2aec6497d0d67b436bbee9bc1d225f1e
625 .P
626 By respecting the
627 .Ev VISUAL /\c
628 .Ev EDITOR
629 and
630 .Ev PAGER
631 environment variables,
632 the new behavior confirms better to the common style on Unix systems.
633 Additionally, the new approach is more uniform and clearer to users.
635 .U3 "Locale
636 .P
637 The configure option
638 .Sw --disable-locale
639 was removed because today there's hardly any need to disable locale
640 support.
641 .Ci ccf4f175ef4c4e7522f9510a4a1149c15d810dd9
643 .U3 "ndbm
644 .P
645 .Pn slocal
646 used to depend on
647 .I ndbm ,
648 a database library.
649 The database is used to store the `\fLMessage-ID\fP's of all
650 messages delivered.
651 This enables
652 .Pn slocal
653 to suppress delivering the same message to the same user twice.
654 (This features was enabled by the
655 .Sw -suppressdup
656 switch.)
657 .P
658 A variety of version of the database library exist.
659 .[
660 wolter unix incompat notes dbm
661 .]
662 Complicated autoconf code was needed to detect them correctly.
663 Further more, the configure switches
664 .Sw --with-ndbm=ARG
665 and
666 .Sw --with-ndbmheader=ARG
667 were added to help with difficult setups that would
668 not be detected automatically or correctly.
669 .P
670 By removing the suppress duplicates feature of
671 .Pn slocal ,
672 the dependency on
673 .I ndbm
674 vanished and 120 lines of complex autoconf code could be saved.
675 .Ci ecd6d6a20cb7a1507e3a20d6c4cb3a1cf14c6bbf
676 The change removed funtionality too, but that is minor to the
677 improvement by dropping the dependency and the complex autoconf code.
679 .U3 "mh-e Support
680 .P
681 The configure option
682 .Sw --disable-mhe
683 was removed when the mh-e support was reworked.
684 Mh-e is the Emacs front-end to MH.
685 It requires MH to provide minor additional functions.
686 The
687 .Sw --disable-mhe
688 configure option could switch these extensions off.
689 After removing the support for old versions of mh-e,
690 only the
691 .Sw -build
692 switches of
693 .Pn forw
694 and
695 .Pn repl
696 are left to be mh-e extensions.
697 They are now always built in because they add little code and complexity.
698 In consequence, the
699 .Sw --disable-mhe
700 configure option was removed
701 .Ci a7ce7b4a580d77b6c2c4d980812beb589aa4c643
702 Removing the option removed a second code setup that would have
703 needed to be tested.
704 This change was first done in nmh and thereafter merged into mmh.
705 .P
706 The interface changes in mmh require mh-e to be adjusted in order
707 to be able to use mmh as back-end.
708 This will require minor changes to mh-e, but removing the
709 .Sw -build
710 switches would require more rework.
712 .U3 "Masquerading
713 .P
714 The configure option
715 .Sw --enable-masquerade
716 could take up to three arguments:
717 `draft_from', `mmailid', and `username_extension'.
718 They activated different types of address masquerading.
719 All of them were implemented in the SMTP-speaking
720 .Pn post
721 command, which provided an MSA.
722 Address masquerading is an MTA's task and mmh does not cover
723 this field anymore.
724 Hence, true masquerading needs to be implemented in the external MTA.
725 .P
726 The
727 .I mmailid
728 masquerading type is the oldest one of the three and the only one
729 available in the original MH.
730 It provided a
731 .I username
732 to
733 .I fakeusername
734 mapping, based on the password file's GECOS field.
735 The man page
736 .Mp mh-tailor(5)
737 described the use case as being the following:
738 .QP
739 This is useful if you want the messages you send to always
740 appear to come from the name of an MTA alias rather than your
741 actual account name. For instance, many organizations set up
742 `First.Last' sendmail aliases for all users. If this is
743 the case, the GECOS field for each user should look like:
744 ``First [Middle] Last <First.Last>''
745 .P
746 As mmh sends outgoing mail via the local MTA only,
747 the best location to do such global rewrites is there.
748 Besides, the MTA is conceptionally the right location because it
749 does the reverse mapping for incoming mail (aliasing), too.
750 Further more, masquerading set up there is readily available for all
751 mail software on the system.
752 Hence, mmailid masquerading was removed.
753 .Ci 0836c8000ccb34b59410ef1c15b1b7feac70ce5f
754 .P
755 The
756 .I username_extension
757 masquerading type did not replace the username but would append a suffix,
758 specified by the
759 .Ev USERNAME_EXTENSION
760 environment variable, to it.
761 This provided support for the
762 .I user-extension
763 feature of qmail and the similar
764 .I "plussed user
765 processing of sendmail.
766 The decision to remove this username_extension masquerading was
767 motivated by the fact that
768 .Pn spost
769 hadn't supported it already.
770 .Ci 2abae0bfd0ad5bf898461e50aa4b466d641f23d9
771 Username extensions are possible in mmh, but less convenient to use.
772 .\" XXX format file %(getenv USERNAME_EXTENSION)
773 .P
774 The
775 .I draft_from
776 masquerading type instructed
777 .Pn post
778 to use the value of the `From:' header as SMTP envelope sender.
779 Sender addresses could be replaced completely.
780 .Ci b14ea6073f77b4359aaf3fddd0e105989db9
781 Mmh offers a kind of masquerading similar in effect, but
782 with technical differences.
783 As mmh does not transfer messages itself, the local MTA has final control
784 over the sender's address. Any masquerading mmh introduces may be reverted
785 by the MTA.
786 In times of pedantic spam checking, an MTA will take care to use
787 sensible envelope sender addresses to keep its own reputation up.
788 Nonetheless, the MUA can set the `From:' header and thereby propose
789 a sender address to the MTA.
790 The MTA may then decide to take that one or generate the canonical sender
791 address for use as envelope sender address.
792 .P
793 In mmh, the MTA will always extract the recipient and sender from the
794 message headers (\c
795 .Pn sendmail 's
796 .Sw -t
797 switch).
798 The `From:' header of the draft may be set arbitrary by the user.
799 If it is missing, the canonical sender address will be generated by the MTA.
801 .U3 "Remaining Options
802 .P
803 Two configure options remain in mmh.
804 One is the locking method to use:
805 .Sw --with-locking=[dot|fcntl|flock|lockf] .
806 The idea of removing all methods except the portable dot locking
807 and having that one as the default is appealing, but this change
808 requires deeper technical investigation into the topic.
809 The other option,
810 .Sw --enable-debug ,
811 compiles the programs with debugging symbols and does not strip them.
812 This option is likely to stay.
817 .H2 "Removal of switches
818 .P
823 .H1 "Modernizing
826 .H2 "Removal of Code Relicts
827 .P
828 The code base of mmh originates from the late Seventies,
829 had been extensively
830 worked on in the mid Eighties, and had been partly reorganized and extended
831 in the Nineties. Relicts of all those times had gathered in the code base.
832 My goal was to remove any ancient code parts. One part of the task was
833 converting obsolete code constructs to standard constructs, the other part
834 was dropping obsolete functions.
835 .P
836 As I'm not even thirty years old and have no more than seven years of
837 Unix experience, I needed to learn about the history in retrospective.
838 Older people likely have used those ancient constructs themselves
839 and have suffered from their incompatibilities and have longed for
840 standardization. Unfortunately, I have only read that others had done so.
841 This put me in a much more difficult positions when working on the old
842 code. I needed to recherche what other would have known by heart from
843 experience. All my programming experience comes from a time past ANSI C
844 and past POSIX. Although I knew about the times before, I took the
845 current state implicitly for granted most of the time.
846 .P
847 Being aware of
848 these facts, I rather let people with more historic experience solve the
849 task of converting the ancient code constructs to standardized ones.
850 Luckily, Lyndon Nerenberg focused on this task at the nmh project.
851 He converted large parts of the code to POSIX constructs, removing
852 the conditionals compilation for now standardized features.
853 I'm thankful for this task being solved. I only pulled the changes into
854 mmh.
855 .P
856 The other task \(en dropping ancient functionality to remove old code \(en
857 I did myself, though. My position to strip mmh to the bare minimum of
858 frequently used features is much more revolutional than the nmh community
859 likes it. Without the need to justify my decisions, I was able to quickly
860 remove functionality I considered ancient.
861 The need to discuss my decisions with
862 peers likely would have slowed this process down. Of course, I researched
863 if a particular feature really should be dropped. Having not had any
864 contact to this feature within my computer life was a first indicator to
865 drop it, but I also asked others and searched the literature for modern
866 usage of the feature. If it appeared to be truly ancient, I dropped it.
867 The reason for dropping is always part of the commit message in the
868 version control system. Thus, it is easy for others to check their
869 view on the topic with mine and possibly to argue for reinclusion.
871 .U2 "MMDF maildrop support
872 .P
873 I did drop any support for the MMDF maildrop format. This type of format
874 is conceptionally similar to the mbox format, but uses four bytes with
875 value 1 (\fL^A^A^A^A\fP) as message delimiter,
876 instead of the string ``\fLFrom\ \fP''.
877 Due to the similarity and mbox being the de-facto standard maildrop
878 format on Unix, but also due to the larger influence of Sendmail than MMDF,
879 the MMDF maildrop format had vanished.
880 .P
881 The simplifications within the code were only moderate. Switches could
882 be removed from tools like
883 .L packf ,
884 which generate packed mailboxes. Only one packed mailbox format remained:
885 mbox.
886 The most important changes affect the equally named mail parsing routine in
887 .L sbr/m_getfld.c .
888 The direct MMDF code had been removed, but as now only one packed mailbox
889 format is left, code structure simplifications are likely possible.
890 The reason why they are still outstanding is the heavily optimized code
891 of
892 .Fu m_getfld() .
893 Changes beyond a small local scope \(en
894 which restructuring in its core is \(en cause a high risk of damaging
895 the intricate workings of the optimized code. This problem is know
896 to the developers of nmh, too. They also avoid touching this minefield
897 if possible.
899 .U2 "UUCP Bang Paths
900 .P
901 More questionably than the former topic is the removal of support for the
902 UUCP bang path address style. However, the user may translate the bang
903 paths on retrieval to Internet addresses and the other way on posting
904 messages. The former can be done my an MDA like procmail; the latter
905 by a sendmail wrapper. This would ensure that any address handling would
906 work as expected. However, it might just work well without any
907 such modifications, as mmh does not touch addresses much, in general.
908 But I can't ensure as I have never used an environment with bang paths.
909 Also, the behavior might break at any point in further development.
911 .U2 "Hardcopy terminal support
912 .P
913 More of a funny anecdote is the remaining of a check for printing to a
914 hardcopy terminal until Spring 2012, when I finally removed it.
915 I surely would be very happy to see such a terminal in action, maybe
916 actually being able to work on it, but I fear my chances are null.
917 .P
918 The check only prevented a pager to be placed between the outputting
919 program (\c
920 .Pn mhl )
921 and the terminal. This could have been ensured with
922 the
923 .Sw -nomoreproc
924 at the command line statically, too.
926 .U2 "Removed support for header fields
927 .P
928 The `Encrypted' header had been introduced by RFC\^822, but already
929 marked legacy in RFC 2822. It was superseded by FIXME.
930 Mmh does no more support this header.
931 .P
932 Native support for `Face' headers
933 had been removed, as well.
934 The feature is similar to the `X-Face' header in its intent,
935 but takes a different approach to store the image.
936 Instead of encoding the image data directly into the header,
937 the the header contains the hostname and UDP port where the image
938 date could be retrieved.
939 Neither `X-Face' nor the here described `Face' system
940 \**
941 .FS
942 There is also a newer but different system, invented 2005,
943 using `Face' headers.
944 It is the successor of `X-Face' providing colored PNG images.
945 .FE
946 became well used in the large scale.
947 It's still possible to use a Face systems,
948 although mmh does not provide support for any of the different systems
949 anymore. It's fairly easy to write a small shell script to
950 extract the embedded or fetch the external Face data and display the image.
951 Own Face headers can be added into the draft template files.
952 .P
953 `Content-MD5' headers were introduced by RFC\^1864. They provide only
954 a verification of data corruption during the transfer. By no means can
955 they ensure verbatim end-to-end delivery of the contents. This is clearly
956 stated in the RFC. The proper approach to provide verificationability
957 of content in an end-to-end relationship is the use of digital cryptography
958 (RFCs FIXME). On the other hand, transfer protocols should ensure the
959 integrity of the transmission. In combinations these two approaches
960 make the `Content-MD5' header field useless. In consequence, I removed
961 the support for it. By this removal, MD5 computation is not needed
962 anywhere in mmh. Hence, over 500 lines of code were removed by this one
963 change. Even if the `Content-MD5' header field is useful sometimes,
964 I value its usefulnes less than the improvement in maintainability, caused
965 by the removal.
967 .U2 "Prompter's Control Keys
968 .P
969 The program
970 .Pn prompter
971 queries the user to fill in a message form. When used by
972 .Pn comp
973 as:
974 .VS
975 comp -editor prompter
976 VE
977 the resulting behavior is similar to
978 .Pn mailx .
979 Apparently,
980 .Pn prompter
981 hadn't been touched lately. Otherwise it's hardly explainable why it
982 still offered the switches
983 .Sw "-erase \f(CIchr\fP
984 and
985 .Sw "-kill \f(CIchr\fP
986 to name the characters for command line editing.
987 The times when this had been necessary are long time gone.
988 Today these things work out-of-the-box, and if not, are configured
989 with the standard tool
990 .Pn stty .
992 .U2 "Vfork and Retry Loops
993 .P
994 MH creates many processes, which is a consequence of the tool chest approach.
995 In earlier times
996 .Fu fork()
997 had been an expensive system call, as the process's whole image needed
998 to be duplicated. One common case is replacing the image with
999 .Fu exec()
1000 right after having forked the child process.
1001 To speed up this case, the
1002 .Fu vfork()
1003 system call was invented at Berkeley. It completely omits copying the
1004 image. If the image gets replaced right afterwards then unnecessary
1005 work is omited. On old systems this results in large speed ups.
1006 MH uses
1007 .Fu vfork()
1008 whenever possible.
1009 .P
1010 Memory management units that support copy-on-write semantics make
1011 .Fu fork()
1012 almost as fast as
1013 .Fu vfork()
1014 in the cases when they can be exchanged.
1015 With
1016 .Fu vfork()
1017 being more error-prone and hardly faster, it's preferable to simply
1018 use
1019 .Fu fork()
1020 instead.
1021 .P
1022 Related to the costs of
1023 .Fu fork()
1024 is the probability of its success.
1025 Today on modern systems, the system call will succeed almost always.
1026 In the Eighties on heavy loaded systems, as they were common at
1027 universities, this had been different. Thus, many of the
1028 .Fu fork()
1029 calls were wrapped into loops to retry to fork several times in
1030 short intervals, in case of previous failure.
1031 In mmh, the program aborts at once if the fork failed.
1032 The user can reexecute the command then. This is expected to be a
1033 very rare case on modern systems, especially personal ones, which are
1034 common today.
1037 .H2 "Attachments
1038 .P
1039 MIME
1042 .H2 "Digital Cryptography
1043 .P
1044 Signing and encryption.
1047 .H2 "Good Defaults
1048 .P
1049 foo
1054 .H1 "Code style
1055 .P
1056 foo
1059 .H2 "Standard Code
1060 .P
1061 POSIX
1064 .H2 "Separation
1066 .U2 "MH Directory Split
1067 .P
1068 In MH and nmh, a personal setup had consisted of two parts:
1069 The MH profile, named
1070 .Fn \&.mh_profile
1071 and being located directly in the user's home directory.
1072 And the MH directory, where all his mail messages and also his personal
1073 forms, scan formats, other configuration files are stored. The location
1074 of this directory could be user-chosen. The default was to name it
1075 .Fn Mail
1076 and have it directly in the home directory.
1077 .P
1078 I've never liked the data storage and the configuration to be intermixed.
1079 They are different kinds of data. One part, are the messages,
1080 which are the data to operate on. The other part, are the personal
1081 configuration files, which are able to change the behavior of the operations.
1082 The actual operations are defined in the profile, however.
1083 .P
1084 When storing data, one should try to group data by its type.
1085 There's sense in the Unix file system hierarchy, where configuration
1086 file are stored separate (\c
1087 .Fn /etc )
1088 to the programs (\c
1089 .Fn /bin
1090 and
1091 .Fn /usr/bin )
1092 to their sources (\c
1093 .Fn /usr/src ).
1094 Such separation eases the backup management, for instance.
1095 .P
1096 In mmh, I've reorganized the file locations.
1097 Still there are two places:
1098 There's the mail storage directory, which, like in MH, contains all the
1099 messages, but, unlike in MH, nothing else.
1100 Its location still is user-chosen, with the default name
1101 .Fn Mail ,
1102 in the user's home directory. This is much similar to the case in nmh.
1103 The configuration files, however, are grouped together in the new directory
1104 .Fn \&.mmh
1105 in the user's home directory.
1106 The user's profile now is a file, named
1107 .Fn profile ,
1108 in this mmh directory.
1109 Consistently, the context file and all the personal forms, scan formats,
1110 and the like, are also there.
1111 .P
1112 The naming changed with the relocation.
1113 The directory where everything, except the profile, had been stored (\c
1114 .Fn $HOME/Mail ),
1115 used to be called \fIMH directory\fP. Now, this directory is called the
1116 user's \fImail storage\fP. The name \fImmh directory\fP is now given to
1117 the new directory
1118 (\c
1119 .Fn $HOME/.mmh ),
1120 containing all the personal configuration files.
1121 .P
1122 The separation of the files by type of content is logical and convenient.
1123 There are no functional differences as any possible setup known to me
1124 can be implemented with both approaches, although likely a bit easier
1125 with the new approach. The main goal of the change had been to provide
1126 sensible storage locations for any type of personal mmh file.
1127 .P
1128 In order for one user to have multiple MH setups, he can use the
1129 environment variable
1130 .Ev MH
1131 the point to a different profile file.
1132 The MH directory (mail storage plus personal configuration files) is
1133 defined by the
1134 .Pe Path
1135 profile entry.
1136 The context file could be defined by the
1137 .Pe context
1138 profile entry or by the
1139 .Ev MHCONTEXT
1140 environment variable.
1141 The latter is useful to have a distinct context (e.g. current folders)
1142 in each terminal window, for instance.
1143 In mmh, there are three environment variables now.
1144 .Ev MMH
1145 may be used to change the location of the mmh directory.
1146 .Ev MMHP
1147 and
1148 .Ev MMHC
1149 change the profile and context files, respectively.
1150 Besides providing a more consistent feel (which simply is the result
1151 of being designed anew), the set of personal configuration files can
1152 be chosen independently from the profile (including mail storage location)
1153 and context, now. Being it relevant for practical use or not, it
1154 de-facto is an improvement. However, the main achievement is the
1155 split between mail storage and personal configuration files.
1158 .H2 "Modularization
1159 .P
1160 whatnowproc
1161 .P
1162 The \fIMH library\fP
1163 .Fn libmh.a
1164 collects a bunch of standard functions that many of the MH tools need,
1165 like reading the profile or context files.
1166 This doesn't hurt the separation.
1169 .H2 "Style
1170 .P
1171 Code layout, goto, ...
1176 .H1 "Concept Exploitation/Homogeniety
1179 .H2 "Draft Folder
1180 .P
1181 Historically, MH provided exactly one draft message, named
1182 .Fn draft
1183 and
1184 being located in the MH directory. When starting to compose another message
1185 before the former one was sent, the user had been questioned whether to use,
1186 refile or replace the old draft. Working on multiple drafts at the same time
1187 was impossible. One could only work on them in alteration by refiling the
1188 previous one to some directory and fetching some other one for reediting.
1189 This manual draft management needed to be done each time the user wanted
1190 to switch between editing one draft to editing another.
1191 .P
1192 To allow true parallel editing of drafts, in a straight forward way, the
1193 draft folder facility exists. It had been introduced already in July 1984
1194 by Marshall T. Rose. The facility was deactivated by default.
1195 Even in nmh, the draft folder facility remained deactivated by default.
1196 At least, Richard Coleman added the man page
1197 .Mp mh-draft(5)
1198 to document
1199 the feature well.
1200 .P
1201 The only advantage of not using the draft folder facility is the static
1202 name of the draft file. This could be an issue for MH frontends like mh-e.
1203 But as they likely want to provide working on multiple drafts in parallel,
1204 the issue is only concerning compatibility. The aim of nmh to stay compatible
1205 prevented the default activation of the draft folder facility.
1206 .P
1207 On the other hand, a draft folder is the much more natural concept than
1208 a draft message. MH's mail storage consists of folders and messages,
1209 the messages named with ascending numbers. A draft message breaks with this
1210 concept by introducing a message in a file named
1211 .Fn draft .
1212 This draft
1213 message is special. It can not be simply listed with the available tools,
1214 but instead requires special switches. I.e. corner-cases were
1215 introduced. A draft folder, in contrast, does not introduce such
1216 corner-cases. The available tools can operate on the messages within that
1217 folder like on any messages within any mail folders. The only difference
1218 is the fact that the default folder for
1219 .Pn send
1220 is the draft folder,
1221 instead of the current folder, like for all other tools.
1222 .P
1223 The trivial part of the change was activating the draft folder facility
1224 by default and setting a default name for this folder. Obviously, I chose
1225 the name
1226 .Fn +drafts .
1227 This made the
1228 .Sw -draftfolder
1229 and
1230 .Sw -draftmessage
1231 switches useless, and I could remove them.
1232 The more difficult but also the part that showed the real improvement,
1233 was updating the tools to the new concept.
1234 .Sw -draft
1235 switches could
1236 be dropped, as operating on a draft message became indistinguishable to
1237 operating on any other message for the tools.
1238 .Pn comp
1239 still has its
1240 .Sw -use
1241 switch for switching between its two modes: (1) Compose a new
1242 draft, possibly by taking some existing message as a form. (2) Modify
1243 an existing draft. In either case, the behavior of
1244 .Pn comp is
1245 deterministic. There is no more need to query the user. I consider this
1246 a major improvement. By making
1247 .Pn send
1248 simply operate on the current
1249 message in the draft folder by default, with message and folder both
1250 overridable by specifying them on the command line, it is now possible
1251 to send a draft anywhere within the storage by simply specifying its folder
1252 and name.
1253 .P
1254 All theses changes converted special cases to regular cases, thus
1255 simplifying the tools and increasing the flexibility.
1258 .H2 "Trash Folder
1259 .P
1260 Similar to the situation for drafts is the situation for removed messages.
1261 Historically, a message was deleted by renaming. A specific
1262 \fIbackup prefix\fP, often comma (\c
1263 .Fn , )
1264 or hash (\c
1265 .Fn # ),
1266 being prepended to the file name. Thus, MH wouldn't recognize the file
1267 as a message anymore, as only files whose name consists of digits only
1268 are treated as messages. The removed messages remained as files in the
1269 same directory and needed some maintenance job to truly delete them after
1270 some grace time. Usually, by running a command similar to
1271 .VS
1272 find /home/user/Mail -ctime +7 -name ',*' | xargs rm
1273 VE
1274 in a cron job. Within the grace time interval
1275 the original message could be restored by stripping the
1276 the backup prefix from the file name. If however, the last message of
1277 a folder is been removed \(en say message
1278 .Fn 6
1279 becomes file
1280 .Fn ,6
1281 \(en and a new message enters the same folder, thus the same
1282 numbered being given again \(en in our case
1283 .Fn 6
1284 \(en, if that one
1285 is removed too, then the backup of the former message gets overwritten.
1286 Thus, the ability to restore removed messages does not only depend on
1287 the ``sweeping cron job'' but also on the removing of further messages.
1288 This is undesirable, because the real mechanism is hidden from the user
1289 and the consequences of further removals are not always obvious.
1290 Further more, the backup files are scattered within the whole mail
1291 storage, instead of being collected at one place.
1292 .P
1293 To improve the situation, the profile entry
1294 .Pe rmmproc
1295 (previously named
1296 .Pe Delete-Prog )
1297 was introduced, very early.
1298 It could be set to any command, which would care for the mail removal
1299 instead of taking the default action, described above.
1300 Refiling the to-be-removed files to some garbage folder was a common
1301 example. Nmh's man page
1302 .Mp rmm(1)
1303 proposes
1304 .Cl "refile +d
1305 to move messages to the garbage folder and
1306 .Cl "rm `mhpath +d all`
1307 the empty the garbage folder.
1308 Managing the message removal this way is a sane approach. It keeps
1309 the removed messages in one place, makes it easy to remove the backup
1310 files, and, most important, enables the user to use the tools of MH
1311 itself to operate on the removed messages. One can
1312 .Pn scan
1313 them,
1314 .Pn show
1315 them, and restore them with
1316 .Pn refile .
1317 There's no more
1318 need to use
1319 .Pn mhpath
1320 to switch over from MH tools to Unix tools \(en MH can do it all itself.
1321 .P
1322 This approach matches perfect with the concepts of MH, thus making
1323 it powerful. Hence, I made it the default. And even more, I also
1324 removed the old backup prefix approach, as it is clearly less powerful.
1325 Keeping unused alternative in the code is a bad choice as they likely
1326 gather bugs, by not being constantly tested. Also, the increased code
1327 size and more conditions crease the maintenance costs. By strictly
1328 converting to the trash folder approach, I simplified the code base.
1329 .Pn rmm
1330 calls
1331 .Pn refile
1332 internally to move the to-be-removed
1333 message to the trash folder (\c
1334 .Fn +trash
1335 by default). Messages
1336 there can be operated on like on any other message in the storage.
1337 The sweep clean, one can use
1338 .Cl "rmm -unlink +trash a" ,
1339 where the
1340 .Sw -unlink
1341 switch causes the files to be truly unliked instead
1342 of moved to the trash folder.
1345 .H2 "Path Notations
1346 .P
1347 foo
1350 .H2 "MIME Integration
1351 .P
1352 user-visible access to whole messages and MIME parts are inherently
1353 different
1356 .H2 "Of One Cast
1357 .P