docs/master
view ch03.roff @ 51:49cf68506b5d
Spell checking.
| author | markus schnalke <meillo@marmaro.de> |
|---|---|
| date | Sun, 20 May 2012 11:40:19 +0200 |
| parents | d3a02f5e63b3 |
| children | 814c33b96d89 |
line source
1 .H0 "Work Report
2 .P
3 foo
4 .P
5 bar
7 .H1 "Removal of Code Relicts
8 .P
9 The code base of mmh originates from the late Seventies,
10 had been extensively
11 worked on in the mid Eighties, and had been partly reorganized and extended
12 in the Nineties. Relicts of all those times had gathered in the code base.
13 My goal was to remove any ancient code parts. One part of the task was
14 converting obsolete code constructs to standard constructs, the other part
15 was dropping obsolete functions.
16 .P
17 As I'm not even thirty years old and have no more than seven years of
18 Unix experience, I needed to learn about the history in retrospective.
19 Older people likely have used those ancient constructs themselves
20 and have suffered from their incompatibilities and have longed for
21 standardization. Unfortunately, I have only read that others had done so.
22 This put me in a much more difficult positions when working on the old
23 code. I needed to recherche what other would have known by heart from
24 experience. All my programming experience comes from a time past ANSI C
25 and past POSIX. Although I knew about the times before, I took the
26 current state implicitly for granted most of the time.
27 .P
28 Being aware of
29 these facts, I rather let people with more historic experience solve the
30 task of converting the ancient code constructs to standardized ones.
31 Luckily, Lyndon Nerenberg focused on this task at the nmh project.
32 He converted large parts of the code to POSIX constructs, removing
33 the conditionals compilation for now standardized features.
34 I'm thankful for this task being solved. I only pulled the changes into
35 mmh.
36 .P
37 The other task \(en dropping ancient functionality to remove old code \(en
38 I did myself, though. My position to strip mmh to the bare minimum of
39 frequently used features is much more revolutional than the nmh community
40 likes it. Without the need to justify my decisions, I was able to quickly
41 remove functionality I considered ancient.
42 The need to discuss my decisions with
43 peers likely would have slowed this process down. Of course, I researched
44 if a particular feature really should be dropped. Having not had any
45 contact to this feature within my computer life was a first indicator to
46 drop it, but I also asked others and searched the literature for modern
47 usage of the feature. If it appeared to be truly ancient, I dropped it.
48 The reason for dropping is always part of the commit message in the
49 version control system. Thus, it is easy for others to check their
50 view on the topic with mine and possibly to argue for reinclusion.
52 .U2 "MMDF maildrop support
53 .P
54 I did drop any support for the MMDF maildrop format. This type of format
55 is conceptionally similar to the mbox format, but uses four bytes with
56 value 1 (\fL^A^A^A^A\fP) as message delimiter,
57 instead of the string ``\fLFrom\ \fP''.
58 Due to the similarity and mbox being the de-facto standard maildrop
59 format on Unix, but also due to the larger influence of Sendmail than MMDF,
60 the MMDF maildrop format had vanished.
61 .P
62 The simplifications within the code were only moderate. Switches could
63 be removed from tools like
64 .L packf ,
65 which generate packed mailboxes. Only one packed mailbox format remained:
66 mbox.
67 The most important changes affect the equally named mail parsing routine in
68 .L sbr/m_getfld.c .
69 The direct MMDF code had been removed, but as now only one packed mailbox
70 format is left, code structure simplifications are likely possible.
71 The reason why they are still outstanding is the heavily optimized code
72 of
73 .Fu m_getfld() .
74 Changes beyond a small local scope \(en
75 which restructuring in its core is \(en cause a high risk of damaging
76 the intricate workings of the optimized code. This problem is know
77 to the developers of nmh, too. They also avoid touching this minefield
78 if possible.
80 .U2 "UUCP Bang Paths
81 .P
82 More questionably than the former topic is the removal of support for the
83 UUCP bang path address style. However, the user may translate the bang
84 paths on retrieval to Internet addresses and the other way on posting
85 messages. The former can be done my an MDA like procmail; the latter
86 by a sendmail wrapper. This would ensure that any address handling would
87 work as expected. However, it might just work well without any
88 such modifications, as mmh does not touch addresses much, in general.
89 But I can't ensure as I have never used an environment with bang paths.
90 Also, the behavior might break at any point in further development.
92 .U2 "Hardcopy terminal support
93 .P
94 More of a funny anecdote is the remaining of a check for printing to a
95 hardcopy terminal until Spring 2012, when I finally removed it.
96 I surely would be very happy to see such a terminal in action, maybe
97 actually being able to work on it, but I fear my chances are null.
98 .P
99 The check only prevented a pager to be placed between the outputting
100 program (\c
101 .Pn mhl )
102 and the terminal. This could have been ensured with
103 the
104 .Sw \-nomoreproc
105 at the command line statically, too.
107 .U2 "Removed support for header fields
108 .P
109 The `Encrypted' header had been introduced by RFC\^822, but already
110 marked legacy in RFC 2822. It was superseded by FIXME.
111 Mmh does no more support this header.
112 .P
113 Native support for `Face' headers
114 had been removed, as well.
115 The feature is similar to the `X-Face' header in its intent,
116 but takes a different approach to store the image.
117 Instead of encoding the image data directly into the header,
118 the the header contains the hostname and UDP port where the image
119 date could be retrieved.
120 Neither `X-Face' nor the here described `Face' system
121 \**
122 .FS
123 There is also a newer but different system, invented 2005,
124 using `Face' headers.
125 It is the successor of `X-Face' providing colored PNG images.
126 .FE
127 became well used in the large scale.
128 It's still possible to use a Face systems,
129 although mmh does not provide support for any of the different systems
130 anymore. It's fairly easy to write a small shell script to
131 extract the embedded or fetch the external Face data and display the image.
132 Own Face headers can be added into the draft template files.
133 .P
134 `Content-MD5' headers were introduced by RFC\^1864. They provide only
135 a verification of data corruption during the transfer. By no means can
136 they ensure verbatim end-to-end delivery of the contents. This is clearly
137 stated in the RFC. The proper approach to provide verificationability
138 of content in an end-to-end relationship is the use of digital cryptography
139 (RFCs FIXME). On the other hand, transfer protocols should ensure the
140 integrity of the transmission. In combinations these two approaches
141 make the `Content-MD5' header field useless. In consequence, I removed
142 the support for it. By this removal, MD5 computation is not needed
143 anywhere in mmh. Hence, over 500 lines of code were removed by this one
144 change. Even if the `Content-MD5' header field is useful sometimes,
145 I value its usefulnes less than the improvement in maintainability, caused
146 by the removal.
148 .U2 "Prompter's Control Keys
149 .P
150 The program
151 .Pn prompter
152 queries the user to fill in a message form. When used by
153 .Pn comp
154 as:
155 .DS
156 comp \-editor prompter
157 .DE
158 the resulting behavior is similar to
159 .Pn mailx .
160 Apparently,
161 .Pn prompter
162 hadn't been touched lately. Otherwise it's hardly explainable why it
163 still offered the switches
164 .Sn \-erase \fUchr\fP
165 and
166 .Sn \-kill \fUchr\fP
167 to name the characters for command line editing.
168 The times when this had been necessary are long time gone.
169 Today these things work out-of-the-box, and if not, are configured
170 with the standard tool
171 .Pn stty .
173 .U2 "Vfork and Retry Loops
174 .P
175 MH creates many processes, which is a consequence of the tool chest approach.
176 In earlier times
177 .Fu fork()
178 had been an expensive system call, as the process's whole image needed
179 to be duplicated. One common case is replacing the image with
180 .Fu exec()
181 right after having forked the child process.
182 To speed up this case, the
183 .Fu vfork()
184 system call was invented at Berkeley. It completely omits copying the
185 image. If the image gets replaced right afterwards then unnecessary
186 work is omited. On old systems this results in large speed ups.
187 MH uses
188 .Fu vfork()
189 whenever possible.
190 .P
191 Memory management units that support copy-on-write semantics make
192 .Fu fork()
193 almost as fast as
194 .Fu vfork()
195 in the cases when they can be exchanged.
196 With
197 .Fu vfork()
198 being more error-prone and hardly faster, it's preferable to simply
199 use
200 .Fu fork()
201 instead.
202 .P
203 Related to the costs of
204 .Fu fork()
205 is the probability of its success.
206 Today on modern systems, the system call will succeed almost always.
207 In the Eighties on heavy loaded systems, as they were common at
208 universities, this had been different. Thus, many of the
209 .Fu fork()
210 calls were wrapped into loops to retry to fork several times in
211 short intervals, in case of previous failure.
212 In mmh, the program aborts at once if the fork failed.
213 The user can reexecute the command then. This is expected to be a
214 very rare case on modern systems, especially personal ones, which are
215 common today.
218 .H1 "Removal of Tools
219 .P
220 MH had been considered an all-in-one system for mail handling.
221 The community around nmh has a similar understanding.
222 In fundamental difference, I believe that mmh should be a MUA but
223 nothing more. I believe that all-in-one mail systems are not the way
224 to go. There are excellent specialized MTAs, like Postfix;
225 there are specialized MDAs, like Procmail; there are specialized
226 MRAs, like Fetchmail. I believe it's best to use them instead of
227 providing the same function ourselves. Doing something well requires to
228 focus on this particular aspect or a small set of aspects. The more
229 it is possible to focus, the better the result in this particular
230 area will be. The limiting resource in Free Software community development
231 usually is human power. If the low development power is even parted
232 into multiple development areas, it will hardly be possible to
233 compete with the specialists in the various fields. This is even
234 increased, given the small community \(en developers and users \(en
235 that MH-based mail systems have. In consequence, I believe that the
236 available resources should be concentrated at the point where MH is
237 most unique. This is clearly the MUA part.
238 .P
239 Several of nmh's tools were removed from mmh because they didn't
240 match the main focus of adding to the MUA's task.
241 .P
242 .Pn conflict
243 was removed because it is a mail system maintenance tool.
244 Besides, it also checks the
245 .Fn /etc/passwd
246 and
247 .Fn /etc/group
248 files.
249 The tool might be useful, but it should not be shipped with mmh.
250 .P
251 .Pn rcvtty
252 was removed because its usecase of writing to the user's terminal
253 on receiving of mail is hardly wanted today. If users like to be
254 informed of new mail, then using the shell's
255 .Ev MAILPATH
256 variable or different (graphical) notifications are likely more
257 appealing. Writing directly to other terminals is hardly ever wanted
258 today. If though one wants to have it this way, the standard tool
259 .Pn write
260 can be used in a way similar to:
261 .DS
262 scan -file - | write `id -un`
263 .DE
264 .P
265 When the new attachment system was introduced,
266 .Pn viamail
267 was removed because then
268 .Pn forw
269 could cover the task itself.
270 The wrapper program
271 .Pn sendfiles
272 was rewritten as a shell script to use
273 .Pn forw .
274 .P
275 .Pn msgchk
276 was removed as it became hardly useful when POP support was removed.
277 It is questionable if
278 .Pn msgchk
279 provides more information than:
280 .DS
281 ls -l /var/mail/meillo
282 .DE
283 It does separate between old and new mail, but that's not very
284 useful and can be found out with
285 .Pn stat (1)
286 too. A very small shell script could care for the form of output.
287 As mmh's inc only incorporates mail from the user's local maildrop
288 and thus no long data transfers are involved,
289 there's no need to check for new mail before incorporating it.
290 .P
291 .Pn msh
292 was removed because the tool was in conflict with the original
293 philosophy of MH. It provided an interactive shell to access the
294 features of MH. One major feature of MH is being a tool chest.
295 .Pn msh
296 wouldn't be just another shell, tailored to the needs of mail
297 handling, but one large program to have the MH tools built in.
298 It's main use was for accessing Bulletin Boards, which have seized to
299 be popular. Removing
300 .Pn msh ,
301 together with the truly obsolete programs
302 .Pn vmh
303 and
304 .Pn wmh ,
305 saved more than 7\|000 lines of C code \(en a major achievement.
308 .H1 "Draft and Trash Folders
309 .U2 "Draft Folder
310 .P
311 Historically, MH provided exactly one draft message, named
312 .Fn draft
313 and
314 being located in the MH directory. When starting to compose another message
315 before the former one was sent, the user had been questioned whether to use,
316 refile or replace the old draft. Working on multiple drafts at the same time
317 was impossible. One could only work on them in alteration by refiling the
318 previous one to some directory and fetching some other one for reediting.
319 This manual draft management needed to be done each time the user wanted
320 to switch between editing one draft to editing another.
321 .P
322 To allow true parallel editing of drafts, in a straight forward way, the
323 draft folder facility exists. It had been introduced already in July 1984
324 by Marshall T. Rose. The facility was deactivated by default.
325 Even in nmh, the draft folder facility remained deactivated by default.
326 At least, Richard Coleman added the man page
327 .Mp mh-draft(5)
328 to document
329 the feature well.
330 .P
331 The only advantage of not using the draft folder facility is the static
332 name of the draft file. This could be an issue for MH frontends like mh-e.
333 But as they likely want to provide working on multiple drafts in parallel,
334 the issue is only concerning compatibility. The aim of nmh to stay compatible
335 prevented the default activation of the draft folder facility.
336 .P
337 On the other hand, a draft folder is the much more natural concept than
338 a draft message. MH's mail storage consists of folders and messages,
339 the messages named with ascending numbers. A draft message breaks with this
340 concept by introducing a message in a file named
341 .Fn draft .
342 This draft
343 message is special. It can not be simply listed with the available tools,
344 but instead requires special switches. I.e. corner-cases were
345 introduced. A draft folder, in contrast, does not introduce such
346 corner-cases. The available tools can operate on the messages within that
347 folder like on any messages within any mail folders. The only difference
348 is the fact that the default folder for
349 .Pn send
350 is the draft folder,
351 instead of the current folder, like for all other tools.
352 .P
353 The trivial part of the change was activating the draft folder facility
354 by default and setting a default name for this folder. Obviously, I chose
355 the name
356 .Fn +drafts .
357 This made the
358 .Sw \-draftfolder
359 and
360 .Sw \-draftmessage
361 switches useless, and I could remove them.
362 The more difficult but also the part that showed the real improvement,
363 was updating the tools to the new concept.
364 .Sw \-draft
365 switches could
366 be dropped, as operating on a draft message became indistinguishable to
367 operating on any other message for the tools.
368 .Pn comp
369 still has its
370 .Sw \-use
371 switch for switching between its two modes: (1) Compose a new
372 draft, possibly by taking some existing message as a form. (2) Modify
373 an existing draft. In either case, the behavior of
374 .Pn comp is
375 deterministic. There is no more need to query the user. I consider this
376 a major improvement. By making
377 .Pn send
378 simply operate on the current
379 message in the draft folder by default, with message and folder both
380 overridable by specifying them on the command line, it is now possible
381 to send a draft anywhere within the storage by simply specifying its folder
382 and name.
383 .P
384 All theses changes converted special cases to regular cases, thus
385 simplifying the tools and increasing the flexibility.
387 .U2 "Trash Folder
388 .P
389 Similar to the situation for drafts is the situation for removed messages.
390 Historically, a message was deleted by renaming. A specific
391 \fIbackup prefix\fP, often comma (\c
392 .Fn , )
393 or hash (\c
394 .Fn # ),
395 being prepended to the file name. Thus, MH wouldn't recognize the file
396 as a message anymore, as only files whose name consists of digits only
397 are treated as messages. The removed messages remained as files in the
398 same directory and needed some maintenance job to truly delete them after
399 some grace time. Usually, by running a command similar to
400 .DS
401 find /home/user/Mail \-ctime +7 \-name ',*' | xargs rm
402 .DE
403 in a cron job. Within the grace time interval
404 the original message could be restored by stripping the
405 the backup prefix from the file name. If however, the last message of
406 a folder is been removed \(en say message
407 .Fn 6
408 becomes file
409 .Fn ,6
410 \(en and a new message enters the same folder, thus the same
411 numbered being given again \(en in our case
412 .Fn 6
413 \(en, if that one
414 is removed too, then the backup of the former message gets overwritten.
415 Thus, the ability to restore removed messages does not only depend on
416 the ``sweeping cron job'' but also on the removing of further messages.
417 This is undesirable, because the real mechanism is hidden from the user
418 and the consequences of further removals are not always obvious.
419 Further more, the backup files are scattered within the whole mail
420 storage, instead of being collected at one place.
421 .P
422 To improve the situation, the profile entry
423 .Pe rmmproc
424 (previously named
425 .Pe Delete-Prog )
426 was introduced, very early.
427 It could be set to any command, which would care for the mail removal
428 instead of taking the default action, described above.
429 Refiling the to-be-removed files to some garbage folder was a common
430 example. Nmh's man page
431 .Mp rmm(1)
432 proposes
433 .Cl "refile +d
434 to move messages to the garbage folder and
435 .Cl "rm `mhpath +d all`
436 the empty the garbage folder.
437 Managing the message removal this way is a sane approach. It keeps
438 the removed messages in one place, makes it easy to remove the backup
439 files, and, most important, enables the user to use the tools of MH
440 itself to operate on the removed messages. One can
441 .Pn scan
442 them,
443 .Pn show
444 them, and restore them with
445 .Pn refile .
446 There's no more
447 need to use
448 .Pn mhpath
449 to switch over from MH tools to Unix tools \(en MH can do it all itself.
450 .P
451 This approach matches perfect with the concepts of MH, thus making
452 it powerful. Hence, I made it the default. And even more, I also
453 removed the old backup prefix approach, as it is clearly less powerful.
454 Keeping unused alternative in the code is a bad choice as they likely
455 gather bugs, by not being constantly tested. Also, the increased code
456 size and more conditions crease the maintenance costs. By strictly
457 converting to the trash folder approach, I simplified the code base.
458 .Pn rmm
459 calls
460 .Pn refile
461 internally to move the to-be-removed
462 message to the trash folder (\c
463 .Fn +trash
464 by default). Messages
465 there can be operated on like on any other message in the storage.
466 The sweep clean, one can use
467 .Cl "rmm \-unlink +trash a" ,
468 where the
469 .Sw \-unlink
470 switch causes the files to be truly unliked instead
471 of moved to the trash folder.
474 .H1 "MH Directory Split
475 .P
476 In MH and nmh, a personal setup had consisted of two parts:
477 The MH profile, named
478 .Fn \&.mh_profile
479 and being located directly in the user's home directory.
480 And the MH directory, where all his mail messages and also his personal
481 forms, scan formats, other configuration files are stored. The location
482 of this directory could be user-chosen. The default was to name it
483 .Fn Mail
484 and have it directly in the home directory.
485 .P
486 I've never liked the data storage and the configuration to be intermixed.
487 They are different kinds of data. One part, are the messages,
488 which are the data to operate on. The other part, are the personal
489 configuration files, which are able to change the behavior of the operations.
490 The actual operations are defined in the profile, however.
491 .P
492 When storing data, one should try to group data by its type.
493 There's sense in the Unix file system hierarchy, where configuration
494 file are stored separate (\c
495 .Fn /etc )
496 to the programs (\c
497 .Fn /bin
498 and
499 .Fn /usr/bin )
500 to their sources (\c
501 .Fn /usr/src ).
502 Such separation eases the backup management, for instance.
503 .P
504 In mmh, I've reorganized the file locations.
505 Still there are two places:
506 There's the mail storage directory, which, like in MH, contains all the
507 messages, but, unlike in MH, nothing else.
508 Its location still is user-chosen, with the default name
509 .Fn Mail ,
510 in the user's home directory. This is much similar to the case in nmh.
511 The configuration files, however, are grouped together in the new directory
512 .Fn \&.mmh
513 in the user's home directory.
514 The user's profile now is a file, named
515 .Fn profile ,
516 in this mmh directory.
517 Consistently, the context file and all the personal forms, scan formats,
518 and the like, are also there.
519 .P
520 The naming changed with the relocation.
521 The directory where everything, except the profile, had been stored (\c
522 .Fn $HOME/Mail ),
523 used to be called \fIMH directory\fP. Now, this directory is called the
524 user's \fImail storage\fP. The name \fImmh directory\fP is now given to
525 the new directory
526 (\c
527 .Fn $HOME/.mmh ),
528 containing all the personal configuration files.
529 .P
530 The separation of the files by type of content is logical and convenient.
531 There are no functional differences as any possible setup known to me
532 can be implemented with both approaches, although likely a bit easier
533 with the new approach. The main goal of the change had been to provide
534 sensible storage locations for any type of personal mmh file.
535 .P
536 In order for one user to have multiple MH setups, he can use the
537 environment variable
538 .Ev MH
539 the point to a different profile file.
540 The MH directory (mail storage plus personal configuration files) is
541 defined by the
542 .Pe Path
543 profile entry.
544 The context file could be defined by the
545 .Pe context
546 profile entry or by the
547 .Ev MHCONTEXT
548 environment variable.
549 The latter is useful to have a distinct context (e.g. current folders)
550 in each terminal window, for instance.
551 In mmh, there are three environment variables now.
552 .Ev MMH
553 may be used to change the location of the mmh directory.
554 .Ev MMHP
555 and
556 .Ev MMHC
557 change the profile and context files, respectively.
558 Besides providing a more consistent feel (which simply is the result
559 of being designed anew), the set of personal configuration files can
560 be chosen independently from the profile (including mail storage location)
561 and context, now. Being it relevant for practical use or not, it
562 de-facto is an improvement. However, the main achievement is the
563 split between mail storage and personal configuration files.
566 .H1 "Path Notations
567 .P
568 foo
570 .H1 "Attachments
571 .P
572 foo
574 .H1 "mhshow to show Transition
575 .P
576 Since the very beginning, already in the first concept paper,
577 .Pn show
578 had been MH's mail display program.
579 .Pn show
580 found out which pathnames the relevant messages had and invoked
581 .Pn mhl
582 then to let it render the content.
583 With the advent of MIME, this approach wasn't sufficient anymore.
584 MIME messages can consist of multiple parts, some of which aren't
585 directly displayable, and text content can be encoded in
586 foreign charsets.
587 .Pn show 's
588 simple approach and
589 .Pn mhl 's
590 limited display facilities couldn't cope with the task any longer.
591 Instead of extending these tools, new ones were written from scratch
592 and then added to the MH tool chest. Doing so is encouraged by the
593 tool chest approach. The new tools could be added without interfering
594 with the existing ones. This is great. It allowed MH to be the
595 first MUA to implement MIME.
596 .P
597 The new MIME features were added in form of the single program
598 .Pn mhn .
599 The command
600 .DS
601 mhn \-show 42
602 .DE
603 would show the MIME message numbered 42.
604 With the 1.0 release of nmh in February 1999, Richard Coleman finished
605 the split of
606 .Pn mhn
607 into a set of specialized programs, which together covered the
608 aspects of MIME. One of these resulting tools was
609 .Pn mhshow .
612 .H1 "Blind Carbon Copies
613 .P
614 foo
616 .H1 "Good Defaults
617 .P
618 foo
620 .H1 "Modularization
621 .P
622 foo
624 .H1 "Code style
625 .P
626 foo
630 (e.g. the
631 user-visible access to whole messages and MIME parts are inherently
632 different).
634 The \fIMH library\fP
635 .Fn libmh.a
636 collects a bunch of standard functions that many of the MH tools need,
637 like reading the profile or context files.
638 This doesn't hurt the separation.