docs/master: 348b92755bef discussion.roff

docs/master

view discussion.roff @ 233:348b92755bef

Some more minor adjustments to the acknowledgements.

author	markus schnalke <meillo@marmaro.de>
date	Mon, 16 Jul 2012 08:26:17 +0200
parents	96778c1afc3e
children

line source

1 .H0 "Discussion

2 .P

3 This main chapter discusses the practical work accomplished in the

4 mmh project.

5 It is structured along the goals chosen for the project.

6 A selection of the work undertaken

7 is described.

8 .P

9 This discussion compares the present version of mmh with the state of

10 nmh at the time when the mmh project had started, i.e. fall 2011.

11 Recent changes in nmh are rarely part of the discussion.

12 .P

13 Whenever lines of code are counted, David A. Wheeler's \fIsloccount\fP

14 was used to measure the amount in a comparable way.

15 .P

16 For the reader's convenience, the structure of modern email systems

17 is depicted in the following figure.

18 It illustrates the path a message takes from sender to recipient.

20 .sp 1.5

21 .KS

22 .in 2c

23 .so input/mail-agents.pic

24 .KE

25 .sp 1.5

27 .LP

28 The ellipses denote mail agents, i.e. different jobs in email processing.

29 These are:

30 .IP "Mail User Agent (MUA)

31 The only program users directly interact with.

32 It includes functions to compose new mail, display received mail,

33 and to manage the mail storage.

34 It is called a \fImail client\fP as well.

35 .IP "Mail Submission Agent (MSA)

36 A special kind of Mail Transfer Agent, used to submit mail into the

37 mail transport system.

38 Often it is also called an MTA.

39 .IP "Mail Transfer Agent (MTA)

40 A node in the mail transport system.

41 It transfers incoming mail to a transport node nearer to the

42 final destination.

43 An MTA may be the final destination itself.

44 .IP "Mail Delivery Agent (MDA)

45 Delivers mail according to a set of rules.

46 Usually, the messages are stored to disk.

47 .IP "Mail Retrieval Agent (MRA)

48 Initiates the transfer of mail from a remote location to the local machine.

49 (The dashed arrow in the figure represents the pull request.)

50 .LP

51 The dashed boxes represent entities that usually reside on single machines.

52 The box on the lower left represents the sender's system.

53 The box on the upper left represents the first mail transfer node.

54 The box on the upper right represents the transfer node responsible

55 for the destination address.

56 The box on the lower right represents the recipient's system.

57 Often, the boxes above the dotted line are servers on the Internet.

58 Many mail clients, including nmh, include all of the components below

59 the dotted line.

60 This is not the case for mmh; it implements the MUA only.

67 .\" --------------------------------------------------------------

68 .H1 "Streamlining

70 .P

71 MH once provided a complete email system.

72 The community around nmh tries to keep nmh in similar shape.

73 In fundamental contrast, mmh shall be an MUA only.

74 I believe that the development of all-in-one mail systems is obsolete.

75 Today, email is too complex to be fully covered by a single project.

76 Such a project will not be able to excel in all aspects.

77 Instead, the aspects of email should be covered by multiple projects,

78 which then can be combined to form a complete system.

79 Excellent implementations for the various aspects of email already exist.

80 Just to name three examples: Postfix is a specialized MTA, Procmail

81 is a specialized MDA, and Fetchmail is a specialized MRA.

82 I believe that it is best to use such specialized tools instead of

83 providing the same function once more as a side component.

84 .P

85 Doing something well requires focusing on a small set of specific aspects.

86 Under the assumption that development which is focussed on a particular

87 area produces better results there, specialized projects will be superior

88 in their field of focus.

89 Hence, all-in-one mail system projects \(en no matter if monolithic

90 or modular \(en will never be the best choice in any of the fields.

91 Even in providing the most consistent all-in-one system, they are likely

92 to be beaten by projects that focus exclusively on the creation

93 of a homogeneous system by integrating existing mail components.

94 .P

95 Usually, the limiting resource in the community development of

96 free software is man power.

97 If the development effort is spread over a large development area,

98 it becomes more difficult to compete with the specialists in the

99 various fields.

100 The concrete situation for MH-based mail systems is even tougher,

101 given their small and aged community, concerning both developers and users.

102 .P

103 In consequence, I believe that the available development resources

104 should focus on the point where MH is most unique.

105 This is clearly the user interface \(en the MUA.

106 Peripheral parts should be removed to streamline mmh for the MUA task.

107

108

109 .H2 "Mail Transfer Facilities

110 .Id mail-transfer-facilities

111 .P

112 The removal of the mail transfer facilities, effectively dropping the

113 MSA and MRA, had been the first work task in the mmh project.

114 The desire for this change initiated the creation of the mmh project.

115 .P

116 Focusing on one mail agent role only, is motivated by Eric Allman's

117 experience with Sendmail.

118 He identified the limitation of Sendmail

119 to the MTA task as one reason for its success:

120 .[ [

121 costales sendmail

122 .], p. xviii]

123 .QS

124 Second, I limited myself to the routing function \(en

125 I wouldn't write user agents or delivery back-ends.

126 This was a departure of the dominant thought of the time,

127 in which routing logic, local delivery, and often the network code

128 were incorporated directly into the user agents.

129 .QE

130 .P

131 In nmh, the MSA is called \fIMessage Transfer Service\fP (MTS).

132 This facility, implemented by the

133 .Pn post

134 command, establishes network connections and spoke SMTP to submit

135 messages to be relayed to the outside world.

136 When email transfer changed, this part needed to be changed as well.

137 Encryption and authentication for network connections

138 needed to be supported, hence TLS and SASL were introduced into nmh.

139 This added complexity without improving the core functions.

140 Furthermore, keeping up with recent developments in the field of

141 mail transfer requires development power and specialists.

142 In mmh, this whole facility was simply cut off

143 .Ci f6aa95b724fd8c791164abe7ee5468bf5c34f226

144 .Ci fecd5d34f65597a4dfa16aeabea7d74b191532c3

145 .Ci 156d35f6425bea4c1ed3c4c79783dc613379c65b .

146 Instead, mmh depends on an external MSA.

147 All outgoing mail in mmh goes through the

148 .Pn sendmail

149 command, which almost any MSA provides.

150 If not, a wrapper program can be written.

151 It must read the message from the standard input, extract the

152 recipient addresses from the message header, and hand the message

153 over to the MSA.

154 For example, a wrapper script for qmail would be:

155 .VS

156 #!/bin/sh

157 exec qmail-inject # ignore command line arguments

158 VE

159 The requirement to parse the recipient addresses out of the message header

160 may be removed in the future.

161 Mmh could pass the recipient addresses as command line arguments.

162 This appears to be the better interface.

163 .P

164 To retrieve mail, the

165 .Pn inc

166 command in nmh acts as MRA.

167 It establishes network connections

168 and speaks POP3 to retrieve mail from remote servers.

169 As with mail submission, the network connections required encryption and

170 authentication, thus TLS and SASL were added to nmh.

171 Support for message retrieval through IMAP will soon become necessary

172 additions, too, and likewise for any other changes in mail transfer.

173 But not in mmh because it has dropped the support for retrieving mail

174 from remote locations

175 .Ci ab7b48411962d26439f92f35ed084d3d6275459c .

176 Instead, it depends on an external tool to cover this task.

177 Mmh has two paths for messages to enter mmh's mail storage:

178 (1) Mail can be incorporated with

179 .Pn inc

180 from the system maildrop, or (2) with

181 .Pn rcvstore

182 by reading them, one at a time, from the standard input.

183 .P

184 With the removal of the MSA and MRA, mmh converted from a complete

185 mail system to only an MUA.

186 Now, of course, mmh depends on third-party software.

187 An external MSA is required to transfer mail to the outside world;

188 an external MRA is required to retrieve mail from remote machines.

189 Excellent implementations of such software exist.

190 They likely are superior to the internal versions that were removed.

191 Additionally, the best suiting programs can be chosen freely.

192 .P

193 As it had already been possible to use an external MSA and MRA,

194 why should the internal version not be kept for convenience?

195 Transferred to a different area, the question,

196 whether there is sense in having a fall-back pager in all

197 the command line tools for the cases when

198 .Pn more

199 or

200 .Pn less

201 are not available, appears to be ridiculous.

202 Of course, MSAs and MRAs are more complex than text pagers

203 and not necessarily available but still the concept of orthogonal

204 design holds: ``Write programs that do one thing and do it well''.

205 .[

206 mcilroy unix phil

207 p. 53

208 .]

209 .[

210 mcilroy bstj foreword

211 .]

212 Here, this part of the Unix philosophy was applied not only

213 to the programs but to the project itself.

214 In other words:

215 Develop projects that focus on one thing and do it well.

216 Projects which have grown complex should be split, for the same

217 reasons that programs which have grown complex should be split.

218 If it is conceptionally more elegant to have the MSA and MRA as

219 separate projects then they should be separated.

220 In my opinion, this is the case.

221 The RFCs suggest this separation by clearly distinguishing the

222 different mail handling tasks [RFC\|821].

223 The small interfaces between the mail agents support the

224 separation as well.

225 .P

226 Once, email had been small and simple.

227 At that time,

228 .Pn /bin/mail

229 had covered everything there was to email and still was small and simple.

230 Later, the essential complexity of email increased.

231 (Essential complexity is the complexity defined by the problem itself

232 .[ [

233 brooks no silver bullet

234 .]].)

235 Consequently, email systems grew.

236 RFCs started to introduce the concept of mail agents to separate the

237 various roles because they became more extensive and because

238 new roles appeared.

239 As mail system implementations grew, parts of them were split off.

240 For instance, a POP server was included in the original MH;

241 it was removed in nmh.

242 Now is the time to go one step further and split off the MSA and MRA,

243 as well.

244 Not only does this decrease the code size of the project,

245 more importantly, it unburdens mmh of the whole field of

246 message transfer, with all its implications for the project.

247 There is no more need for concern with changes in network transfer.

248 This independence is gained by depending on external components

249 that cover the field.

250 .P

251 In general, functionality can be added in three different ways:

252 .LI 1

253 By implementing the function in the project itself.

254 .LI 2

255 By depending on a library that provides the function.

256 .LI 3

257 By depending on a program that provides the function.

258 .LP

259 While implementing the function in the project itself leads to the

260 largest increase in code size and requires the most maintenance

261 and development work,

262 it keeps the project's dependence on other software lowest.

263 Using libraries or external programs requires less maintenance work

264 but introduces dependencies on external projects.

265 Programs have the smallest interfaces and provide the best separation,

266 but possibly limit the information exchange.

267 External libraries are more strongly connected than external programs,

268 thus information can be exchanged in a more flexible manner.

269 Obviously, adding code to a project increases the maintenance work.

270 As implementing complex functions in the project itself adds

271 a lot of code, this should be avoided if possible.

272 Thus, the dependencies only change in their character,

273 not in their existence.

274 In mmh, library dependencies on

275 .Pn libsasl2

276 and

277 .Pn libcrypto /\c

278 .Pn libssl

279 were traded against program dependencies on an MSA and an MRA.

280 This also meant trading build-time dependencies against run-time

281 dependencies.

282 Besides providing stronger separation and greater flexibility,

283 program dependencies also allowed

284 over 6\|000 lines of code to be removed from mmh.

285 This made mmh's code base about 12\|% smaller.

286 Reducing the project's code size by such an amount without actually

287 losing functionality is a convincing argument.

288 Actually, as external MSAs and MRAs are likely superior to the

289 project's internal versions, the common user even gains functionality.

290 .P

291 Users of MH should not have problems setting up an external MSA and MRA.

292 Also, the popular MSAs and MRAs have large communities and a lot

293 of available documentation.

294

295 Choices for MSAs range from small forwarders such as \fIssmtp\fP and

296 \fInullmailer\fP, over mid-size MTAs including \fImasqmail\fP and

297 \fIdma\fP, up to full-featured MTAs as for instance \fIPostfix\fP.

298 MRAs are provided for example by \fIfetchmail\fP, \fIgetmail\fP,

299 \fImpop\fP, and \fIfdm\fP.

300

301

302 .H2 "Non-MUA Tools

303 .P

304 One goal of mmh is to remove the tools that do not significantly

305 contribute to the MUA's job.

306 Loosely related and rarely used tools distract from a lean appearance,

307 and require maintenance work without adding much to the core task.

308 By removing these tools, mmh became more streamlined and focused.

309 .BU

310 .Pn conflict

311 was removed

312 .Ci 8b235097cbd11d728c07b966cf131aa7133ce5a9

313 because it is a mail system maintenance tool and not MUA-related.

314 It even checked

315 .Fn /etc/passwd

316 and

317 .Fn /etc/group

318 for consistency, which is completely unrelated to email.

319 A tool like

320 .Pn conflict

321 is surely useful, but it should not be shipped with mmh.

322 .BU

323 .Pn rcvtty

324 was removed

325 .Ci 14767c94b3827be7c867196467ed7aea5f6f49b0

326 because its use case of writing to the user's terminal

327 on reception of mail is obsolete.

328 If users like to be informed of new mail, the shell's

329 .Ev MAILPATH

330 variable or graphical notifications are technically more appealing.

331 Writing to terminals directly is hardly ever desired today.

332 If, though, one prefers this approach, the standard tool

333 .Pn write

334 can be used in a way similar to:

335 .VS

336 scan -file - | write `id -un`

337 VE

338 .BU

339 .Pn viamail

340 was removed

341 .Ci eda72d6a7a7c20ff123043fb7f19c509ea01f932

342 when the new attachment system was activated, because

343 .Pn forw

344 could then cover the task itself.

345 The

346 .Pn sendfiles

347 shell script was rewritten as a wrapper around

348 .Pn forw

349 .Ci 0e82199cf3c991a173e0ac8aa776efdb3ded61e6 .

350 .BU

351 .Pn msgchk

352 was removed

353 .Ci bb9360ead7eb7a3fedcce2eeedfc660014e41dbe ,

354 because it lost its use case when POP support was removed.

355 A call to

356 .Pn msgchk

357 provided hardly more information than:

358 .VS

359 ls -l /var/mail/meillo

360 VE

361 Yet, it distinguished between old and new mail, but

362 these details can be retrieved with

363 .Pn stat (1),

364 too.

365 A small shell script could be written to print the information

366 in a similar way, if truly necessary.

367 As mmh's

368 .Pn inc

369 only incorporates mail from the user's local maildrop,

370 and thus no data transfers over slow networks are involved,

371 there is hardly any need to check for new mail before incorporating it.

372 .BU

373 .Pn msh

374 was removed

375 .Ci 916690191222433a6923a4be54b0d8f6ac01bd02

376 because the tool was in conflict with the philosophy of MH.

377 It provided an interactive shell to access the features of MH.

378 However, it was not just a shell tailored to the needs of mail handling,

379 but one large program that had several MH tools built in.

380 This conflicted with the major feature of MH of being a tool chest.

381 .Pn msh 's

382 main use case had been accessing Bulletin Boards, which have ceased to

383 be popular.

384 .P

385 Removing

386 .Pn msh

387 together with the truly archaic code relics

388 .Pn vmh

389 and

390 .Pn wmh

391 saved more than 7\|000 lines of C code \(en

392 about 15\|% of the project's original source code amount.

393 Having less code \(en with equal readability, of course \(en

394 for the same functionality is an advantage.

395 Less code means less bugs and less maintenance work.

396 As

397 .Pn rcvtty

398 and

399 .Pn msgchk

400 are assumed to be rarely used and can be implemented in different ways,

401 why should one keep them?

402 Removing them streamlined mmh.

403 .Pn viamail 's

404 use case is now partly obsolete and partly covered by

405 .Pn forw ,

406 hence there is no reason to still maintain it.

407 .Pn conflict

408 is not related to the mail client, and

409 .Pn msh

410 conflicts with the basic concept of MH.

411 These two tools might still be useful, but they should not be part of mmh.

412 .P

413 .Id slocal

414 Finally, there is

415 .Pn slocal ,

416 which is an MDA and thus not directly MUA-related.

417 It should be removed from mmh because including it conflicts with

418 the idea that mmh is an MUA only.

419 However,

420 .Pn slocal

421 provides rule-based processing of messages, like filing them into

422 different folders, which is otherwise not available in mmh.

423 Although

424 .Pn slocal

425 neither pulls in dependencies, nor does it include a separate

426 technical area (cf. Sec.

427 .Cf mail-transfer-facilities ),

428 it still accounts for about 1\|000 lines of code that need to be maintained.

429 As

430 .Pn slocal

431 is almost self-standing, it should be split off into a separate project.

432 This would cut the strong connection between the MUA mmh and the MDA

433 .Pn slocal .

434 For anyone not using MH,

435 .Pn slocal

436 would become yet another independent MDA, like

437 .I procmail .

438 Then

439 .Pn slocal

440 could be installed without a complete MH system.

441 Likewise, mmh users could decide to use

442 .I procmail

443 without having a second, unused MDA, i.e.

444 .Pn slocal ,

445 installed.

446 That appears to be conceptionally the best solution.

447 Yet,

448 .Pn slocal

449 is not split off.

450 I defer the decision over

451 .Pn slocal

452 out of a need for deeper investigation.

453 In the meanwhile, it remains part of mmh

454 as its continued existence is not significant;

455 .Pn slocal

456 is unrelated to the rest of the project.

457

458

459

460 .H2 "Displaying Messages

461 .Id mhshow

462 .P

463 Since the very beginning, already in the first concept paper,

464 .[

465 original memo rand mh shapiro gaines

466 .]

467 .Pn show

468 had been MH's message display program.

469 .Pn show

470 mapped message numbers and sequences to files and invoked

471 .Pn mhl

472 to have the files formatted.

473 With MIME, this approach was not sufficient anymore.

474 MIME messages can consist of multiple parts.

475 Some parts, like binary attachments or text content in foreign charsets,

476 are not directly displayable.

477 .Pn show 's

478 understanding of messages and

479 .Pn mhl 's

480 display capabilities could not cope with the task any longer.

481 .P

482 Instead of extending these tools, additional tools were written from

483 scratch and were added to the MH tool chest.

484 Doing so is encouraged by the tool chest approach.

485 Modular design is a great advantage for extending a system,

486 as new tools can be added without interfering with existing ones.

487 First, the new MIME features were added in form of the single program

488 .Pn mhn .

489 The command

490 .Cl "mhn -show 42

491 had then shown the message number

492 .Fn 42 ,

493 interpreting MIME.

494 With the 1.0 release of nmh in February 1999, Richard Coleman finished

495 the split of

496 .Pn mhn

497 into a set of specialized tools, which together covered the

498 multiple aspects of MIME.

499 One of them was

500 .Pn mhshow ,

501 which replaced

502 .Cl "mhn -show" .

503 It was capable of displaying MIME messages appropriately.

504 .P

505 .ZZ

506 From then on, two message display tools were part of nmh,

507 .Pn show

508 and

509 .Pn mhshow .

510 To ease the life of users,

511 .Pn show

512 was extended to automatically hand the job over to

513 .Pn mhshow

514 if displaying the message would be beyond

515 .Pn show 's

516 abilities.

517 In consequence, the user would simply invoke

518 .Pn show

519 (possibly through

520 .Pn next

521 or

522 .Pn prev )

523 and get the message printed with either

524 .Pn show

525 or

526 .Pn mhshow ,

527 whatever was more appropriate.

528 .P

529 Having two similar tools for basically the same task is redundancy.

530 Usually, users do not distinguish between

531 .Pn show

532 and

533 .Pn mhshow

534 in their daily mail reading.

535 Having two separate display programs was therefore unnecessary

536 from a user's point of view.

537 Besides, the development of both programs needed to be in sync,

538 to ensure that the programs behaved in a similar way,

539 because they were used like a single tool.

540 Different behavior would have surprised the user.

541 .P

542 Today, non-MIME messages are rather seen to be a special case of

543 MIME messages, although it is the other way round.

544 As

545 .Pn mhshow

546 already had been able to display non-MIME messages, it appeared natural

547 to drop

548 .Pn show

549 in favor of using

550 .Pn mhshow

551 exclusively

552 .Ci 4c1efddfd499300c7e74263e57d8aa137e84c853 .

553 Removing

554 .Pn show

555 is no loss in function, because

556 .Pn mhshow

557 covers it completely.

558 Yet, the old behavior of

559 .Pn show

560 can still be emulated with the simple command line:

561 .VS

562 mhl `mhpath c`

563 VE

564 .P

565 For convenience,

566 .Pn mhshow

567 was renamed to

568 .Pn show

569 after

570 .Pn show

571 was gone.

572 It is clear that such a rename may confuse future developers when

573 trying to understand the history.

574 Nevertheless, I consider the convenience on the user's side,

575 to outweigh the inconvenience for understanding the evolution

576 of the tools.

577 .P

578 To prepare for the transition,

579 .Pn mhshow

580 was reworked to behave more like

581 .Pn show

582 first (cf. Sec.

583 .Cf mhshow ).

584 Once the tools behaved more alike, the replacing appeared to be

585 even more natural.

586 Today, mmh's new

587 .Pn show

588 has become the one single message display program once again,

589 with the difference

590 that today it handles MIME messages as well as non-MIME messages.

591 The outcomes of the transition are one program less to maintain,

592 no second display program for users to deal with,

593 and less system complexity.

594 .P

595 Still, removing the old

596 .Pn show

597 hurts in one regard: It had been such a simple program.

598 Its lean elegance is missing from the new

599 .Pn show ,

600 but there is no alternative;

601 supporting MIME demands higher essential complexity.

602

603

604

605 .H2 "Configure Options

606 .P

607 Customization is a double-edged sword.

608 It allows better suiting setups, but not for free.

609 There is the cost of code complexity to be able to customize.

610 There is the cost of less tested setups, because there are

611 more possible setups and especially corner cases.

612 Steve Johnson confirms:

613 .[ [

614 eric raymond the art of unix programming

615 .], p. 233]

616 .QS

617 Unless it is done very carefully, the addition of an on/off

618 configuration option can lead to a need to double the amount of

619 testing. Since in practice one never does double the amount of testing,

620 the practical effect is to reduce the amount of testing that any

621 given configuration receives. Ten options leads to 1024 times as much

622 testing, and pretty soon you are talking real reliability problems.

623 .QE

624 .LP

625 Additionally, there is the cost of choice itself.

626 The code complexity directly affects the developers.

627 Less tested code affects both users and developers.

628 The problem of choice affects the users, for once by having to choose

629 but also by more complex interfaces that require more documentation.

630 Whenever options add few advantages but increase the complexity of the

631 system, they should be considered for removal.

632 I have reduced the number of project-specific configure options from

633 15 to 3.

634

635 .U3 "Mail Transfer Facilities

636 .P

637 With the removal of the mail transfer facilities (cf. Sec.

638 .Cf mail-transfer-facilities )

639 five configure options vanished:

640 .P

641 The switches

642 .Sw --with-tls

643 and

644 .Sw --with-cyrus-sasl

645 had activated the support for transfer encryption and authentication.

646 They are not needed anymore.

647 .Ci fecd5d34f65597a4dfa16aeabea7d74b191532c3

648 .Ci 156d35f6425bea4c1ed3c4c79783dc613379c65b

649 .P

650 The configure switch

651 .Sw --enable-pop

652 had activated the message retrieval facility.

653 Whereas the code area that had been conditionally compiled in

654 for TLS and SASL support was small,

655 the conditionally compiled code area for POP support was much larger.

656 The code base had only changed slightly on toggling TLS or SASL

657 support but it had changed much on toggling POP support.

658 The changes in the code base could hardly be overviewed.

659 By having POP support togglable, a second code base had been created,

660 one that needed to be tested.

661 This situation is basically similar for the conditional TLS and SASL

662 code, but there the changes are minor and can yet be overviewed.

663 Still, conditional compilation of a code base creates variations

664 of the original program.

665 More variations require more testing and maintenance work.

666 .P

667 Two other options had only specified default configuration values:

668 .Sw --with-mts

669 defined the default transport service

670 .Ci f6aa95b724fd8c791164abe7ee5468bf5c34f226 .

671 With

672 .Sw --with-smtpservers

673 default SMTP servers could be set

674 .Ci 128545e06224233b7e91fc4c83f8830252fe16c9 .

675 Both of them became irrelevant when the SMTP transport service was removed.

676 In mmh, all messages are handed over to

677 .Pn sendmail

678 for transportation.

679

680

681 .U3 "Backup Prefix

682 .P

683 The backup prefix is the string that was prepended to message

684 filenames to tag them as deleted.

685 By default it had been the comma character (`\fL,\fP').

686 In July 2000, Kimmo Suominen introduced

687 the configure option

688 .Sw --with-hash-backup

689 to change the default to the hash character `\f(CW#\fP'.

690 This choice was probably personal preference, but,

691 being related or not, words that start with the hash character

692 introduce a comment in the Unix shell.

693 Thus, the command line

694 .Cl "rm #13 #15

695 calls

696 .Pn rm

697 without arguments because the first hash character starts a comment

698 that reaches until the end of the line.

699 To delete the backup files,

700 .Cl "rm ./#13 ./#15"

701 needs to be used.

702 Thus, using the hash as backup prefix may be seen as a precaution

703 against backup loss.

704 .P

705 First, I removed the configure option but added the profile entry

706 .Pe Backup-Prefix ,

707 which allowed to specify an arbitrary string as backup prefix

708 .Ci 6c40d481d661d532dd527eaf34cebb6d3f8ed086 .

709 This change did not remove the choice but moved it to a location where

710 it suited better, in my eyes.

711 .P

712 Eventually however, the new trash folder concept

713 (cf. Sec.

714 .Cf trash-folder )

715 removed the need for the backup prefix completely.

716 .Ci 8edc5aaf86f9f77124664f6801bc6c6cdf258173

717 .Ci ca0b3e830b86700d9e5e31b1784de2bdcaf58fc5

718

719

720 .U3 "Editor and Pager

721 .Id editor-pager

722 .P

723 The two configure options

724 .CW --with-editor=EDITOR

725 .CW --with-pager=PAGER

726 were used to specify the default editor and pager at configure time.

727 Doing so at configure time made sense in the eighties,

728 when the set of available editors and pagers varied much across

729 different systems.

730 Today, the situation is more homogeneous.

731 The programs

732 .Pn vi

733 and

734 .Pn more

735 can be expected to be available on every Unix system,

736 as they are specified by POSIX since two decades.

737 (The specifications for

738 .Pn vi

739 and

740 .Pn more

741 appeared in

742 .[

743 posix 1987

744 .]

745 and,

746 .[

747 posix 1992

748 .]

749 respectively.)

750 As a first step, these two tools were hard-coded as defaults

751 .Ci 5d43a99db70c12a673028c7758c20cbe3e13ef5f .

752 Not changed were the

753 .Pe editor

754 and

755 .Pe moreproc

756 profile entries, which allowed the user to override the system defaults.

757 Later, the concept was reworked again to respect the standard

758 environment variables

759 .Ev VISUAL

760 and

761 .Ev PAGER

762 if they are set.

763 Today, mmh determines the editor to use in the following order,

764 taking the first available and non-empty item

765 .Ci f85f4b7ae62e3d05a945dcd46ead51f0a2a89a9b :

766 .LI 1

767 Environment variable

768 .Ev MMHEDITOR

769 .LI 2

770 Profile entry

771 .Pe Editor

772 .LI 3

773 Environment variable

774 .Ev VISUAL

775 .LI 4

776 Environment variable

777 .Ev EDITOR

778 .LI 5

779 Command

780 .Pn vi .

781 .LP

782 The pager to use is determined in a similar order

783 .Ci 0c4214ea2aec6497d0d67b436bbee9bc1d225f1e :

784 .LI 1

785 Environment variable

786 .Ev MMHPAGER

787 .LI 2

788 Profile entry

789 .Pe Pager

790 (replaces

791 .Pe moreproc )

792 .LI 3

793 Environment variable

794 .Ev PAGER

795 .LI 4

796 Command

797 .Pn more .

798 .LP

799 By respecting the

800 .Ev VISUAL /\c

801 .Ev EDITOR

802 and

803 .Ev PAGER

804 environment variables,

805 the new behavior complies with the common style on Unix systems.

806 It is more uniform and clearer for users.

807

808

809 .U3 "ndbm

810 .P

811 .Pn slocal

812 used to depend on the database library

813 .I ndbm .

814 The database is used to store the

815 .Hd Message-ID

816 header field values of all messages delivered.

817 This enabled

818 .Pn slocal

819 to suppress delivering the same message to the same user twice.

820 This features was enabled by the

821 .Sw -suppressdup

822 switch.

823 .P

824 Because a variety of versions of the database library exist,

825 .[

826 wolter unix incompat notes dbm

827 .]

828 complicated autoconf code was needed to detect them correctly.

829 Furthermore, the configure switches

830 .Sw --with-ndbm=ARG

831 and

832 .Sw --with-ndbmheader=ARG

833 were added to help with difficult setups that would

834 not be detected automatically or not correctly.

835 .P

836 By removing the suppress duplicates feature of

837 .Pn slocal ,

838 the dependency on

839 .I ndbm

840 vanished and 120 lines of complex autoconf code could be saved

841 .Ci ecd6d6a20cb7a1507e3a20d6c4cb3a1cf14c6bbf .

842 The change removed functionality but that is considered minor to the

843 improvement of dropping the dependency and the complex autoconf code.

844

845

846 .U3 "MH-E Support

847 .P

848 The configure option

849 .Sw --disable-mhe

850 was removed when the MH-E support was reworked.

851 MH-E is the Emacs front-end to MH.

852 It requires MH to provide minor additional functions.

853 The

854 .Sw --disable-mhe

855 configure option had switched off these extensions.

856 After removing the support for old versions of MH-E,

857 only the

858 .Sw -build

859 switches of

860 .Pn forw

861 and

862 .Pn repl

863 are left to be MH-E extensions.

864 They are now always built in because they add little code and complexity.

865 In consequence, the

866 .Sw --disable-mhe

867 configure option was removed

868 .Ci a7ce7b4a580d77b6c2c4d980812beb589aa4c643 .

869 Dropping the option also removed a variant of the code base

870 that would have needed to be tested.

871 This change was undertaken in January 2012 in nmh and

872 thereafter merged into mmh.

873

874

875 .U3 "Masquerading

876 .P

877 The configure option

878 .Sw --enable-masquerade

879 could take up to three arguments:

880 .CW draft_from ,

881 .CW mmailid ,

882 and

883 .CW username_extension .

884 They activated different types of address masquerading.

885 All of them were implemented in the SMTP-speaking

886 .Pn post

887 command.

888 Address masquerading is an MTA's task and mmh does not cover

889 this field anymore.

890 Hence, true masquerading needs to be implemented in the external MTA.

891 .P

892 The

893 .I mmailid

894 masquerading type is the oldest one of the three and the only one

895 available in the original MH.

896 It provided a

897 .I username

898 to

899 .I fakeusername

900 mapping, based on the

901 .Fn passwd 's

902 GECOS field.

903 Nmh's man page

904 .Mp mh-tailor (5)

905 described the use case as being the following:

906 .QS

907 This is useful if you want the messages you send to always

908 appear to come from the name of an MTA alias rather than your

909 actual account name. For instance, many organizations set up

910 `First.Last' sendmail aliases for all users. If this is

911 the case, the GECOS field for each user should look like:

912 ``First [Middle] Last <First.Last>''

913 .QE

914 .P

915 As mmh sends outgoing mail via the local MTA only,

916 the best location to do such global rewrites is there.

917 Besides, the MTA is conceptionally the right location because it

918 does the reverse mapping for incoming mail (aliasing), too.

919 Furthermore, masquerading set up there is readily available for all

920 mail software on the system.

921 Hence, mmailid masquerading was removed.

922 .Ci 0836c8000ccb34b59410ef1c15b1b7feac70ce5f

923 .P

924 The

925 .I username_extension

926 masquerading type did not replace the username but would append a suffix,

927 specified by the

928 .Ev USERNAME_EXTENSION

929 environment variable, to it.

930 This provided support for the

931 .I user-extension

932 feature of qmail

933 .[ [

934 sill qmail handbook

935 .], p. 141]

936 and the similar

937 .I "plussed user

938 processing of Sendmail.

939 .[ [

940 sendmail costales

941 .], p. 476]

942 The decision to remove this username_extension masquerading

943 was motivated by the fact that

944 .Pn spost

945 had not supported it yet.

946 Username extensions can be used in mmh, but less convenient.

947 .\" XXX In the format file: %(getenv USERNAME_EXTENSION)

948 .Ci 2abae0bfd0ad5bf898461e50aa4b466d641f23d9

949 .P

950 The

951 .I draft_from

952 masquerading type instructed

953 .Pn post

954 to use the value of the

955 .Hd From

956 header field as SMTP envelope sender.

957 Sender addresses could be replaced completely.

958 Mmh offers a kind of masquerading similar in effect, but

959 with technical differences.

960 As mmh does not transfer messages itself, the local MTA has final control

961 over the sender's address.

962 Any masquerading mmh introduces may be reverted by the MTA.

963 In times of pedantic spam checking, an MTA will take care to use

964 sensible envelope sender addresses to keep its own reputation up.

965 Nonetheless, the MUA can set the

966 .Hd From

967 header field and thereby propose a sender address to the MTA.

968 The MTA may then decide to take that one or generate the canonical sender

969 address for use as envelope sender address.

970 .Ci b14ea6073f77b4359aaf3fddd0e105989db9

971 .P

972 In mmh, the MTA will always extract the recipient and sender from the

973 message header (\c

974 .Pn sendmail 's

975 .Sw -t

976 switch).

977 The

978 .Hd From

979 header field of the draft may be set arbitrary by the user.

980 If it is missing, the canonical sender address will be generated by the MTA.

981

982 .U3 "Remaining Options

983 .P

984 Two configure options remain in mmh.

985 One of them is the file locking method to use:

986 .Sw --with-locking=[dot|fcntl|flock|lockf] .

987 The idea of removing all methods except the portable

988 .I "dot locking

989 and having that one as the default is appealing, but this change

990 requires deeper technical investigation into the topic.

991 The other remaining option,

992 .Sw --enable-debug ,

993 compiles the programs with debugging symbols.

994 This option is likely to stay.

995

996

997

998

999 .H2 "Command Line Switches

1000 .P

1001 The command line switches of MH tools are similar in style to the

1002 switches in the X Window System.

1003 They consist of a single dash (`\fL-\fP') followed by a word.

1004 For example

1005 .Cl -truncate .

1006 To ease typing, the switch can be abbreviated, given the remaining

1007 prefix is unambiguous.

1008 If no other switch starts with the letter `t', then any of

1009 .Cl "-truncate" ,

1010 .Cl "-trunc" ,

1011 .Cl "-tr" ,

1012 and

1013 .Cl "-t

1014 is equal.

1015 As a result, switches can neither be grouped (as in

1016 .Cl "ls -ltr" )

1017 nor can switch arguments be appended directly to the switch itself

1018 (as in

1019 .Cl "sendmail -q30m" ).

1020 Many switches have negating counter-parts, which start with `no'.

1021 For example

1022 .Cl "-notruncate

1023 inverts the

1024 .Cl "-truncate

1025 switch.

1026 They exist to override the effect of default switches in the profile.

1027 Every program in mmh has two generic switches:

1028 .Sw -help ,

1029 to print a short message on how to use the program, and

1030 .Sw -Version

1031 (with capital `V'), to tell what version of mmh the program belongs to.

1032 .P

1033 Switches change the behavior of programs.

1034 Programs that do one thing in one way require no switches.

1035 In most cases, doing something in exactly one way is too limiting.

1036 If one task should be accomplished in various ways,

1037 switches are a good approach to alter the behavior of a program.

1038 Changing the behavior of programs provides flexibility and customization

1039 to users, but at the same time it complicates the code,

1040 the documentation, and the usage of the program.

1041 Therefore, the number of switches should be kept small.

1042 A small set of well-chosen switches is best.

1043 Usually, the number of switches increases over time.

1044 Already in 1985, Rose and Romine have identified this as a major

1045 problem of MH:

1046 .[ [

1047 rose romine real work

1048 .], p. 12]

1049 .QS

1050 A complaint often heard about systems which undergo substantial development

1051 by many people over a number of years, is that more and more options are

1052 introduced which add little to the functionality but greatly increase the

1053 amount of information a user needs to know in order to get useful work done.

1054 This is usually referred to as creeping featurism.

1055 .QP

1056 Unfortunately MH, having undergone six years of off-and-on development by

1057 ten or so well-meaning programmers (the present authors included),

1058 suffers mightily from this.

1059 .QE

1060 .P

1061 Being reluctant to adding new switches (or \fIoptions\fP,

1062 as Rose and Romine call them) is one part of a counter-action,

1063 the other part is removing hardly used switches.

1064 Nmh's tools have lots of switches already implemented.

1065 Hence, cleaning up by removing some of them was the more important part

1066 of the counter-action.

1067 Removing existing functionality is always difficult because it

1068 breaks programs that use these functions.

1069 Also, for every obsolete feature, there'll always be someone who still

1070 uses it and thus opposes its removal.

1071 This puts the developer into the position,

1072 where sensible improvements to style are regarded as destructive acts.

1073 Yet, living with the featurism is far worse, in my eyes, because

1074 future needs will demand adding further features,

1075 worsening the situation more and more.

1076 Rose and Romine added in a footnote,

1077 ``[...]

1078 .Pn send

1079 will no doubt acquire an endless number of switches in the years to come''

1080 .[ [

1081 rose romine real work

1082 .], p. 12].

1083 Although clearly humorous, the comment points to the nature of the problem.

1084 Refusing to add any new switches would encounter the problem at its root,

1085 but this is not practical.

1086 New needs will require new switches and it would be unwise to block

1087 them strictly.

1088 Nevertheless, removing obsolete switches still is an effective approach

1089 to deal with the problem.

1090 Working on an experimental branch without an established user base,

1091 eased my work because I did not offend users when I removed existing

1092 functions.

1093 .P

1094 Rose and Romine counted 24 visible and 9 more hidden switches for

1095 .Pn send .

1096 In nmh, they increased up to 32 visible and 12 hidden ones.

1097 At the time of writing, no more than 4 visible switches and 1 hidden switch

1098 have remained in mmh's

1099 .Pn send .

1100 These numbers include the two generic switches,

1101 .Sw -help

1102 and

1103 .Sw -Version .

1104 .P

1105 Hidden switches are ones not documented.

1106 In mmh, 12 tools have hidden switches.

1107 9 of them are

1108 .Sw -debug

1109 switches, the other 6 provide special interfaces for internal use.

1110 .P

1111 The following figure displays the number of switches for each of the tools

1112 that is available in both nmh and mmh.

1113 The tools are sorted by the number of switches they had in nmh.

1114 Both visible and hidden switches were counted,

1115 but not the generic help and version switches.

1116 Whereas in the beginning of the project, the average tool had 11 switches,

1117 now it has no more than 5 \(en only half as many.

1118 If the `no' switches and similar inverse variant are folded onto

1119 their counter-parts, the average tool had 8 switches in pre-mmh times and

1120 has 4 now.

1121 The total number of functional switches in mmh dropped from 465

1122 to 233.

1123

1124 .sp

1125 .KS

1126 .in 1c

1127 .so input/switches.grap

1128 .KE

1129 .sp

1130

1131 .P

1132 .ZZ

1133 A part of the switches vanished after functions were removed.

1134 This was the case for network mail transfer, for instance.

1135 Sometimes, however, the work flow was the other way:

1136 I looked through the

1137 .Mp mh-chart (7)

1138 man page to identify the tools with apparently too many switches.

1139 Then I valued the benefit of each switch by examining

1140 the tool's man page and source code, aided by literature research

1141 and testing.

1142

1143

1144 .U3 "Draft Folder Facility

1145 .P

1146 A change early in the project was the complete transition from

1147 the single draft message to the draft folder facility

1148 .Ci 337338b404931f06f0db2119c9e145e8ca5a9860

1149 (cf. Sec.

1150 .Cf draft-folder ).

1151 The draft folder facility was introduced in the mid-eighties, when

1152 Rose and Romine called it a ``relatively new feature''.

1153 .[

1154 rose romine real work

1155 .]

1156 Since then, the facility was included, inactive by default.

1157 By making it permanently active and by related rework of the tools, the

1158 .Sw -[no]draftfolder ,

1159 and

1160 .Sw -draftmessage

1161 switches could be removed from

1162 .Pn comp ,

1163 .Pn repl ,

1164 .Pn forw ,

1165 .Pn dist ,

1166 .Pn whatnow ,

1167 and

1168 .Pn send

1169 .Ci 337338b404931f06f0db2119c9e145e8ca5a9860 .

1170 The only flexibility lost with this change is having multiple

1171 draft folders within one profile.

1172 I consider this a theoretical problem only.

1173 At the same time, the

1174 .Sw -draft

1175 switch of

1176 .Pn anno ,

1177 .Pn refile ,

1178 and

1179 .Pn send

1180 was removed.

1181 The special treatment of \fIthe\fP draft message became irrelevant after

1182 the rework of the draft system

1183 (cf. Sec.

1184 .Cf draft-folder ).

1185

1186

1187 .U3 "In Place Editing

1188 .P

1189 .Pn anno

1190 had the switches

1191 .Sw -[no]inplace

1192 to either annotate the message in place and thus preserve hard links,

1193 or annotate a copy to replace the original message.

1194 The latter approach broke hard links.

1195 Following the assumption that linked messages should truly be the

1196 same message and annotating it should not break the link, the

1197 .Sw -[no]inplace

1198 switches were removed and the previous default

1199 .Sw -inplace

1200 was made the definitive behavior

1201 .Ci c8195849d2e366c569271abb0f5f60f4ebf0b4d0 .

1202 The

1203 .Sw -[no]inplace

1204 switches of

1205 .Pn repl ,

1206 .Pn forw ,

1207 and

1208 .Pn dist

1209 could be removed, as well, as they were simply passed through to

1210 .Pn anno .

1211 .P

1212 .Pn burst

1213 also had

1214 .Sw -[no]inplace

1215 switches, but with a different meaning.

1216 With

1217 .Sw -inplace ,

1218 the digest had been replaced by the table of contents (i.e. the

1219 introduction text) and the burst messages were placed right

1220 after this message, renumbering all following messages.

1221 Also, any trailing text of the digest was lost, though,

1222 in practice, it usually consists of an end-of-digest marker only.

1223 Nonetheless, this behavior appeared less elegant than the

1224 .Sw -noinplace

1225 behavior, which already had been the default.

1226 Nmh's

1227 .Mp burst (1)

1228 man page reads:

1229 .QS

1230 If

1231 .Sw -noinplace

1232 is given, each digest is preserved, no table

1233 of contents is produced, and the messages contained within

1234 the digest are placed at the end of the folder. Other messages

1235 are not tampered with in any way.

1236 .QE

1237 .LP

1238 The decision to drop the

1239 .Sw -inplace

1240 behavior was supported by the code complexity and the possible data loss

1241 it caused.

1242 .Sw -noinplace

1243 was chosen to be the definitive behavior.

1244 .Ci 68a686adeb39223a5e1ad35e4a24890ec053679d

1245

1246

1247 .U3 "Forms and Format Strings

1248 .P

1249 Historically, the tools that had

1250 .Sw -form

1251 switches to supply a form file had

1252 .Sw -format

1253 switches as well to supply the contents of a form file as a string

1254 on the command line directly.

1255 In consequence, the following two lines equaled:

1256 .VS

1257 scan -form scan.mailx

1258 scan -format "`cat /path/to/scan.mailx`"

1259 VE

1260 The

1261 .Sw -format

1262 switches were dropped in favor for extending the

1263 .Sw -form

1264 switches

1265 .Ci f51956be123db66b00138f80464d06f030dbb88d .

1266 If their argument starts with an equal sign (`\fL=\fP'),

1267 then the rest of the argument is taken as a format string,

1268 otherwise the arguments is treated as the name of a format file.

1269 Thus, now the following two lines equal:

1270 .VS

1271 scan -form scan.mailx

1272 scan -form "=`cat /path/to/scan.mailx`"

1273 VE

1274 This rework removed the prefix collision between

1275 .Sw -form

1276 and

1277 .Sw -format .

1278 Typing `\fL-fo\fP' is sufficient to specify form file or format string.

1279 .P

1280 The different meaning of

1281 .Sw -format

1282 for

1283 .Pn forw

1284 and

1285 .Pn repl

1286 was removed in mmh.

1287 .Pn forw

1288 was completely switched to MIME-type forwarding, thus removing the

1289 .Sw -[no]format

1290 .Ci 6e271608b7b9c23771523f88d23a4d3593010cf1 .

1291 For

1292 .Pn repl ,

1293 the

1294 .Sw -[no]format

1295 switches were reworked to

1296 .Sw -[no]filter

1297 switches

1298 .Ci 67411b1f95d6ec987b4c732459e1ba8a8ac192c6 .

1299 The

1300 .Sw -format

1301 switches of

1302 .Pn send

1303 and

1304 .Pn post ,

1305 which had a third meaning, were removed likewise

1306 .Ci f3cb7cde0e6f10451b6848678d95860d512224b9 .

1307 Eventually, the ambiguity of the

1308 .Sw -format

1309 switches is resolved by not having such switches anymore in mmh.

1310

1311

1312 .U3 "MIME Tools

1313 .P

1314 The MIME tools, which once were part of

1315 .Pn mhn

1316 (whatever that stood for),

1317 had several switches that added little practical value to the programs.

1318 The

1319 .Sw -[no]realsize

1320 switches of

1321 .Pn mhbuild

1322 and

1323 .Pn mhlist

1324 were removed

1325 .Ci 8d8f1c3abc586c005c904e52c4adbfe694d2201c .

1326 Real size calculations are done always now because nmh's

1327 .Mp mhbuild (1)

1328 man page states that

1329 ``This provides an accurate count at the expense of a small delay''

1330 with the small delay not being noticeable on modern systems.

1331 .P

1332 The

1333 .Sw -[no]check

1334 switches were removed together with the support for

1335 .Hd Content-MD5

1336 header fields [RFC\|1864]

1337 (cf. Sec.

1338 .Cf content-md5 )

1339 .Ci 31dc797eb5178970d68962ca8939da3fd9a8efda .

1340 .P

1341 The

1342 .Sw -[no]ebcdicsafe

1343 and

1344 .Sw -[no]rfc934mode

1345 switches of

1346 .Pn mhbuild

1347 were removed because they are considered obsolete

1348 .Ci 01a3480928da485b4d6109d36d751dfa71799d58

1349 .Ci 3363e2624dce0eb8164cf8b3f1ab385c8ff72e88 .

1350 .P

1351 Content caching of external MIME parts, activated with the

1352 .Sw -rcache

1353 and

1354 .Sw -wcache

1355 switches was completely removed

1356 .Ci d1fefd9f614e4dc3cda16da6c69133c1b2005269 .

1357 External MIME parts are rare today, having a caching facility

1358 for them appears to be unnecessary.

1359 .P

1360 In pre-MIME times,

1361 .Pn mhl

1362 had covered many tasks that are part of MIME handling today.

1363 Therefore,

1364 .Pn mhl

1365 could be simplified to a large extend, reducing the number of its

1366 switches from 21 to 6

1367 .Ci 350ad6d3542a07639213cf2a4fe524e829c1e7b6

1368 .Ci 0e46503be3c855bddaeae3843e1b659279c35d70 .

1369

1370

1371

1372

1373 .U3 "Header Printing

1374 .P

1375 .Pn folder 's

1376 data output is self-explaining enough that

1377 displaying the header line makes little sense.

1378 Hence, the

1379 .Sw -[no]header

1380 switch was removed and headers are never printed

1381 .Ci 601cc73d1fa05ce96faa728f036d6c51b91701c7 .

1382 .P

1383 In

1384 .Pn mhlist ,

1385 the

1386 .Sw -[no]header

1387 switches were removed, as well

1388 .Ci b24f96523aaf60e44e04a3ffb1d22e69a13a602f .

1389 In this case, the headers are printed always because the output

1390 is not self-explaining.

1391 .P

1392 .Pn scan

1393 also had

1394 .Sw -[no]header

1395 switches.

1396 Printing this header had been sensible until the introduction of

1397 format strings made it impossible to display column headings.

1398 Only the folder name and the current date remained to be printed.

1399 As this information can be perfectly generated with

1400 .Pn folder

1401 and

1402 .Pn date ,

1403 the switches were removed

1404 .Ci c477dc5d1d03fa6d9a8ab3dd3508c63cbddc044e .

1405 .P

1406 By removing all

1407 .Sw -header

1408 switches, the collision with

1409 .Sw -help

1410 on the first two letters was resolved.

1411 Currently,

1412 .Sw -h

1413 evaluates to

1414 .Sw -help

1415 for all tools of mmh.

1416

1417

1418 .U3 "Suppressing Edits or the Invocation of the WhatNow Shell

1419 .P

1420 The

1421 .Sw -noedit

1422 switch of

1423 .Pn comp ,

1424 .Pn repl ,

1425 .Pn forw ,

1426 .Pn dist ,

1427 and

1428 .Pn whatnow

1429 was removed and replaced by the ability to specify

1430 .Sw -editor

1431 with an empty argument

1432 .Ci 75fca31a5b9d5c1a99c74ab14c94438d8852fba9 .

1433 (Using

1434 .Cl "-editor /bin/true

1435 is nearly the same. It differs only in setting the previous editor.)

1436 .P

1437 The more important change is the removal of the

1438 .Sw -nowhatnowproc

1439 switch

1440 .Ci ee4f43cf2ef0084ec698e4e87159a94c01940622 .

1441 This switch had once introduced an awkward behavior,

1442 as explained in nmh's man page for

1443 .Mp comp (1):

1444 .QS

1445 The

1446 .Sw -editor

1447 .Ar editor

1448 switch indicates the editor to use for

1449 the initial edit. Upon exiting from the editor,

1450 .Pn comp

1451 will invoke the

1452 .Pn whatnow

1453 program. See

1454 .Mp whatnow (1)

1455 for a discussion of available options.

1456 The invocation of this program can be

1457 inhibited by using the

1458 .Sw -nowhatnowproc

1459 switch. (In truth of fact, it is the

1460 .Pn whatnow

1461 program which starts the initial edit.

1462 Hence,

1463 .Sw \%-nowhatnowproc

1464 will prevent any edit from occurring.)

1465 .QE

1466 .P

1467 Effectively, the

1468 .Sw -nowhatnowproc

1469 switch caused only only a draft message to be created.

1470 As

1471 .Cl "-whatnowproc /bin/true

1472 does the same, the

1473 .Sw -nowhatnowproc

1474 switch was removed for being redundant.

1475

1476

1477

1478 .U3 "Various

1479 .BU

1480 With the removal of MMDF maildrop format support,

1481 .Pn packf

1482 and

1483 .Pn rcvpack

1484 no longer needed the

1485 .Sw -mbox

1486 and

1487 .Sw -mmdf

1488 switches.

1489 The behavior of

1490 .Sw -mbox

1491 became the definitive behavior

1492 .Ci 3916ab66ad5d183705ac12357621ea8661afd3c0 .

1493 Further rework in both tools made the

1494 .Sw -file

1495 switch unnecessary

1496 .Ci ca1023716d4c2ab890696f3e41fa0d94267a940e .

1497

1498 .BU

1499 Mmh's tools do no longer clear the screen (\c

1500 .Pn scan 's

1501 and

1502 .Pn mhl 's

1503 .Sw -[no]clear

1504 switches

1505 .Ci e57b17343dcb3ff373ef4dd089fbe778f0c7c270

1506 .Ci 943765e7ac5693ae177fd8d2b5a2440e53ce816e ).

1507 The message formating tool

1508 .Pn mhl

1509 does neither ring the bell (\c

1510 .Sw -[no]bell

1511 .Ci e11983f44e59d8de236affa5b0d0d3067c192e24 )

1512 nor does it page the output itself (\c

1513 .Sw -length

1514 .Ci 5b9d883db0318ed2b84bb82dee880d7381f99188 )

1515 anymore.

1516 Generally, the pager to use is no longer specified with the

1517 .Sw -[no]moreproc

1518 command line switches for

1519 .Pn mhl

1520 and

1521 .Pn show /\c

1522 .Pn mhshow

1523 .Ci 39e87a75b5c2d3572ec72e717720b44af291e88a .

1524

1525 .BU

1526 In order to avoid prefix collisions among switch names, the

1527 .Sw -version

1528 switch was renamed to

1529 .Sw -Version

1530 (with capital `V')

1531 .Ci 32b2354dbaf4bf934936eb5b102a4a3d2fdd209a .

1532 Every program has the

1533 .Sw -version

1534 switch but its first three letters collided with the

1535 .Sw -verbose

1536 switch, present in many programs.

1537 The rename solved this problem once for all.

1538 Although this rename breaks a basic interface, having the

1539 .Sw -V

1540 abbreviation to display the version information, isn't all too bad.

1541

1542 .BU

1543 .Sw -[no]preserve

1544 of

1545 .Pn refile

1546 was removed

1547 .Ci 8edc5aaf86f9f77124664f6801bc6c6cdf258173

1548 because what use was it anyway?

1549 Quoting nmh's man page

1550 .Mp refile (1):

1551 .QS

1552 Normally when a message is refiled, for each destination

1553 folder it is assigned the number which is one above the current

1554 highest message number in that folder. Use of the

1555 .Sw -preserv

1556 [sic!] switch will override this message renaming, and try

1557 to preserve the number of the message. If a conflict for a

1558 particular folder occurs when using the

1559 .Sw -preserve

1560 switch, then

1561 .Pn refile

1562 will use the next available message number which

1563 is above the message number you wish to preserve.

1564 .QE

1565

1566 .BU

1567 The removal of the

1568 .Sw -[no]reverse

1569 switches of

1570 .Pn scan

1571 .Ci 8edc5aaf86f9f77124664f6801bc6c6cdf258173

1572 is a bug fix.

1573 This is supported by the comments

1574 ``\-[no]reverse under #ifdef BERK (I really HATE this)''

1575 by Rose and

1576 ``Lists messages in reverse order with the `\-reverse' switch.

1577 This should be considered a bug'' by Romine in the changelogs.

1578 The question remains why neither Rose nor Romine have fixed this

1579 bug in the eighties when they wrote these comments.

1580

1581

1582

1583

1584

1585

1586

1587 .\" --------------------------------------------------------------

1588 .H1 "Modernizing

1589 .P

1590 In the more than thirty years of MH's existence, its code base was

1591 increasingly extended.

1592 New features entered the project and became alternatives to the

1593 existing behavior.

1594 Relics from several decades have gathered in the code base

1595 but seldom obsolete features were dropped.

1596 This section describes the removing of old code

1597 and the modernizing of the default setup.

1598 It focuses on the functional aspect only;

1599 the non-functional aspects of code style are discussed in Sec.

1600 .Cf code-style .

1601

1602

1603 .H2 "Code Relics

1604 .P

1605 My position regarding the removal of obsolete code

1606 is much more revolutional than the nmh community appreciates.

1607 Working on an experimental version, I was able to quickly drop

1608 functionality that I considered ancient.

1609 The need for consensus with peers would have slowed this process down.

1610 Without the need to justify my decisions, I was able to rush forward.

1611 .P

1612 In December 2011, Paul Vixie motivated the nmh developers to just

1613 do the work:

1614 .[

1615 paul vixie edginess nmh-workers

1616 .]

1617 .QS

1618 let's stop walking on egg shells with this code base. there's no need to

1619 discuss whether to keep using vfork, just note in [sic!] passing, [...]

1620 we don't need a separate branch for removing vmh

1621 or ridding ourselves of #ifdef's or removing posix replacement functions

1622 or depending on pure ansi/posix ``libc''.

1623 .QP

1624 these things should each be a day or two of work and the ``main branch''

1625 should just be modern. [...]

1626 let's push forward, aggressively.

1627 .QE

1628 .LP

1629 I did so already in the months before.

1630 I pushed forward.

1631 I simply dropped the cruft.

1632 .P

1633 The decision to drop a feature was based on literature research and

1634 careful thinking, but whether having had contact with this particular

1635 feature within my own computer life served as a rule of thumb.

1636 I explained my reasons in the commit messages

1637 in the version control system.

1638 Hence, others can comprehend my view and argue for undoing the change

1639 if I have missed an important aspect.

1640 I was quick in dropping parts.

1641 I rather include falsely dropped parts again, than going at a slower pace.

1642 Mmh is experimental work; it requires tough decisions.

1643

1644

1645 .U3 "Process Forking

1646 .P

1647 Being a tool chest, MH creates many processes.

1648 In earlier times

1649 .Fu fork()

1650 had been an expensive system call, because the process's image needed

1651 to be completely duplicated at once.

1652 This expensive work was especially unnecessary in the commonly occurring

1653 case wherein the image is replaced by a call to

1654 .Fu exec()

1655 right after having forked the child process.

1656 The

1657 .Fu vfork()

1658 system call was invented to speed up this particular case.

1659 It completely omits the duplication of the image.

1660 On old systems this resulted in significant speed ups.

1661 Therefore MH used

1662 .Fu vfork()

1663 whenever possible.

1664 .P

1665 Modern memory management units support copy-on-write semantics, which make

1666 .Fu fork()

1667 almost as fast as

1668 .Fu vfork() .

1669 The man page of

1670 .Mp vfork (2)

1671 in FreeBSD 8.0 states:

1672 .QS

1673 This system call will be eliminated when proper system sharing mechanisms

1674 are implemented. Users should not depend on the memory sharing semantics

1675 of vfork() as it will, in that case, be made synonymous to fork(2).

1676 .QE

1677 .LP

1678 Vixie supports the removal with the note that ``the last

1679 system on which fork was so slow that an mh user would notice it, was

1680 Eunice. that was 1987''.

1681 .[

1682 nmh-workers vixie edginess

1683 .]

1684 I replaced all calls to

1685 .Fu vfork()

1686 with calls to

1687 .Fu fork()

1688 .Ci 40821f5c1316e9205a08375e7075909cc9968e7d .

1689 .P

1690 Related to the costs of

1691 .Fu fork()

1692 is the probability of its success.

1693 In the eighties, on heavy loaded systems, calls to

1694 .Fu fork()

1695 were prone to failure.

1696 Hence, many of the

1697 .Fu fork()

1698 calls in the code were wrapped into loops to retry the

1699 .Fu fork()

1700 several times, to increase the chances to succeed eventually.

1701 On modern systems, a failing

1702 .Fu fork()

1703 call is unusual.

1704 Hence, in the rare case when

1705 .Fu fork()

1706 fails, mmh programs simply abort

1707 .Ci 5fbf37ee68e018998ada61eeab73e035b26834b6 .

1708

1709

1710 .U3 "Header Fields

1711 .BU

1712 The

1713 .Hd Encrypted

1714 header field was introduced by RFC\|822,

1715 but already marked as legacy in RFC\|2822.

1716 Today, OpenPGP provides the basis for standardized exchange of encrypted

1717 messages [RFC\|4880, RFC\|3156].

1718 Hence, the support for

1719 .Hd Encrypted

1720 header fields is removed in mmh

1721 .Ci 064527f7b57ab050e5af13e15ad99aeeab125857 .

1722 .BU

1723 The native support for

1724 .Hd Face

1725 header fields has been removed, as well

1726 .Ci 8e5be81f784682822f5e868c1bf3c8624682bd23 .

1727 This feature is similar to the

1728 .Hd X-Face

1729 header field in its intent,

1730 but takes a different approach to store the image.

1731 Instead of encoding the image data directly into the header field,

1732 it contains the hostname and UDP port where the image

1733 date can be retrieved.

1734 There is even a third Face system,

1735 which is the successor of

1736 .Hd X-Face ,

1737 although it re-uses the

1738 .Hd Face

1739 header field name.

1740 It was invented in 2005 and supports colored PNG images.

1741 None of the Face systems described here is popular today.

1742 Hence, mmh has no direct support for them.

1743 .BU

1744 .Id content-md5

1745 The

1746 .Hd Content-MD5

1747 header field was introduced by RFC\|1864.

1748 It provides detection of data corruption during the transfer.

1749 But it can not ensure verbatim end-to-end delivery of the contents

1750 [RFC\|1864].

1751 The proper approach to verify content integrity in an

1752 end-to-end relationship is the use of digital signatures [RFC\|4880].

1753 On the other hand, transfer protocols should detect corruption during

1754 the transmission.

1755 The TCP includes a checksum field therefore.

1756 These two approaches in combinations render the

1757 .Hd Content-MD5

1758 header field superfluous.

1759 Not a single one out of 4\|200 messages from two decades

1760 in the nmh-workers mailing list archive

1761 .[

1762 nmh-workers mailing list archive website

1763 .]

1764 contained a

1765 .Hd Content-MD5

1766 header field.

1767 Neither did any of the 60\|000 messages in my personal mail storage.

1768 Removing the support for this header field

1769 .Ci 31dc797eb5178970d68962ca8939da3fd9a8efda ,

1770 removed the last place where MD5 computation was needed.

1771 Hence, the MD5 code could be removed as well.

1772 Over 500 lines of code vanished by this one change.

1773

1774

1775 .U3 "MMDF maildrop support

1776 .P

1777 This type of maildrop format is conceptionally similar to the mbox format,

1778 but uses a different message delimiter (`\fL\\1\\1\\1\\1\fP',

1779 commonly written as `\fL^A^A^A^A\fP', instead of `\fLFrom\0\fP').

1780 Mbox is the de-facto standard maildrop format on Unix,

1781 whereas the MMDF maildrop format is now forgotten.

1782 Mbox remains as the only packed mailbox format, supported in mmh.

1783 .P

1784 The simplifications within the code were moderate.

1785 Mainly, the reading and writing of MMDF mailbox files was removed.

1786 But also, switches of

1787 .Pn packf

1788 and

1789 .Pn rcvpack

1790 could be removed

1791 .Ci 3916ab66ad5d183705ac12357621ea8661afd3c0 .

1792 In the message parsing function

1793 .Fn sbr/m_getfld.c ,

1794 knowledge of MMDF packed mail boxes was removed

1795 .Ci 684ec30d81e1223a282764452f4902ed4ad1c754 .

1796 Further code structure simplifications may be possible there,

1797 because only one single packed mailbox format is left to be supported.

1798 I have not worked on them yet because

1799 .Fu m_getfld()

1800 is heavily optimized and thus dangerous to touch.

1801 The risk of damaging the intricate workings of the optimized code is

1802 too high.

1803

1804

1805 .U3 "Prompter's Control Keys

1806 .P

1807 The program

1808 .Pn prompter

1809 queries the user to fill in a message form.

1810 When used as

1811 .Cl "comp -editor prompter" ,

1812 the resulting behavior is similar to

1813 .Pn mailx .

1814 Apparently,

1815 .Pn prompter

1816 had not been touched lately.

1817 Otherwise it's hardly explainable why it

1818 still offered the switches

1819 .Sw -erase

1820 .Ar chr

1821 and

1822 .Sw -kill

1823 .Ar chr

1824 to name the characters for command line editing.

1825 The times when this had been necessary are long time gone.

1826 Today these things work out-of-the-box, and if not, are configured

1827 with the standard tool

1828 .Pn stty .

1829 The switches are removed now

1830 .Ci 0bd9750710cdbab80cfb4036dd87af20afe1552f .

1831

1832

1833 .U3 "Hardcopy Terminal Support

1834 .P

1835 More of a funny anecdote is a check for being connected to a

1836 hardcopy terminal.

1837 It remained in the code until spring 2012, when I finally removed it

1838 .Ci b7764c4a6b71d37918a97594d866258f154017ca .

1839 The check only prevented a pager to be placed between the printing

1840 program (\c

1841 .Pn mhl )

1842 and the terminal.

1843 In nmh, this could have been ensured statically with the

1844 .Sw -nomoreproc

1845 at the command line, too.

1846 In mmh, setting the profile entry

1847 .Pe Pager

1848 or the environment variable

1849 .Ev PAGER

1850 to

1851 .Pn cat

1852 is sufficient.

1853

1854

1855

1856

1857 .H2 "Attachments

1858 .P

1859 The mind model of email attachments is unrelated to MIME.

1860 Although the MIME RFCs [RFC\|2045\(en2049] define the technical

1861 requirements for having attachments, they do not mention the term.

1862 Instead of attachments, MIME talks about ``multi-part message bodies''

1863 [RFC\|2045], a more general concept.

1864 Multi-part messages are messages

1865 ``in which one or more different

1866 sets of data are combined in a single body''

1867 [RFC\|2046].

1868 MIME keeps its descriptions generic;

1869 it does not imply specific usage models.

1870 Today, one usage model is prevalent: attachments.

1871 The idea is having a main text document with files of arbitrary kind

1872 attached to it.

1873 In MIME terms, this is a multi-part message having a text part first

1874 and parts of arbitrary type following.

1875 .P

1876 .ZZ

1877 MH's MIME support is a direct implementation of the RFCs.

1878 The perception of the topic described in the RFCs is clearly visible

1879 in MH's implementation.

1880 As a result,

1881 MH had all the MIME features but no idea of attachments.

1882 But users do not need all the MIME features,

1883 they want convenient attachment handling.

1884

1885

1886 .U3 "Composing MIME Messages

1887 .P

1888 In order to improve the situation on the message composing side,

1889 Jon Steinhart had added an attachment system to nmh in 2002

1890 .Ci 7480dbc14bc90f2d872d434205c0784704213252 .

1891 In the file

1892 .Fn docs/README-ATTACHMENTS ,

1893 he described his motivation to do so:

1894 .QS

1895 Although nmh contains the necessary functionality for MIME message

1896 handing [sic!], the interface to this functionality is pretty obtuse.

1897 There's no way that I'm ever going to convince my partner to write

1898 .Pn mhbuild

1899 composition files!

1900 .QE

1901 .LP

1902 With this change, the mind model of attachments entered nmh.

1903 In the same document:

1904 .QS

1905 These changes simplify the task of managing attachments on draft files.

1906 They allow attachments to be added, listed, and deleted.

1907 MIME messages are automatically created when drafts with attachments

1908 are sent.

1909 .QE

1910 .LP

1911 Unfortunately, the attachment system, like every new facilities in nmh,

1912 was inactive by default.

1913 .P

1914 During my time in Argentina, I tried to improve the attachment system.

1915 But, after long discussions my patch died as a proposal on the

1916 mailing list because of great opposition in the nmh community.

1917 .[

1918 nmh-workers attachment proposal

1919 .]

1920 In January 2012, I extended the patch and applied it to mmh

1921 .Ci 8ff284ff9167eff8f5349481529332d59ed913b1 .

1922 In mmh, the attachment system is active by default.

1923 Instead of command line switches, the

1924 .Pe Attachment-Header

1925 profile entry is used to specify

1926 the name of the attachment header field.

1927 It is pre-defined to

1928 .Hd Attach .

1929 .P

1930 To add an attachment to a draft, a header line needs to be added:

1931 .VS

1932 To: bob

1933 Subject: The file you wanted

1934 Attach: /path/to/the/file-bob-wanted

1935 --------

1936 Here it is.

1937 VE

1938 The header field can be added to the draft manually in the editor,

1939 or by using the `attach' command at the WhatNow prompt, or

1940 non-interactively with

1941 .Pn anno :

1942 .VS

1943 anno -append -nodate -component Attach -text /path/to/attachment

1944 VE

1945 Drafts with attachment headers are converted to MIME automatically by

1946 .Pn send .

1947 The conversion to MIME is invisible to the user.

1948 The draft stored in the draft folder is always in source form with

1949 attachment headers.

1950 If the MIMEification fails (e.g. because the file to attach

1951 is not accessible) the original draft is not changed.

1952 .P

1953 The attachment system handles the forwarding of messages, too.

1954 If the attachment header value starts with a plus character (`\fL+\fP'),

1955 like in

1956 .Cl "Attach: +bob 30 42" ,

1957 the given messages in the specified folder will be attached.

1958 This allowed to simplify

1959 .Pn forw

1960 .Ci f41f04cf4ceca7355232cf7413e59afafccc9550 .

1961 .P

1962 Closely related to attachments is non-ASCII text content,

1963 because it requires MIME as well.

1964 In nmh, the user needed to call `mime' at the WhatNow prompt

1965 to have the draft converted to MIME.

1966 This was necessary whenever the draft contained non-ASCII characters.

1967 If the user did not call `mime', a broken message would be sent.

1968 Therefore, the

1969 .Pe automimeproc

1970 profile entry could be specified to have the `mime' command invoked

1971 automatically each time.

1972 Unfortunately, this approach conflicted with the attachment system

1973 because the draft would already be in MIME format at the time

1974 when the attachment system wanted to MIMEify it.

1975 To use nmh's attachment system, `mime' must not be called at the

1976 WhatNow prompt and

1977 .Pe automimeproc

1978 must not be set in the profile.

1979 But then the case of non-ASCII text without attachment headers was

1980 not caught.

1981 All in all, the solution was complex and irritating.

1982 My patch from December 2010

1983 .[

1984 nmh-workers attachment proposal

1985 .]

1986 would have simplified the situation.

1987 .P

1988 Mmh's current solution is even more elaborate.

1989 Any necessary MIMEification is done automatically.

1990 There is no `mime' command at the WhatNow prompt anymore.

1991 The draft will be converted automatically to MIME when either an

1992 attachment header or non-ASCII text is present.

1993 Furthermore, the hash character (`\fL#\fP') is not special any more

1994 at line beginnings in the draft message.

1995 Users need not concern themselves with the whole topic at all.

1996 The approach taken in mmh is tailored towards today's most common case:

1997 a text part, possibly with attachments.

1998 This case was simplified.

1999 .P

2000 Although the new approach does not anymore support arbitrary MIME

2001 compositions directly, the full power of

2002 .Pn mhbuild

2003 can still be accessed.

2004 Given no attachment headers are included, users can create

2005 .Pn mhbuild

2006 composition drafts like in nmh.

2007 Then, at the WhatNow prompt, they can invoke

2008 .Cl "edit mhbuild

2009 to convert the draft to MIME.

2010 Because the resulting draft neither contains non-ASCII characters

2011 nor has it attachment headers, the attachment system will not touch it.

2012

2013

2014 .U3 "MIME Type Guessing

2015 .P

2016 From the programmer's point of view, the use of

2017 .Pn mhbuild

2018 composition drafts had one notable advantage over attachment headers:

2019 The user provides the appropriate MIME types for files to include.

2020 The new attachment system needs to find out the correct MIME type itself.

2021 This is a difficult task.

2022 Determining the correct MIME type of content is partly mechanical,

2023 partly intelligent work.

2024 Forcing the user to find out the correct MIME type,

2025 forces him to do partly mechanical work.

2026 Letting the computer do the work can lead to bad choices for difficult

2027 content.

2028 For mmh, the latter option was chosen to spare the user the work

2029 .Ci 3baec236a39c5c89a9bda8dbd988d643a21decc6 .

2030 .P

2031 Determining the MIME type by the suffix of the file name is a dumb

2032 approach, yet it is simple to implement and provides good results

2033 for the common cases.

2034 If no MIME type can be determined, text content is sent as `text/plain',

2035 anything else under the generic fall-back type `application/octet-stream'.

2036 Mmh implements this approach in the

2037 .Pn print-mimetype

2038 script

2039 .Ci 4b5944268ea0da7bb30598a27857304758ea9b44 .

2040 .P

2041 A far better, though less portable, approach is the use of

2042 .Pn file .

2043 This standard tool tries to determine the type of files.

2044 Unfortunately, its capabilities and accuracy varies from system to system.

2045 Additionally, its output was only intended for human beings,

2046 but not to be used by programs.

2047 Nevertheless, modern versions of GNU

2048 .Pn file ,

2049 which are prevalent on the popular GNU/Linux systems,

2050 provide MIME type output in machine-readable form.

2051 Although this solution is system-dependent,

2052 it solves the difficult problem well.

2053 On systems where GNU

2054 .Pn file ,

2055 version 5.04 or higher, is available it should be used.

2056 One needs to specify the following profile entry to do so:

2057 .VS

2058 Mime-Type-Query: file -b --mime

2059 VE

2060 .LP

2061 Other versions of

2062 .Pn file

2063 might possibly be usable with wrapper scripts that reformat the output.

2064 The diversity among

2065 .Pn file

2066 implementations is great; one needs to check the local variant.

2067 .P

2068 It is not possible in mmh to override the automatic MIME type guessing

2069 for a specific file.

2070 To do so, either the user would need to know in advance for which file

2071 the automatic guessing fails or the system would require interaction.

2072 I consider both cases impractical.

2073 The existing solution should be sufficient.

2074 If not, the user may always fall back to

2075 .Pn mhbuild

2076 composition drafts and bypass the attachment system.

2077

2078

2079 .U3 "Storing Attachments

2080 .P

2081 Extracting MIME parts of a message and storing them to disk is performed by

2082 .Pn mhstore .

2083 The program has two operation modes,

2084 .Sw -auto

2085 and

2086 .Sw -noauto .

2087 With the former one, each part is stored under the filename given in the

2088 MIME part's meta information, if available.

2089 This naming information is usually available for modern attachments.

2090 If no filename is available, this MIME part is stored as if

2091 .Sw -noauto

2092 would have been specified.

2093 In the

2094 .Sw -noauto

2095 mode, the parts are processed according to the rules that are defined by

2096 .Pe mhstore-store-*

2097 profile entries.

2098 These rules define generic filename templates for storing

2099 or commands to post-process the contents in arbitrary ways.

2100 If no matching rule is available the part is stored under a generic

2101 filename, built from message number, MIME part number, and MIME type.

2102 .P

2103 The

2104 .Sw -noauto

2105 mode had been the default in nmh because it was considered safe,

2106 in contrast to the

2107 .Sw -auto

2108 mode.

2109 In mmh,

2110 .Sw -auto

2111 is not dangerous anymore.

2112 Two changes were necessary:

2113 .LI 1

2114 Any directory path is removed from the proposed filename.

2115 Thus, the files are always stored in the expected directory.

2116 .Ci 41b6eadbcecf63c9a66aa5e582011987494abefb

2117 .LI 2

2118 Tar files are not extracted automatically any more.

2119 Thus, the rest of the file system will not be touched.

2120 .Ci 94c80042eae3383c812d9552089953f9846b1bb6

2121 .P

2122 In mmh, the result of

2123 .Cl "mhstore -auto

2124 can be foreseen from the output of

2125 .Cl "mhlist -verbose" .

2126 Although the

2127 .Sw -noauto

2128 mode is considered to be more powerful, it is less convenient and

2129 .Sw -auto

2130 is safe now.

2131 Additionally, storing attachments under their original name

2132 is intuitive.

2133 Hence,

2134 .Sw -auto

2135 serves better as the default option

2136 .Ci 3410b680416c49a7617491af38bc1929855a331d .

2137 .P

2138 Files are stored into the directory given by the

2139 .Pe Nmh-Storage

2140 profile entry, if set, or

2141 into the current working directory, otherwise.

2142 Storing to different directories is only possible with

2143 .Pe mhstore-store-*

2144 profile entries.

2145 .P

2146 Still existing files get overwritten silently in both modes.

2147 This can be considered a bug.

2148 Yet, each other behavior has its draw-backs, too.

2149 Refusing to replace files requires adding a

2150 .Sw -force

2151 switch.

2152 Users will likely need to invoke

2153 .Pn mhstore

2154 a second time with

2155 .Sw -force .

2156 Eventually, only the user can decide in the specific case.

2157 This requires interaction, which I like to avoid if possible.

2158 Appending a unique suffix to the filename is another bad option.

2159 For now, the behavior remains as it is.

2160 .P

2161 In mmh, only MIME parts of type message are special in

2162 .Pn mhstore 's

2163 .Sw -auto

2164 mode.

2165 Instead of storing message/rfc822 parts as files to disk,

2166 they are stored as messages into the current mail folder.

2167 The same applies to message/partial, although the parts are

2168 automatically reassembled beforehand.

2169 MIME parts of type message/external-body are not automatically retrieved

2170 anymore.

2171 Instead, information on how to retrieve them is output.

2172 Not supporting this rare case saved nearly one thousand lines of code

2173 .Ci 55e1d8c654ee0f7c45b9361ce34617983b454c32 .

2174 The MIME type `application/octet-stream; type=tar' is not special anymore.

2175 The automatically extracting of such MIME parts had been the

2176 dangerous part of the

2177 .Sw -auto

2178 mode

2179 .Ci 94c80042eae3383c812d9552089953f9846b1bb6 .

2180

2181

2182

2183 .U3 "Showing MIME Messages

2184 .Id showing-mime-msgs

2185 .P

2186 The program

2187 .Pn mhshow

2188 was written to display MIME messages.

2189 It implemented the conceptional view of the MIME RFCs.

2190 Nmh's

2191 .Pn mhshow

2192 handles each MIME part independently, presenting them separately

2193 to the user.

2194 This does not match today's understanding of email attachments,

2195 where displaying a message is seen to be a single, integrated operation.

2196 Today, email messages are expected to consist of a main text part

2197 plus possibly attachments.

2198 They are no more seen to be arbitrary MIME hierarchies with

2199 information on how to display the individual parts.

2200 I adjusted

2201 .Pn mhshow 's

2202 behavior to the modern view on the topic.

2203 .P

2204 One should note that this section completely ignores the original

2205 .Pn show

2206 program, because it was not capable to display MIME messages

2207 and is no longer part of mmh

2208 (cf. Sec.

2209 .Cf mhshow ).

2210 Although

2211 .Pn mhshow

2212 was renamed to

2213 .Pn show

2214 in mmh, this section uses the name

2215 .Pn mhshow ,

2216 in order to avoid confusion.

2217 .P

2218 In mmh, the basic idea is that

2219 .Pn mhshow

2220 should display a message in one single pager session.

2221 Therefore,

2222 .Pn mhshow

2223 invokes a pager session for all its output,

2224 whenever it prints to a terminal

2225 .Ci a4197ea6ffc5c1550e8b52d5a654bcaaaee04a4e .

2226 In consequence,

2227 .Pn mhl

2228 does no more invoke a pager

2229 .Ci 0e46503be3c855bddaeae3843e1b659279c35d70 .

2230 With

2231 .Pn mhshow

2232 replacing the original

2233 .Pn show ,

2234 the output of

2235 .Pn mhl

2236 no longer goes to the terminal directly, but through

2237 .Pn mhshow .

2238 Hence,

2239 .Pn mhl

2240 does not need to invoke a pager.

2241 The one and only job of

2242 .Pn mhl

2243 is to format messages or parts of them.

2244 The only place in mmh, where a pager is invoked is

2245 .Pn mhshow .

2246 .P

2247 Only text content is displayed.

2248 Other kinds of attachments are ignored.

2249 Non-text content needs to be converted to text by appropriate

2250 .Pe mhshow-show-*

2251 profile entries before, if this is possible and wanted.

2252 A common example for this are PDF files.

2253 .P

2254 MIME parts are always displayed serially.

2255 The request to display the MIME type `multipart/parallel' in parallel

2256 is ignored.

2257 It is simply treated as `multipart/mixed'

2258 .Ci d0581ba306a7299113a346f9b4c46ce97bc4cef6 .

2259 This was already possible to request with the

2260 .Sw -serialonly

2261 switch of

2262 .Pn mhshow ,

2263 which is now removed.

2264 As MIME parts are always processed exclusively, i.e. serially,

2265 the `\fL%e\fP' escape in

2266 .Pe mhshow-show-*

2267 profile entries became useless and was thus removed

2268 .Ci a20d405db09b7ccca74d3e8c57550883da49e1ae .

2269 For parallel display, the attachments need to be stored to disk first.

2270 .P

2271 To display text content in foreign charsets, they need to be converted

2272 to the native charset.

2273 Therefore,

2274 .Pe mhshow-charset-*

2275 profile entries were needed.

2276 In mmh, the conversion is performed automatically by piping the

2277 text through the

2278 .Pn iconv

2279 command, if necessary

2280 .Ci 2433122c20baccb10b70b49c04c6b0497b5b3b60 .

2281 Custom

2282 .Pe mhshow-show-*

2283 rules for textual content might need a

2284 .Cl "iconv -f %c %f |

2285 prefix to have the text converted to the native charset.

2286 .P

2287 Although the conversion of foreign charsets to the native one

2288 has improved, it is not consistent enough.

2289 Further work needs to be done and

2290 the basic concepts in this field need to be re-thought.

2291 Though, the default setup of mmh displays message in foreign charsets

2292 correctly without the need to configure anything.

2293

2294

2295 .ig

2296

2297 .P

2298 mhshow/mhstore: Removed support for retrieving message/external-body parts.

2299 These tools will not download the contents automatically anymore. Instead,

2300 they print the information needed to get the contents. If someone should

2301 really receive one of those rare message/external-body messages, he can

2302 do the job manually. We save nearly a thousand lines of code. That's worth

2303 it!

2304 (The profile entry `nmh-access-ftp' and sbr/ruserpass.c for reading

2305 ~/.netrc are gone now.)

2306 .Ci 55e1d8c654ee0f7c45b9361ce34617983b454c32

2307

2308 ..

2309

2310

2311

2312 .H2 "Signing and Encrypting

2313 .P

2314 Nmh offers no direct support for digital signatures and message encryption.

2315 This functionality needed to be added through third-party software.

2316 In mmh, the functionality is included because it

2317 is a part of modern email and is likely wanted by users of mmh.

2318 A fresh mmh installation supports signing and encrypting

2319 out-of-the-box.

2320 Therefore, Neil Rickert's

2321 .Pn mhsign

2322 and

2323 .Pn mhpgp

2324 scripts

2325 .[

2326 neil rickert mhsign mhpgp

2327 .]

2328 were included

2329 .Ci f45cdc98117a84f071759462c7ae212f4bc5ab2e

2330 .Ci 58cf09aa36e9f7f352a127158bbf1c5678bc6ed8 .

2331 The scripts fit well because they are lightweight and

2332 similar of style to the existing tools.

2333 Additionally, no licensing difficulties appeared

2334 as they are part of the public domain.

2335 .P

2336 .Pn mhsign

2337 handles the signing and encrypting part.

2338 It comprises about 250 lines of shell code and interfaces between

2339 .Pn gnupg

2340 and the MH system.

2341 It was meant to be invoked manually at the WhatNow prompt, but in mmh,

2342 .Pn send

2343 invokes

2344 .Pn mhsign

2345 automatically

2346 .Ci c7b5e1df086bcc37ff40163ee67571f076cf6683 .

2347 Special header fields were introduced to request this action.

2348 If a draft contains the

2349 .Hd Sign

2350 header field,

2351 .Pn send

2352 will initiate the signing.

2353 The signing key is either chosen automatically or it is specified by the

2354 .Pe Pgpkey

2355 profile entry.

2356 .Pn send

2357 always create signatures using the PGP/MIME standard [RFC\|4880],

2358 but by invoking

2359 .Pn mhsign

2360 manually, old-style non-MIME signatures can be created as well.

2361 To encrypt an outgoing message, the draft needs to contain an

2362 .Hd Enc

2363 header field.

2364 Public keys of all recipients are searched for in the gnupg keyring and

2365 in a file called

2366 .Fn pgpkeys ,

2367 which contains exceptions and overrides.

2368 Unless public keys are found for all recipients,

2369 .Pn mhsign

2370 will refuse to encrypt it.

2371 Currently, messages with hidden (BCC) recipients can not be encrypted.

2372 This work is pending because it requires a structurally more complex

2373 approach.

2374 .P

2375 .Pn mhpgp

2376 is the companion to

2377 .Pn mhsign .

2378 It verifies signatures and decrypts messages.

2379 Encrypted messages can be either temporarily decrypted and displayed

2380 or permanently decrypted and stored into the current folder.

2381 Currently,

2382 .Pn mhpgp

2383 needs to be invoked manually.

2384 The integration into

2385 .Pn show

2386 and

2387 .Pn mhstore

2388 to verify signatures and decrypt messages as needed

2389 is planned but not yet realized.

2390 .P

2391 Both scripts were written for nmh.

2392 Hence they needed to be adjust

2393 according to the differences between nmh and mmh.

2394 For instance, they use the backup prefix no longer.

2395 Furthermore, compatibility support for old PGP features was dropped.

2396 .P

2397 The integrated message signing and encrypting support is one of the

2398 most recent features in mmh.

2399 It has not had the time to mature.

2400 User feedback and personal experience need to be accumulated to

2401 direct the further development of the facility.

2402 Already it seems to be worthwhile to consider adding

2403 .Sw -[no]sign

2404 and

2405 .Sw -[no]enc

2406 switches to

2407 .Pn send ,

2408 to be able to override the corresponding header fields.

2409 A profile entry:

2410 .VS

2411 send: -sign

2412 VE

2413 would then activate signing for all outgoing messages.

2414 With the present approach, a

2415 .Hd Send

2416 header component needs to be added to each draft template

2417 to achieve the same result.

2418 Adding the switches would ease the work greatly and keep the

2419 template files clean.

2420

2421

2422

2423

2424 .H2 "Draft and Trash Folder

2425 .P

2426

2427 .U3 "Draft Folder

2428 .Id draft-folder

2429 .P

2430 In the beginning, MH had the concept of a draft message.

2431 This was a file named

2432 .Fn draft

2433 in the MH directory, which was treated special.

2434 On composing a message, this draft file was used.

2435 When starting to compose another message before the former one was sent,

2436 the user had to decide among:

2437 .LI 1

2438 Using the old draft to finish and send it before starting with a new one.

2439 .LI 2

2440 Discarding the old draft and replacing it with a new one.

2441 .LI 3

2442 Preserving the old draft by refiling it to a folder.

2443 .LP

2444 Working on multiple drafts was only possible in alternation.

2445 For that, the current draft needed to be refiled to a folder and

2446 another one re-used for editing.

2447 Working on multiple drafts at the same time was impossible.

2448 The usual approach of switching to a different MH context did not

2449 help anything.

2450 .P

2451 The draft folder facility exists to

2452 allow true parallel editing of drafts, in a straight forward way.

2453 It was introduced by Marshall T. Rose, already in 1984.

2454 Similar to other new features, the draft folder was inactive by default.

2455 Even in nmh, the highly useful draft folder was not available

2456 out-of-the-box.

2457 At least, Richard Coleman added the man page

2458 .Mp mh-draft (5)

2459 to better document the feature.

2460 .P

2461 Not using the draft folder facility has the single advantage of having

2462 the draft file at a static location.

2463 This is simple in simple cases but the concept does not scale for more

2464 complex cases.

2465 The concept of the draft message is too limited for the problem

2466 it tries to solve.

2467 Therefore the draft folder was introduced.

2468 It is the more powerful and more natural concept.

2469 The draft folder is a folder like any other folder in MH.

2470 Its messages can be listed like any other messages.

2471 A draft message is no longer a special case.

2472 Tools do not need special switches to work on the draft message.

2473 Hence corner cases were removed.

2474 .P

2475 The trivial part of the work was activating the draft folder with a

2476 default name.

2477 I chose the name

2478 .Fn +drafts ,

2479 for obvious reasons.

2480 In consequence, the command line switches

2481 .Sw -draftfolder

2482 and

2483 .Sw -draftmessage

2484 could be removed.

2485 More difficult, but also more improving, was the updating of the

2486 tools to the new concept.

2487 By fully switching to the draft folder, the tools could be

2488 simplified by dropping the awkward draft message handling code.

2489 .Sw -draft

2490 switches were removed because operating on a draft message is no longer

2491 special.

2492 It became indistinguishable to operating on any other message.

2493 .Ci 337338b404931f06f0db2119c9e145e8ca5a9860

2494 .P

2495 There is no more need to query the user for draft handling

2496 .Ci 2d48b455c303a807041c35e4248955f8bec59eeb .

2497 It is always possible to add another new draft.

2498 Refiling drafts is without difference to refiling other messages.

2499 All of these special cases are gone.

2500 Yet, one draft-related switch remained.

2501 .Pn comp

2502 still has

2503 .Sw -[no]use

2504 for switching between two modes:

2505 .LI 1

2506 Modifying an existing draft, with

2507 .Sw -use .

2508 .LI 2

2509 Composing a new draft, possibly taking some existing message as template,

2510 with

2511 .Sw -nouse ,

2512 the default.

2513 .ZZ

2514 .RT

2515 .sp \n(PDu

2516 In either case, the behavior of

2517 .Pn comp

2518 is deterministic.

2519 .P

2520 .Pn send

2521 now operates on the current message in the draft folder by default.

2522 As message and folder can both be overridden by specifying them on

2523 the command line, it is possible to send any message in the mail storage

2524 by simply specifying its number and folder.

2525 In contrast to the other tools,

2526 .Pn send

2527 takes the draft folder as its default folder.

2528 .P

2529 Dropping the draft message concept in favor for the draft folder concept,

2530 replaced special cases with regular cases.

2531 This simplified the source code of the tools, as well as the concepts.

2532 In mmh, draft management does not break with the MH concepts

2533 but applies them.

2534 .Cl "scan +drafts" ,

2535 for instance, is a truly natural request.

2536 .P

2537 Most of the work was already performed by Rose in the eighties.

2538 The original improvement of mmh is dropping the old draft message approach

2539 and thus simplifying the tools, the documentation,

2540 and the system as a whole.

2541 Although my part in the draft handling improvement was small,

2542 it was important.

2543

2544

2545 .U3 "Trash Folder

2546 .Id trash-folder

2547 .P

2548 Similar to the situation for drafts is the situation for removed messages.

2549 Historically, a message was ``deleted'' by prepending a specific

2550 \fIbackup prefix\fP, usually the comma character,

2551 to the file name.

2552 The specific file would then be ignored by MH because only files with

2553 names consisting of digits only are treated as messages.

2554 Although files remained in the file system,

2555 the messages were no longer visible in MH.

2556 To truly delete them, a maintenance job was needed.

2557 Usually a cron job was installed to delete them after a grace time.

2558 For instance:

2559 .VS

2560 find $HOME/Mail -type f -name ',*' -ctime +7 -delete

2561 VE

2562 In such a setup, the original message could be restored

2563 within the grace time interval by stripping the

2564 backup prefix from the file name \(en usually but not always.

2565 If the last message of a folder with six messages (\fL1-6\fP) was removed,

2566 message

2567 .Fn 6 ,

2568 became file

2569 .Fn ,6 .

2570 If then a new message entered the same folder, it would be named with

2571 the number one above the highest existing message number.

2572 In this case the message would be named

2573 .Fn 6 ,

2574 reusing the number.

2575 If this new message would be removed as well,

2576 then the backup of the former message becomes overwritten.

2577 Hence, the ability to restore removed messages did not only depend on

2578 the sweeping cron job but also on the removing of further messages.

2579 It is undesirable to have such obscure and complex mechanisms.

2580 The user should be given a small set of clear assertions, such as

2581 ``Removed files are restorable within a seven-day grace time.''

2582 With the addition ``... unless a message with the same name in the

2583 same folder is removed before.'' the statement becomes complex.

2584 A user will hardly be able to keep track of all removals to know

2585 if the assertion still holds true for a specific file.

2586 In practice, the real mechanism is unclear to the user.

2587 .P

2588 Furthermore, the backup files were scattered within the whole mail storage.

2589 This complicated managing them.

2590 It was possible with the help of

2591 .Pn find ,

2592 but everything is more convenient

2593 if the deleted messages are collected in one place.

2594 .P

2595 The profile entry

2596 .Pe rmmproc

2597 (previously named

2598 .Pe Delete-Prog )

2599 was introduced very early to improve the situation.

2600 It could be set to any command, which would be executed to remove

2601 the specified messages.

2602 This had overridden the default action, described above.

2603 Refiling the to-be-removed files to a trash folder was the usual example.

2604 Nmh's man page

2605 .Mp rmm (1)

2606 proposes to set the

2607 .Pe rmmproc

2608 to

2609 .Cl "refile +d

2610 to move messages to the trash folder

2611 .Fn +d

2612 instead of renaming them with the backup prefix.

2613 The man page additionally proposes the expunge command

2614 .Cl "rm `mhpath +d all`

2615 to empty the trash folder.

2616 .P

2617 Removing messages in such a way has advantages:

2618 .LI 1

2619 The mail storage is prevented from being cluttered with removed messages

2620 because they are all collected in one place.

2621 Existing and removed messages are thus separated more strictly.

2622 .LI 2

2623 No backup files are silently overwritten.

2624 .LI 3

2625 Most important, however, removed messages are kept in the MH domain.

2626 Messages in the trash folder can be listed like those in any other folder.

2627 Deleted messages can be displayed like any other messages.

2628 .Pn refile

2629 can restore deleted messages.

2630 All operations on deleted files are still covered by the MH tools.

2631 The trash folder is just like any other folder in the mail storage.

2632 .P

2633 Similar to the draft folder case, I dropped the old backup prefix approach

2634 in favor for replacing it by the better suiting trash folder system.

2635 Hence,

2636 .Pn rmm

2637 calls

2638 .Pn refile

2639 to move the to-be-removed message to the trash folder,

2640 .Fn +trash

2641 by default.

2642 To sweep it clean, the user can use

2643 .Cl "rmm -unlink +trash a" ,

2644 where the

2645 .Sw -unlink

2646 switch causes the files to be unlinked.

2647 .Ci 8edc5aaf86f9f77124664f6801bc6c6cdf258173

2648 .Ci ca0b3e830b86700d9e5e31b1784de2bdcaf58fc5

2649 .P

2650 Dropping the legacy approach and converting to the new approach

2651 completely, simplified the code base.

2652 The relationship between

2653 .Pn rmm

2654 and

2655 .Pn refile

2656 was inverted.

2657 In mmh,

2658 .Pn rmm

2659 invokes

2660 .Pn refile .

2661 That used to be the other way round.

2662 Yet, the relationship is simpler now.

2663 Loops, like described in nmh's man page for

2664 .Mp refile (1),

2665 can no longer occur:

2666 .QS

2667 Since

2668 .Pn refile

2669 uses your

2670 .Pe rmmproc

2671 to delete the message, the

2672 .Pe rmmproc

2673 must NOT call

2674 .Pn refile

2675 without specifying

2676 .Sw -normmproc

2677 or you will create an infinite loop.

2678 .QE

2679 .LP

2680 .Pn rmm

2681 either unlinks a message with

2682 .Fu unlink()

2683 or invokes

2684 .Pn refile

2685 to move it to the trash folder.

2686 .Pn refile

2687 does not invoke any tools.

2688 .P

2689 By generalizing the message removal in the way that it became covered

2690 by the MH concepts made the whole system more powerful.

2691

2692

2693

2694

2695

2696 .H2 "Modern Defaults

2697 .P

2698 Nmh has a bunch of convenience-improving features inactive by default,

2699 although one can expect every new user to want them active.

2700 The reason they are inactive by default is the wish to stay compatible

2701 with old versions.

2702 But what are old versions?

2703 Still, the highly useful draft folder facility has not been activated

2704 by default although it was introduced over twenty-five years ago.

2705 .[

2706 rose romine real work

2707 .]

2708 The community seems not to care.

2709 .P

2710 In nmh, new users are required to first build up

2711 a profile before they can access the modern features.

2712 Without an extensive profile, the setup is hardly usable

2713 for modern emailing.

2714 The point is not the customization of the setup,

2715 but the need to activate generally useful facilities.

2716 Yet, the real problem lies less in enabling the features,

2717 as this is straight forward as soon as one knows what he wants.

2718 The real problem is that new users need deep insight into the project

2719 to discover the available but inactive features.

2720 To give an example, I needed one year of using nmh

2721 before I became aware of the existence of the attachment system.

2722 One could argue that this fact disqualifies my reading of the

2723 documentation.

2724 If I would have installed nmh from source back then, I could agree.

2725 Yet, I had used a pre-packaged version and had expected that it would

2726 just work.

2727 Nevertheless, I had been convinced by the concepts of MH already

2728 and I am a software developer,

2729 still I required a lot of time to discover the cool features.

2730 How can we expect users to be even more advanced than me,

2731 just to enable them to use MH in a convenient and modern way?

2732 Unless they are strongly convinced of the concepts, they will fail.

2733 I have seen friends of me giving up disappointed

2734 before they truly used the system,

2735 although they had been motivated in the beginning.

2736 New users suffer hard enough to get used to the tool chest approach,

2737 we developers should spare them further inconveniences.

2738 .P

2739 Maintaining compatibility for its own sake is bad,

2740 because the code base will collect more and more compatibility code.

2741 Sticking to the compatibility code means remaining limited;

2742 whereas adjusting to the changes renders the compatibility unnecessary.

2743 Keeping unused alternatives in the code for longer than a short

2744 grace time is a bad choice as they likely

2745 gather bugs by not being constantly tested.

2746 Also, the increased code size and the greater number of conditions

2747 increase the maintenance costs.

2748 If any MH implementation would be the back-end of widespread

2749 email clients with large user bases, compatibility would be more

2750 important.

2751 Yet, it appears as if this is not the case.

2752 Hence, compatibility is hardly important for technical reasons.

2753 Its importance originates from personal reasons rather.

2754 Nmh's user base is small and old.

2755 Changing the interfaces causes inconvenience to long-term users of MH.

2756 It forces them to change their many years old MH configurations.

2757 I do understand this aspect, but by sticking to the old users,

2758 new users are kept from entering the world of MH.

2759 But the future lies in new users.

2760 In consequence, mmh invites new users by providing a convenient

2761 and modern setup, readily usable out-of-the-box.

2762 .P

2763 In mmh, all modern features are active by default and many previous

2764 approaches are removed or only accessible in a manual way.

2765 New default features include:

2766 .BU

2767 The attachment system (\c

2768 .Hd Attach )

2769 .Ci 8ff284ff9167eff8f5349481529332d59ed913b1 .

2770 .BU

2771 The draft folder facility (\c

2772 .Fn +drafts )

2773 .Ci 337338b404931f06f0db2119c9e145e8ca5a9860 .

2774 .BU

2775 The unseen sequence (`u')

2776 .Ci c2360569e1d8d3678e294eb7c1354cb8bf7501c1

2777 and the sequence negation prefix (`!')

2778 .Ci db74c2bd004b2dc9bf8086a6d8bf773ac051f3cc .

2779 .BU

2780 Quoting the original message in the reply

2781 .Ci 67411b1f95d6ec987b4c732459e1ba8a8ac192c6 .

2782 .BU

2783 Forwarding messages using MIME

2784 .Ci 6e271608b7b9c23771523f88d23a4d3593010cf1 .

2785 .LP

2786 An mmh setup with a profile that defines only the path to the

2787 mail storage, is already convenient to use.

2788 Again, Paul Vixie's supports the direction I took:

2789 ``the `main branch' should just be modern''.

2790 .[

2791 paul vixie edginess nmh-workers

2792 .]

2793

2794

2795

2796

2797

2798 .\" --------------------------------------------------------------

2799 .H1 "Styling

2800 .P

2801 Kernighan and Pike have emphasized the importance of style in the

2802 preface of \fIThe Practice of Programming\fP:

2803 .[ [

2804 kernighan pike practice of programming

2805 .], p. x]

2806 .QS

2807 Chapter 1 discusses programming style.

2808 Good style is so important to good programming that we have chosen

2809 to cover it first.

2810 .QE

2811 This section covers changes in mmh that were guided by the desire

2812 to improve on style.

2813 Many of them follow the advice given in the quoted book.

2814

2815

2816

2817

2818 .H2 "Code Style

2819 .Id code-style

2820 .P

2821 .U3 "Indentation Style

2822 .P

2823 Indentation styles are the holy cow of programming.

2824 Kernighan and Pike write:

2825 .[ [

2826 kernighan pike practice of programming

2827 .], p. 10]

2828 .QS

2829 Programmers have always argued about the layout of programs,

2830 but the specific style is much less important than its consistent

2831 application.

2832 Pick one style, preferably ours, use it consistently, and don't waste

2833 time arguing.

2834 .QE

2835 .P

2836 I agree that the constant application is most important,

2837 but I believe that some styles have advantages over others.

2838 For instance the indentation with tab characters only.

2839 The number of tabs corresponds to the nesting level \(en

2840 one tab, one level.

2841 Tab characters provide flexible visual appearance because developers

2842 can adjust their width as preferred.

2843 There is no more need to check for the correct mixture of

2844 tabs and spaces.

2845 Two simple rules ensure the integrity and flexibility of the visual

2846 appearance:

2847 .LI 1

2848 Leading whitespace must consist of tabs only.

2849 .LI 2

2850 All other whitespace should be spaces.

2851 .LP

2852 Although reformatting existing code should be avoided, I did it.

2853 I did not waste time arguing; I just reformatted the code.

2854 .Ci a485ed478abbd599d8c9aab48934e7a26733ecb1

2855

2856 .U3 "Comments

2857 .P

2858 Kernighan and Pike demand: ``Don't belabor the obvious''.

2859 .[ [

2860 kernighan pike practice of programming

2861 .], p. 23]

2862 Following the advice, I removed unnecessary comments.

2863 For instance, I removed all comments in the following code excerpt

2864 .Ci 426543622b377fc5d091455cba685e114b6df674 :

2865 .VS

2866 context_replace(curfolder, folder); /* update current folder */

2867 seq_setcur(mp, mp->lowsel); /* update current message */

2868 seq_save(mp); /* synchronize message sequences */

2869 folder_free(mp); /* free folder/message structure */

2870 context_save(); /* save the context file */

2871

2872 [...]

2873

2874 int c; /* current character */

2875 char *cp; /* miscellaneous character pointer */

2876

2877 [...]

2878

2879 /* NUL-terminate the field */

2880 *cp = '\0';

2881 VE

2882 .P

2883 The information in each of the comments was present in the code

2884 statements already, except for the NUL-termination, which became

2885 obvious from the context.

2886

2887

2888 .U3 "Names

2889 .P

2890 Regarding this topic, Kernighan and Pike suggest:

2891 ``Use active names for functions''.

2892 .[ [

2893 kernighan pike practice of programming

2894 .], p. 4]

2895 One application of this rule was the rename of

2896 .Fu check_charset()

2897 to

2898 .Fu is_native_charset()

2899 .Ci 8d77b48284c58c135a6b2787e721597346ab056d .

2900 The same change additionally fixed a violation of ``Be accurate'',

2901 .[ [

2902 kernighan pike practice of programming

2903 .], p. 4]

2904 as the code did not match the expectation the function suggested.

2905 It did not compare charset names but prefixes of them only.

2906 In case the native charset was `ISO-8859-1', then

2907 .VS

2908 check_charset("ISO-8859-11", strlen("ISO-8859-11"))

2909 VE

2910 had returned true although the upper halves of the code pages

2911 are different.

2912 .P

2913 More important than using active names is using descriptive names.

2914 .VS

2915 m_unknown(in); /* the MAGIC invocation... */

2916 VE

2917 Renaming the obscure

2918 .Fu m_unknown()

2919 function was a delightful event, although it made the code less funny

2920 .Ci 611d68d19204d7cbf5bd585391249cb5bafca846 .

2921 .P

2922 Magic numbers are generally considered bad style.

2923 Obviously, Kernighan and Pike agree:

2924 ``Give names to magic numbers''.

2925 .[ [

2926 kernighan pike practice of programming

2927 .], p. 19]

2928 .P

2929 The argument

2930 .CW outnum

2931 of the function

2932 .Fu scan()

2933 in

2934 .Fn uip/scansbr.c

2935 holds the number of the message to be created.

2936 As well it encodes program logic with negative numbers and zero.

2937 This led to obscure code.

2938 I clarified the code by introducing two variables that extracted

2939 the hidden information:

2940 .VS

2941 int incing = (outnum > 0);

2942 int ismbox = (outnum != 0);

2943 VE

2944 The readable names are thus used in conditions;

2945 the variable

2946 .CW outnum

2947 is used only to extract ordinary message numbers

2948 .Ci b8b075c77be7794f3ae9ff0e8cedb12b48fd139f .

2949 .P

2950 Through the clarity improvement of the change detours in the program

2951 logic of related code parts became apparent.

2952 The implementation was simplified.

2953 This possibility to improve had been invisible before

2954 .Ci aa60b0ab5e804f8befa890c0a6df0e3143ce0723 .

2955 .P

2956 The names just described were a first step, yet the situation

2957 was further improved by giving names to the magic values of

2958 .CW outnum :

2959 .VS

2960 #define SCN_MBOX (-1)

2961 #define SCN_FOLD 0

2962 VE

2963 The two variables were updated thereafter as well:

2964 .VS

2965 int incing = (outnum != SCN_MBOX && outnum != SCN_FOLD);

2966 int scanfolder = (outnum == SCN_FOLD);

2967 VE

2968 Furthermore,

2969 .CW ismbox

2970 was replaced by

2971 .CW scanfolder

2972 because that matched better to the program logic.

2973 .Ci 7ffb36d28e517a6f3a10272056fc127592ab1c19

2974

2975

2976

2977

2978 .H2 "Structural Rework

2979 .P

2980 .ZZ

2981 Although the stylistic changes described already improve the

2982 readability of the source code, all of them were changes ``in the small''.

2983 Structural changes, in contrast, affect much larger code areas.

2984 They are more difficult to accomplish but lead to larger improvements,

2985 especially as they often influence the outer shape of the tools as well.

2986 .P

2987 At the end of their chapter on style,

2988 Kernighan and Pike ask: ``But why worry about style?''

2989 .[ [

2990 kernighan pike practice of programming

2991 .], p. 28].

2992 Following are two examples of structural rework that demonstrate

2993 why style is important in the first place.

2994

2995

2996 .U3 "Rework of \f(CWanno\fP

2997 .P

2998 Until 2002,

2999 .Pn anno

3000 had six functional command line switches:

3001 .Sw -component

3002 and

3003 .Sw -text ,

3004 each with an argument,

3005 and the two pairs of flags,

3006 .Sw -[no]date

3007 and

3008 .Sw -[no]inplace .

3009 Then Jon Steinhart introduced his attachment system.

3010 In need for more advanced annotation handling, he extended

3011 .Pn anno

3012 and added five more switches:

3013 .Sw -draft ,

3014 .Sw -list ,

3015 .Sw \%-delete ,

3016 .Sw -append ,

3017 and

3018 .Sw -number ,

3019 the last one taking an argument

3020 .Ci 7480dbc14bc90f2d872d434205c0784704213252 .

3021 Later,

3022 .Sw -[no]preserve

3023 was added as well

3024 .Ci d9b1d57351d104d7ec1a5621f090657dcce8cb7f .

3025 Then, the Synopsis section of the man page

3026 .Mp anno (1)

3027 read:

3028 .VS

3029 anno [+folder] [msgs] [-component f(CIfieldfP] [-inplace | -noinplace]

3030 [-date | -nodate] [-draft] [-append] [-list] [-delete]

3031 [-number [f(CInumfP|fPallfP]] [-preserve | -nopreserve] [-version]

3032 [-help] [-text f(CIbodyfP]

3033 VE

3034 .LP

3035 The implementation followed the same structure.

3036 Problems became visible when

3037 .Cl "anno -list -number 42

3038 worked on the current message instead of on message number 42,

3039 and

3040 .Cl "anno -list -number l:5

3041 did not work on the last five messages but failed with the mysterious

3042 error message: ``anno: missing argument to -list''.

3043 Yet, the invocation matched the specification in the man page.

3044 There, the correct use of

3045 .Sw -number

3046 was defined as being

3047 .Cl "[-number [num|all]]

3048 and the textual description for the combination with

3049 .Sw -list

3050 read:

3051 .QS

3052 The

3053 .Sw -list

3054 option produces a listing of the field bodies for

3055 header fields with names matching the specified component,

3056 one per line. The listing is numbered, starting at 1, if the

3057 .Sw -number

3058 option is also used.

3059 .QE

3060 .LP

3061 The problem was manifold.

3062 Semantically, the argument to the

3063 .Sw -number

3064 switch is only necessary in combination with

3065 .Sw -delete ,

3066 but not with

3067 .Sw -list .

3068 The code, however, required a numeric argument in any case.

3069 If the argument was missing or non-numeric,

3070 .Pn anno

3071 aborted with an error message that additionally had an off-by-one error.

3072 It printed the name of the switch one before the concerned one.

3073 .P

3074 Trying to fix these problems on the surface would not have solved them.

3075 They originate from a discrepance between the

3076 structure of the problem and the structure implemented in the program.

3077 Such structural differences can only be solved by adjusting the

3078 structure of the implementation to the structure of the problem.

3079 .P

3080 Steinhart had added the

3081 .Sw -list

3082 and

3083 .Sw -delete

3084 switches in the same way as the other switches though

3085 they are of structural different type.

3086 Semantically,

3087 .Sw -list

3088 and

3089 .Sw \%-delete

3090 introduce operation modes.

3091 Historically,

3092 .Pn anno

3093 had only one operation mode: adding header fields.

3094 With the extension, two more modes were added:

3095 listing and deleting header fields.

3096 The structure of the code changes did not pay respect to this

3097 fundamental change.

3098 Neither the implementation nor the documentation did clearly

3099 declare the exclusive operation modes as such.

3100 Having identified the problem, I solved it by putting structure into

3101 .Pn anno

3102 and its documentation

3103 .Ci d54c8db8bdf01e8381890f7729bc0ef4a055ea11 .

3104 .P

3105 The difference is visible in both the code and the documentation.

3106 For instance in the following code excerpt:

3107 .VS

3108 int delete = -2; /* delete header element if set */

3109 int list = 0; /* list header elements if set */

3110 [...]

3111 case DELETESW: /* delete annotations */

3112 delete = 0;

3113 continue;

3114 case LISTSW: /* produce a listing */

3115 list = 1;

3116 continue;

3117 VE

3118 .LP

3119 which was replaced by:

3120 .VS

3121 static enum { MODE_ADD, MODE_DEL, MODE_LIST } mode = MODE_ADD;

3122 [...]

3123 case DELETESW: /* delete annotations */

3124 mode = MODE_DEL;

3125 continue;

3126 case LISTSW: /* produce a listing */

3127 mode = MODE_LIST;

3128 continue;

3129 VE

3130 .LP

3131 The replacement code does not only reflect the problem's structure better,

3132 it is easier to understand as well.

3133 The same applies to the documentation.

3134 The man page was completely reorganized to propagate the same structure.

3135 This is already visible in the Synopsis section:

3136 .VS

3137 anno [+folder] [msgs] [-component f(CIfieldfP] [-text fPbodyfP]

3138 [-append] [-date | -nodate] [-preserve | -nopreserve]

3139 [-Version] [-help]

3140

3141 anno -delete [+folder] [msgs] [-component fPfieldfP] [-text

3142 fPbodyfP] [-number fPnum fP| fPall fP] [-preserve | -nopreserve]

3143 [-Version] [-help]

3144

3145 anno -list [+folder] [msgs] [-component fPfieldfP] [-number]

3146 [-Version] [-help]

3147 VE

3148

3149

3150

3151 .U3 "Path Conversion

3152 .P

3153 Four kinds of path names can appear in MH:

3154 .LI 1

3155 Absolute Unix directory paths, like

3156 .Fn /etc/passwd .

3157 .LI 2

3158 Relative Unix directory paths, like

3159 .Fn ./foo/bar .

3160 .LI 3

3161 Absolute MH folder paths, like

3162 .Fn +projects/mmh .

3163 .LI 4

3164 Relative MH folder paths, like

3165 .Fn @subfolder .

3166 .LP

3167 Relative MH folder paths, are hardly documented

3168 although they are useful for large mail storages.

3169 The current mail folder is specified as `\c

3170 .Fn @ ',

3171 just like the current directory is specified as `\c

3172 .Fn . '.

3173 .P

3174 To allow MH tools to understand all four notations,

3175 they need to be able to convert between them.

3176 In nmh, these path name conversion functions were located in the files

3177 .Fn sbr/path.c

3178 (``return a pathname'') and

3179 .Fn sbr/m_maildir.c

3180 (``get the path for the mail directory'').

3181 The seven functions in the two files were documented with no more

3182 than two comments, which described obvious information.

3183 The signatures of the four exported functions did not explain their

3184 semantics:

3185 .LI 1

3186 .CW "char *path(char *, int);

3187 .LI 2

3188 .CW "char *pluspath(char *);

3189 .LI 3

3190 .CW "char *m_mailpath(char *);

3191 .LI 4

3192 .CW "char *m_maildir(char *);

3193 .P

3194 My investigations provided the following descriptions:

3195 .LI 1

3196 The second parameter of

3197 .Fu path()

3198 defines the type as which the path given in the first parameter should

3199 be treated.

3200 Directory paths are converted to absolute directory paths.

3201 Folder paths are converted to absolute folder paths.

3202 Folder paths must not include a leading `\fL@\fP' character.

3203 Leading plus characters are preserved.

3204 The result is a pointer to newly allocated memory.

3205 .LI 2

3206 .Fu pluspath()

3207 is a convenience-wrapper to

3208 .Fu path() ,

3209 to convert folder paths only.

3210 This function can not be used for directory paths.

3211 An empty string parameter causes a buffer overflow.

3212 .LI 3

3213 .Fu m_mailpath()

3214 converts directory paths to absolute directory paths.

3215 The characters `\fL+\fP' or `\fL@\fP' at the beginning of the path name are

3216 treated literal, i.e. as the first character of a relative directory path.

3217 Hence, this function can not be used for folder paths.

3218 In any case, the result is an absolute directory path,

3219 returned as a pointer to newly allocated memory.

3220 .LI 4

3221 .Fu m_maildir()

3222 returns the parameter unchanged if it is an absolute directory path

3223 or begins with the entry `\fL.\fP' or `\fL..\fP'.

3224 All other strings are prepended with the current working directory.

3225 Hence, this function can not be used for folder paths.

3226 The result is either an absolute directory path or a relative

3227 directory path, starting with dot or dot-dot.

3228 In contrast to the other functions, the result is a pointer to

3229 static memory.

3230 .P

3231 The situation was obscure, irritating, error-prone, and non-orthogonal.

3232 Additionally, no clear terminology was used to name the different

3233 kinds of path names.

3234 Sometimes, the names were even misleading, much as the first argument of

3235 .Fu m_mailpath() ,

3236 which was named

3237 .CW folder ,

3238 although

3239 .Fu m_mailpath()

3240 could not be used with MH folder arguments.

3241 .P

3242 I clarified the path name conversion by complete rework.

3243 First of all, the terminology needed to be defined.

3244 A path name is either in the Unix domain, then it is called

3245 \fIdirectory path\fP or it is in the MH domain,

3246 then it is called \fIfolder path\fP.

3247 The two terms need to be used with strict distinction.

3248 Second, I exploited the concept of path type indicators.

3249 By requiring every path name to start with a distinct type identifier,

3250 the conversion between the types could be fully automated.

3251 This allows the tools to accept path names of any type from the user.

3252 Therefore, it was necessary to require relative directory paths to be

3253 prefixed with a dot character.

3254 In consequence, the dot character could no longer be an alias for the

3255 current message

3256 .Ci cff0e16925e7edbd25b8b9d6d4fbdf03e0e60c01 .

3257 Third, I created three new functions to replace the previous mess:

3258 .LI 1

3259 .Fu expandfol()

3260 converts folder paths to absolute folder paths.

3261 Directory paths are simply passed through.

3262 This function is to be used for folder paths only, thus the name.

3263 The result is a pointer to static memory.

3264 .LI 2

3265 .Fu expanddir()

3266 converts directory paths to absolute directory paths.

3267 Folder paths are treated as relative directory paths.

3268 This function is to be used for directory paths only, thus the name.

3269 The result is a pointer to static memory.

3270 .LI 3

3271 .Fu toabsdir()

3272 converts any type of path to an absolute directory path.

3273 This is the function of choice for path conversion.

3274 Absolute directory paths are the most general representation of a

3275 path name.

3276 The result is a pointer to static memory.

3277 .P

3278 The new functions have names that indicate their use.

3279 Two of the functions convert relative to absolute path names of the

3280 same type.

3281 The third function converts any path name type to the most general one,

3282 the absolute directory path.

3283 All of the functions return pointers to static memory.

3284 The file

3285 .Fn sbr/path.c

3286 contains the implementation of the functions;

3287 .Fn sbr/m_maildir.c

3288 was removed.

3289 .Ci d39e2c447b0d163a5a63f480b23d06edb7a73aa0

3290 .P

3291 Along with the path conversion rework, I also replaced

3292 .Fu getfolder(FDEF)

3293 with

3294 .Fu getdeffol()

3295 and

3296 .Fu getfolder(FCUR)

3297 with

3298 .Fu getcurfol() ,

3299 which only wraps

3300 .Fu expandfol(""@"")

3301 for convenience.

3302 This code was moved from

3303 .Fn sbr/getfolder.c

3304 into

3305 .Fn sbr/path.c

3306 as well.

3307 .Ci d39e2c447b0d163a5a63f480b23d06edb7a73aa0

3308 .P

3309 The related function

3310 .Fu etcpath()

3311 is now included in

3312 .Fn sbr/path.c ,

3313 too

3314 .Ci b4c29794c12099556151d93a860ee51badae2e35 .

3315 Previously, it had been located in

3316 .Fn config/config.c .

3317 .P

3318 Now,

3319 .Fn sbr/path.c

3320 contains all path handling code.

3321 Besides being less code, its readability is highly improved.

3322 The functions follow a common style and are well documented.

3323

3324

3325

3326

3327 .H2 "Profile Reading

3328 .P

3329 The MH profile contains the configuration of a user-specific MH setup.

3330 MH tools read the profile right after starting up

3331 because it contains the location of the user's mail storage

3332 and similar settings that influence the whole setup.

3333 Furthermore, the profile contains the default switches for the tools

3334 as well.

3335 The context file is read along with the profile.

3336 .P

3337 For historic reasons, some MH tools did not read the profile and context.

3338 Among them were

3339 .Pn post /\c

3340 .Pn spost ,

3341 .Pn mhmail ,

3342 and

3343 .Pn slocal .

3344 The reason why these tools ignored the profile were not clearly stated.

3345 During a discussion on the nmh-workers mailing list,

3346 David Levine posted an explanation, quoting John Romine:

3347 .[

3348 nmh-workers levine post profile

3349 .]

3350

3351 .QS

3352 I asked John Romine and here's what he had to say, which

3353 agrees and provides an example that convinces me:

3354 .QS

3355 My take on this is that

3356 .Pn post

3357 should not be called by users directly, and it doesn't read the

3358 .Fn .mh_profile

3359 (only front-end UI programs read the profile).

3360 .QP

3361 For example, there can be contexts where

3362 .Pn post

3363 is called by a helper program (like `\c

3364 .Pn mhmail ')

3365 which may be run by a non-MH user.

3366 We don't want this to prompt the user to create an MH profile, etc.

3367 .QP

3368 My suggestion would be to have

3369 .Pn send

3370 pass a (hidden) `\c

3371 .Sw -fileproc

3372 .Ar proc '

3373 option to

3374 .Pn post

3375 if needed.

3376 You could also

3377 use an environment variable (I think

3378 .Pn send /\c

3379 .Pn whatnow

3380 do this).

3381 .QE

3382 .sp \n(PDu

3383 I think that's the way to go.

3384 My personal preference is to use a command line option,

3385 not an environment variable.

3386 .QE

3387

3388 .P

3389 To solve the problem that

3390 .Pn post

3391 does not honor the

3392 .Pe fileproc

3393 profile entry,

3394 the community roughly agreed that a switch

3395 .Sw -fileproc

3396 should be added to

3397 .Pn post

3398 to be able to pass a different fileproc.

3399 I strongly disagree with this approach because it does not solve

3400 the problem; it only removes a single symptom.

3401 The actual problem is that

3402 .Pn post

3403 does not behave as expected,

3404 though all programs should behave as expected.

3405 Clear and general concepts are a precondition for this.

3406 Thus, there should be no separation into ``front-end UI programs''

3407 and ones that ``should not be called by users directly''.

3408 The real solution is having all MH tools read the profile.

3409 .P

3410 But the problem has a further aspect,

3411 which originates from

3412 .Pn mhmail

3413 mainly.

3414 .Pn mhmail

3415 was intended to be a replacement for

3416 .Pn mailx

3417 on systems with MH installations.

3418 In difference to

3419 .Pn mailx ,

3420 .Pn mhmail

3421 used MH's

3422 .Pn post

3423 to send the message.

3424 The idea was that using

3425 .Pn mhmail

3426 should not be influenced whether the user had

3427 MH set up for himself or not.

3428 Therefore

3429 .Pn mhmail

3430 had not read the profile.

3431 As

3432 .Pn mhmail

3433 used

3434 .Pn post ,

3435 .Pn post

3436 was not allowed to read the profile neither.

3437 This is the reason for the actual problem.

3438 Yet, this was not considered much of a problem because

3439 .Pn post

3440 was not intended to be used by users directly.

3441 To invoke

3442 .Pn post ,

3443 .Pn send

3444 was used an a front-end.

3445 .Pn send

3446 read the profile and passed all relevant values on the command line to

3447 .Pn post

3448 \(en an awkward solution.

3449 .P

3450 The important insight is that

3451 .Pn mhmail

3452 is a wolf in sheep's clothing.

3453 This alien tool broke the concepts because it was treated like

3454 a normal MH tool.

3455 Instead it should have been treated accordingly to its foreign style.

3456 .P

3457 The solution is not to prevent the tools from reading the profile but

3458 to instruct them to read a different profile.

3459 .Pn mhmail

3460 could have set up a well-defined profile and caused the following

3461 .Pn post

3462 to use this profile by exporting an environment variable.

3463 With this approach, no special cases would have been introduced

3464 and no surprises would have been caused.

3465 By writing a wrapper program to provide a clean temporary profile,

3466 the concept could have been generalized orthogonally to the whole

3467 MH tool chest.

3468 .P

3469 In mmh, the wish to have

3470 .Pn mhmail

3471 as a replacement for

3472 .Pn mailx

3473 is considered obsolete.

3474 Mmh's

3475 .Pn mhmail

3476 does no longer cover this use-case

3477 .Ci d36e56e695fe1c482c7920644bfbb6386ac9edb0 .

3478 Currently,

3479 .Pn mhmail

3480 is in a transition state

3481 .Ci 32d4f9daaa70519be3072479232ff7be0500d009 .

3482 It may become a front-end to

3483 .Pn comp ,

3484 which provides an alternative interface which can be more convenient

3485 in some cases.

3486 This would convert

3487 .Pn mhmail

3488 into an ordinary MH tool.

3489 If, however, this idea does not convince, then

3490 .Pn mhmail

3491 will be removed.

3492 .P

3493 .ZZ -1

3494 In the mmh tool chest, every program reads the profile.

3495 (\c

3496 .Pn slocal

3497 is not considered part of the mmh tool chest (cf. Sec.

3498 .Cf slocal ).)

3499 Mmh has no

3500 .Pn post

3501 program, but it has

3502 .Pn spost ,

3503 which now does read the profile

3504 .Ci 3e017a7abbdf69bf0dff7a4073275961eda1ded8 .

3505 Following this change,

3506 .Pn send

3507 and

3508 .Pn spost

3509 can be considered for merging.

3510 Besides

3511 .Pn send ,

3512 .Pn spost

3513 is only invoked directly by the to-be-changed

3514 .Pn mhmail

3515 implementation and by

3516 .Pn rcvdist ,

3517 which requires rework anyway.

3518

3519 .P

3520 Jeffrey Honig quoted Marshall T. Rose explaining the decision that

3521 .Pn post

3522 ignores the profile:

3523 .[

3524 nmh-workers honig post profile

3525 .]

3526 .QS

3527 when you run mh commands in a script, you want all the defaults to be

3528 what the man page says.

3529 when you run a command by hand, then you want your own defaults...

3530 .QE

3531 .LP

3532 The explanation neither matches the problem concerned exactly

3533 nor is the interpretation clear.

3534 If the described desire addresses the technical level,

3535 then it conflicts with the Unix philosophy,

3536 precisely because the indistinquishability of human and script

3537 input is the main reason for the huge software leverage in Unix.

3538 If, however, the described desire addresses the user's view,

3539 then different technical solutions are more appropriate.

3540 The two cases can be regarded simply as two different MH setups.

3541 Hence, mapping the problem of different behavior between interactive and

3542 automated use on the concept of switching between different profiles,

3543 marks it already solved.

3544

3545

3546

3547 .H2 "Standard Libraries

3548 .P

3549 MH is one decade older than the POSIX and ANSI C standards.

3550 Hence, MH included own implementations of functions

3551 that were neither standardized nor widely available, back then.

3552 Today, twenty years after POSIX and ANSI C were published,

3553 developers can expect that systems comply with these standards.

3554 In consequence, MH-specific replacements for standard functions

3555 can and should be dropped.

3556 Kernighan and Pike advise: ``Use standard libraries''.

3557 .[ [

3558 kernighan pike practice of programming

3559 .], p. 196]

3560 Actually, MH had followed this advice in history,

3561 but it had not adjusted to more recent changes in this field.

3562 The

3563 .Fu snprintf()

3564 function, for instance, was standardized with C99 and is available

3565 almost everywhere because of its high usefulness.

3566 Thus, the project's own implementation of

3567 .Fu snprintf()

3568 was dropped in March 2012 in favor for using the one of the

3569 standard library

3570 .Ci 0052f1024deb0a0a2fc2e5bacf93d45a5a9c9b32 .

3571 Such decisions limit the portability of mmh

3572 if systems do not support these standardized and widespread functions.

3573 This compromise is made because mmh focuses on the future.

3574 .P

3575 .ZZ

3576 As I am still in my twenties, have no programming experience from

3577 past decades.

3578 I have not followed the evolution of C through time.

3579 I have not suffered from the the Unix wars.

3580 I have not longed for standardization.

3581 All my programming experience is from a time when ANSI C and POSIX

3582 were well established already.

3583 Thus, I needed to learn about the history in retrospective.

3584 I have only read a lot of books about the (good) old times.

3585 This put me in a difficult position when working with old code.

3586 I need to freshly acquire knowledge about old code constructs and ancient

3587 programming styles, whereas older programmers know these things by

3588 heart from their own experience.

3589 Being aware of the situation, I rather let people with more historic

3590 experience do the transition from ancient code constructs to

3591 standardized ones.

3592 Lyndon Nerenberg covered large parts of this task for the nmh project.

3593 He converted project-specific functions to POSIX replacements,

3594 also removing the conditionals compilation of now standardized features.

3595 Ken Hornstein and David Levine had their part in this work, as well.

3596 Often, I only pulled the changes over from nmh into mmh.

3597 These changes include many commits, among them:

3598 .Ci 768b5edd9623b7238e12ec8dfc409b82a1ed9e2d

3599 .Ci 0052f1024deb0a0a2fc2e5bacf93d45a5a9c9b32 .

3600 .P

3601 Nevertheless, I worked on the task as well, tidying up the

3602 \fIMH standard library\fP,

3603 .Fn libmh.a .

3604 It is located in the

3605 .Fn sbr

3606 (``subroutines'') directory in the source tree and

3607 includes functions that mmh tools usually need.

3608 Among them are MH-specific functions for profile, context, sequence,

3609 and folder handling, but as well

3610 MH-independent functions, such as auxiliary string functions,

3611 portability interfaces and error-checking wrappers for critical

3612 functions of the standard library.

3613 .BU

3614 I have replaced the

3615 .Fu atooi()

3616 function with calls to

3617 .Fu strtoul() ,

3618 setting the third parameter, the base, to eight.

3619 .Fu strtoul()

3620 is part of C89 and thus considered safe to use

3621 .Ci c490c51b3c0f8871b6953bd0c74551404f840a74 .

3622 .BU

3623 I did remove project-included fallback implementations of

3624 .Fu memmove()

3625 and

3626 .Fu strerror()

3627 .Ci b067ff5c465a5d243ce5a19e562085a9a1a97215 ,

3628 although Peter Maydell had re-included them into nmh in 2008

3629 to support SunOS 4.

3630 Nevertheless, these functions are part of ANSI C.

3631 Systems that do not even provide full ANSI C support should not

3632 put a load on mmh.

3633 .BU

3634 The

3635 .Fu copy()

3636 function copies the string in parameter one to the location in

3637 parameter two.

3638 In contrast to

3639 .Fu strcpy() ,

3640 it returns a pointer to the terminating null-byte in the destination area.

3641 The code was adjusted to replace

3642 .Fu copy()

3643 with

3644 .Fu strcpy() ,

3645 except within

3646 .Fu concat() ,

3647 where

3648 .Fu copy()

3649 was more convenient.

3650 Therefore, the definition of

3651 .Fu copy()

3652 was moved into the source file of

3653 .Fu concat()

3654 and its visibility it limited to that

3655 .Ci 552fd7253e5ee9e554c5c7a8248a6322aa4363bb .

3656 .BU

3657 The function

3658 .Fu r1bindex()

3659 had been a generalized version of

3660 .Fu basename()

3661 with minor differences.

3662 As all calls to

3663 .Fu r1bindex()

3664 had the slash (`\fL/\fP') as delimiter anyway,

3665 replacing

3666 .Fu r1bindex()

3667 with the more specific and better-named function

3668 .Fu basename()

3669 became desirable.

3670 Unfortunately, many of the 54 calls to

3671 .Fu r1bindex()

3672 depended on a special behavior,

3673 which differed from the POSIX specification for

3674 .Fu basename() .

3675 Hence,

3676 .Fu r1bindex()

3677 was kept but renamed to

3678 .Fu mhbasename() ,

3679 setting the delimiter to the slash

3680 .Ci 240013872c392fe644bd4f79382d9f5314b4ea60 .

3681 For possible uses of

3682 .Fu r1bindex()

3683 with a different delimiter,

3684 the ANSI C function

3685 .Fu strrchr()

3686 provides the core functionality.

3687 .BU

3688 .ZZ

3689 The

3690 .Fu ssequal()

3691 function \(en apparently for ``substring equal'' \(en

3692 was renamed to

3693 .Fu isprefix() ,

3694 because this is what it actually checked

3695 .Ci c20b4fa14515c7ab388ce35411d89a7a92300711.

3696 Its source file had included both of the following comments, no joke.

3697 .in -\n(PIu

3698 .VS

3699 /*

3700 * THIS CODE DOES NOT WORK AS ADVERTISED.

3701 * It is actually checking if s1 is a PREFIX of s2.

3702 * All calls to this function need to be checked to see

3703 * if that needs to be changed. Prefix checking is cheaper, so

3704 * should be kept if it's sufficient.

3705 */

3706 sp .5

3707 /*

3708 * Check if s1 is a substring of s2.

3709 * If yes, then return 1, else return 0.

3710 */

3711 VE

3712 .in +\n(PIu

3713 Eventually, the function was completely replaced with calls to

3714 .Fu strncmp()

3715 .Ci b0b1dd37ff515578cf7cba51625189eb34a196cb .

3716

3717

3718

3719

3720

3721 .H2 "User Data Locations

3722 .P

3723 In nmh, a personal setup consists of the MH profile and the MH directory.

3724 The profile is a file named

3725 .Fn \&.mh_profile

3726 in the user's home directory.

3727 It contains the static configuration.

3728 It also contains the location of the MH directory in the profile entry

3729 .Pe Path .

3730 The MH directory contains the mail storage and is the first

3731 place to search for form files, scan formats, and similar

3732 configuration files.

3733 The location of the MH directory can be chosen freely by the user.

3734 The usual name is a directory named

3735 .Fn Mail

3736 in the user's home directory.

3737 .P

3738 The way MH data is split between profile and MH directory is a legacy.

3739 It is only sensible in a situation where the profile is the only

3740 configuration file.

3741 Why else should the mail storage and the configuration files be intermixed?

3742 They are of different kind:

3743 One kind is the data to be operated on and the other kind is

3744 the configuration to change how tools operate.

3745 Splitting the configuration between the profile and the MH directory

3746 is inappropriate, as well.

3747 I improved the situation by breaking compatibility.

3748 .P

3749 In mmh, personal data is grouped by type.

3750 This results in two distinct parts:

3751 the mail storage and the configuration.

3752 The mail storage directory still contains all the messages,

3753 but, in exception of public sequences files, nothing else.

3754 In difference to nmh, the auxiliary configuration files are no longer

3755 located there.

3756 Therefore, the directory is no longer called the user's \fIMH directory\fP

3757 but the user's \fImail storage\fP.

3758 Its location is still user-chosen, with the default name

3759 .Fn Mail

3760 in the user's home directory.

3761 The configuration is grouped together in the hidden directory

3762 .Fn \&.mmh

3763 in the user's home directory.

3764 This \fImmh directory\fP contains the context file, personal forms,

3765 scan formats, and the like, but also the user's profile, now named

3766 .Fn profile .

3767 The path to the profile is no longer

3768 .Fn $HOME/.mh_profile

3769 but

3770 .Fn $HOME/.mmh/profile .

3771 (The alternative of having file

3772 .Fn $HOME/.mh_profile

3773 and a configuration directory

3774 .Fn $HOME/.mmh

3775 appeared to be inconsistent.)

3776 .P

3777 The approach chosen for mmh is consistent, simple, and familiar to

3778 Unix users.

3779 The main achievement of the change is the clear and sensible separation

3780 of the mail storage and the configuration.

3781 .Ci 7030d7edb099bff36ded7548bb5380f7acab4f9b

3782 .P

3783 As MH allows users to have multiple MH setups,

3784 it is necessary to switch the profile.

3785 The profile is the single entry point to access the rest of a

3786 personal MH setup.

3787 In nmh, the environment variable

3788 .Ev MH

3789 is used to specify a different profile.

3790 To operate in the same MH setup with a separate context, the

3791 .Ev MHCONTEXT

3792 environment variable is used.

3793 This allows having a separate current folder in each terminal at

3794 the same time, for instance.

3795 In mmh, three environment variables replace the two of nmh.

3796 .Ev MMH

3797 overrides the default location of the mmh directory (\c

3798 .Fn .mmh ).

3799 .Ev MMHP

3800 and

3801 .Ev MMHC

3802 override the paths to the profile and context file, respectively.

3803 This approach allows the set of personal configuration files to be chosen

3804 independently of the profile, context, and mail storage.

3805 The new approach has no functional disadvantages,

3806 as every setup I can imagine can be implemented with both approaches,

3807 possibly even easier with the new one.

3808 .Ci 7030d7edb099bff36ded7548bb5380f7acab4f9b

3809

3810

3811

3812

3813

3814 .H2 "Modularization

3815 .Id modularization

3816 .P

3817 The source code of the mmh tools is located in the

3818 .Fn uip

3819 (``user interface programs'') directory.

3820 Each tool has a source file with the name of the command.

3821 For example,

3822 .Pn rmm

3823 is built from

3824 .Fn uip/rmm.c .

3825 Some source files are used for multiple programs.

3826 For example

3827 .Fn uip/scansbr.c

3828 is used for both

3829 .Pn scan

3830 and

3831 .Pn inc .

3832 In nmh, 49 tools were built from 76 source files.

3833 This is a ratio of 1.6 source files per program.

3834 32 programs depended on multiple source files;

3835 17 programs depended on one source file only.

3836 In mmh, 39 tools are built from 51 source files.

3837 This is a ratio of 1.3 source files per program.

3838 18 programs depend on multiple source files;

3839 21 programs depend on one source file only.

3840 (These numbers and the ones in the following text ignore the MH library

3841 as well as shell scripts and multiple names for the same program.)

3842 .P

3843 Splitting the source code of a large program into multiple files can

3844 increase the readability of its source code,

3845 but most of the mmh tools are small and straight-forward programs.

3846 In exception of the MIME handling tools (i.e.

3847 .Pn mhbuild ,

3848 .Pn mhstore ,

3849 .Pn show ,

3850 etc.),

3851 .Pn pick

3852 is the only tool with more than one thousand lines of source code.

3853 Splitting programs with less than one thousand lines of code into

3854 multiple source files leads seldom to better readability.

3855 For such tools, splitting still makes sense

3856 when parts of the code are reused in other programs

3857 and the reused code fragment is (1) not general enough

3858 for including it in the MH library

3859 or (2) has dependencies on a library that only few programs need.

3860 .Fn uip/packsbr.c ,

3861 for instance, provides the core program logic for the

3862 .Pn packf

3863 and

3864 .Pn rcvpack

3865 programs.

3866 .Fn uip/packf.c

3867 and

3868 .Fn uip/rcvpack.c

3869 mainly wrap the core function appropriately.

3870 No other tools use the folder packing functions.

3871 As another example,

3872 .Fn uip/termsbr.c

3873 accesses terminal properties, which requires linking with the

3874 \fItermcap\fP or a \fIcurses\fP library.

3875 If

3876 .Fn uip/termsbr.c

3877 is included in the MH library, then every program needs to be linked

3878 with termcap or curses, although only few of the programs use

3879 the library.

3880 .P

3881 The task of MIME handling is complex enough that splitting its code

3882 into multiple source files improves the readability.

3883 The program

3884 .Pn mhstore ,

3885 for instance, is compiled out of seven source files with 2\|500

3886 lines of code in summary.

3887 The main code file

3888 .Fn uip/mhstore.c

3889 consists of 800 lines; the other 1\|700 lines are code reused in

3890 other MIME handling tools.

3891 It seems to be worthwhile to bundle the generic MIME handling code into

3892 a MH-MIME library, as a companion to the MH standard library.

3893 This is left to be done.

3894 .P

3895 The work already accomplished focussed on the non-MIME tools.

3896 The amount of code compiled into each program was reduced.

3897 This eases the understanding of the code base.

3898 In nmh,

3899 .Pn comp

3900 was built from six source files:

3901 .Fn comp.c ,

3902 .Fn whatnowproc.c ,

3903 .Fn whatnowsbr.c ,

3904 .Fn sendsbr.c ,

3905 .Fn annosbr.c ,

3906 and

3907 .Fn distsbr.c .

3908 In mmh, it builds from only two:

3909 .Fn comp.c

3910 and

3911 .Fn whatnowproc.c .

3912 In nmh's

3913 .Pn comp ,

3914 the core function of

3915 .Pn whatnow ,

3916 .Pn send ,

3917 and

3918 .Pn anno

3919 were all compiled into

3920 .Pn comp .

3921 This saved the need to execute these programs with

3922 the expensive system calls

3923 .Fu fork()

3924 and

3925 .Fu exec() .

3926 Whereas this approach improved the time performance,

3927 it interwove the source code.

3928 Core functionalities were not encapsulated into programs but into

3929 function, which were then wrapped by programs.

3930 For example,

3931 .Fn uip/annosbr.c

3932 included the function

3933 .Fu annotate() .

3934 Each program that wanted to annotate messages, included the source file

3935 .Fn uip/annosbr.c

3936 and called

3937 .Fu annotate() .

3938 Because the function

3939 .Fu annotate()

3940 was used like the tool

3941 .Pn anno ,

3942 it had seven parameters, reflecting the command line switches of the tool.

3943 When another pair of command line switches was added to

3944 .Pn anno ,

3945 a rather ugly hack was implemented to avoid adding another parameter

3946 to the function

3947 .Ci d9b1d57351d104d7ec1a5621f090657dcce8cb7f .

3948 .P

3949 In mmh, the relevant code of

3950 .Pn comp

3951 comprises the two files

3952 .Fn uip/comp.c

3953 and

3954 .Fn uip/whatnowproc.c ,

3955 together 210 lines of code,

3956 whereas in nmh,

3957 .Pn comp

3958 comprises six files with 2\|450 lines.

3959 Not all of the code in these six files is actually used by

3960 .Pn comp ,

3961 but the reader needed to read it all to know which parts are relevant.

3962 Understanding nmh's

3963 .Pn comp ,

3964 required understanding the inner workings of

3965 .Fn uip/annosbr.c

3966 first.

3967 To be sure to fully understand a program, its whole source code needs

3968 to be examined.

3969 Not doing so is a leap of faith, assuming that the developers

3970 have avoided obscure programming techniques.

3971 Here, it should be recalled that information passed in obscure ways

3972 through the program's source base, due to the aforementioned hack

3973 to save an additional parameter in nmh's

3974 .Pn anno .

3975 .P

3976 In mmh, understanding

3977 .Pn comp

3978 requires to read only 210 lines of code to read, whereas the amount

3979 is ten times more for nmh's

3980 .Pn comp .

3981 .P

3982 By separating the tools on the program-level,

3983 the boundaries are clearly visible, as the interfaces are calls to

3984 .Fu exec()

3985 rather than arbitrary function calls.

3986 Additionally, this kind of separation is more strict because

3987 it is technically enforced by the operating system;

3988 it can not be simply bypassed with global variables.

3989 Good separation simplifies the understanding of program code

3990 because the area influenced by any particular statement is small.

3991 As I have read a lot in nmh's code base during the last two years,

3992 I have learned about the easy and the difficult parts.

3993 In my observation, the understanding of code is enormously eased

3994 if the influenced area is small and clearly bounded.

3995 .P

3996 Yet, the real problem is another:

3997 Nmh violates the golden ``one tool, one job'' rule of the Unix philosophy.

3998 Understanding

3999 .Pn comp

4000 requires understanding

4001 .Fn uip/annosbr.c

4002 and

4003 .Fn uip/sendsbr.c

4004 because

4005 .Pn comp

4006 annotates and sends messages.

4007 In nmh, there surely exist the tools

4008 .Pn anno

4009 and

4010 .Pn send ,

4011 which cover these jobs,

4012 but

4013 .Pn comp

4014 and

4015 .Pn repl

4016 and

4017 .Pn forw

4018 and

4019 .Pn dist

4020 and

4021 .Pn whatnow

4022 and

4023 .Pn viamail

4024 \(en they all (!) \(en

4025 have the same annotating and sending functions included, once more.

4026 As a result,

4027 .Pn comp

4028 sends messages without using

4029 .Pn send .

4030 The situation is the same as if

4031 .Pn grep

4032 would page its output without using

4033 .Pn more

4034 just because both programs are part of the same code base.

4035 .P

4036 The clear separation on the surface of nmh

4037 \(en the tool chest approach \(en

4038 is violated on the level below.

4039 This violation is for the sake of time performance.

4040 Decades ago, sacrificing readability and conceptional beauty

4041 for speed might have been necessary to prevent MH from being

4042 unusably slow, but today this is not the case anymore.

4043 No longer should speed improvements that became unnecessary be kept.

4044 No longer should readability or conceptional beauty be sacrificed.

4045 No longer should the Unix philosophy's ``one tool, one job''

4046 guideline be violated.

4047 Therefore, mmh's

4048 .Pn comp

4049 no longer sends messages.

4050 .P

4051 In mmh, different jobs are divided among separate programs that

4052 invoke each other as needed.

4053 In consequence,

4054 .Pn comp

4055 invokes

4056 .Pn whatnow

4057 which thereafter invokes

4058 .Pn send

4059 .Ci 3df5ab3c116e6d4a2fb4bb5cc9dfc5f781825815

4060 .Ci c73c00bfccd22ec77e9593f47462aeca4a8cd9c0 .

4061 The clear separation on the surface is maintained on the level below.

4062 Human users and other tools use the same interface \(en

4063 annotations, for example, are made by invoking

4064 .Pn anno ,

4065 no matter if requested by programs or by human beings

4066 .Ci 469a4163c2a1a43731d412eaa5d9cae7d670c48b

4067 .Ci aed384169af5204b8002d06e7a22f89197963d2d

4068 .Ci 3caf9e298a8861729ca8b8a84f57022b6f3ea742 .

4069 .P

4070 .ZZ -1

4071 The decrease of tools built from multiple source files and thus

4072 the decrease of

4073 .Fn uip/*sbr.c

4074 files confirm the improvement

4075 .Ci 9e6d91313f01c96b4058d6bf419a8ca9a207bc33

4076 .ci 81744a46ac9f845d6c2b9908074d269275178d2e

4077 .Ci f0f858069d21111f0dbea510044593f89c9b0829

4078 .Ci 0503a6e9be34f24858b55b555a5c948182b9f24b

4079 .Ci 27826f9353e0f0b04590b7d0f8f83e60462b90f0

4080 .Ci d1da1f94ce62160aebb30df4063ccbc53768656b

4081 .Ci c42222869e318fff5dec395eca3e776db3075455 .

4082 This is also visible in the complexity of the build dependency graphs:

4083

4084 .sp

4085 Nmh:

4086 .BP input/deps-nmh.eps .5i

4087 .EP

4088 .sp

4089 Mmh:

4090 .BP input/deps-mmh.eps .8i

4091 .EP

4092

4093 The figures display all program to source file relationships where

4094 programs (ellipses) are built from multiple source files (rectangles).

4095 The primary source file of each program is omited from the graph.