docs/master: 1fa5a74bf138 discussion.roff

docs/master

view discussion.roff @ 223:1fa5a74bf138

Reworked References structure: Collect them all in one Chapter. Have list of websites separately.

author	markus schnalke <meillo@marmaro.de>
date	Sun, 15 Jul 2012 19:16:35 +0200
parents	95257474a123
children	b0e83cfde2bf

line source

1 .H0 "Discussion

2 .P

3 This main chapter discusses the practical work accomplished in the

4 mmh project.

5 It is structured along the goals chosen for the project.

6 A selection of the work undertaken

7 is described.

8 .P

9 This discussion compares the present version of mmh with the state of

10 nmh at the time when the mmh project had started, i.e. fall 2011.

11 Recent changes in nmh are seldom part of the discussion.

12 Sometimes they are mentioned in side notes.

13 .P

14 For the reader's convenience, the structure of modern email systems

15 is depicted in the following figure.

16 It illustrates the path a message takes from sender to recipient.

18 .sp 1.5

19 .KS

20 .in 2c

21 .so input/mail-agents.pic

22 .KE

23 .sp 1.5

25 .LP

26 The ellipses denote mail agents, i.e. different jobs in email processing.

27 These are:

28 .IP "Mail User Agent (MUA)

29 The only program users directly interact with.

30 It includes functions to compose new mail, display received mail,

31 and to manage the mail storage.

32 It is called a \fImail client\fP as well.

33 .IP "Mail Submission Agent (MSA)

34 A special kind of Mail Transfer Agent, used to submit mail into the

35 mail transport system.

36 Often it is also called an MTA.

37 .IP "Mail Transfer Agent (MTA)

38 A node in the mail transport system.

39 It transfers incoming mail to a transport node nearer to the

40 final destination.

41 An MTA may be the final destination itself.

42 .IP "Mail Delivery Agent (MDA)

43 Delivers mail according to a set of rules.

44 Usually, the messages are stored to disk.

45 .IP "Mail Retrieval Agent (MRA)

46 Initiates the transfer of mail from a remote location to the local machine.

47 (The dashed arrow in the figure represents the pull request.)

48 .LP

49 The dashed boxes represent entities that usually reside on single machines.

50 The box on the lower left represents the sender's system.

51 The box on the upper left represents the first mail transfer node.

52 The box on the upper right represents the transfer node responsible for the

53 destination address.

54 The box on the lower right represents the recipient's system.

55 Often, the boxes above the dotted line are servers on the Internet.

56 Many mail clients, including nmh, include all of the components below

57 the dotted line.

58 This is not the case for mmh; it implements the MUA only.

65 .\" --------------------------------------------------------------

66 .H1 "Streamlining

68 .P

69 MH once provided a complete email system.

70 The community around nmh tries to keep nmh in similar shape.

71 In fundamental contrast, mmh shall be an MUA only.

72 I believe that the development of all-in-one mail systems is obsolete.

73 Today, email is too complex to be fully covered by a single project.

74 Such a project will not be able to excel in all aspects.

75 Instead, the aspects of email should be covered by multiple projects,

76 which then can be combined to form a complete system.

77 Excellent implementations for the various aspects of email already exist.

78 Just to name three examples: Postfix is a specialized MTA, Procmail

79 is a specialized MDA, and Fetchmail is a specialized MRA.

80 I believe that it is best to use such specialized tools instead of

81 providing the same function once more as a side component.

82 .P

83 Doing something well requires focusing on a small set of specific aspects.

84 Under the assumption that development which is focussed on a particular

85 area produces better results there, specialized projects will be superior

86 in their field of focus.

87 Hence, all-in-one mail system projects \(en no matter if monolithic

88 or modular \(en will never be the best choice in any of the fields.

89 Even in providing the most consistent all-in-one system, they are likely

90 to be beaten by projects that focus exclusively on the creation

91 of a homogeneous system by integrating existing mail components.

92 .P

93 Usually, the limiting resource in the community development of

94 free software is man power.

95 .\" XXX FIXME ref!

96 If the development effort is spread over a large development area,

97 it becomes more difficult to compete with the specialists in the

98 various fields.

99 The concrete situation for MH-based mail systems is even tougher,

100 given their small and aged community, concerning both developers and users.

101 .P

102 In consequence, I believe that the available development resources

103 should focus on the point where MH is most unique.

104 This is clearly the user interface \(en the MUA.

105 Peripheral parts should be removed to streamline mmh for the MUA task.

106

107

108 .H2 "Mail Transfer Facilities

109 .Id mail-transfer-facilities

110 .P

111 The removal of the mail transfer facilities, effectively dropping the

112 MSA and MRA, had been the first work task in the mmh project.

113 The desire for this change initiated the creation of the mmh project.

114 .P

115 Focusing on one mail agent role only, is motivated by Eric Allman's

116 experience with Sendmail.

117 He identified the limitation of Sendmail

118 to the MTA task as one reason for its success:

119 .[ [

120 costales sendmail

121 .], p. xviii]

122 .QS

123 Second, I limited myself to the routing function \(en

124 I wouldn't write user agents or delivery back-ends.

125 This was a departure of the dominant thought of the time,

126 in which routing logic, local delivery, and often the network code

127 were incorporated directly into the user agents.

128 .QE

129 .P

130 In nmh, the MSA is called \fIMessage Transfer Service\fP (MTS).

131 This facility, implemented by the

132 .Pn post

133 command, establishes network connections and spoke SMTP to submit

134 messages to be relayed to the outside world.

135 When email transfer changed, this part needed to be changed as well.

136 Encryption and authentication for network connections

137 needed to be supported, hence TLS and SASL were introduced into nmh.

138 This added complexity without improving the core functions.

139 Furthermore, keeping up with recent developments in the field of

140 mail transfer requires development power and specialists.

141 In mmh, this whole facility was simply cut off

142 .Ci f6aa95b724fd8c791164abe7ee5468bf5c34f226

143 .Ci fecd5d34f65597a4dfa16aeabea7d74b191532c3

144 .Ci 156d35f6425bea4c1ed3c4c79783dc613379c65b .

145 Instead, mmh depends on an external MSA.

146 All outgoing mail in mmh goes through the

147 .Pn sendmail

148 command, which almost any MSA provides.

149 If not, a wrapper program can be written.

150 It must read the message from the standard input, extract the

151 recipient addresses from the message header, and hand the message

152 over to the MSA.

153 For example, a wrapper script for qmail would be:

154 .VS

155 #!/bin/sh

156 exec qmail-inject # ignore command line arguments

157 VE

158 The requirement to parse the recipient addresses out of the message header

159 may be removed in the future.

160 Mmh could pass the recipient addresses as command line arguments.

161 This appears to be the better interface.

162 .P

163 To retrieve mail, the

164 .Pn inc

165 command in nmh acts as MRA.

166 It establishes network connections

167 and speaks POP3 to retrieve mail from remote servers.

168 As with mail submission, the network connections required encryption and

169 authentication, thus TLS and SASL were added to nmh.

170 Support for message retrieval through IMAP will soon become necessary

171 additions, too, and likewise for any other changes in mail transfer.

172 But not in mmh because it has dropped the support for retrieving mail

173 from remote locations

174 .Ci ab7b48411962d26439f92f35ed084d3d6275459c .

175 Instead, it depends on an external tool to cover this task.

176 Mmh has two paths for messages to enter mmh's mail storage:

177 (1) Mail can be incorporated with

178 .Pn inc

179 from the system maildrop, or (2) with

180 .Pn rcvstore

181 by reading them, one at a time, from the standard input.

182 .P

183 With the removal of the MSA and MRA, mmh converted from a complete

184 mail system to only an MUA.

185 Now, of course, mmh depends on third-party software.

186 An external MSA is required to transfer mail to the outside world;

187 an external MRA is required to retrieve mail from remote machines.

188 Excellent implementations of such software exist.

189 They likely are superior to the internal versions that were removed.

190 Additionally, the best suiting programs can be chosen freely.

191 .P

192 As it had already been possible to use an external MSA and MRA,

193 why should the internal version not be kept for convenience?

194 .\" XXX commas correct?

195 Transfered to a different area,

196 the question whether there is sense in having a fall-back pager in all

197 the command line tools, for the cases when

198 .Pn more

199 or

200 .Pn less

201 are not available, appears to be ridiculous.

202 Of course, MSAs and MRAs are more complex than text pagers

203 and not necessarily available but still the concept of orthogonal

204 design holds: ``Write programs that do one thing and do it well''.

205 .[

206 mcilroy unix phil

207 p. 53

208 .]

209 .[

210 mcilroy bstj foreword

211 .]

212 Here, this part of the Unix philosophy was applied not only

213 to the programs but to the project itself.

214 In other words:

215 Develop projects that focus on one thing and do it well.

216 Projects which have grown complex should be split, for the same

217 reasons that programs which have grown complex should be split.

218 If it is conceptionally more elegant to have the MSA and MRA as

219 separate projects then they should be separated.

220 In my opinion, this is the case.

221 The RFCs suggest this separation by clearly distinguishing the

222 different mail handling tasks [RFC\|821].

223 The small interfaces between the mail agents support the

224 separation as well.

225 .P

226 Once, email had been small and simple.

227 At that time,

228 .Pn /bin/mail

229 had covered everything there was to email and still was small and simple.

230 Later, the essential complexity of email increased.

231 (Essential complexity is the complexity defined by the problem itself.\&

232 .[ [

233 brooks no silver bullet

234 .]])

235 Consequently, email systems grew.

236 RFCs started to introduce the concept of mail agents to separate the

237 various roles because they became more extensive and because

238 new roles appeared.

239 As mail system implementations grew, parts of them were split off.

240 For instance, a POP server was included in the original MH;

241 it was removed in nmh.

242 Now is the time to go one step further and split off the MSA and MRA,

243 as well.

244 Not only does this decrease the code size of the project,

245 more importantly, it unburdens mmh of the whole field of

246 message transfer, with all its implications for the project.

247 There is no more need for concern with changes in network transfer.

248 This independence is gained by depending on external components

249 that cover the field.

250 .P

251 In general, functionality can be added in three different ways:

252 .LI 1

253 By implementing the function in the project itself.

254 .LI 2

255 By depending on a library that provides the function.

256 .LI 3

257 By depending on a program that provides the function.

258 .LP

259 .\" XXX Rework sentence

260 While implementing the function in the project itself leads to the

261 largest increase in code size and requires the most maintenance

262 and development work,

263 it keeps the project's dependence on other software lowest.

264 Using libraries or external programs requires less maintenance work

265 but introduces dependencies on external projects.

266 Programs have the smallest interfaces and provide the best separation,

267 but possibly limit the information exchange.

268 External libraries are more strongly connected than external programs,

269 thus information can be exchanged in a more flexible manner.

270 Adding code to a project increases maintenance work.

271 .\" XXX ref

272 As implementing complex functions in the project itself adds

273 a lot of code, this should be avoided if possible.

274 Thus, the dependencies only change in their character,

275 not in their existence.

276 In mmh, library dependencies on

277 .Pn libsasl2

278 and

279 .Pn libcrypto /\c

280 .Pn libssl

281 were traded against program dependencies on an MSA and an MRA.

282 This also meant trading build-time dependencies against run-time

283 dependencies.

284 Besides providing stronger separation and greater flexibility,

285 program dependencies also allowed

286 over 6\|000 lines of code to be removed from mmh.

287 This made mmh's code base about 12\|% smaller.

288 Reducing the project's code size by such an amount without actually

289 losing functionality is a convincing argument.

290 Actually, as external MSAs and MRAs are likely superior to the

291 project's internal versions, the common user even gains functionality.

292 .P

293 Users of MH should not have problems setting up an external MSA and MRA.

294 Also, the popular MSAs and MRAs have large communities and a lot

295 of available documentation.

296

297 Choices for MSAs range from small forwarders such as \fIssmtp\fP and

298 \fInullmailer\fP, over mid-size MTAs including \fImasqmail\fP and

299 \fIdma\fP, up to full-featured MTAs as for instance \fIPostfix\fP.

300 MRAs are provided for example by \fIfetchmail\fP, \fIgetmail\fP,

301 \fImpop\fP, and \fIfdm\fP.

302

303

304 .H2 "Non-MUA Tools

305 .P

306 One goal of mmh is to remove the tools that do not significantly

307 contribute to the MUA's job.

308 Loosely related and rarely used tools distract from a lean appearance,

309 and require maintenance work without adding much to the core task.

310 By removing these tools, mmh became more streamlined and focused.

311 .BU

312 .Pn conflict

313 was removed

314 .Ci 8b235097cbd11d728c07b966cf131aa7133ce5a9

315 because it is a mail system maintenance tool and not MUA-related.

316 It even checked

317 .Fn /etc/passwd

318 and

319 .Fn /etc/group

320 for consistency, which is completely unrelated to email.

321 A tool like

322 .Pn conflict

323 is surely useful, but it should not be shipped with mmh.

324 .\" XXX historic reasons?

325 .BU

326 .Pn rcvtty

327 was removed

328 .Ci 14767c94b3827be7c867196467ed7aea5f6f49b0

329 because its use case of writing to the user's terminal

330 on reception of mail is obsolete.

331 If users like to be informed of new mail, the shell's

332 .Ev MAILPATH

333 variable or graphical notifications are technically more appealing.

334 Writing to terminals directly is hardly ever desired today.

335 If, though, one prefers this approach, the standard tool

336 .Pn write

337 can be used in a way similar to:

338 .VS

339 scan -file - | write `id -un`

340 VE

341 .BU

342 .Pn viamail

343 .\" XXX was macht viamail

344 was removed

345 .Ci eda72d6a7a7c20ff123043fb7f19c509ea01f932

346 when the new attachment system was activated, because

347 .Pn forw

348 could then cover the task itself.

349 The program

350 .Pn sendfiles

351 was rewritten as a shell script wrapper around

352 .Pn forw .

353 .Ci 0e82199cf3c991a173e0ac8aa776efdb3ded61e6

354 .BU

355 .Pn msgchk

356 .\" XXX was macht msgchk

357 was removed

358 .Ci bb9360ead7eb7a3fedcce2eeedfc660014e41dbe ,

359 because it lost its use case when POP support was removed.

360 A call to

361 .Pn msgchk

362 provided hardly more information than:

363 .VS

364 ls -l /var/mail/meillo

365 VE

366 Yet, it distinguished between old and new mail, but

367 these details can be retrieved with

368 .Pn stat (1),

369 too.

370 A small shell script could be written to print the information

371 in a similar way, if truly necessary.

372 As mmh's

373 .Pn inc

374 only incorporates mail from the user's local maildrop,

375 and thus no data transfers over slow networks are involved,

376 there is hardly any need to check for new mail before incorporating it.

377 .BU

378 .Pn msh

379 was removed

380 .Ci 916690191222433a6923a4be54b0d8f6ac01bd02

381 because the tool was in conflict with the philosophy of MH.

382 It provided an interactive shell to access the features of MH.

383 However, it was not just a shell tailored to the needs of mail handling,

384 but one large program that had several MH tools built in.

385 This conflicted with the major feature of MH of being a tool chest.

386 .Pn msh 's

387 main use case had been accessing Bulletin Boards, which have ceased to

388 be popular.

389 .P

390 Removing

391 .Pn msh

392 together with the truly archaic code relics

393 .Pn vmh

394 and

395 .Pn wmh

396 saved more than 7\|000 lines of C code \(en

397 about 15\|% of the project's original source code amount.

398 Having less code \(en with equal readability, of course \(en

399 for the same functionality is an advantage.

400 Less code means less bugs and less maintenance work.

401 As

402 .Pn rcvtty

403 and

404 .Pn msgchk

405 are assumed to be rarely used and can be implemented in different ways,

406 why should one keep them?

407 Removing them streamlined mmh.

408 .Pn viamail 's

409 use case is now partly obsolete and partly covered by

410 .Pn forw ,

411 hence there is no reason to still maintain it.

412 .Pn conflict

413 is not related to the mail client, and

414 .Pn msh

415 conflicts with the basic concept of MH.

416 These two tools might still be useful, but they should not be part of mmh.

417 .P

418 .Id slocal

419 Finally, there is

420 .Pn slocal ,

421 which is an MDA and thus not directly MUA-related.

422 It should be removed from mmh because including it conflicts with

423 the idea that mmh is an MUA only.

424 However,

425 .Pn slocal

426 provides rule-based processing of messages, like filing them into

427 different folders, which is otherwise not available in mmh.

428 Although

429 .Pn slocal

430 neither pulls in dependencies, nor does it include a separate

431 technical area (cf. Sec.

432 .Cf mail-transfer-facilities ),

433 it still accounts for about 1\|000 lines of code that need to be maintained.

434 As

435 .Pn slocal

436 is almost self-standing, it should be split off into a separate project.

437 This would cut the strong connection between the MUA mmh and the MDA

438 .Pn slocal .

439 For anyone not using MH,

440 .Pn slocal

441 would become yet another independent MDA, like

442 .I procmail .

443 Then

444 .Pn slocal

445 could be installed without a complete MH system.

446 Likewise, mmh users could decide to use

447 .I procmail

448 without having a second, unused MDA, i.e.

449 .Pn slocal ,

450 installed.

451 That appears to be conceptionally the best solution.

452 Yet,

453 .Pn slocal

454 is not split off.

455 I defer the decision over

456 .Pn slocal

457 out of a need for deeper investigation.

458 In the meanwhile, it remains part of mmh

459 as its continued existence is not significant;

460 .Pn slocal

461 is unrelated to the rest of the project.

462

463

464

465 .H2 "Displaying Messages

466 .Id mhshow

467 .P

468 Since the very beginning, already in the first concept paper,

469 .\" XXX ref!!!

470 .Pn show

471 had been MH's message display program.

472 .Pn show

473 mapped message numbers and sequences to files and invoked

474 .Pn mhl

475 to have the files formatted.

476 With MIME, this approach was not sufficient anymore.

477 MIME messages can consist of multiple parts.

478 Some parts, like binary attachments or text content in foreign charsets,

479 are not directly displayable.

480 .Pn show 's

481 understanding of messages and

482 .Pn mhl 's

483 display capabilities could not cope with the task any longer.

484 .P

485 Instead of extending these tools, additional tools were written from

486 scratch and were added to the MH tool chest.

487 Doing so is encouraged by the tool chest approach.

488 Modular design is a great advantage for extending a system,

489 as new tools can be added without interfering with existing ones.

490 First, the new MIME features were added in form of the single program

491 .Pn mhn .

492 The command

493 .Cl "mhn -show 42

494 had then shown the message number

495 .Fn 42 ,

496 interpreting MIME.

497 With the 1.0 release of nmh in February 1999, Richard Coleman finished

498 the split of

499 .Pn mhn

500 into a set of specialized tools, which together covered the

501 multiple aspects of MIME.

502 One of them was

503 .Pn mhshow ,

504 which replaced

505 .Cl "mhn -show" .

506 It was capable of displaying MIME messages appropriately.

507 .P

508 From then on, two message display tools were part of nmh,

509 .Pn show

510 and

511 .Pn mhshow .

512 To ease the life of users,

513 .Pn show

514 was extended to automatically hand the job over to

515 .Pn mhshow

516 if displaying the message would be beyond

517 .Pn show 's

518 abilities.

519 In consequence, the user would simply invoke

520 .Pn show

521 (possibly through

522 .Pn next

523 or

524 .Pn prev )

525 and get the message printed with either

526 .Pn show

527 or

528 .Pn mhshow ,

529 whatever was more appropriate.

530 .P

531 Having two similar tools for basically the same task is redundancy.

532 Usually, users do not distinguish between

533 .Pn show

534 and

535 .Pn mhshow

536 in their daily mail reading.

537 Having two separate display programs was therefore unnecessary

538 from a user's point of view.

539 Besides, the development of both programs needed to be in sync,

540 to ensure that the programs behaved in a similar way,

541 because they were used like a single tool.

542 Different behavior would have surprised the user.

543 .P

544 Today, non-MIME messages are rather seen to be a special case of

545 MIME messages, although it is the other way round.

546 As

547 .Pn mhshow

548 already had been able to display non-MIME messages, it appeared natural

549 to drop

550 .Pn show

551 in favor of using

552 .Pn mhshow

553 exclusively

554 .Ci 4c1efddfd499300c7e74263e57d8aa137e84c853 .

555 Removing

556 .Pn show

557 is no loss in function, because

558 .Pn mhshow

559 covers it completely.

560 Yet, the old behavior of

561 .Pn show

562 can still be emulated with the simple command line:

563 .VS

564 mhl `mhpath c`

565 VE

566 .P

567 For convenience,

568 .Pn mhshow

569 was renamed to

570 .Pn show

571 after

572 .Pn show

573 was gone.

574 It is clear that such a rename may confuse future developers when

575 trying to understand the history.

576 Nevertheless, I consider the convenience on the user's side,

577 to outweigh the inconvenience for understanding the evolution

578 of the tools.

579 .P

580 To prepare for the transition,

581 .Pn mhshow

582 was reworked to behave more like

583 .Pn show

584 first (cf. Sec.

585 .Cf mhshow ).

586 .\" XXX code commits?

587 Once the tools behaved more alike, the replacing appeared to be

588 even more natural.

589 Today, mmh's new

590 .Pn show

591 has become the one single message display program once again,

592 with the difference

593 that today it handles MIME messages as well as non-MIME messages.

594 The outcomes of the transition are one program less to maintain,

595 no second display program for users to deal with,

596 and less system complexity.

597 .P

598 Still, removing the old

599 .Pn show

600 hurts in one regard: It had been such a simple program.

601 Its lean elegance is missing from the new

602 .Pn show ,

603 .\" XXX

604 however there is no alternative;

605 supporting MIME demands higher essential complexity.

606

607 .ig

608 XXX

609 Consider including text on scan listings here

610

611 Scan listings shall not contain body content. Hence, removed this feature.

612 Scan listings shall operator on message headers and non-message information

613 only. Displaying the beginning of the body complicates everything too much.

614 That's no surprise, because it's something completely different. If you

615 want to examine the body, then use show(1)/mhshow(1).

616 Changed the default scan formats accordingly.

617 .Ci 70b2643e0da8485174480c644ad9785c84f5bff4

618 ..

619

620

621

622

623 .H2 "Configure Options

624 .P

625 Customization is a double-edged sword.

626 It allows better suiting setups, but not for free.

627 There is the cost of code complexity to be able to customize.

628 There is the cost of less tested setups, because there are

629 more possible setups and especially corner cases.

630 Additionally, there is the cost of choice itself.

631 The code complexity directly affects the developers.

632 Less tested code affects both users and developers.

633 The problem of choice affects the users, for once by having to choose

634 but also by more complex interfaces that require more documentation.

635 Whenever options add few advantages but increase the complexity of the

636 system, they should be considered for removal.

637 I have reduced the number of project-specific configure options from

638 15 to 3.

639

640 .U3 "Mail Transfer Facilities

641 .P

642 With the removal of the mail transfer facilities 5 configure

643 options vanished:

644 .P

645 The switches

646 .Sw --with-tls

647 and

648 .Sw --with-cyrus-sasl

649 had activated the support for transfer encryption and authentication.

650 .\" XXX cf

651 .\" XXX gruende kurz wiederholen

652 They are not needed anymore.

653 .Ci fecd5d34f65597a4dfa16aeabea7d74b191532c3

654 .Ci 156d35f6425bea4c1ed3c4c79783dc613379c65b

655 .P

656 .\" XXX cf

657 .\" XXX ``For the same reason ...''

658 The configure switch

659 .Sw --enable-pop

660 activated the message retrieval facility.

661 Whereas the code area that had been conditionally compiled in

662 for TLS and SASL support was small,

663 the conditionally compiled code area for POP support was much larger.

664 The code base had only changed slightly on toggling TLS or SASL

665 support but it had changed much on toggling POP support.

666 The changes in the code base could hardly be overviewed.

667 By having POP support togglable, a second code base had been created,

668 one that needed to be tested.

669 This situation is basically similar for the conditional TLS and SASL

670 code, but there the changes are minor and can yet be overviewed.

671 Still, conditional compilation of a code base creates variations

672 of the original program.

673 More variations require more testing and maintenance work.

674 .P

675 Two other options only specified default configuration values:

676 .Sw --with-mts

677 defined the default transport service

678 .Ci f6aa95b724fd8c791164abe7ee5468bf5c34f226 .

679 With

680 .Sw --with-smtpservers

681 default SMTP servers could be specified

682 .Ci 128545e06224233b7e91fc4c83f8830252fe16c9 .

683 Both of them became irrelevant when the SMTP transport service was removed.

684 .\" XXX code ref

685 In mmh, all messages are handed over to

686 .Pn sendmail

687 for transportation.

688

689

690 .U3 "Backup Prefix

691 .P

692 The backup prefix is the string that was prepended to message

693 filenames to tag them as deleted.

694 By default it had been the comma character (`\fL,\fP').

695 .\" XXX Zeitlich ordnen

696 In July 2000, Kimmo Suominen introduced

697 the configure option

698 .Sw --with-hash-backup

699 to change the default to the hash character `\f(CW#\fP'.

700 This choice was probably personal preference, but,

701 being related or not, words that start with the hash character

702 introduce a comment in the Unix shell.

703 Thus, the command line

704 .Cl "rm #13 #15

705 calls

706 .Pn rm

707 without arguments because the first hash character starts a comment

708 that reaches until the end of the line.

709 To delete the backup files,

710 .Cl "rm ./#13 ./#15"

711 needs to be used.

712 Thus, using the hash as backup prefix may be seen as a precaution

713 against backup loss.

714 .P

715 First, I removed the configure option but added the profile entry

716 .Pe Backup-Prefix ,

717 which allowed to specify an arbitrary string as backup prefix

718 .Ci 6c40d481d661d532dd527eaf34cebb6d3f8ed086 .

719 This change did not remove the choice but moved it to a location where

720 it suited better, in my eyes.

721 .P

722 Eventually however, the new trash folder concept

723 (cf. Sec.

724 .Cf trash-folder )

725 removed the need for the backup prefix completely.

726 .Ci 8edc5aaf86f9f77124664f6801bc6c6cdf258173

727 .Ci ca0b3e830b86700d9e5e31b1784de2bdcaf58fc5

728

729

730 .U3 "Editor and Pager

731 .P

732 The two configure options

733 .CW --with-editor=EDITOR

734 .CW --with-pager=PAGER

735 were used to specify the default editor and pager at configure time.

736 Doing so at configure time made sense in the eighties,

737 when the set of available editors and pagers varied much across

738 different systems.

739 Today, the situation is more homogeneous.

740 The programs

741 .Pn vi

742 and

743 .Pn more

744 can be expected to be available on every Unix system,

745 as they are specified by POSIX since two decades.

746 (The specifications for

747 .Pn vi

748 and

749 .Pn more

750 appeared in

751 .[

752 posix 1987

753 .]

754 and,

755 .[

756 posix 1992

757 .]

758 respectively.)

759 As a first step, these two tools were hard-coded as defaults

760 .Ci 5d43a99db70c12a673028c7758c20cbe3e13ef5f .

761 Not changed were the

762 .Pe editor

763 and

764 .Pe moreproc

765 profile entries, which allowed the user to override the system defaults.

766 Later, the concept was reworked again to respect the standard

767 environment variables

768 .Ev VISUAL

769 and

770 .Ev PAGER

771 if they are set.

772 Today, mmh determines the editor to use in the following order,

773 taking the first available and non-empty item

774 .Ci f85f4b7ae62e3d05a945dcd46ead51f0a2a89a9b :

775 .LI 1

776 Environment variable

777 .Ev MMHEDITOR

778 .LI 2

779 Profile entry

780 .Pe Editor

781 .LI 3

782 Environment variable

783 .Ev VISUAL

784 .LI 4

785 Environment variable

786 .Ev EDITOR

787 .LI 5

788 Command

789 .Pn vi .

790 .LP

791 The pager to use is determined in a similar order

792 .Ci 0c4214ea2aec6497d0d67b436bbee9bc1d225f1e :

793 .LI 1

794 Environment variable

795 .Ev MMHPAGER

796 .LI 2

797 Profile entry

798 .Pe Pager

799 (replaces

800 .Pe moreproc )

801 .LI 3

802 Environment variable

803 .Ev PAGER

804 .LI 4

805 Command

806 .Pn more .

807 .LP

808 By respecting the

809 .Ev VISUAL /\c

810 .Ev EDITOR

811 and

812 .Ev PAGER

813 environment variables,

814 the new behavior complies with the common style on Unix systems.

815 It is more uniform and clearer for users.

816

817

818 .U3 "ndbm

819 .P

820 .Pn slocal

821 used to depend on the database library

822 .I ndbm .

823 The database is used to store the

824 .Hd Message-ID

825 header field values of all messages delivered.

826 This enabled

827 .Pn slocal

828 to suppress delivering the same message to the same user twice.

829 This features was enabled by the

830 .Sw -suppressdup

831 switch.

832 .P

833 As a variety of versions of the database library exist,

834 .[

835 wolter unix incompat notes dbm

836 .]

837 complicated autoconf code was needed to detect them correctly.

838 Furthermore, the configure switches

839 .Sw --with-ndbm=ARG

840 and

841 .Sw --with-ndbmheader=ARG

842 were added to help with difficult setups that would

843 not be detected automatically or not correctly.

844 .P

845 By removing the suppress duplicates feature of

846 .Pn slocal ,

847 the dependency on

848 .I ndbm

849 vanished and 120 lines of complex autoconf code could be saved

850 .Ci ecd6d6a20cb7a1507e3a20d6c4cb3a1cf14c6bbf .

851 The change removed functionality but that is considered minor to the

852 improvement of dropping the dependency and the complex autoconf code.

853 .\" XXX argument: slocal ist sowieso nicht teil vom mmh kern

854

855 .U3 "MH-E Support

856 .P

857 The configure option

858 .Sw --disable-mhe

859 was removed when the MH-E support was reworked.

860 MH-E is the Emacs front-end to MH.

861 It requires MH to provide minor additional functions.

862 The

863 .Sw --disable-mhe

864 configure option had switched off these extensions.

865 After removing the support for old versions of MH-E,

866 only the

867 .Sw -build

868 switches of

869 .Pn forw

870 and

871 .Pn repl

872 are left to be MH-E extensions.

873 They are now always built in because they add little code and complexity.

874 In consequence, the

875 .Sw --disable-mhe

876 configure option was removed

877 .Ci a7ce7b4a580d77b6c2c4d980812beb589aa4c643 .

878 Dropping the option also removed a variant of the code base

879 that would have needed to be tested.

880 This change was undertaken in January 2012 in nmh and

881 thereafter merged into mmh.

882

883

884 .U3 "Masquerading

885 .P

886 The configure option

887 .Sw --enable-masquerade

888 could take up to three arguments:

889 .Ar draft_from ,

890 .Ar mmailid ,

891 and

892 .Ar username_extension .

893 They activated different types of address masquerading.

894 All of them were implemented in the SMTP-speaking

895 .Pn post

896 command.

897 Address masquerading is an MTA's task and mmh does not cover

898 this field anymore.

899 Hence, true masquerading needs to be implemented in the external MTA.

900 .P

901 The

902 .I mmailid

903 masquerading type is the oldest one of the three and the only one

904 available in the original MH.

905 It provided a

906 .I username

907 to

908 .I fakeusername

909 mapping, based on the

910 .Fn passwd 's

911 GECOS field.

912 Nmh's man page

913 .Mp mh-tailor (5)

914 described the use case as being the following:

915 .QS

916 This is useful if you want the messages you send to always

917 appear to come from the name of an MTA alias rather than your

918 actual account name. For instance, many organizations set up

919 `First.Last' sendmail aliases for all users. If this is

920 the case, the GECOS field for each user should look like:

921 ``First [Middle] Last <First.Last>''

922 .QE

923 .P

924 As mmh sends outgoing mail via the local MTA only,

925 the best location to do such global rewrites is there.

926 Besides, the MTA is conceptionally the right location because it

927 does the reverse mapping for incoming mail (aliasing), too.

928 Furthermore, masquerading set up there is readily available for all

929 mail software on the system.

930 Hence, mmailid masquerading was removed.

931 .Ci 0836c8000ccb34b59410ef1c15b1b7feac70ce5f

932 .P

933 The

934 .I username_extension

935 masquerading type did not replace the username but would append a suffix,

936 specified by the

937 .Ev USERNAME_EXTENSION

938 environment variable, to it.

939 This provided support for the

940 .I user-extension

941 feature of qmail

942 .[ [

943 sill qmail handbook

944 .], p. 141]

945 and the similar

946 .I "plussed user

947 processing of Sendmail.

948 .[ [

949 sendmail costales

950 .], p. 476]

951 The decision to remove this username_extension masquerading

952 was motivated by the fact that

953 .Pn spost

954 had not supported it yet.

955 Username extensions can be used in mmh, but less convenient.

956 .\" XXX covered by next paragraph

957 .\" XXX format file %(getenv USERNAME_EXTENSION)

958 .Ci 2abae0bfd0ad5bf898461e50aa4b466d641f23d9

959 .P

960 The

961 .I draft_from

962 masquerading type instructed

963 .Pn post

964 to use the value of the

965 .Hd From

966 header field as SMTP envelope sender.

967 Sender addresses could be replaced completely.

968 Mmh offers a kind of masquerading similar in effect, but

969 with technical differences.

970 As mmh does not transfer messages itself, the local MTA has final control

971 over the sender's address.

972 Any masquerading mmh introduces may be reverted by the MTA.

973 In times of pedantic spam checking, an MTA will take care to use

974 sensible envelope sender addresses to keep its own reputation up.

975 Nonetheless, the MUA can set the

976 .Hd From

977 header field and thereby propose a sender address to the MTA.

978 The MTA may then decide to take that one or generate the canonical sender

979 address for use as envelope sender address.

980 .Ci b14ea6073f77b4359aaf3fddd0e105989db9

981 .P

982 In mmh, the MTA will always extract the recipient and sender from the

983 message header (\c

984 .Pn sendmail 's

985 .Sw -t

986 switch).

987 The

988 .Hd From

989 header field of the draft may be set arbitrary by the user.

990 If it is missing, the canonical sender address will be generated by the MTA.

991

992 .U3 "Remaining Options

993 .P

994 Two configure options remain in mmh.

995 One is the locking method to use:

996 .Sw --with-locking=[dot|fcntl|flock|lockf] .

997 The idea of removing all methods except the portable

998 .I "dot locking

999 and having that one as the default is appealing, but this change

1000 requires deeper technical investigation into the topic.

1001 The other option,

1002 .Sw --enable-debug ,

1003 compiles the programs with debugging symbols.

1004 This option is likely to stay.

1005

1006

1007

1008

1009 .H2 "Command Line Switches

1010 .P

1011 The command line switches of MH tools follow a style similar to

1012 the X Window System style.

1013 .\" XXX ref

1014 The switches consist of a single dash (`\fL-\fP') followed by a word.

1015 For example

1016 .Cl -truncate .

1017 To ease typing, the word can be abbreviated, given the remaining

1018 prefix is unambiguous.

1019 If no other switch starts with the letter `t', then any of

1020 .Cl "-truncate" ,

1021 .Cl "-trunc" ,

1022 .Cl "-tr" ,

1023 and

1024 .Cl "-t

1025 is equal.

1026 As a result, switches can neither be grouped (as in

1027 .Cl "ls -ltr" )

1028 nor can switch arguments be appended directly to the switch (as in

1029 .Cl "sendmail -q30m" ).

1030 Many switches have negating counter-parts, which start with `no'.

1031 For example

1032 .Cl "-notruncate

1033 inverts the

1034 .Cl "-truncate

1035 switch.

1036 They exist to override the effect of default switches in the profile.

1037 Every program in mmh has two generic switches:

1038 .Sw -help ,

1039 to print a short message on how to use the program, and

1040 .Sw -Version

1041 (with capital `V'), to tell what version of mmh the program belongs to.

1042 .P

1043 Switches change the behavior of programs.

1044 Programs that do one thing in one way require no switches.

1045 In most cases, doing something in exactly one way is too limiting.

1046 If one task should be accomplished in various ways,

1047 switches are a good approach to alter the behavior of a program.

1048 Changing the behavior of programs provides flexibility and customization

1049 to users, but at the same time it complicates the code,

1050 the documentation, and the usage of the program.

1051 .\" XXX: Ref

1052 Therefore, the number of switches should be kept small.

1053 A small set of well-chosen switches is best.

1054 Usually, the number of switches increases over time.

1055 Already in 1985, Rose and Romine have identified this as a major

1056 problem of MH:

1057 .[ [

1058 rose romine real work

1059 .], p. 12]

1060 .QS

1061 A complaint often heard about systems which undergo substantial development

1062 by many people over a number of years, is that more and more options are

1063 introduced which add little to the functionality but greatly increase the

1064 amount of information a user needs to know in order to get useful work done.

1065 This is usually referred to as creeping featurism.

1066 .QP

1067 Unfortunately MH, having undergone six years of off-and-on development by

1068 ten or so well-meaning programmers (the present authors included),

1069 suffers mightily from this.

1070 .QE

1071 .P

1072 Being reluctant to adding new switches (or \fIoptions\fP,

1073 as Rose and Romine call them) is one part of a counter-action,

1074 the other part is removing hardly used switches.

1075 Nmh's tools have lots of switches already implemented.

1076 Hence, cleaning up by removing some of them was the more important part

1077 of the counter-action.

1078 Removing existing functionality is always difficult because it

1079 breaks programs that use these functions.

1080 Also, for every obsolete feature, there'll always be someone who still

1081 uses it and thus opposes its removal.

1082 This puts the developer into the position,

1083 where sensible improvements to style are regarded as destructive acts.

1084 Yet, living with the featurism is far worse, in my eyes, because

1085 future needs will demand adding further features,

1086 worsening the situation more and more.

1087 Rose and Romine added in a footnote,

1088 ``[...]

1089 .Pn send

1090 will no doubt acquire an endless number of switches in the years to come''

1091 .[ [

1092 rose romine real work

1093 .], p. 12].

1094 Although clearly humorous, the comment points to the nature of the problem.

1095 Refusing to add any new switches would encounter the problem at its root,

1096 but this is not practical.

1097 New needs will require new switches and it would be unwise to block

1098 them strictly.

1099 Nevertheless, removing obsolete switches still is an effective approach

1100 to deal with the problem.

1101 Working on an experimental branch without an established user base,

1102 eased my work because I did not offend users when I removed existing

1103 functions.

1104 .P

1105 Rose and Romine counted 24 visible and 9 more hidden switches for

1106 .Pn send .

1107 In nmh, they increased up to 32 visible and 12 hidden ones.

1108 At the time of writing, no more than 4 visible switches and 1 hidden switch

1109 have remained in mmh's

1110 .Pn send .

1111 These numbers include the two generic switches,

1112 .Sw -help

1113 and

1114 .Sw -Version .

1115 .P

1116 Hidden switches are ones not documented.

1117 In mmh, 12 tools have hidden switches.

1118 9 of them are

1119 .Sw -debug

1120 switches, the other 6 provide special interfaces for internal use.

1121 .P

1122 The following figure displays the number of switches for each of the tools

1123 that is available in both nmh and mmh.

1124 The tools are sorted by the number of switches they had in nmh.

1125 Both visible and hidden switches were counted,

1126 but not the generic help and version switches.

1127 Whereas in the beginning of the project, the average tool had 11 switches,

1128 now it has no more than 5 \(en only half as many.

1129 If the `no' switches and similar inverse variant are folded onto

1130 their counter-parts, the average tool had 8 switches in pre-mmh times and

1131 has 4 now.

1132 The total number of functional switches in mmh dropped from 465

1133 to 233.

1134

1135 .KS

1136 .in 1c

1137 .so input/switches.grap

1138 .KE

1139

1140 .P

1141 A part of the switches vanished after functions were removed.

1142 This was the case for network mail transfer, for instance.

1143 Sometimes, however, the work flow was the other way:

1144 I looked through the

1145 .Mp mh-chart (7)

1146 man page to identify the tools with apparently too many switches.

1147 Then I considered the benefit of each switch by examining

1148 the tool's man page and source code, aided by literature research

1149 and testing.

1150

1151

1152 .U3 "Draft Folder Facility

1153 .P

1154 A change early in the project was the complete transition from

1155 the single draft message to the draft folder facility

1156 .Ci 337338b404931f06f0db2119c9e145e8ca5a9860 .

1157 .\" XXX ref to section ...

1158 The draft folder facility was introduced in the mid-eighties, when

1159 Rose and Romine called it a ``relatively new feature''.

1160 .[

1161 rose romine real work

1162 .]

1163 Since then, the facility was included, inactive by default.

1164 By making it permanently active and by related rework of the tools, the

1165 .Sw -[no]draftfolder ,

1166 and

1167 .Sw -draftmessage

1168 switches could be removed from

1169 .Pn comp ,

1170 .Pn repl ,

1171 .Pn forw ,

1172 .Pn dist ,

1173 .Pn whatnow ,

1174 and

1175 .Pn send

1176 .Ci 337338b404931f06f0db2119c9e145e8ca5a9860 .

1177 The only flexibility lost with this change is having multiple

1178 draft folders within one profile.

1179 I consider this a theoretical problem only.

1180 At the same time, the

1181 .Sw -draft

1182 switch of

1183 .Pn anno ,

1184 .Pn refile ,

1185 and

1186 .Pn send

1187 was removed.

1188 The special treatment of \fIthe\fP draft message became irrelevant after

1189 the rework of the draft system

1190 (cf. Sec.

1191 .Cf draft-folder ).

1192 Furthermore,

1193 .Pn comp

1194 no longer needs a

1195 .Sw -file

1196 switch as the draft folder facility together with the

1197 .Sw -form

1198 switch are sufficient.

1199

1200

1201 .U3 "In Place Editing

1202 .P

1203 .Pn anno

1204 had the switches

1205 .Sw -[no]inplace

1206 to either annotate the message in place and thus preserve hard links,

1207 or annotate a copy to replace the original message.

1208 The latter approach broke hard links.

1209 Following the assumption that linked messages should truly be the

1210 same message and annotating it should not break the link, the

1211 .Sw -[no]inplace

1212 switches were removed and the previous default

1213 .Sw -inplace

1214 was made the definitive behavior

1215 .Ci c8195849d2e366c569271abb0f5f60f4ebf0b4d0 .

1216 The

1217 .Sw -[no]inplace

1218 switches of

1219 .Pn repl ,

1220 .Pn forw ,

1221 and

1222 .Pn dist

1223 could be removed, as well, as they were simply passed through to

1224 .Pn anno .

1225 .P

1226 .Pn burst

1227 also had

1228 .Sw -[no]inplace

1229 switches, but with a different meaning.

1230 With

1231 .Sw -inplace ,

1232 the digest had been replaced by the table of contents (i.e. the

1233 introduction text) and the burst messages were placed right

1234 after this message, renumbering all following messages.

1235 Also, any trailing text of the digest was lost, though,

1236 in practice, it usually consists of an end-of-digest marker only.

1237 Nonetheless, this behavior appeared less elegant than the

1238 .Sw -noinplace

1239 behavior, which already had been the default.

1240 Nmh's

1241 .Mp burst (1)

1242 man page reads:

1243 .QS

1244 If

1245 .Sw -noinplace

1246 is given, each digest is preserved, no table

1247 of contents is produced, and the messages contained within

1248 the digest are placed at the end of the folder. Other messages

1249 are not tampered with in any way.

1250 .QE

1251 .LP

1252 The decision to drop the

1253 .Sw -inplace

1254 behavior was supported by the code complexity and the possible data loss

1255 it caused.

1256 .Sw -noinplace

1257 was chosen to be the definitive behavior.

1258 .Ci 68a686adeb39223a5e1ad35e4a24890ec053679d

1259

1260

1261 .U3 "Forms and Format Strings

1262 .P

1263 Historically, the tools that had

1264 .Sw -form

1265 switches to supply a form file had

1266 .Sw -format

1267 switches as well to supply the contents of a form file as a string

1268 on the command line directly.

1269 In consequence, the following two lines equaled:

1270 .VS

1271 scan -form scan.mailx

1272 scan -format "`cat /path/to/scan.mailx`"

1273 VE

1274 The

1275 .Sw -format

1276 switches were dropped in favor for extending the

1277 .Sw -form

1278 switches

1279 .Ci f51956be123db66b00138f80464d06f030dbb88d .

1280 If their argument starts with an equal sign (`\fL=\fP'),

1281 then the rest of the argument is taken as a format string,

1282 otherwise the arguments is treated as the name of a format file.

1283 Thus, now the following two lines equal:

1284 .VS

1285 scan -form scan.mailx

1286 scan -form "=`cat /path/to/scan.mailx`"

1287 VE

1288 This rework removed the prefix collision between

1289 .Sw -form

1290 and

1291 .Sw -format .

1292 Typing `\fL-fo\fP' is sufficient to specify form file or format string.

1293 .P

1294 The different meaning of

1295 .Sw -format

1296 for

1297 .Pn forw

1298 and

1299 .Pn repl

1300 was removed in mmh.

1301 .Pn forw

1302 was completely switched to MIME-type forwarding, thus removing the

1303 .Sw -[no]format

1304 .Ci 6e271608b7b9c23771523f88d23a4d3593010cf1 .

1305 For

1306 .Pn repl ,

1307 the

1308 .Sw -[no]format

1309 switches were reworked to

1310 .Sw -[no]filter

1311 switches

1312 .Ci 67411b1f95d6ec987b4c732459e1ba8a8ac192c6 .

1313 The

1314 .Sw -format

1315 switches of

1316 .Pn send

1317 and

1318 .Pn post ,

1319 which had a third meaning, were removed likewise

1320 .Ci f3cb7cde0e6f10451b6848678d95860d512224b9 .

1321 Eventually, the ambiguity of the

1322 .Sw -format

1323 switches is resolved by not having such switches anymore in mmh.

1324

1325

1326 .U3 "MIME Tools

1327 .P

1328 The MIME tools, which once were part of

1329 .Pn mhn

1330 (whatever that stood for),

1331 had several switches that added little practical value to the programs.

1332 The

1333 .Sw -[no]realsize

1334 switches of

1335 .Pn mhbuild

1336 and

1337 .Pn mhlist

1338 were removed

1339 .Ci 8d8f1c3abc586c005c904e52c4adbfe694d2201c .

1340 Real size calculations are done always now because nmh's

1341 .Mp mhbuild (1)

1342 man page states that

1343 ``This provides an accurate count at the expense of a small delay''

1344 with the small delay not being noticable on modern systems.

1345 .P

1346 The

1347 .Sw -[no]check

1348 switches were removed together with the support for

1349 .Hd Content-MD5

1350 header fields [RFC\|1864]

1351 (cf. Sec.

1352 .Cf content-md5 )

1353 .Ci 31dc797eb5178970d68962ca8939da3fd9a8efda .

1354 .P

1355 The

1356 .Sw -[no]ebcdicsafe

1357 and

1358 .Sw -[no]rfc934mode

1359 switches of

1360 .Pn mhbuild

1361 were removed because they are considered obsolete

1362 .Ci 01a3480928da485b4d6109d36d751dfa71799d58

1363 .Ci 3363e2624dce0eb8164cf8b3f1ab385c8ff72e88 .

1364 .P

1365 Content caching of external MIME parts, activated with the

1366 .Sw -rcache

1367 and

1368 .Sw -wcache

1369 switches was completely removed

1370 .Ci d1fefd9f614e4dc3cda16da6c69133c1b2005269 .

1371 External MIME parts are rare today, having a caching facility

1372 for them appears to be unnecessary.

1373 .P

1374 In pre-MIME times,

1375 .Pn mhl

1376 had covered many tasks that are part of MIME handling today.

1377 Therefore,

1378 .Pn mhl

1379 could be simplified to a large extend, reducing the number of its

1380 switches from 21 to 6

1381 .Ci 350ad6d3542a07639213cf2a4fe524e829c1e7b6

1382 .Ci 0e46503be3c855bddaeae3843e1b659279c35d70 .

1383

1384

1385

1386

1387 .U3 "Header Printing

1388 .P

1389 .Pn folder 's

1390 data output is self-explaining enough that

1391 displaying the header line makes little sense.

1392 Hence, the

1393 .Sw -[no]header

1394 switch was removed and headers are never printed

1395 .Ci 601cc73d1fa05ce96faa728f036d6c51b91701c7 .

1396 .P

1397 In

1398 .Pn mhlist ,

1399 the

1400 .Sw -[no]header

1401 switches were removed, as well

1402 .Ci b24f96523aaf60e44e04a3ffb1d22e69a13a602f .

1403 In this case, the headers are printed always because the output

1404 is not self-explaining.

1405 .P

1406 .Pn scan

1407 also had

1408 .Sw -[no]header

1409 switches.

1410 Printing this header had been sensible until the introduction of

1411 format strings made it impossible to display column headings.

1412 Only the folder name and the current date remained to be printed.

1413 As this information can be perfectly generated with

1414 .Pn folder

1415 and

1416 .Pn date ,

1417 the switches were removed

1418 .Ci c477dc5d1d03fa6d9a8ab3dd3508c63cbddc044e .

1419 .P

1420 By removing all

1421 .Sw -header

1422 switches, the collision with

1423 .Sw -help

1424 on the first two letters was resolved.

1425 Currently,

1426 .Sw -h

1427 evaluates to

1428 .Sw -help

1429 for all tools of mmh.

1430

1431

1432 .U3 "Suppressing Edits or the Invocation of the WhatNow Shell

1433 .P

1434 The

1435 .Sw -noedit

1436 switch of

1437 .Pn comp ,

1438 .Pn repl ,

1439 .Pn forw ,

1440 .Pn dist ,

1441 and

1442 .Pn whatnow

1443 was removed and replaced by specifying

1444 .Sw -editor

1445 with an empty argument

1446 .Ci 75fca31a5b9d5c1a99c74ab14c94438d8852fba9 .

1447 (Specifying

1448 .Cl "-editor /bin/true

1449 is nearly the same. It differs only in setting the previous editor.)

1450 .P

1451 The more important change is the removal of the

1452 .Sw -nowhatnowproc

1453 switch

1454 .Ci ee4f43cf2ef0084ec698e4e87159a94c01940622 .

1455 This switch had once introduced an awkward behavior,

1456 as explained in nmh's man page for

1457 .Mp comp (1):

1458 .QS

1459 The

1460 .Sw -editor

1461 .Ar editor

1462 switch indicates the editor to use for

1463 the initial edit. Upon exiting from the editor,

1464 .Pn comp

1465 will invoke the

1466 .Pn whatnow

1467 program. See

1468 .Mp whatnow (1)

1469 for a discussion of available options.

1470 The invocation of this program can be

1471 inhibited by using the

1472 .Sw -nowhatnowproc

1473 switch. (In truth of fact, it is the

1474 .Pn whatnow

1475 program which starts the initial edit.

1476 Hence,

1477 .Sw -nowhatnowproc

1478 will prevent any edit from occurring.)

1479 .QE

1480 .P

1481 Effectively, the

1482 .Sw -nowhatnowproc

1483 switch caused only only a draft message to be created.

1484 As

1485 .Cl "-whatnowproc /bin/true

1486 does the same, the

1487 .Sw -nowhatnowproc

1488 switch was removed for being redundant.

1489

1490

1491

1492 .U3 "Various

1493 .BU

1494 With the removal of MMDF maildrop format support,

1495 .Pn packf

1496 and

1497 .Pn rcvpack

1498 no longer needed their

1499 .Sw -mbox

1500 and

1501 .Sw -mmdf

1502 switches.

1503 The behavior of

1504 .Sw -mbox

1505 is the sole behavior now

1506 .Ci 3916ab66ad5d183705ac12357621ea8661afd3c0 .

1507 Further rework in both tools made the

1508 .Sw -file

1509 switch unnecessary

1510 .Ci ca1023716d4c2ab890696f3e41fa0d94267a940e .

1511

1512 .BU

1513 Mmh's tools do no longer clear the screen (\c

1514 .Pn scan 's

1515 and

1516 .Pn mhl 's

1517 .Sw -[no]clear

1518 switches

1519 .Ci e57b17343dcb3ff373ef4dd089fbe778f0c7c270

1520 .Ci 943765e7ac5693ae177fd8d2b5a2440e53ce816e ).

1521 Neither does

1522 .Pn mhl

1523 ring the bell (\c

1524 .Sw -[no]bell

1525 .Ci e11983f44e59d8de236affa5b0d0d3067c192e24 )

1526 nor does it page the output itself (\c

1527 .Sw -length

1528 .Ci 5b9d883db0318ed2b84bb82dee880d7381f99188 ).

1529 .\" XXX Ref

1530 Generally, the pager to use is no longer specified with the

1531 .Sw -[no]moreproc

1532 command line switches for

1533 .Pn mhl

1534 and

1535 .Pn show /\c

1536 .Pn mhshow

1537 .Ci 39e87a75b5c2d3572ec72e717720b44af291e88a .

1538

1539 .BU

1540 In order to avoid prefix collisions among switch names, the

1541 .Sw -version

1542 switch was renamed to

1543 .Sw -Version

1544 (with capital `V')

1545 .Ci 32b2354dbaf4bf934936eb5b102a4a3d2fdd209a .

1546 Every program has the

1547 .Sw -version

1548 switch but its first three letters collided with the

1549 .Sw -verbose

1550 switch, present in many programs.

1551 The rename solved this problem once for all.

1552 Although this rename breaks a basic interface, having the

1553 .Sw -V

1554 abbreviation to display the version information, isn't all too bad.

1555

1556 .BU

1557 .Sw -[no]preserve

1558 of

1559 .Pn refile

1560 was removed

1561 .Ci 8edc5aaf86f9f77124664f6801bc6c6cdf258173

1562 because what use was it anyway?

1563 Quoting nmh's man page

1564 .Mp refile (1):

1565 .QS

1566 Normally when a message is refiled, for each destination

1567 folder it is assigned the number which is one above the current

1568 highest message number in that folder. Use of the

1569 .Sw -preserv

1570 [sic!] switch will override this message renaming, and try

1571 to preserve the number of the message. If a conflict for a

1572 particular folder occurs when using the

1573 .Sw -preserve

1574 switch, then

1575 .Pn refile

1576 will use the next available message number which

1577 is above the message number you wish to preserve.

1578 .QE

1579

1580 .BU

1581 The removal of the

1582 .Sw -[no]reverse

1583 switches of

1584 .Pn scan

1585 .Ci 8edc5aaf86f9f77124664f6801bc6c6cdf258173

1586 is a bug fix.

1587 This is supported by the comments

1588 ``\-[no]reverse under #ifdef BERK (I really HATE this)''

1589 by Rose and

1590 ``Lists messages in reverse order with the `\-reverse' switch.

1591 This should be considered a bug'' by Romine in the changelogs.

1592 The question remains why neither Rose nor Romine have fixed this

1593 bug in the eighties when they wrote these comments.

1594

1595

1596

1597

1598

1599

1600

1601 .\" --------------------------------------------------------------

1602 .H1 "Modernizing

1603 .P

1604 In the more than thirty years of MH's existence, its code base was

1605 increasingly extended.

1606 New features entered the project and became alternatives to the

1607 existing behavior.

1608 Relics from several decades have gathered in the code base

1609 but seldom obsolete features were dropped.

1610 This section describes the removing of old code

1611 and the modernizing of the default setup.

1612 It focuses on the functional aspect only;

1613 the non-functional aspects of code style are discussed in Sec.

1614 .Cf code-style .

1615

1616

1617 .H2 "Code Relics

1618 .P

1619 My position regarding the removal of obsolete code

1620 is much more revolutional than the nmh community appreciates.

1621 Working on an experimental version, I was able to quickly drop

1622 functionality that I considered ancient.

1623 The need for consensus with peers would have slowed this process down.

1624 Without the need to justify my decisions, I was able to rush forward.

1625 .P

1626 In December 2011, Paul Vixie motivated the nmh developers to just

1627 do the work:

1628 .[

1629 paul vixie edginess nmh-workers

1630 .]

1631 .QS

1632 let's stop walking on egg shells with this code base. there's no need to

1633 discuss whether to keep using vfork, just note in [sic!] passing, [...]

1634 we don't need a separate branch for removing vmh

1635 or ridding ourselves of #ifdef's or removing posix replacement functions

1636 or depending on pure ansi/posix ``libc''.

1637 .QP

1638 these things should each be a day or two of work and the ``main branch''

1639 should just be modern. [...]

1640 let's push forward, aggressively.

1641 .QE

1642 .LP

1643 I did so already in the months before.

1644 I pushed forward.

1645 .\" XXX semicolon ?

1646 I simply dropped the cruft.

1647 .P

1648 The decision to drop a feature was based on literature research and

1649 careful thinking, but whether having had contact with this particular

1650 feature within my own computer life served as a rule of thumb.

1651 I explained my reasons in the commit messages

1652 in the version control system.

1653 Hence, others can comprehend my view and argue for undoing the change

1654 if I have missed an important aspect.

1655 I was quick in dropping parts.

1656 I rather include falsely dropped parts again, than going at a slower pace.

1657 Mmh is experimental work; it requires tough decisions.

1658 .\" XXX ``exp. work'' schon oft gesagt

1659

1660

1661 .U3 "Process Forking

1662 .P

1663 Being a tool chest, MH creates many processes.

1664 In earlier times

1665 .Fu fork()

1666 had been an expensive system call, because the process's image needed

1667 to be completely duplicated at once.

1668 This expensive work was especially unnecessary in the commonly occurring

1669 case wherein the image is replaced by a call to

1670 .Fu exec()

1671 right after having forked the child process.

1672 The

1673 .Fu vfork()

1674 system call was invented to speed up this particular case.

1675 It completely omits the duplication of the image.

1676 On old systems this resulted in significant speed ups.

1677 Therefore MH used

1678 .Fu vfork()

1679 whenever possible.

1680 .P

1681 Modern memory management units support copy-on-write semantics, which make

1682 .Fu fork()

1683 almost as fast as

1684 .Fu vfork() .

1685 The man page of

1686 .Mp vfork (2)

1687 in FreeBSD 8.0 states:

1688 .QS

1689 This system call will be eliminated when proper system sharing mechanisms

1690 are implemented. Users should not depend on the memory sharing semantics

1691 of vfork() as it will, in that case, be made synonymous to fork(2).

1692 .QE

1693 .LP

1694 Vixie supports the removal with the note that ``the last

1695 system on which fork was so slow that an mh user would notice it, was

1696 Eunice. that was 1987''.

1697 .[

1698 nmh-workers vixie edginess

1699 .]

1700 I replaced all calls to

1701 .Fu vfork()

1702 with calls to

1703 .Fu fork()

1704 .Ci 40821f5c1316e9205a08375e7075909cc9968e7d .

1705 .P

1706 Related to the costs of

1707 .Fu fork()

1708 is the probability of its success.

1709 In the eighties, on heavy loaded systems, calls to

1710 .Fu fork()

1711 were prone to failure.

1712 Hence, many of the

1713 .Fu fork()

1714 calls in the code were wrapped into loops to retry the

1715 .Fu fork()

1716 several times, to increase the chances to succeed eventually.

1717 On modern systems, a failing

1718 .Fu fork()

1719 call is unusual.

1720 Hence, in the rare case when

1721 .Fu fork()

1722 fails, mmh programs simply abort

1723 .Ci 5fbf37ee68e018998ada61eeab73e035b26834b6 .

1724

1725

1726 .U3 "Header Fields

1727 .BU

1728 The

1729 .Hd Encrypted

1730 header field was introduced by RFC\|822,

1731 but already marked as legacy in RFC\|2822.

1732 Today, OpenPGP provides the basis for standardized exchange of encrypted

1733 messages [RFC\|4880, RFC\|3156].

1734 Hence, the support for

1735 .Hd Encrypted

1736 header fields is removed in mmh

1737 .Ci 064527f7b57ab050e5af13e15ad99aeeab125857 .

1738 .BU

1739 The native support for

1740 .Hd Face

1741 header fields has been removed, as well

1742 .Ci 8e5be81f784682822f5e868c1bf3c8624682bd23 .

1743 This feature is similar to the

1744 .Hd X-Face

1745 header field in its intent,

1746 but takes a different approach to store the image.

1747 Instead of encoding the image data directly into the header field,

1748 it contains the hostname and UDP port where the image

1749 date can be retrieved.

1750 There is even a third Face system,

1751 which is the successor of

1752 .Hd X-Face ,

1753 although it re-uses the

1754 .Hd Face

1755 header field name.

1756 It was invented in 2005 and supports colored PNG images.

1757 None of the Face systems described here is popular today.

1758 Hence, mmh has no direct support for them.

1759 .BU

1760 .Id content-md5

1761 The

1762 .Hd Content-MD5

1763 header field was introduced by RFC\|1864.

1764 It provides detection of data corruption during the transfer.

1765 But it can not ensure verbatim end-to-end delivery of the contents

1766 [RFC\|1864].

1767 The proper approach to verify content integrity in an

1768 end-to-end relationship is the use of digital signatures.

1769 .\" XXX (RFCs FIXME).

1770 On the other hand, transfer protocols should detect corruption during

1771 the transmission.

1772 The TCP includes a checksum field therefore.

1773 These two approaches in combinations render the

1774 .Hd Content-MD5

1775 header field superfluous.

1776 Not a single one out of 4\|200 messages from two decades

1777 in the nmh-workers mailing list archive

1778 .[

1779 nmh-workers mailing list archive website

1780 .]

1781 contains a

1782 .Hd Content-MD5

1783 header field.

1784 Neither did any of the 60\|000 messages in my personal mail storage.

1785 Removing the support for this header field

1786 .Ci 31dc797eb5178970d68962ca8939da3fd9a8efda ,

1787 removed the last place where MD5 computation was needed.

1788 Hence, the MD5 code could be removed as well.

1789 Over 500 lines of code vanished by this one change.

1790

1791

1792 .U3 "MMDF maildrop support

1793 .P

1794 This type of maildrop format is conceptionally similar to the mbox format,

1795 but uses a different message delimiter (`\fL\\1\\1\\1\\1\fP',

1796 commonly written as `\fL^A^A^A^A\fP', instead of `\fLFrom\0\fP').

1797 Mbox is the de-facto standard maildrop format on Unix,

1798 whereas the MMDF maildrop format is now forgotten.

1799 Mbox remains as the only packed mailbox format, supported in mmh.

1800 .P

1801 The simplifications within the code were moderate.

1802 Mainly, the reading and writing of MMDF mailbox files was removed.

1803 But also, switches of

1804 .Pn packf

1805 and

1806 .Pn rcvpack

1807 could be removed

1808 .Ci 3916ab66ad5d183705ac12357621ea8661afd3c0 .

1809 In the message parsing function

1810 .Fn sbr/m_getfld.c ,

1811 knowledge of MMDF packed mail boxes was removed

1812 .Ci 684ec30d81e1223a282764452f4902ed4ad1c754 .

1813 Further code structure simplifications may be possible there,

1814 because only one single packed mailbox format is left to be supported.

1815 I have not worked on them yet because

1816 .Fu m_getfld()

1817 is heavily optimized and thus dangerous to touch.

1818 The risk of damaging the intricate workings of the optimized code is

1819 too high.

1820

1821

1822 .U3 "Prompter's Control Keys

1823 .P

1824 The program

1825 .Pn prompter

1826 queries the user to fill in a message form.

1827 When used as

1828 .Cl "comp -editor prompter" ,

1829 the resulting behavior is similar to

1830 .Pn mailx .

1831 Apparently,

1832 .Pn prompter

1833 had not been touched lately.

1834 Otherwise it's hardly explainable why it

1835 still offered the switches

1836 .Sw -erase

1837 .Ar chr

1838 and

1839 .Sw -kill

1840 .Ar chr

1841 to name the characters for command line editing.

1842 The times when this had been necessary are long time gone.

1843 Today these things work out-of-the-box, and if not, are configured

1844 with the standard tool

1845 .Pn stty .

1846 The switches are removed now

1847 .Ci 0bd9750710cdbab80cfb4036dd87af20afe1552f .

1848

1849

1850 .U3 "Hardcopy Terminal Support

1851 .P

1852 More of a funny anecdote is a check for being connected to a

1853 hardcopy terminal.

1854 It remained in the code until spring 2012, when I finally removed it

1855 .Ci b7764c4a6b71d37918a97594d866258f154017ca .

1856 .P

1857 The check only prevented a pager to be placed between the printing

1858 program (\c

1859 .Pn mhl )

1860 and the terminal.

1861 In nmh, this could have been ensured statically with the

1862 .Sw -nomoreproc

1863 at the command line, too.

1864 In mmh, setting the profile entry

1865 .Pe Pager

1866 or the environment variable

1867 .Ev PAGER

1868 to

1869 .Pn cat

1870 is sufficient.

1871

1872

1873

1874

1875 .H2 "Attachments

1876 .P

1877 The mind model of email attachments is unrelated to MIME.

1878 Although the MIME RFCs [RFC\|2045\(en2049] define the technical

1879 requirements for having attachments, they do not mention the term.

1880 Instead of attachments, MIME talks about ``multi-part message bodies''

1881 [RFC\|2045], a more general concept.

1882 Multi-part messages are messages

1883 ``in which one or more different

1884 sets of data are combined in a single body''

1885 [RFC\|2046].

1886 MIME keeps its descriptions generic;

1887 it does not imply specific usage models.

1888 Today, one usage model is prevalent: attachments.

1889 The idea is having a main text document with files of arbitrary kind

1890 attached to it.

1891 In MIME terms, this is a multi-part message having a text part first

1892 and parts of arbitrary type following.

1893 .P

1894 MH's MIME support is a direct implementation of the RFCs.

1895 The perception of the topic described in the RFCs is clearly visible

1896 in MH's implementation.

1897 .\" XXX rewrite ``no idea''.

1898 As a result,

1899 MH had all the MIME features but no idea of attachments.

1900 But users do not need all the MIME features,

1901 they want convenient attachment handling.

1902

1903

1904 .U3 "Composing MIME Messages

1905 .P

1906 In order to improve the situation on the message composing side,

1907 Jon Steinhart had added an attachment system to nmh in 2002

1908 .Ci 7480dbc14bc90f2d872d434205c0784704213252 .

1909 In the file

1910 .Fn docs/README-ATTACHMENTS ,

1911 he described his motivation to do so:

1912 .QS

1913 Although nmh contains the necessary functionality for MIME message

1914 handing [sic!], the interface to this functionality is pretty obtuse.

1915 There's no way that I'm ever going to convince my partner to write

1916 .Pn mhbuild

1917 composition files!

1918 .QE

1919 .LP

1920 With this change, the mind model of attachments entered nmh.

1921 In the same document:

1922 .QS

1923 These changes simplify the task of managing attachments on draft files.

1924 They allow attachments to be added, listed, and deleted.

1925 MIME messages are automatically created when drafts with attachments

1926 are sent.

1927 .QE

1928 .LP

1929 Unfortunately, the attachment system, like every new facilities in nmh,

1930 was inactive by default.

1931 .P

1932 During my time in Argentina, I tried to improve the attachment system.

1933 But, after long discussions my patch died as a proposal on the

1934 mailing list because of great opposition in the nmh community.

1935 .[

1936 nmh-workers attachment proposal

1937 .]

1938 In January 2012, I extended the patch and applied it to mmh

1939 .Ci 8ff284ff9167eff8f5349481529332d59ed913b1 .

1940 In mmh, the attachment system is active by default.

1941 Instead of command line switches, the

1942 .Pe Attachment-Header

1943 profile entry is used to specify

1944 the name of the attachment header field.

1945 It is pre-defined to

1946 .Hd Attach .

1947 .P

1948 To add an attachment to a draft, a header line needs to be added:

1949 .VS

1950 To: bob

1951 Subject: The file you wanted

1952 Attach: /path/to/the/file-bob-wanted

1953 --------

1954 Here it is.

1955 VE

1956 The header field can be added to the draft manually in the editor,

1957 or by using the `attach' command at the WhatNow prompt, or

1958 non-interactively with

1959 .Pn anno :

1960 .VS

1961 anno -append -nodate -component Attach -text /path/to/attachment

1962 VE

1963 Drafts with attachment headers are converted to MIME automatically by

1964 .Pn send .

1965 The conversion to MIME is invisible to the user.

1966 The draft stored in the draft folder is always in source form with

1967 attachment headers.

1968 If the MIMEification fails (e.g. because the file to attach

1969 is not accessible) the original draft is not changed.

1970 .P

1971 The attachment system handles the forwarding of messages, too.

1972 If the attachment header value starts with a plus character (`\fL+\fP'),

1973 like in

1974 .Cl "Attach: +bob 30 42" ,

1975 the given messages in the specified folder will be attached.

1976 This allowed to simplify

1977 .Pn forw

1978 .Ci f41f04cf4ceca7355232cf7413e59afafccc9550 .

1979 .P

1980 Closely related to attachments is non-ASCII text content,

1981 because it requires MIME as well.

1982 In nmh, the user needed to call `mime' at the WhatNow prompt

1983 to have the draft converted to MIME.

1984 This was necessary whenever the draft contained non-ASCII characters.

1985 If the user did not call `mime', a broken message would be sent.

1986 Therefore, the

1987 .Pe automimeproc

1988 profile entry could be specified to have the `mime' command invoked

1989 automatically each time.

1990 Unfortunately, this approach conflicted with the attachment system

1991 because the draft would already be in MIME format at the time

1992 when the attachment system wanted to MIMEify it.

1993 To use nmh's attachment system, `mime' must not be called at the

1994 WhatNow prompt and

1995 .Pe automimeproc

1996 must not be set in the profile.

1997 But then the case of non-ASCII text without attachment headers was

1998 not caught.

1999 All in all, the solution was complex and irritating.

2000 My patch from December 2010

2001 .[

2002 nmh-workers attachment proposal

2003 .]

2004 would have simplified the situation.

2005 .P

2006 Mmh's current solution is even more elaborate.

2007 Any necessary MIMEification is done automatically.

2008 There is no `mime' command at the WhatNow prompt anymore.

2009 The draft will be converted automatically to MIME when either an

2010 attachment header or non-ASCII text is present.

2011 Furthermore, the hash character (`\fL#\fP') is not special any more

2012 at line beginnings in the draft message.

2013 .\" XXX REF ?

2014 Users need not concern themselves with the whole topic at all.

2015 .P

2016 Although the new approach does not anymore support arbitrary MIME

2017 compositions directly, the full power of

2018 .Pn mhbuild

2019 can still be accessed.

2020 Given no attachment headers are included, users can create

2021 .Pn mhbuild

2022 composition drafts like in nmh.

2023 Then, at the WhatNow prompt, they can invoke

2024 .Cl "edit mhbuild

2025 to convert the draft to MIME.

2026 Because the resulting draft neither contains non-ASCII characters

2027 nor has it attachment headers, the attachment system will not touch it.

2028 .P

2029 The approach taken in mmh is tailored towards today's most common case:

2030 a text part, possibly with attachments.

2031 This case was simplified.

2032

2033

2034 .U3 "MIME Type Guessing

2035 .P

2036 From the programmer's point of view, the use of

2037 .Pn mhbuild

2038 composition drafts had one notable advantage over attachment headers:

2039 The user provides the appropriate MIME types for files to include.

2040 The new attachment system needs to find out the correct MIME type itself.

2041 This is a difficult task.

2042 Determining the correct MIME type of content is partly mechanical,

2043 partly intelligent work.

2044 Forcing the user to find out the correct MIME type,

2045 forces him to do partly mechanical work.

2046 Letting the computer do the work can lead to bad choices for difficult

2047 content.

2048 For mmh, the latter option was chosen to spare the user the work

2049 .Ci 3baec236a39c5c89a9bda8dbd988d643a21decc6 .

2050 .P

2051 Determining the MIME type by the suffix of the file name is a dumb

2052 approach, yet it is simple to implement and provides good results

2053 for the common cases.

2054 If no MIME type can be determined, text content is sent as `text/plain',

2055 anything else under the generic fall-back type `application/octet-stream'.

2056 Mmh implements this approach in the

2057 .Pn print-mimetype

2058 script

2059 .Ci 4b5944268ea0da7bb30598a27857304758ea9b44 .

2060 .P

2061 A far better, though less portable, approach is the use of

2062 .Pn file .

2063 This standard tool tries to determine the type of files.

2064 Unfortunately, its capabilities and accuracy varies from system to system.

2065 Additionally, its output was only intended for human beings,

2066 but not to be used by programs.

2067 Nevertheless, modern versions of GNU

2068 .Pn file ,

2069 which are prevalent on the popular GNU/Linux systems,

2070 provide MIME type output in machine-readable form.

2071 Although this solution is system-dependent,

2072 it solves the difficult problem well.

2073 On systems where GNU

2074 .Pn file ,

2075 version 5.04 or higher, is available it should be used.

2076 One needs to specify the following profile entry to do so:

2077 .VS

2078 Mime-Type-Query: file -b --mime

2079 VE

2080 .LP

2081 Other versions of

2082 .Pn file

2083 might possibly be usable with wrapper scripts that reformat the output.

2084 The diversity among

2085 .Pn file

2086 implementations is great; one needs to check the local variant.

2087 .P

2088 It is not possible in mmh to override the automatic MIME type guessing

2089 for a specific file.

2090 To do so, either the user would need to know in advance for which file

2091 the automatic guessing fails or the system would require interaction.

2092 I consider both cases impractical.

2093 The existing solution should be sufficient.

2094 If not, the user may always fall back to

2095 .Pn mhbuild

2096 composition drafts and bypass the attachment system.

2097

2098

2099 .U3 "Storing Attachments

2100 .P

2101 Extracting MIME parts of a message and storing them to disk is performed by

2102 .Pn mhstore .

2103 The program has two operation modes,

2104 .Sw -auto

2105 and

2106 .Sw -noauto .

2107 With the former one, each part is stored under the filename given in the

2108 MIME part's meta information, if available.

2109 This naming information is usually available for modern attachments.

2110 If no filename is available, this MIME part is stored as if

2111 .Sw -noauto

2112 would have been specified.

2113 In the

2114 .Sw -noauto

2115 mode, the parts are processed according to rules, defined by

2116 .Pe mhstore-store-*

2117 profile entries.

2118 These rules define generic filename templates for storing

2119 or commands to post-process the contents in arbitrary ways.

2120 If no matching rule is available the part is stored under a generic

2121 filename, built from message number, MIME part number, and MIME type.

2122 .P

2123 The

2124 .Sw -noauto

2125 mode had been the default in nmh because it was considered safe,

2126 in contrast to the

2127 .Sw -auto

2128 mode.

2129 In mmh,

2130 .Sw -auto

2131 is not dangerous anymore.

2132 Two changes were necessary:

2133 .LI 1

2134 Any directory path is removed from the proposed filename.

2135 Thus, the files are always stored in the expected directory.

2136 .Ci 41b6eadbcecf63c9a66aa5e582011987494abefb

2137 .LI 2

2138 Tar files are not extracted automatically any more.

2139 Thus, the rest of the file system will not be touched.

2140 .Ci 94c80042eae3383c812d9552089953f9846b1bb6

2141 .P

2142 In mmh, the result of

2143 .Cl "mhstore -auto

2144 can be foreseen from the output of

2145 .Cl "mhlist -verbose" .

2146 Although the

2147 .Sw -noauto

2148 mode is considered to be more powerful, it is less convenient and

2149 .Sw -auto

2150 is safe now.

2151 Additionally, storing attachments under their original name

2152 is intuitive.

2153 Hence,

2154 .Sw -auto

2155 serves better as the default option

2156 .Ci 3410b680416c49a7617491af38bc1929855a331d .

2157 .P

2158 Files are stored into the directory given by the

2159 .Pe Nmh-Storage

2160 profile entry, if set, or

2161 into the current working directory, otherwise.

2162 Storing to different directories is only possible with

2163 .Pe mhstore-store-*

2164 profile entries.

2165 .P

2166 Still existing files get overwritten silently in both modes.

2167 This can be considered a bug.

2168 Yet, each other behavior has its draw-backs, too.

2169 Refusing to replace files requires adding a

2170 .Sw -force

2171 switch.

2172 Users will likely need to invoke

2173 .Pn mhstore

2174 a second time with

2175 .Sw -force .

2176 Eventually, only the user can decide in the specific case.

2177 This requires interaction, which I like to avoid if possible.

2178 Appending a unique suffix to the filename is another bad option.

2179 For now, the behavior remains as it is.

2180 .P

2181 In mmh, only MIME parts of type message are special in

2182 .Pn mhstore 's

2183 .Sw -auto

2184 mode.

2185 Instead of storing message/rfc822 parts as files to disk,

2186 they are stored as messages into the current mail folder.

2187 The same applies to message/partial, although the parts are

2188 automatically reassembled beforehand.

2189 MIME parts of type message/external-body are not automatically retrieved

2190 anymore.

2191 Instead, information on how to retrieve them is output.

2192 Not supporting this rare case saved nearly one thousand lines of code

2193 .Ci 55e1d8c654ee0f7c45b9361ce34617983b454c32 .

2194 .\" XXX mention somewhere else too: (The profile entry `nmh-access-ftp'

2195 .\" and sbr/ruserpass.c for reading ~/.netrc are gone now.)

2196 The MIME type `application/octet-stream; type=tar' is not special anymore.

2197 The automatically extracting of such MIME parts had been the

2198 dangerous part of the

2199 .Sw -auto

2200 mode

2201 .Ci 94c80042eae3383c812d9552089953f9846b1bb6 .

2202

2203

2204

2205 .U3 "Showing MIME Messages

2206 .P

2207 The program

2208 .Pn mhshow

2209 was written to display MIME messages.

2210 It implemented the conceptional view of the MIME RFCs.

2211 Nmh's

2212 .Pn mhshow

2213 handles each MIME part independently, presenting them separately

2214 to the user.

2215 This does not match today's understanding of email attachments,

2216 where displaying a message is seen to be a single, integrated operation.

2217 Today, email messages are expected to consist of a main text part

2218 plus possibly attachments.

2219 They are no more seen to be arbitrary MIME hierarchies with

2220 information on how to display the individual parts.

2221 I adjusted

2222 .Pn mhshow 's

2223 behavior to the modern view on the topic.

2224 .P

2225 (One should note that this section completely ignores the original

2226 .Pn show

2227 program, because it was not capable to display MIME messages

2228 and is no longer part of mmh.

2229 .\" XXX ref to other section

2230 Although

2231 .Pn mhshow

2232 was renamed to

2233 .Pn show

2234 in mmh, this section uses the name

2235 .Pn mhshow ,

2236 in order to avoid confusion.)

2237 .P

2238 In mmh, the basic idea is that

2239 .Pn mhshow

2240 should display a message in one single pager session.

2241 Therefore,

2242 .Pn mhshow

2243 invokes a pager session for all its output,

2244 whenever it prints to a terminal

2245 .Ci a4197ea6ffc5c1550e8b52d5a654bcaaaee04a4e .

2246 In consequence,

2247 .Pn mhl

2248 does no more invoke a pager

2249 .Ci 0e46503be3c855bddaeae3843e1b659279c35d70 .

2250 With

2251 .Pn mhshow

2252 replacing the original

2253 .Pn show ,

2254 the output of

2255 .Pn mhl

2256 no longer goes to the terminal directly, but through

2257 .Pn mhshow .

2258 Hence,

2259 .Pn mhl

2260 does not need to invoke a pager.

2261 The one and only job of

2262 .Pn mhl

2263 is to format messages or parts of them.

2264 The only place in mmh, where a pager is invoked is

2265 .Pn mhshow .

2266 .P

2267 In the intended setup, only text content is be displayed,

2268 in a single pager session.

2269 Non-text content needs to be converted to text by appropriate

2270 .Pe mhshow-show-*

2271 profile entries before, if this is possible and wanted.

2272 A common example for this are PDF files.

2273 .ig \"XXX

2274 .Pe mhshow-show-*

2275 profile entries can be used to display MIME parts in a specific way.

2276 to display them in the terminal.

2277 ..

2278 In mmh, MIME parts are always displayed serially.

2279 The request to display the MIME type `multipart/parallel' in parallel

2280 is ignored.

2281 It is simply treated as `multipart/mixed'

2282 .Ci d0581ba306a7299113a346f9b4c46ce97bc4cef6 .

2283 This was already possible to requested with the, now removed,

2284 .Sw -serialonly

2285 switch of

2286 .Pn mhshow .

2287 As MIME parts are always processed exclusively, i.e. serially,

2288 the `\fL%e\fP' escape in

2289 .Pe mhshow-show-*

2290 profile entries became useless and was thus removed

2291 .Ci a20d405db09b7ccca74d3e8c57550883da49e1ae .

2292 .P

2293 Other kinds of attachments are ignored.

2294 With

2295 .Pe mhshow-show-*

2296 profile entries for them, they can be displayed serially along

2297 the message.

2298 For parallel display, the attachments need to be stored to disk first.

2299 .P

2300 To display text content in foreign charsets, they need to be converted

2301 to the native charset.

2302 Therefore,

2303 .Pe mhshow-charset-*

2304 profile entries were needed.

2305 In mmh, the conversion is performed automatically by piping the

2306 text through the

2307 .Pn iconv

2308 command, if necessary

2309 .Ci 2433122c20baccb10b70b49c04c6b0497b5b3b60 .

2310 Custom

2311 .Pe mhshow-show-*

2312 rules for textual content might need a

2313 .Cl "iconv -f %c %f |

2314 prefix to have the text converted to the native charset.

2315 .P

2316 Although the conversion of foreign charsets to the native one

2317 has improved, it is not consistent enough.

2318 Further work needs to be done and

2319 the basic concepts in this field need to be re-thought.

2320 Though, the default setup of mmh displays message in foreign charsets

2321 correctly without the need to configure anything.

2322

2323

2324 .ig

2325

2326 .P

2327 mhshow/mhstore: Removed support for retrieving message/external-body parts.

2328 These tools will not download the contents automatically anymore. Instead,

2329 they print the information needed to get the contents. If someone should

2330 really receive one of those rare message/external-body messages, he can

2331 do the job manually. We save nearly a thousand lines of code. That's worth

2332 it!

2333 (The profile entry `nmh-access-ftp' and sbr/ruserpass.c for reading

2334 ~/.netrc are gone now.)

2335 .Ci 55e1d8c654ee0f7c45b9361ce34617983b454c32

2336

2337 ..

2338

2339

2340

2341 .H2 "Signing and Encrypting

2342 .P

2343 Nmh offers no direct support for digital signatures and message encryption.

2344 This functionality needed to be added through third-party software.

2345 In mmh, the functionality is included because it

2346 is a part of modern email and is likely wanted by users of mmh.

2347 A fresh mmh installation supports signing and encrypting

2348 out-of-the-box.

2349 Therefore, Neil Rickert's

2350 .Pn mhsign

2351 and

2352 .Pn mhpgp

2353 scripts

2354 .[

2355 neil rickert mhsign mhpgp

2356 .]

2357 were included

2358 .Ci f45cdc98117a84f071759462c7ae212f4bc5ab2e

2359 .Ci 58cf09aa36e9f7f352a127158bbf1c5678bc6ed8 .

2360 The scripts fit well because they are lightweight and

2361 similar of style to the existing tools.

2362 Additionally, no licensing difficulties appeared

2363 as they are part of the public domain.

2364 .P

2365 .Pn mhsign

2366 handles the signing and encrypting part.

2367 It comprises about 250 lines of shell code and interfaces between

2368 .Pn gnupg

2369 and the MH system.

2370 It was meant to be invoked manually at the WhatNow prompt, but in mmh,

2371 .Pn send

2372 invokes

2373 .Pn mhsign

2374 automatically

2375 .Ci c7b5e1df086bcc37ff40163ee67571f076cf6683 .

2376 Special header fields were introduced to request this action.

2377 If a draft contains the

2378 .Hd Sign

2379 header field,

2380 .Pn send

2381 will initiate the signing.

2382 The signing key is either chosen automatically or it is specified by the

2383 .Pe Pgpkey

2384 profile entry.

2385 .Pn send

2386 always create signatures using the PGP/MIME standard [RFC\|4880],

2387 but by invoking

2388 .Pn mhsign

2389 manually, old-style non-MIME signatures can be created as well.

2390 To encrypt an outgoing message, the draft needs to contain an

2391 .Hd Enc

2392 header field.

2393 Public keys of all recipients are searched for in the gnupg keyring and

2394 in a file called

2395 .Fn pgpkeys ,

2396 which contains exceptions and overrides.

2397 Unless public keys are found for all recipients,

2398 .Pn mhsign

2399 will refuse to encrypt it.

2400 Currently, messages with hidden (BCC) recipients can not be encrypted.

2401 This work is pending because it requires a structurally more complex

2402 approach.

2403 .P

2404 .Pn mhpgp

2405 is the companion to

2406 .Pn mhsign .

2407 It verifies signatures and decrypts messages.

2408 Encrypted messages can be either temporarily decrypted and displayed

2409 or permanently decrypted and stored into the current folder.

2410 Currently,

2411 .Pn mhpgp

2412 needs to be invoked manually.

2413 The integration into

2414 .Pn show

2415 and

2416 .Pn mhstore

2417 to verify signatures and decrypt messages as needed

2418 is planned but not yet realized.

2419 .P

2420 Both scripts were written for nmh.

2421 Hence they needed to be adjust

2422 according to the differences between nmh and mmh.

2423 For instance, they use the backup prefix no longer.

2424 Furthermore, compatibility support for old PGP features was dropped.

2425 .P

2426 The integrated message signing and encrypting support is one of the

2427 most recent features in mmh.

2428 It has not had the time to mature.

2429 User feedback and personal experience need to be accumulated to

2430 direct the further development of the facility.

2431 Already it seems to be worthwhile to consider adding

2432 .Sw -[no]sign

2433 and

2434 .Sw -[no]enc

2435 switches to

2436 .Pn send ,

2437 to be able to override the corresponding header fields.

2438 A profile entry:

2439 .VS

2440 send: -sign

2441 VE

2442 would then activate signing for all outgoing messages.

2443 With the present approach, a

2444 .Hd Send

2445 header component needs to be added to each draft template

2446 to achieve the same result.

2447 Adding the switches would ease the work greatly and keep the

2448 template files clean.

2449

2450

2451

2452

2453 .H2 "Draft and Trash Folder

2454 .P

2455

2456 .U3 "Draft Folder

2457 .Id draft-folder

2458 .P

2459 In the beginning, MH had the concept of a draft message.

2460 This was a file named

2461 .Fn draft

2462 in the MH directory, which was treated special.

2463 On composing a message, this draft file was used.

2464 When starting to compose another message before the former one was sent,

2465 the user had to decide among:

2466 .LI 1

2467 Using the old draft to finish and send it before starting with a new one.

2468 .LI 2

2469 Discarding the old draft and replacing it with a new one.

2470 .LI 3

2471 Preserving the old draft by refiling it to a folder.

2472 .LP

2473 Working on multiple drafts was only possible in alternation.

2474 For that, the current draft needed to be refiled to a folder and

2475 another one re-used for editing.

2476 Working on multiple drafts at the same time was impossible.

2477 The usual approach of switching to a different MH context did not

2478 help anything.

2479 .P

2480 The draft folder facility exists to

2481 allow true parallel editing of drafts, in a straight forward way.

2482 It was introduced by Marshall T. Rose, already in 1984.

2483 Similar to other new features, the draft folder was inactive by default.

2484 Even in nmh, the highly useful draft folder was not available

2485 out-of-the-box.

2486 At least, Richard Coleman added the man page

2487 .Mp mh-draft (5)

2488 to better document the feature.

2489 .P

2490 Not using the draft folder facility has the single advantage of having

2491 the draft file at a static location.

2492 This is simple in simple cases but the concept does not scale for more

2493 complex cases.

2494 The concept of the draft message is too limited for the problem

2495 it tries to solve.

2496 Therefore the draft folder was introduced.

2497 It is the more powerful and more natural concept.

2498 The draft folder is a folder like any other folder in MH.

2499 Its messages can be listed like any other messages.

2500 A draft message is no longer a special case.

2501 Tools do not need special switches to work on the draft message.

2502 Hence corner cases were removed.

2503 .P

2504 The trivial part of the work was activating the draft folder with a

2505 default name.

2506 I chose the name

2507 .Fn +drafts ,

2508 for obvious reasons.

2509 In consequence, the command line switches

2510 .Sw -draftfolder

2511 and

2512 .Sw -draftmessage

2513 could be removed.

2514 More difficult but also more improving was updating the tools to the

2515 new concept.

2516 For nearly three decades, the tools needed to support two draft handling

2517 approaches.

2518 By fully switching to the draft folder, the tools could be

2519 simplified by dropping the awkward draft message handling code.

2520 .Sw -draft

2521 switches were removed because operating on a draft message is no longer

2522 special.

2523 It became indistinguishable to operating on any other message.

2524 .Ci 337338b404931f06f0db2119c9e145e8ca5a9860

2525 .P

2526 There is no more need to query the user for draft handling

2527 .Ci 2d48b455c303a807041c35e4248955f8bec59eeb .

2528 It is always possible to add another new draft.

2529 Refiling drafts is without difference to refiling other messages.

2530 All of these special cases are gone.

2531 Yet, one draft-related switch remained.

2532 .Pn comp

2533 still has

2534 .Sw -[no]use

2535 for switching between two modes:

2536 .LI 1

2537 Modifying an existing draft, with

2538 .Sw -use .

2539 .LI 2

2540 Composing a new draft, possibly taking some existing message as template,

2541 with

2542 .Sw -nouse ,

2543 the default.

2544 .LP

2545 In either case, the behavior of

2546 .Pn comp

2547 is deterministic.

2548 .P

2549 .Pn send

2550 now operates on the current message in the draft folder by default.

2551 As message and folder can both be overridden by specifying them on

2552 the command line, it is possible to send any message in the mail storage

2553 by simply specifying its number and folder.

2554 In contrast to the other tools,

2555 .Pn send

2556 takes the draft folder as its default folder.

2557 .P

2558 Dropping the draft message concept in favor for the draft folder concept,

2559 replaced special cases with regular cases.

2560 This simplified the source code of the tools, as well as the concepts.

2561 In mmh, draft management does not break with the MH concepts

2562 but applies them.

2563 .Cl "scan +drafts" ,

2564 for instance, is a truly natural request.

2565 .P

2566 Most of the work was already performed by Rose in the eighties.

2567 The original improvement of mmh is dropping the old draft message approach

2568 and thus simplifying the tools, the documentation,

2569 and the system as a whole.

2570 Although my part in the draft handling improvement was small,

2571 it was important.

2572

2573

2574 .U3 "Trash Folder

2575 .Id trash-folder

2576 .P

2577 Similar to the situation for drafts is the situation for removed messages.

2578 Historically, a message was ``deleted'' by prepending a specific

2579 \fIbackup prefix\fP, usually the comma character,

2580 to the file name.

2581 The specific file would then be ignored by MH because only files with

2582 names consisting of digits only are treated as messages.

2583 Although files remained in the file system,

2584 the messages were no longer visible in MH.

2585 To truly delete them, a maintenance job was needed.

2586 Usually a cron job was installed to delete them after a grace time.

2587 For instance:

2588 .VS

2589 find $HOME/Mail -type f -name ',*' -ctime +7 -delete

2590 VE

2591 In such a setup, the original message could be restored

2592 within the grace time interval by stripping the

2593 backup prefix from the file name \(en usually but not always.

2594 If the last message of a folder with six messages (\fL1-6\fP) was removed,

2595 message

2596 .Fn 6 ,

2597 became file

2598 .Fn ,6 .

2599 If then a new message entered the same folder, it would be named with

2600 the number one above the highest existing message number.

2601 In this case the message would be named

2602 .Fn 6 ,

2603 reusing the number.

2604 If this new message would be removed as well,

2605 then the backup of the former message becomes overwritten.

2606 Hence, the ability to restore removed messages did not only depend on

2607 the sweeping cron job but also on the removing of further messages.

2608 It is undesirable to have such obscure and complex mechanisms.

2609 The user should be given a small set of clear assertions, such as

2610 ``Removed files are restorable within a seven-day grace time.''

2611 With the addition ``... unless a message with the same name in the

2612 same folder is removed before.'' the statement becomes complex.

2613 A user will hardly be able to keep track of all removals to know

2614 if the assertion still holds true for a specific file.

2615 In practice, the real mechanism is unclear to the user.

2616 .P

2617 Furthermore, the backup files were scattered within the whole mail storage.

2618 This complicated managing them.

2619 It was possible with the help of

2620 .Pn find ,

2621 but everything is more convenient

2622 if the deleted messages are collected in one place.

2623 .P

2624 The profile entry

2625 .Pe rmmproc

2626 (previously named

2627 .Pe Delete-Prog )

2628 was introduced very early to improve the situation.

2629 It could be set to any command, which would be executed to remove

2630 the specified messages.

2631 This had overridden the default action, described above.

2632 Refiling the to-be-removed files to a trash folder was the usual example.

2633 Nmh's man page

2634 .Mp rmm (1)

2635 proposes to set the

2636 .Pe rmmproc

2637 to

2638 .Cl "refile +d

2639 to move messages to the trash folder

2640 .Fn +d

2641 instead of renaming them with the backup prefix.

2642 The man page additionally proposes the expunge command

2643 .Cl "rm `mhpath +d all`

2644 to empty the trash folder.

2645 .P

2646 Removing messages in such a way has advantages:

2647 .LI 1

2648 The mail storage is prevented from being cluttered with removed messages

2649 because they are all collected in one place.

2650 Existing and removed messages are thus separated more strictly.

2651 .LI 2

2652 No backup files are silently overwritten.

2653 .LI 3

2654 Most important, however, removed messages are kept in the MH domain.

2655 Messages in the trash folder can be listed like those in any other folder.

2656 Deleted messages can be displayed like any other messages.

2657 .Pn refile

2658 can restore deleted messages.

2659 All operations on deleted files are still covered by the MH tools.

2660 The trash folder is just like any other folder in the mail storage.

2661 .P

2662 Similar to the draft folder case, I dropped the old backup prefix approach

2663 in favor for replacing it by the better suiting trash folder system.

2664 Hence,

2665 .Pn rmm

2666 calls

2667 .Pn refile

2668 to move the to-be-removed message to the trash folder,

2669 .Fn +trash

2670 by default.

2671 To sweep it clean, the user can use

2672 .Cl "rmm -unlink +trash a" ,

2673 where the

2674 .Sw -unlink

2675 switch causes the files to be unlinked.

2676 .Ci 8edc5aaf86f9f77124664f6801bc6c6cdf258173

2677 .Ci ca0b3e830b86700d9e5e31b1784de2bdcaf58fc5

2678 .P

2679 Dropping the legacy approach and converting to the new approach

2680 completely, simplified the code base.

2681 The relationship between

2682 .Pn rmm

2683 and

2684 .Pn refile

2685 was inverted.

2686 In mmh,

2687 .Pn rmm

2688 invokes

2689 .Pn refile .

2690 That used to be the other way round.

2691 Yet, the relationship is simpler now.

2692 Loops, like described in nmh's man page for

2693 .Mp refile (1),

2694 can no longer occur:

2695 .QS

2696 Since

2697 .Pn refile

2698 uses your

2699 .Pe rmmproc

2700 to delete the message, the

2701 .Pe rmmproc

2702 must NOT call

2703 .Pn refile

2704 without specifying

2705 .Sw -normmproc

2706 or you will create an infinite loop.

2707 .QE

2708 .LP

2709 .Pn rmm

2710 either unlinks a message with

2711 .Fu unlink()

2712 or invokes

2713 .Pn refile

2714 to move it to the trash folder.

2715 .Pn refile

2716 does not invoke any tools.

2717 .P

2718 By generalizing the message removal in the way that it became covered

2719 by the MH concepts made the whole system more powerful.

2720

2721

2722

2723

2724

2725 .H2 "Modern Defaults

2726 .P

2727 Nmh has a bunch of convenience-improving features inactive by default,

2728 although one can expect every new user to want them active.

2729 The reason they are inactive by default is the wish to stay compatible

2730 with old versions.

2731 But what are old versions?

2732 Still, the highly useful draft folder facility has not been activated

2733 by default although it was introduced over twenty-five years ago.

2734 .[

2735 rose romine real work

2736 .]

2737 The community seems not to care.

2738 .P

2739 In nmh, new users are required to first build up

2740 a profile before they can access the modern features.

2741 Without an extensive profile, the setup is hardly usable

2742 for modern emailing.

2743 The point is not the customization of the setup,

2744 but the need to activate generally useful facilities.

2745 Yet, the real problem lies less in enabling the features,

2746 as this is straight forward as soon as one knows what he wants.

2747 The real problem is that new users need deep insight into the project

2748 to discover the available but inactive features.

2749 To give an example, I needed one year of using nmh

2750 before I became aware of the existence of the attachment system.

2751 One could argue that this fact disqualifies my reading of the

2752 documentation.

2753 If I would have installed nmh from source back then, I could agree.

2754 Yet, I had used a pre-packaged version and had expected that it would

2755 just work.

2756 Nevertheless, I had been convinced by the concepts of MH already

2757 and I am a software developer,

2758 still I required a lot of time to discover the cool features.

2759 How can we expect users to be even more advanced than me,

2760 just to enable them to use MH in a convenient and modern way?

2761 Unless they are strongly convinced of the concepts, they will fail.

2762 I have seen friends of me giving up disappointed

2763 before they truly used the system,

2764 although they had been motivated in the beginning.

2765 New users suffer hard enough to get used to the tool chest approach,

2766 we developers should spare them further inconveniences.

2767 .P

2768 Maintaining compatibility for its own sake is bad,

2769 because the code base will collect more and more compatibility code.

2770 Sticking to the compatibility code means remaining limited;

2771 whereas adjusting to the changes renders the compatibility unnecessary.

2772 Keeping unused alternatives in the code for longer than a short

2773 grace time is a bad choice as they likely

2774 gather bugs by not being constantly tested.

2775 Also, the increased code size and the greater number of conditions

2776 increase the maintenance costs.

2777 If any MH implementation would be the back-end of widespread

2778 email clients with large user bases, compatibility would be more

2779 important.

2780 Yet, it appears as if this is not the case.

2781 Hence, compatibility is hardly important for technical reasons.

2782 Its importance originates from personal reasons rather.

2783 Nmh's user base is small and old.

2784 Changing the interfaces causes inconvenience to long-term users of MH.

2785 It forces them to change their many years old MH configurations.

2786 I do understand this aspect, but by sticking to the old users,

2787 new users are kept from entering the world of MH.

2788 But the future lies in new users.

2789 In consequence, mmh invites new users by providing a convenient

2790 and modern setup, readily usable out-of-the-box.

2791 .P

2792 In mmh, all modern features are active by default and many previous

2793 approaches are removed or only accessible in a manual way.

2794 New default features include:

2795 .BU

2796 The attachment system (\c

2797 .Hd Attach )

2798 .Ci 8ff284ff9167eff8f5349481529332d59ed913b1 .

2799 .BU

2800 The draft folder facility (\c

2801 .Fn +drafts )

2802 .Ci 337338b404931f06f0db2119c9e145e8ca5a9860 .

2803 .BU

2804 The unseen sequence (`u')

2805 .Ci c2360569e1d8d3678e294eb7c1354cb8bf7501c1

2806 and the sequence negation prefix (`!')

2807 .Ci db74c2bd004b2dc9bf8086a6d8bf773ac051f3cc .

2808 .BU

2809 Quoting the original message in the reply

2810 .Ci 67411b1f95d6ec987b4c732459e1ba8a8ac192c6 .

2811 .BU

2812 Forwarding messages using MIME

2813 .Ci 6e271608b7b9c23771523f88d23a4d3593010cf1 .

2814 .LP

2815 An mmh setup with a profile that defines only the path to the

2816 mail storage, is already convenient to use.

2817 Again, Paul Vixie's supports the direction I took:

2818 ``the `main branch' should just be modern''.

2819 .[

2820 paul vixie edginess nmh-workers

2821 .]

2822

2823

2824

2825

2826

2827 .\" --------------------------------------------------------------

2828 .H1 "Styling

2829 .P

2830 Kernighan and Pike have emphasized the importance of style in the

2831 preface of \fPThe Practice of Programming\fP:

2832 .[ [

2833 kernighan pike practice of programming

2834 .], p. x]

2835 .QS

2836 Chapter 1 discusses programming style.

2837 Good style is so important to good programming that we have chosen

2838 to cover it first.

2839 .QE

2840 This section covers changes in mmh that were guided by the desire

2841 to improve on style.

2842 Many of them follow the advice given in the quoted book.

2843

2844

2845

2846

2847 .H2 "Code Style

2848 .Id code-style

2849 .P

2850 .U3 "Indentation Style

2851 .P

2852 Indentation styles are the holy cow of programming.

2853 Kernighan and Pike write:

2854 .[ [

2855 kernighan pike practice of programming

2856 .], p. 10]

2857 .QS

2858 Programmers have always argued about the layout of programs,

2859 but the specific style is much less important than its consistent

2860 application.

2861 Pick one style, preferably ours, use it consistently, and don't waste

2862 time arguing.

2863 .QE

2864 .P

2865 I agree that the constant application is most important,

2866 but I believe that some styles have advantages over others.

2867 For instance the indentation with tab characters only.

2868 The number of tabs corresponds to the nesting level \(en

2869 one tab, one level.

2870 Tab characters provide flexible visual appearance because developers

2871 can adjust their width as prefered.

2872 There is no more need to check for the correct mixture of

2873 tabs and spaces.

2874 Two simple rules ensure the integrity and flexibility of the visual

2875 appearance:

2876 .LI 1

2877 Leading whitespace must consist of tabs only.

2878 .LI 2

2879 All other whitespace should be spaces.

2880 .LP

2881 Although reformatting existing code should be avoided, I did it.

2882 I did not waste time arguing; I just reformatted the code.

2883 .Ci a485ed478abbd599d8c9aab48934e7a26733ecb1

2884

2885 .U3 "Comments

2886 .P

2887 Kernighan and Pike demand: ``Don't belabor the obvious''.

2888 .[ [

2889 kernighan pike practice of programming

2890 .], p. 23]

2891 Following the advice, I removed unnecessary comments.

2892 For instance, I removed all comments in the following code excerpt

2893 .Ci 426543622b377fc5d091455cba685e114b6df674 :

2894 .VS

2895 context_replace(curfolder, folder); /* update current folder */

2896 seq_setcur(mp, mp->lowsel); /* update current message */

2897 seq_save(mp); /* synchronize message sequences */

2898 folder_free(mp); /* free folder/message structure */

2899 context_save(); /* save the context file */

2900

2901 [...]

2902

2903 int c; /* current character */

2904 char *cp; /* miscellaneous character pointer */

2905

2906 [...]

2907

2908 /* NUL-terminate the field */

2909 *cp = '\0';

2910 VE

2911 .P

2912 The information in each of the comments was present in the code

2913 statements already, except for the NUL-termination, which became

2914 obvious from the context.

2915

2916

2917 .U3 "Names

2918 .P

2919 Regarding this topic, Kernighan and Pike suggest:

2920 ``Use active names for functions''.

2921 .[ [

2922 kernighan pike practice of programming

2923 .], p. 4]

2924 One application of this rule was the rename of

2925 .Fu check_charset()

2926 to

2927 .Fu is_native_charset()

2928 .Ci 8d77b48284c58c135a6b2787e721597346ab056d .

2929 The same change additionally fixed a violation of ``Be accurate'',

2930 .[ [

2931 kernighan pike practice of programming

2932 .], p. 4]

2933 as the code did not match the expectation the function suggested.

2934 It did not compare charset names but prefixes of them only.

2935 In case the native charset was `ISO-8859-1', then

2936 .VS

2937 check_charset("ISO-8859-11", strlen("ISO-8859-11"))

2938 VE

2939 had returned true although the upper halves of the code pages

2940 are different.

2941 .P

2942 More important than using active names is using descriptive names.

2943 .VS

2944 m_unknown(in); /* the MAGIC invocation... */

2945 VE

2946 Renaming the obscure

2947 .Fu m_unknown()

2948 function was a delightful event, although it made the code less funny

2949 .Ci 611d68d19204d7cbf5bd585391249cb5bafca846 .

2950 .P

2951 Magic numbers are generally considered bad style.

2952 Obviously, Kernighan and Pike agree:

2953 ``Give names to magic numbers''.

2954 .[ [

2955 kernighan pike practice of programming

2956 .], p. 19]

2957 .P

2958 The argument

2959 .CW outnum

2960 of the function

2961 .Fu scan()

2962 in

2963 .Fn uip/scansbr.c

2964 holds the number of the message to be created.

2965 As well it encodes program logic with negative numbers and zero.

2966 This led to obscure code.

2967 I clarified the code by introducing two variables that extracted

2968 the hidden information:

2969 .VS

2970 int incing = (outnum > 0);

2971 int ismbox = (outnum != 0);

2972 VE

2973 The readable names are thus used in conditions;

2974 the variable

2975 .CW outnum

2976 is used only to extract ordinary message numbers

2977 .Ci b8b075c77be7794f3ae9ff0e8cedb12b48fd139f .

2978 .P

2979 Through the clarity improvement of the change detours in the program

2980 logic of related code parts became apparent.

2981 The implementation was simplified.

2982 This possibility to improve had been invisible before

2983 .Ci aa60b0ab5e804f8befa890c0a6df0e3143ce0723 .

2984 .P

2985 The names just described were a first step, yet the situation

2986 was further improved by giving names to the magic values of

2987 .CW outnum :

2988 .VS

2989 #define SCN_MBOX (-1)

2990 #define SCN_FOLD 0

2991 VE

2992 The two variables were updated thereafter as well:

2993 .VS

2994 int incing = (outnum != SCN_MBOX && outnum != SCN_FOLD);

2995 int scanfolder = (outnum == SCN_FOLD);

2996 VE

2997 Furthermore,

2998 .CW ismbox

2999 was replaced by

3000 .CW scanfolder

3001 because that matched better to the program logic.

3002 .Ci 7ffb36d28e517a6f3a10272056fc127592ab1c19

3003

3004

3005

3006

3007 .H2 "Structural Rework

3008 .P

3009 Although the stylistic changes described already improve the

3010 readability of the source code, all of them were changes ``in the small''.

3011 Structural changes, in contrast, affect much larger code areas.

3012 They are more difficult to accomplish but lead to larger improvements,

3013 especially as they often influence the outer shape of the tools as well.

3014 .P

3015 At the end of their chapter on style,

3016 Kernighan and Pike ask: ``But why worry about style?''

3017 .[ [

3018 kernighan pike practice of programming

3019 .], p. 28].

3020 Following are two examples of structural rework that demonstrate

3021 why style is important in the first place.

3022

3023

3024 .U3 "Rework of \f(CWanno\fP

3025 .P

3026 Until 2002,

3027 .Pn anno

3028 had six functional command line switches:

3029 .Sw -component

3030 and

3031 .Sw -text ,

3032 each with an argument,

3033 and the two pairs of flags,

3034 .Sw -[no]date

3035 and

3036 .Sw -[no]inplace .

3037 Then Jon Steinhart introduced his attachment system.

3038 In need for more advanced annotation handling, he extended

3039 .Pn anno .

3040 He added five more switches:

3041 .Sw -draft ,

3042 .Sw -list ,

3043 .Sw -delete ,

3044 .Sw -append ,

3045 and

3046 .Sw -number ,

3047 the last one taking an argument

3048 .Ci 7480dbc14bc90f2d872d434205c0784704213252 .

3049 Later,

3050 .Sw -[no]preserve

3051 was added as well

3052 .Ci d9b1d57351d104d7ec1a5621f090657dcce8cb7f .

3053 Then, the Synopsis section of the man page

3054 .Mp anno (1)

3055 read:

3056 .VS

3057 anno [+folder] [msgs] [-component f(CIfieldfP] [-inplace | -noinplace]

3058 [-date | -nodate] [-draft] [-append] [-list] [-delete]

3059 [-number [f(CInumfP|fPallfP]] [-preserve | -nopreserve] [-version]

3060 [-help] [-text f(CIbodyfP]

3061 VE

3062 .LP

3063 The implementation followed the same structure.

3064 Problems became visible when

3065 .Cl "anno -list -number 42

3066 worked on the current message instead of on message number 42,

3067 and

3068 .Cl "anno -list -number l:5

3069 did not work on the last five messages but failed with the mysterious

3070 error message: ``anno: missing argument to -list''.

3071 Yet, the invocation matched the specification in the man page.

3072 There, the correct use of

3073 .Sw -number

3074 was defined as being

3075 .Cl "[-number [num|all]]

3076 and the textual description for the combination with

3077 .Sw -list

3078 read:

3079 .QS

3080 The

3081 .Sw -list

3082 option produces a listing of the field bodies for

3083 header fields with names matching the specified component,

3084 one per line. The listing is numbered, starting at 1, if the

3085 .Sw -number

3086 option is also used.

3087 .QE

3088 .LP

3089 The problem was manifold.

3090 Semantically, the argument to the

3091 .Sw -number

3092 switch is only necessary in combination with

3093 .Sw -delete ,

3094 but not with

3095 .Sw -list .

3096 The code, however, required a numeric argument in any case.

3097 If the argument was missing or non-numeric,

3098 .Pn anno

3099 aborted with an error message that additionally had an off-by-one error.

3100 It printed the name of the switch one before the concerned one.

3101 .P

3102 Trying to fix these problems on the surface would not have solved them.

3103 They originate from a discrepance between the

3104 structure of the problem and the structure implemented in the program.

3105 Such structural differences can only be solved by adjusting the

3106 structure of the implementation to the structure of the problem.

3107 .P

3108 Steinhart had added the new

3109 .Sw -list

3110 and

3111 .Sw -delete

3112 switches in a style similar to the other switches though

3113 they are of structural different type.

3114 Semantically,

3115 .Sw -list

3116 and

3117 .Sw -delete

3118 introduce operation modes.

3119 Historically,

3120 .Pn anno

3121 had only one operation mode: adding header fields.

3122 With the extension, two more modes were added:

3123 listing and deleting header fields.

3124 The structure of the code changes did not pay respect to this

3125 fundamental change.

3126 Neither the implementation nor the documentation did clearly

3127 declare the exclusive operation modes as such.

3128 Having identified the problem, I solved it by putting structure into

3129 .Pn anno

3130 and its documentation

3131 .Ci d54c8db8bdf01e8381890f7729bc0ef4a055ea11 .

3132 .P

3133 The difference is visible in both the code and the documentation.

3134 For instance in the following code excerpt:

3135 .VS

3136 int delete = -2; /* delete header element if set */

3137 int list = 0; /* list header elements if set */

3138 [...]

3139 case DELETESW: /* delete annotations */

3140 delete = 0;

3141 continue;

3142 case LISTSW: /* produce a listing */

3143 list = 1;

3144 continue;

3145 VE

3146 .LP

3147 which was replaced by:

3148 .VS

3149 static enum { MODE_ADD, MODE_DEL, MODE_LIST } mode = MODE_ADD;

3150 [...]

3151 case DELETESW: /* delete annotations */

3152 mode = MODE_DEL;

3153 continue;

3154 case LISTSW: /* produce a listing */

3155 mode = MODE_LIST;

3156 continue;

3157 VE

3158 .LP

3159 The replacement code does not only reflect the problem's structure better,

3160 it is easier to understand as well.

3161 The same applies to the documentation.

3162 The man page was completely reorganized to propagate the same structure.

3163 This is already visible in the Synopsis section:

3164 .VS

3165 anno [+folder] [msgs] [-component f(CIfieldfP] [-text fPbodyfP]

3166 [-append] [-date | -nodate] [-preserve | -nopreserve]

3167 [-Version] [-help]

3168

3169 anno -delete [+folder] [msgs] [-component fPfieldfP] [-text

3170 fPbodyfP] [-number fPnum fP| fPall fP] [-preserve | -nopreserve]

3171 [-Version] [-help]

3172

3173 anno -list [+folder] [msgs] [-component fPfieldfP] [-number]

3174 [-Version] [-help]

3175 VE

3176

3177

3178

3179 .U3 "Path Conversion

3180 .P

3181 Four kinds of path names can appear in MH:

3182 .LI 1

3183 Absolute Unix directory paths, like

3184 .Fn /etc/passwd .

3185 .LI 2

3186 Relative Unix directory paths, like

3187 .Fn ./foo/bar .

3188 .LI 3

3189 Absolute MH folder paths, like

3190 .Fn +projects/mmh .

3191 .LI 4

3192 Relative MH folder paths, like

3193 .Fn @subfolder .

3194 .LP

3195 Relative MH folder paths, are hardly documented

3196 although they are useful for large mail storages.

3197 The current mail folder is specified as `\c

3198 .Fn @ ',

3199 just like the current directory is specified as `\c

3200 .Fn . '.

3201 .P

3202 To allow MH tools to understand all four notations,

3203 they need to be able to convert between them.

3204 In nmh, these path name conversion functions were located in the files

3205 .Fn sbr/path.c

3206 (``return a pathname'') and

3207 .Fn sbr/m_maildir.c

3208 (``get the path for the mail directory'').

3209 The seven functions in the two files were documented with no more

3210 than two comments, which described obvious information.

3211 The signatures of the four exported functions did not explain their

3212 semantics:

3213 .LI 1

3214 .CW "char *path(char *, int);

3215 .LI 2

3216 .CW "char *pluspath(char *);

3217 .LI 3

3218 .CW "char *m_mailpath(char *);

3219 .LI 4

3220 .CW "char *m_maildir(char *);

3221 .P

3222 My investigations provided the following descriptions:

3223 .LI 1

3224 The second parameter of

3225 .Fu path()

3226 defines the type as which the path given in the first parameter should

3227 be treated.

3228 Directory paths are converted to absolute directory paths.

3229 Folder paths are converted to absolute folder paths.

3230 Folder paths must not include a leading `\fL@\fP' character.

3231 Leading plus characters are preserved.

3232 The result is a pointer to newly allocated memory.

3233 .LI 2

3234 .Fu pluspath()

3235 is a convenience-wrapper to

3236 .Fu path() ,

3237 to convert folder paths only.

3238 This function can not be used for directory paths.

3239 An empty string parameter causes a buffer overflow.

3240 .LI 3

3241 .Fu m_mailpath()

3242 converts directory paths to absolute directory paths.

3243 The characters `\fL+\fP' or `\fL@\fP' at the beginning of the path name are

3244 treated literal, i.e. as the first character of a relative directory path.

3245 Hence, this function can not be used for folder paths.

3246 In any case, the result is an absolute directory path,

3247 returned as a pointer to newly allocated memory.

3248 .LI 4

3249 .Fu m_maildir()

3250 returns the parameter unchanged if it is an absolute directory path

3251 or begins with the entry `\fL.\fP' or `\fL..\fP'.

3252 All other strings are prepended with the current working directory.

3253 Hence, this function can not be used for folder paths.

3254 The result is either an absolute directory path or a relative

3255 directory path, starting with dot or dot-dot.

3256 In contrast to the other functions, the result is a pointer to

3257 static memory.

3258 .P

3259 The situation was obscure, irritating, error-prone, and non-orthogonal.

3260 Additionally, no clear terminology was used to name the different

3261 kinds of path names.

3262 Sometimes, the names were even misleading, much as the first argument of

3263 .Fu m_mailpath() ,

3264 which was named

3265 .CW folder ,

3266 although

3267 .Fu m_mailpath()

3268 could not be used with MH folder arguments.

3269 .P

3270 I clarified the path name conversion by complete rework.

3271 First of all, the terminology needed to be defined.

3272 A path name is either in the Unix domain, then it is called

3273 \fIdirectory path\fP (\fIdirpath\fP for short) or it is in the MH domain,

3274 then it is called \fIfolder path\fP (\fIfolpath\fP for short).

3275 The two terms need to be used with strict distinction.

3276 Often a clear terminology indicates that the problem is understood.

3277 Second, I exploited the concept of path type indicators.

3278 By requiring every path name to start with a distinct type identifier,

3279 the conversion between the types could be fully automated.

3280 This allows the tools to accept paths of any type from the user.

3281 Therefore, it was necessary to require relative directory paths to be

3282 prefixed with a dot character.

3283 In consequence, the dot character could no longer be an alias for the

3284 current message.

3285 .Ci cff0e16925e7edbd25b8b9d6d4fbdf03e0e60c01

3286 Third, I created three new functions to replace the previous mess:

3287 .LI 1

3288 .Fu expandfol()

3289 converts folder paths to absolute folder paths.

3290 Directory paths are simply passed through.

3291 This function is to be used for folder paths only, thus the name.

3292 The result is a pointer to static memory.

3293 .LI 2

3294 .Fu expanddir()

3295 converts directory paths to absolute directory paths.

3296 Folder paths are treated as relative directory paths.

3297 This function is to be used for directory paths only, thus the name.

3298 The result is a pointer to static memory.

3299 .LI 3

3300 .Fu toabsdir()

3301 converts any type of path to an absolute directory path.

3302 This is the function of choice for path conversion.

3303 Absolute directory paths are the most general representation of a

3304 path name.

3305 The result is a pointer to static memory.

3306 .P

3307 .\" XXX ueberfluessig?

3308 The new functions have names that indicate their use.

3309 Two of the functions convert relative to absolute path names of the

3310 same type.

3311 The third function converts any path name type to the most general one,

3312 the absolute directory path.

3313 All of the functions return pointers to static memory.

3314 The file

3315 .Fn sbr/path.c

3316 contains the implementation of the functions;

3317 .Fn sbr/m_maildir.c

3318 was removed.

3319 .Ci d39e2c447b0d163a5a63f480b23d06edb7a73aa0

3320 .P

3321 Along with the path conversion rework, I also replaced

3322 .Fu getfolder(FDEF)

3323 with

3324 .Fu getdeffol()

3325 and

3326 .Fu getfolder(FCUR)

3327 with

3328 .Fu getcurfol() ,

3329 which only wraps

3330 .Fu expandfol(""@"")

3331 for convenience.

3332 This code was moved from

3333 .Fn sbr/getfolder.c

3334 into

3335 .Fn sbr/path.c

3336 as well.

3337 .Ci d39e2c447b0d163a5a63f480b23d06edb7a73aa0

3338 .P

3339 The related function

3340 .Fu etcpath()

3341 is now included in

3342 .Fn sbr/path.c ,

3343 too

3344 .Ci b4c29794c12099556151d93a860ee51badae2e35 .

3345 Previously, it had been located in

3346 .Fn config/config.c .

3347 .P

3348 Now,

3349 .Fn sbr/path.c

3350 contains all path handling code.

3351 Besides being less code, its readability is highly improved.

3352 The functions follow a common style and are well documented.

3353

3354

3355

3356

3357 .H2 "Profile Reading

3358 .P

3359 The MH profile contains the configuration of a user-specific MH setup.

3360 MH tools read the profile right after starting up

3361 because it contains the location of the user's mail storage

3362 and similar settings that influence the whole setup.

3363 Furthermore, the profile contains the default switches for the tools

3364 as well.

3365 The context file is read along with the profile.

3366 .P

3367 For historic reasons, some MH tools did not read the profile and context.

3368 Among them were

3369 .Pn post /\c

3370 .Pn spost ,

3371 .Pn mhmail ,

3372 and

3373 .Pn slocal .

3374 The reason why these tools ignored the profile were not clearly stated.

3375 During a discussion on the nmh-workers mailing list,

3376 David Levine posted an explanation, quoting John Romine:

3377 .[

3378 nmh-workers levine post profile

3379 .]

3380

3381 .QS

3382 I asked John Romine and here's what he had to say, which

3383 agrees and provides an example that convinces me:

3384 .QS

3385 My take on this is that

3386 .Pn post

3387 should not be called by users directly, and it doesn't read the

3388 .Fn .mh_profile

3389 (only front-end UI programs read the profile).

3390 .QP

3391 For example, there can be contexts where

3392 .Pn post

3393 is called by a helper program (like `\c

3394 .Pn mhmail ')

3395 which may be run by a non-MH user.

3396 We don't want this to prompt the user to create an MH profile, etc.

3397 .QP

3398 My suggestion would be to have

3399 .Pn send

3400 pass a (hidden) `\c

3401 .Sw -fileproc

3402 .Ar proc '

3403 option to

3404 .Pn post

3405 if needed.

3406 You could also

3407 use an environment variable (I think

3408 .Pn send /\c

3409 .Pn whatnow

3410 do this).

3411 .QE

3412 .sp \n(PDu

3413 I think that's the way to go.

3414 My personal preference is to use a command line option,

3415 not an environment variable.

3416 .QE

3417

3418 .P

3419 To solve the problem that

3420 .Pn post

3421 does not honor the

3422 .Pe fileproc

3423 profile entry,

3424 the community roughly agreed that a switch

3425 .Sw -fileproc

3426 should be added to

3427 .Pn post

3428 to be able to pass a different fileproc.

3429 I strongly disagree with this approach because it does not solve

3430 the problem; it only removes a single symptom.

3431 The actual problem is that

3432 .Pn post

3433 does not behave as expected,

3434 though all programs should behave as expected.

3435 Clear and general concepts are a precondition for this.

3436 Thus, there should be no separation into ``front-end UI programs''

3437 and ones that ``should not be called by users directly''.

3438 The real solution is having all MH tools read the profile.

3439 .P

3440 But the problem has a further aspect,

3441 which originates from

3442 .Pn mhmail

3443 mainly.

3444 .Pn mhmail

3445 was intended to be a replacement for

3446 .Pn mailx

3447 on systems with MH installations.

3448 In difference to

3449 .Pn mailx ,

3450 .Pn mhmail

3451 used MH's

3452 .Pn post

3453 to send the message.

3454 The idea was that using

3455 .Pn mhmail

3456 should not be influenced whether the user had

3457 MH set up for himself or not.

3458 Therefore

3459 .Pn mhmail

3460 had not read the profile.

3461 As

3462 .Pn mhmail

3463 used

3464 .Pn post ,

3465 .Pn post

3466 was not allowed to read the profile neither.

3467 This is the reason for the actual problem.

3468 Yet, this was not considered much of a problem because

3469 .Pn post

3470 was not intended to be used by users directly.

3471 To invoke

3472 .Pn post ,

3473 .Pn send

3474 was used an a front-end.

3475 .Pn send

3476 read the profile and passed all relevant values on the command line to

3477 .Pn post

3478 \(en an awkward solution.

3479 .P

3480 The important insight is that

3481 .Pn mhmail

3482 is a wolf in sheep's clothing.

3483 This alien tool broke the concepts because it was treated like

3484 a normal MH tool.

3485 Instead it should have been treated accordingly to its foreign style.

3486 .P

3487 The solution is not to prevent the tools from reading the profile but

3488 to instruct them to read a different profile.

3489 .Pn mhmail

3490 could have set up a well-defined profile and caused the following

3491 .Pn post

3492 to use this profile by exporting an environment variable.

3493 With this approach, no special cases would have been introduced

3494 and no surprises would have been caused.

3495 By writing a wrapper program to provide a clean temporary profile,

3496 the concept could have been generalized orthogonally to the whole

3497 MH tool chest.

3498 .P

3499 In mmh, the wish to have

3500 .Pn mhmail

3501 as a replacement for

3502 .Pn mailx

3503 is considered obsolete.

3504 Mmh's

3505 .Pn mhmail

3506 does no longer cover this use-case

3507 .Ci d36e56e695fe1c482c7920644bfbb6386ac9edb0 .

3508 Currently,

3509 .Pn mhmail

3510 is in a transition state

3511 .Ci 32d4f9daaa70519be3072479232ff7be0500d009 .

3512 It may become a front-end to

3513 .Pn comp ,

3514 which provides an alternative interface which can be more convenient

3515 in some cases.

3516 This would convert

3517 .Pn mhmail

3518 into an ordinary MH tool.

3519 If, however, this idea does not convince, then

3520 .Pn mhmail

3521 will be removed.

3522 .P

3523 In the mmh tool chest, every program reads the profile.

3524 (\c

3525 .Pn slocal

3526 is not considered part of the mmh tool chest (cf. Sec.

3527 .Cf slocal ).)

3528 Mmh has no

3529 .Pn post

3530 program, but it has

3531 .Pn spost ,

3532 which now does read the profile

3533 .Ci 3e017a7abbdf69bf0dff7a4073275961eda1ded8 .

3534 Following this change,

3535 .Pn send

3536 and

3537 .Pn spost

3538 can be considered for merging.

3539 Besides

3540 .Pn send ,

3541 .Pn spost

3542 is only invoked directly by the to-be-changed

3543 .Pn mhmail

3544 implementation and by

3545 .Pn rcvdist ,

3546 which requires rework anyway.

3547

3548 .P

3549 Jeffrey Honig quoted Marshall T. Rose explaining the decision that

3550 .Pn post

3551 ignores the profile:

3552 .[

3553 nmh-workers honig post profile

3554 .]

3555 .QS

3556 when you run mh commands in a script, you want all the defaults to be

3557 what the man page says.

3558 when you run a command by hand, then you want your own defaults...

3559 .QE

3560 .LP

3561 The explanation neither matches the problem concered exactly

3562 nor is the interpretation clear.

3563 If the described desire addresses the technical level,

3564 then it conflicts fundametally with the Unix philosophy,

3565 precisely because the indistinquishability of human and script

3566 input is the main reason for the huge software leverage in Unix.

3567 If, however, the described desire addresses the user's view,

3568 then different technical solutions are more appropriate.

3569 The two cases can be regarded simply as two different MH setups.

3570 Hence, mapping the problem of different behavior between interactive and

3571 automated use on the concept of switching between different profiles,

3572 marks it already solved.

3573

3574

3575

3576 .H2 "Standard Libraries

3577 .P

3578 MH is one decade older than the POSIX and ANSI C standards.

3579 Hence, MH included own implementations of functions

3580 that were neither standardized nor widely available, back then.

3581 Today, twenty years after POSIX and ANSI C were published,

3582 developers can expect that systems comply with these standards.

3583 In consequence, MH-specific replacements for standard functions

3584 can and should be dropped.

3585 Kernighan and Pike advise: ``Use standard libraries''.

3586 .[ [

3587 kernighan pike practice of programming

3588 .], p. 196]

3589 Actually, MH had followed this advice in history,

3590 but it had not adjusted to more recent changes in this field.

3591 The

3592 .Fu snprintf()

3593 function, for instance, was standardized with C99 and is available

3594 almost everywhere because of its high usefulness.

3595 Thus, the project's own implementation of

3596 .Fu snprintf()

3597 was dropped in March 2012 in favor for using the one of the

3598 standard library

3599 .Ci 0052f1024deb0a0a2fc2e5bacf93d45a5a9c9b32 .

3600 Such decisions limit the portability of mmh

3601 if systems do not support these standardized and widespread functions.

3602 This compromise is made because mmh focuses on the future.

3603 .P

3604 .\" XXX kuerzen und mit dem naechsten Absatz vereinen

3605 As I am still in my twenties, have no programming experience from

3606 past decades.

3607 I have not followed the evolution of C through time.

3608 I have not suffered from the the Unix wars.

3609 I have not longed for standardization.

3610 All my programming experience is from a time when ANSI C and POSIX

3611 were well established already.

3612 Thus, I needed to learn about the history in retrospective.

3613 I have only read a lot of books about the (good) old times.

3614 This put me in a difficult position when working with old code.

3615 I need to freshly acquire knowledge about old code constructs and ancient

3616 programming styles, whereas older programmers know these things by

3617 heart from their own experience.

3618 Being aware of the situation, I rather let people with more historic

3619 experience do the transition from ancient code constructs to

3620 standardized ones.

3621 Lyndon Nerenberg covered large parts of this task for the nmh project.

3622 He converted project-specific functions to POSIX replacements,

3623 also removing the conditionals compilation of now standardized features.

3624 Ken Hornstein and David Levine had their part in this work, as well.

3625 Often, I only pulled the changes over from nmh into mmh.

3626 These changes include many commits, among them:

3627 .Ci 768b5edd9623b7238e12ec8dfc409b82a1ed9e2d

3628 .Ci 0052f1024deb0a0a2fc2e5bacf93d45a5a9c9b32 .

3629 .P

3630 Nevertheless, I worked on the task as well, tidying up the

3631 \fIMH standard library\fP,

3632 .Fn libmh.a .

3633 It is located in the

3634 .Fn sbr

3635 (``subroutines'') directory in the source tree and

3636 includes functions that mmh tools usually need.

3637 Among them are MH-specific functions for profile, context, sequence,

3638 and folder handling, but as well

3639 MH-independent functions, such as auxiliary string functions,

3640 portability interfaces and error-checking wrappers for critical

3641 functions of the standard library.

3642 .BU

3643 I have replaced the

3644 .Fu atooi()

3645 function with calls to

3646 .Fu strtoul() ,

3647 setting the third parameter, the base, to eight.

3648 .Fu strtoul()

3649 is part of C89 and thus considered safe to use

3650 .Ci c490c51b3c0f8871b6953bd0c74551404f840a74 .

3651 .BU

3652 I did remove project-included fallback implementations of

3653 .Fu memmove()

3654 and

3655 .Fu strerror()

3656 .Ci b067ff5c465a5d243ce5a19e562085a9a1a97215 ,

3657 although Peter Maydell had re-included them into nmh in 2008

3658 to support SunOS 4.

3659 Nevertheless, these functions are part of ANSI C.

3660 Systems that do not even provide full ANSI C support should not

3661 put a load on mmh.

3662 .BU

3663 The

3664 .Fu copy()

3665 function copies the string in parameter one to the location in

3666 parameter two.

3667 In contrast to

3668 .Fu strcpy() ,

3669 it returns a pointer to the terminating null-byte in the destination area.

3670 The code was adjusted to replace

3671 .Fu copy()

3672 with

3673 .Fu strcpy() ,

3674 except within

3675 .Fu concat() ,

3676 where

3677 .Fu copy()

3678 was more convenient.

3679 Therefore, the definition of

3680 .Fu copy()

3681 was moved into the source file of

3682 .Fu concat()

3683 and its visibility it limited to that

3684 .Ci 552fd7253e5ee9e554c5c7a8248a6322aa4363bb .

3685 .BU

3686 The function

3687 .Fu r1bindex()

3688 had been a generalized version of

3689 .Fu basename()

3690 with minor differences.

3691 As all calls to

3692 .Fu r1bindex()

3693 had the slash (`\fL/\fP') as delimiter anyway,

3694 replacing

3695 .Fu r1bindex()

3696 with the more specific and better-named function

3697 .Fu basename()

3698 became desirable.

3699 Unfortunately, many of the 54 calls to

3700 .Fu r1bindex()

3701 depended on a special behavior,

3702 which differed from the POSIX specification for

3703 .Fu basename() .

3704 Hence,

3705 .Fu r1bindex()

3706 was kept but renamed to

3707 .Fu mhbasename() ,

3708 setting the delimiter to the slash

3709 .Ci 240013872c392fe644bd4f79382d9f5314b4ea60 .

3710 For possible uses of

3711 .Fu r1bindex()

3712 with a different delimiter,

3713 the ANSI C function

3714 .Fu strrchr()

3715 provides the core functionality.

3716 .BU

3717 The

3718 .Fu ssequal()

3719 function \(en apparently for ``substring equal'' \(en

3720 was renamed to

3721 .Fu isprefix() ,

3722 because this is what it actually checked

3723 .Ci c20b4fa14515c7ab388ce35411d89a7a92300711.

3724 Its source file had included both of the following comments, no joke.

3725 .in -\n(PIu

3726 .VS

3727 /*

3728 * THIS CODE DOES NOT WORK AS ADVERTISED.

3729 * It is actually checking if s1 is a PREFIX of s2.

3730 * All calls to this function need to be checked to see

3731 * if that needs to be changed. Prefix checking is cheaper, so

3732 * should be kept if it's sufficient.

3733 */

3734

3735 /*

3736 * Check if s1 is a substring of s2.

3737 * If yes, then return 1, else return 0.

3738 */

3739 VE

3740 .in +\n(PIu

3741 Eventually, the function was completely replaced with calls to

3742 .Fu strncmp()

3743 .Ci b0b1dd37ff515578cf7cba51625189eb34a196cb .

3744

3745

3746

3747

3748

3749 .H2 "User Data Locations

3750 .P

3751 In nmh, a personal setup consists of the MH profile and the MH directory.

3752 The profile is a file named

3753 .Fn \&.mh_profile

3754 in the user's home directory.

3755 It contains the static configuration.

3756 It also contains the location of the MH directory in the profile entry

3757 .Pe Path .

3758 The MH directory contains the mail storage and is the first

3759 place to search for form files, scan formats, and similar

3760 configuration files.

3761 The location of the MH directory can be chosen freely by the user.

3762 The usual name is a directory named

3763 .Fn Mail

3764 in the user's home directory.

3765 .P

3766 The way MH data is split between profile and MH directory is a legacy.

3767 It is only sensible in a situation where the profile is the only

3768 configuration file.

3769 Why else should the mail storage and the configuration files be intermixed?

3770 They are of different kind:

3771 One kind is the data to be operated on and the other kind is

3772 the configuration to change how tools operate.

3773 Splitting the configuration between the profile and the MH directory

3774 is inappropriate, as well.

3775 I improved the situation by breaking compatibility.

3776 .P

3777 In mmh, personal data is grouped by type.

3778 This results in two distinct parts:

3779 the mail storage and the configuration.

3780 The mail storage directory still contains all the messages,

3781 but, in exception of public sequences files, nothing else.

3782 In difference to nmh, the auxiliary configuration files are no longer

3783 located there.

3784 Therefore, the directory is no longer called the user's \fIMH directory\fP

3785 but the user's \fImail storage\fP.

3786 Its location is still user-chosen, with the default name

3787 .Fn Mail

3788 in the user's home directory.

3789 The configuration is grouped together in the hidden directory

3790 .Fn \&.mmh

3791 in the user's home directory.

3792 This \fImmh directory\fP contains the context file, personal forms,

3793 scan formats, and the like, but also the user's profile, now named

3794 .Fn profile .

3795 The path to the profile is no longer

3796 .Fn $HOME/.mh_profile

3797 but

3798 .Fn $HOME/.mmh/profile .

3799 (The alternative of having file

3800 .Fn $HOME/.mh_profile

3801 and a configuration directory

3802 .Fn $HOME/.mmh

3803 appeared to be inconsistent.)

3804 .P

3805 The approach chosen for mmh is consistent, simple, and familiar to

3806 Unix users.

3807 The main achievement of the change is the clear and sensible separation

3808 of the mail storage and the configuration.

3809 .Ci 7030d7edb099bff36ded7548bb5380f7acab4f9b

3810 .P

3811 As MH allows users to have multiple MH setups,

3812 it is necessary to switch the profile.

3813 The profile is the single entry point to access the rest of a

3814 personal MH setup.

3815 In nmh, the environment variable

3816 .Ev MH

3817 is used to specify a different profile.

3818 To operate in the same MH setup with a separate context, the

3819 .Ev MHCONTEXT

3820 environment variable is used.

3821 This allows having a separate current folder in each terminal at

3822 the same time, for instance.

3823 In mmh, three environment variables replace the two of nmh.

3824 .Ev MMH

3825 overrides the default location of the mmh directory (\c

3826 .Fn .mmh ).

3827 .Ev MMHP

3828 and

3829 .Ev MMHC

3830 override the paths to the profile and context file, respectively.

3831 This approach allows the set of personal configuration files to be chosen

3832 independently of the profile, context, and mail storage.

3833 The new approach has no functional disadvantages,

3834 as every setup I can imagine can be implemented with both approaches,

3835 possibly even easier with the new one.

3836 .Ci 7030d7edb099bff36ded7548bb5380f7acab4f9b

3837

3838

3839

3840

3841

3842 .H2 "Modularization

3843 .Id modularization

3844 .P

3845 The source code of the mmh tools is located in the

3846 .Fn uip

3847 (``user interface programs'') directory.

3848 Each tool has a source file with the name of the command.

3849 For example,

3850 .Pn rmm

3851 is built from

3852 .Fn uip/rmm.c .

3853 Some source files are used for multiple programs.

3854 For example

3855 .Fn uip/scansbr.c

3856 is used for both

3857 .Pn scan

3858 and

3859 .Pn inc .

3860 In nmh, 49 tools were built from 76 source files.

3861 This is a ratio of 1.6 source files per program.

3862 32 programs depended on multiple source files;

3863 17 programs depended on one source file only.

3864 In mmh, 39 tools are built from 51 source files.

3865 This is a ratio of 1.3 source files per program.

3866 18 programs depend on multiple source files;

3867 21 programs depend on one source file only.

3868 (These numbers and the ones in the following text ignore the MH library

3869 as well as shell scripts and multiple names for the same program.)

3870 .\" XXX graph

3871 .P

3872 Splitting the source code of a large program into multiple files can

3873 increase the readability of its source code,

3874 but most of the mmh tools are small and straight-forward programs.

3875 In exception of the MIME handling tools (i.e.

3876 .Pn mhbuild ,

3877 .Pn mhstore ,

3878 .Pn show ,

3879 etc.),

3880 .Pn pick

3881 is the only tool with more than one thousand lines of source code.

3882 Splitting programs with less than one thousand lines of code into

3883 multiple source files leads seldom to better readability.

3884 For such tools, splitting still makes sense

3885 when parts of the code are reused in other programs

3886 and the reused code fragment is (1) not general enough

3887 for including it in the MH library

3888 or (2) has dependencies on a library that only few programs need.

3889 .Fn uip/packsbr.c ,

3890 for instance, provides the core program logic for the

3891 .Pn packf

3892 and

3893 .Pn rcvpack

3894 programs.

3895 .Fn uip/packf.c

3896 and

3897 .Fn uip/rcvpack.c

3898 mainly wrap the core function appropriately.

3899 No other tools use the folder packing functions.

3900 As another example,

3901 .Fn uip/termsbr.c

3902 accesses terminal properties, which requires linking with the

3903 \fItermcap\fP or a \fIcurses\fP library.

3904 If

3905 .Fn uip/termsbr.c

3906 is included in the MH library, then every program needs to be linked

3907 with termcap or curses, although only few of the programs use

3908 the library.

3909 .P

3910 The task of MIME handling is complex enough that splitting its code

3911 into multiple source files improves the readability.

3912 The program

3913 .Pn mhstore ,

3914 for instance, is compiled out of seven source files with 2\|500

3915 lines of code in summary.

3916 The main code file

3917 .Fn uip/mhstore.c

3918 consists of 800 lines; the other 1\|700 lines are code reused in

3919 other MIME handling tools.

3920 It seems to be worthwhile to bundle the generic MIME handling code into

3921 a MH-MIME library, as a companion to the MH standard library.

3922 This is left to be done.

3923 .P

3924 The work already accomplished focussed on the non-MIME tools.

3925 The amount of code compiled into each program was reduced.

3926 This eases the understanding of the code base.

3927 In nmh,

3928 .Pn comp

3929 was built from six source files:

3930 .Fn comp.c ,

3931 .Fn whatnowproc.c ,

3932 .Fn whatnowsbr.c ,

3933 .Fn sendsbr.c ,

3934 .Fn annosbr.c ,

3935 and

3936 .Fn distsbr.c .

3937 In mmh, it builds from only two:

3938 .Fn comp.c

3939 and

3940 .Fn whatnowproc.c .

3941 In nmh's

3942 .Pn comp ,

3943 the core function of

3944 .Pn whatnow ,

3945 .Pn send ,

3946 and

3947 .Pn anno

3948 were all compiled into

3949 .Pn comp .

3950 This saved the need to execute these programs with

3951 the expensive system calls

3952 .Fu fork()

3953 and

3954 .Fu exec() .

3955 Whereas this approach improved the time performance,

3956 it interwove the source code.

3957 Core functionalities were not encapsulated into programs but into

3958 function, which were then wrapped by programs.

3959 For example,

3960 .Fn uip/annosbr.c

3961 included the function

3962 .Fu annotate() .

3963 Each program that wanted to annotate messages, included the source file

3964 .Fn uip/annosbr.c

3965 and called

3966 .Fu annotate() .

3967 Because the function

3968 .Fu annotate()

3969 was used like the tool

3970 .Pn anno ,

3971 it had seven parameters, reflecting the command line switches of the tool.

3972 When another pair of command line switches was added to

3973 .Pn anno ,

3974 a rather ugly hack was implemented to avoid adding another parameter

3975 to the function

3976 .Ci d9b1d57351d104d7ec1a5621f090657dcce8cb7f .

3977 .P

3978 In mmh, the relevant code of

3979 .Pn comp

3980 comprises the two files

3981 .Fn uip/comp.c

3982 and

3983 .Fn uip/whatnowproc.c ,

3984 together 210 lines of code,

3985 whereas in nmh,

3986 .Pn comp

3987 comprises six files with 2\|450 lines.

3988 Not all of the code in these six files is actually used by

3989 .Pn comp ,

3990 but the reader needed to read it all to know which parts are relevant.

3991 Understanding nmh's

3992 .Pn comp ,

3993 required understanding the inner workings of

3994 .Fn uip/annosbr.c

3995 first.

3996 To be sure to fully understand a program, its whole source code needs

3997 to be examined.

3998 Not doing so is a leap of faith, assuming that the developers

3999 have avoided obscure programming techniques.

4000 Here, it should be recalled that information passed in obscure ways

4001 through the program's source base, due to the aforementioned hack

4002 to save an additional parameter in nmh's

4003 .Pn anno .

4004 .P

4005 In mmh, understanding

4006 .Pn comp

4007 requires to read only 210 lines of code to read, whereas the amount

4008 is ten times more for nmh's

4009 .Pn comp .

4010 .P

4011 By separating the tools on the program-level,

4012 the boundaries are clearly visible, as the interfaces are calls to

4013 .Fu exec()

4014 rather than arbitrary function calls.

4015 Additionally, this kind of separation is more strict because

4016 it is technically enforced by the operating system;

4017 it can not be simply bypassed with global variables.

4018 Good separation simplifies the understanding of program code

4019 because the area influenced by any particular statement is small.

4020 As I have read a lot in nmh's code base during the last two years,

4021 I have learned about the easy and the difficult parts.

4022 In my observation, the understanding of code is enormously eased

4023 if the influenced area is small and clearly bounded.

4024 .P

4025 Yet, the real problem is another:

4026 Nmh violates the golden ``one tool, one job'' rule of the Unix philosophy.

4027 Understanding

4028 .Pn comp

4029 requires understanding

4030 .Fn uip/annosbr.c

4031 and

4032 .Fn uip/sendsbr.c

4033 because

4034 .Pn comp

4035 annotates and sends messages.

4036 In nmh, there surely exist the tools

4037 .Pn anno

4038 and

4039 .Pn send ,

4040 which cover these jobs,

4041 but

4042 .Pn comp

4043 and

4044 .Pn repl

4045 and

4046 .Pn forw

4047 and

4048 .Pn dist

4049 and

4050 .Pn whatnow

4051 and

4052 .Pn viamail

4053 \(en they all (!) \(en

4054 have the same annotating and sending functions included, once more.

4055 As a result,

4056 .Pn comp

4057 sends messages without using

4058 .Pn send .

4059 The situation is the same as if

4060 .Pn grep

4061 would page its output without using

4062 .Pn more

4063 just because both programs are part of the same code base.

4064 .P

4065 The clear separation on the surface of nmh

4066 \(en the tool chest approach \(en

4067 is violated on the level below.

4068 This violation is for the sake of time performance.

4069 Decades ago, sacrificing readability and conceptional beauty

4070 for speed might have been necessary to prevent MH from being

4071 unusably slow, but today this is not the case anymore.

4072 No longer should speed improvements that became unnecessary be kept.

4073 No longer should readability or conceptional beauty be sacrificed.

4074 No longer should the Unix philosophy's ``one tool, one job''

4075 guideline be violated.

4076 Therefore, mmh's

4077 .Pn comp

4078 no longer sends messages.

4079 .P

4080 In mmh, different jobs are divided among separate programs that

4081 invoke each other as needed.

4082 In consequence,

4083 .Pn comp

4084 invokes

4085 .Pn whatnow

4086 which thereafter invokes

4087 .Pn send

4088 .Ci 3df5ab3c116e6d4a2fb4bb5cc9dfc5f781825815

4089 .Ci c73c00bfccd22ec77e9593f47462aeca4a8cd9c0 .

4090 The clear separation on the surface is maintained on the level below.

4091 Human users and other tools use the same interface \(en

4092 annotations, for example, are made by invoking

4093 .Pn anno ,

4094 no matter if requested by programs or by human beings

4095 .Ci 469a4163c2a1a43731d412eaa5d9cae7d670c48b

4096 .Ci aed384169af5204b8002d06e7a22f89197963d2d

4097 .Ci 3caf9e298a8861729ca8b8a84f57022b6f3ea742 .

4098 The decrease of tools built from multiple source files and thus

4099 the decrease of

4100 .Fn uip/*sbr.c

4101 files confirm the improvement

4102 .Ci 9e6d91313f01c96b4058d6bf419a8ca9a207bc33

4103 .ci 81744a46ac9f845d6c2b9908074d269275178d2e

4104 .Ci f0f858069d21111f0dbea510044593f89c9b0829

4105 .Ci 0503a6e9be34f24858b55b555a5c948182b9f24b

4106 .Ci 27826f9353e0f0b04590b7d0f8f83e60462b90f0

4107 .Ci d1da1f94ce62160aebb30df4063ccbc53768656b

4108 .Ci c42222869e318fff5dec395eca3e776db3075455 .

4109 This is also visible in the complexity of the build dependency graphs:

4110

4111 .sp

4112 Nmh:

4113 .BP input/deps-nmh.eps .5i

4114 .EP

4115 .sp

4116 Mmh:

4117 .BP input/deps-mmh.eps .8i

4118 .EP

4119

4120 The figures display all program to source file relationships

4121 that are not one-to-one,

4122 i.e. all programs that are built from multiple source files.

4123 The primary source file of each program is omited from the graph.