docs/master: 27c28990b844 discussion.roff

docs/master

view discussion.roff @ 226:27c28990b844

Added quote by Steve Johnson.

author	markus schnalke <meillo@marmaro.de>
date	Sun, 15 Jul 2012 21:57:51 +0200
parents	d83ab437e3a6
children	157c92fc1597

line source

1 .H0 "Discussion

2 .P

3 This main chapter discusses the practical work accomplished in the

4 mmh project.

5 It is structured along the goals chosen for the project.

6 A selection of the work undertaken

7 is described.

8 .P

9 This discussion compares the present version of mmh with the state of

10 nmh at the time when the mmh project had started, i.e. fall 2011.

11 Recent changes in nmh are rarely part of the discussion.

12 .P

13 For the reader's convenience, the structure of modern email systems

14 is depicted in the following figure.

15 It illustrates the path a message takes from sender to recipient.

17 .sp 1.5

18 .KS

19 .in 2c

20 .so input/mail-agents.pic

21 .KE

22 .sp 1.5

24 .LP

25 The ellipses denote mail agents, i.e. different jobs in email processing.

26 These are:

27 .IP "Mail User Agent (MUA)

28 The only program users directly interact with.

29 It includes functions to compose new mail, display received mail,

30 and to manage the mail storage.

31 It is called a \fImail client\fP as well.

32 .IP "Mail Submission Agent (MSA)

33 A special kind of Mail Transfer Agent, used to submit mail into the

34 mail transport system.

35 Often it is also called an MTA.

36 .IP "Mail Transfer Agent (MTA)

37 A node in the mail transport system.

38 It transfers incoming mail to a transport node nearer to the

39 final destination.

40 An MTA may be the final destination itself.

41 .IP "Mail Delivery Agent (MDA)

42 Delivers mail according to a set of rules.

43 Usually, the messages are stored to disk.

44 .IP "Mail Retrieval Agent (MRA)

45 Initiates the transfer of mail from a remote location to the local machine.

46 (The dashed arrow in the figure represents the pull request.)

47 .LP

48 The dashed boxes represent entities that usually reside on single machines.

49 The box on the lower left represents the sender's system.

50 The box on the upper left represents the first mail transfer node.

51 The box on the upper right represents the transfer node responsible

52 for the destination address.

53 The box on the lower right represents the recipient's system.

54 Often, the boxes above the dotted line are servers on the Internet.

55 Many mail clients, including nmh, include all of the components below

56 the dotted line.

57 This is not the case for mmh; it implements the MUA only.

64 .\" --------------------------------------------------------------

65 .H1 "Streamlining

67 .P

68 MH once provided a complete email system.

69 The community around nmh tries to keep nmh in similar shape.

70 In fundamental contrast, mmh shall be an MUA only.

71 I believe that the development of all-in-one mail systems is obsolete.

72 Today, email is too complex to be fully covered by a single project.

73 Such a project will not be able to excel in all aspects.

74 Instead, the aspects of email should be covered by multiple projects,

75 which then can be combined to form a complete system.

76 Excellent implementations for the various aspects of email already exist.

77 Just to name three examples: Postfix is a specialized MTA, Procmail

78 is a specialized MDA, and Fetchmail is a specialized MRA.

79 I believe that it is best to use such specialized tools instead of

80 providing the same function once more as a side component.

81 .P

82 Doing something well requires focusing on a small set of specific aspects.

83 Under the assumption that development which is focussed on a particular

84 area produces better results there, specialized projects will be superior

85 in their field of focus.

86 Hence, all-in-one mail system projects \(en no matter if monolithic

87 or modular \(en will never be the best choice in any of the fields.

88 Even in providing the most consistent all-in-one system, they are likely

89 to be beaten by projects that focus exclusively on the creation

90 of a homogeneous system by integrating existing mail components.

91 .P

92 Usually, the limiting resource in the community development of

93 free software is man power.

94 If the development effort is spread over a large development area,

95 it becomes more difficult to compete with the specialists in the

96 various fields.

97 The concrete situation for MH-based mail systems is even tougher,

98 given their small and aged community, concerning both developers and users.

99 .P

100 In consequence, I believe that the available development resources

101 should focus on the point where MH is most unique.

102 This is clearly the user interface \(en the MUA.

103 Peripheral parts should be removed to streamline mmh for the MUA task.

104

105

106 .H2 "Mail Transfer Facilities

107 .Id mail-transfer-facilities

108 .P

109 The removal of the mail transfer facilities, effectively dropping the

110 MSA and MRA, had been the first work task in the mmh project.

111 The desire for this change initiated the creation of the mmh project.

112 .P

113 Focusing on one mail agent role only, is motivated by Eric Allman's

114 experience with Sendmail.

115 He identified the limitation of Sendmail

116 to the MTA task as one reason for its success:

117 .[ [

118 costales sendmail

119 .], p. xviii]

120 .QS

121 Second, I limited myself to the routing function \(en

122 I wouldn't write user agents or delivery back-ends.

123 This was a departure of the dominant thought of the time,

124 in which routing logic, local delivery, and often the network code

125 were incorporated directly into the user agents.

126 .QE

127 .P

128 In nmh, the MSA is called \fIMessage Transfer Service\fP (MTS).

129 This facility, implemented by the

130 .Pn post

131 command, establishes network connections and spoke SMTP to submit

132 messages to be relayed to the outside world.

133 When email transfer changed, this part needed to be changed as well.

134 Encryption and authentication for network connections

135 needed to be supported, hence TLS and SASL were introduced into nmh.

136 This added complexity without improving the core functions.

137 Furthermore, keeping up with recent developments in the field of

138 mail transfer requires development power and specialists.

139 In mmh, this whole facility was simply cut off

140 .Ci f6aa95b724fd8c791164abe7ee5468bf5c34f226

141 .Ci fecd5d34f65597a4dfa16aeabea7d74b191532c3

142 .Ci 156d35f6425bea4c1ed3c4c79783dc613379c65b .

143 Instead, mmh depends on an external MSA.

144 All outgoing mail in mmh goes through the

145 .Pn sendmail

146 command, which almost any MSA provides.

147 If not, a wrapper program can be written.

148 It must read the message from the standard input, extract the

149 recipient addresses from the message header, and hand the message

150 over to the MSA.

151 For example, a wrapper script for qmail would be:

152 .VS

153 #!/bin/sh

154 exec qmail-inject # ignore command line arguments

155 VE

156 The requirement to parse the recipient addresses out of the message header

157 may be removed in the future.

158 Mmh could pass the recipient addresses as command line arguments.

159 This appears to be the better interface.

160 .P

161 To retrieve mail, the

162 .Pn inc

163 command in nmh acts as MRA.

164 It establishes network connections

165 and speaks POP3 to retrieve mail from remote servers.

166 As with mail submission, the network connections required encryption and

167 authentication, thus TLS and SASL were added to nmh.

168 Support for message retrieval through IMAP will soon become necessary

169 additions, too, and likewise for any other changes in mail transfer.

170 But not in mmh because it has dropped the support for retrieving mail

171 from remote locations

172 .Ci ab7b48411962d26439f92f35ed084d3d6275459c .

173 Instead, it depends on an external tool to cover this task.

174 Mmh has two paths for messages to enter mmh's mail storage:

175 (1) Mail can be incorporated with

176 .Pn inc

177 from the system maildrop, or (2) with

178 .Pn rcvstore

179 by reading them, one at a time, from the standard input.

180 .P

181 With the removal of the MSA and MRA, mmh converted from a complete

182 mail system to only an MUA.

183 Now, of course, mmh depends on third-party software.

184 An external MSA is required to transfer mail to the outside world;

185 an external MRA is required to retrieve mail from remote machines.

186 Excellent implementations of such software exist.

187 They likely are superior to the internal versions that were removed.

188 Additionally, the best suiting programs can be chosen freely.

189 .P

190 As it had already been possible to use an external MSA and MRA,

191 why should the internal version not be kept for convenience?

192 Transfered to a different area, the question,

193 whether there is sense in having a fall-back pager in all

194 the command line tools for the cases when

195 .Pn more

196 or

197 .Pn less

198 are not available, appears to be ridiculous.

199 Of course, MSAs and MRAs are more complex than text pagers

200 and not necessarily available but still the concept of orthogonal

201 design holds: ``Write programs that do one thing and do it well''.

202 .[

203 mcilroy unix phil

204 p. 53

205 .]

206 .[

207 mcilroy bstj foreword

208 .]

209 Here, this part of the Unix philosophy was applied not only

210 to the programs but to the project itself.

211 In other words:

212 Develop projects that focus on one thing and do it well.

213 Projects which have grown complex should be split, for the same

214 reasons that programs which have grown complex should be split.

215 If it is conceptionally more elegant to have the MSA and MRA as

216 separate projects then they should be separated.

217 In my opinion, this is the case.

218 The RFCs suggest this separation by clearly distinguishing the

219 different mail handling tasks [RFC\|821].

220 The small interfaces between the mail agents support the

221 separation as well.

222 .P

223 Once, email had been small and simple.

224 At that time,

225 .Pn /bin/mail

226 had covered everything there was to email and still was small and simple.

227 Later, the essential complexity of email increased.

228 (Essential complexity is the complexity defined by the problem itself

229 .[ [

230 brooks no silver bullet

231 .]].)

232 Consequently, email systems grew.

233 RFCs started to introduce the concept of mail agents to separate the

234 various roles because they became more extensive and because

235 new roles appeared.

236 As mail system implementations grew, parts of them were split off.

237 For instance, a POP server was included in the original MH;

238 it was removed in nmh.

239 Now is the time to go one step further and split off the MSA and MRA,

240 as well.

241 Not only does this decrease the code size of the project,

242 more importantly, it unburdens mmh of the whole field of

243 message transfer, with all its implications for the project.

244 There is no more need for concern with changes in network transfer.

245 This independence is gained by depending on external components

246 that cover the field.

247 .P

248 In general, functionality can be added in three different ways:

249 .LI 1

250 By implementing the function in the project itself.

251 .LI 2

252 By depending on a library that provides the function.

253 .LI 3

254 By depending on a program that provides the function.

255 .LP

256 While implementing the function in the project itself leads to the

257 largest increase in code size and requires the most maintenance

258 and development work,

259 it keeps the project's dependence on other software lowest.

260 Using libraries or external programs requires less maintenance work

261 but introduces dependencies on external projects.

262 Programs have the smallest interfaces and provide the best separation,

263 but possibly limit the information exchange.

264 External libraries are more strongly connected than external programs,

265 thus information can be exchanged in a more flexible manner.

266 Adding code to a project increases maintenance work.

267 As implementing complex functions in the project itself adds

268 a lot of code, this should be avoided if possible.

269 Thus, the dependencies only change in their character,

270 not in their existence.

271 In mmh, library dependencies on

272 .Pn libsasl2

273 and

274 .Pn libcrypto /\c

275 .Pn libssl

276 were traded against program dependencies on an MSA and an MRA.

277 This also meant trading build-time dependencies against run-time

278 dependencies.

279 Besides providing stronger separation and greater flexibility,

280 program dependencies also allowed

281 over 6\|000 lines of code to be removed from mmh.

282 This made mmh's code base about 12\|% smaller.

283 Reducing the project's code size by such an amount without actually

284 losing functionality is a convincing argument.

285 Actually, as external MSAs and MRAs are likely superior to the

286 project's internal versions, the common user even gains functionality.

287 .P

288 Users of MH should not have problems setting up an external MSA and MRA.

289 Also, the popular MSAs and MRAs have large communities and a lot

290 of available documentation.

291

292 Choices for MSAs range from small forwarders such as \fIssmtp\fP and

293 \fInullmailer\fP, over mid-size MTAs including \fImasqmail\fP and

294 \fIdma\fP, up to full-featured MTAs as for instance \fIPostfix\fP.

295 MRAs are provided for example by \fIfetchmail\fP, \fIgetmail\fP,

296 \fImpop\fP, and \fIfdm\fP.

297

298

299 .H2 "Non-MUA Tools

300 .P

301 One goal of mmh is to remove the tools that do not significantly

302 contribute to the MUA's job.

303 Loosely related and rarely used tools distract from a lean appearance,

304 and require maintenance work without adding much to the core task.

305 By removing these tools, mmh became more streamlined and focused.

306 .BU

307 .Pn conflict

308 was removed

309 .Ci 8b235097cbd11d728c07b966cf131aa7133ce5a9

310 because it is a mail system maintenance tool and not MUA-related.

311 It even checked

312 .Fn /etc/passwd

313 and

314 .Fn /etc/group

315 for consistency, which is completely unrelated to email.

316 A tool like

317 .Pn conflict

318 is surely useful, but it should not be shipped with mmh.

319 .BU

320 .Pn rcvtty

321 was removed

322 .Ci 14767c94b3827be7c867196467ed7aea5f6f49b0

323 because its use case of writing to the user's terminal

324 on reception of mail is obsolete.

325 If users like to be informed of new mail, the shell's

326 .Ev MAILPATH

327 variable or graphical notifications are technically more appealing.

328 Writing to terminals directly is hardly ever desired today.

329 If, though, one prefers this approach, the standard tool

330 .Pn write

331 can be used in a way similar to:

332 .VS

333 scan -file - | write `id -un`

334 VE

335 .BU

336 .Pn viamail

337 was removed

338 .Ci eda72d6a7a7c20ff123043fb7f19c509ea01f932

339 when the new attachment system was activated, because

340 .Pn forw

341 could then cover the task itself.

342 The program

343 .Pn sendfiles

344 was rewritten as a shell script wrapper around

345 .Pn forw .

346 .Ci 0e82199cf3c991a173e0ac8aa776efdb3ded61e6

347 .BU

348 .Pn msgchk

349 was removed

350 .Ci bb9360ead7eb7a3fedcce2eeedfc660014e41dbe ,

351 because it lost its use case when POP support was removed.

352 A call to

353 .Pn msgchk

354 provided hardly more information than:

355 .VS

356 ls -l /var/mail/meillo

357 VE

358 Yet, it distinguished between old and new mail, but

359 these details can be retrieved with

360 .Pn stat (1),

361 too.

362 A small shell script could be written to print the information

363 in a similar way, if truly necessary.

364 As mmh's

365 .Pn inc

366 only incorporates mail from the user's local maildrop,

367 and thus no data transfers over slow networks are involved,

368 there is hardly any need to check for new mail before incorporating it.

369 .BU

370 .Pn msh

371 was removed

372 .Ci 916690191222433a6923a4be54b0d8f6ac01bd02

373 because the tool was in conflict with the philosophy of MH.

374 It provided an interactive shell to access the features of MH.

375 However, it was not just a shell tailored to the needs of mail handling,

376 but one large program that had several MH tools built in.

377 This conflicted with the major feature of MH of being a tool chest.

378 .Pn msh 's

379 main use case had been accessing Bulletin Boards, which have ceased to

380 be popular.

381 .P

382 Removing

383 .Pn msh

384 together with the truly archaic code relics

385 .Pn vmh

386 and

387 .Pn wmh

388 saved more than 7\|000 lines of C code \(en

389 about 15\|% of the project's original source code amount.

390 Having less code \(en with equal readability, of course \(en

391 for the same functionality is an advantage.

392 Less code means less bugs and less maintenance work.

393 As

394 .Pn rcvtty

395 and

396 .Pn msgchk

397 are assumed to be rarely used and can be implemented in different ways,

398 why should one keep them?

399 Removing them streamlined mmh.

400 .Pn viamail 's

401 use case is now partly obsolete and partly covered by

402 .Pn forw ,

403 hence there is no reason to still maintain it.

404 .Pn conflict

405 is not related to the mail client, and

406 .Pn msh

407 conflicts with the basic concept of MH.

408 These two tools might still be useful, but they should not be part of mmh.

409 .P

410 .Id slocal

411 Finally, there is

412 .Pn slocal ,

413 which is an MDA and thus not directly MUA-related.

414 It should be removed from mmh because including it conflicts with

415 the idea that mmh is an MUA only.

416 However,

417 .Pn slocal

418 provides rule-based processing of messages, like filing them into

419 different folders, which is otherwise not available in mmh.

420 Although

421 .Pn slocal

422 neither pulls in dependencies, nor does it include a separate

423 technical area (cf. Sec.

424 .Cf mail-transfer-facilities ),

425 it still accounts for about 1\|000 lines of code that need to be maintained.

426 As

427 .Pn slocal

428 is almost self-standing, it should be split off into a separate project.

429 This would cut the strong connection between the MUA mmh and the MDA

430 .Pn slocal .

431 For anyone not using MH,

432 .Pn slocal

433 would become yet another independent MDA, like

434 .I procmail .

435 Then

436 .Pn slocal

437 could be installed without a complete MH system.

438 Likewise, mmh users could decide to use

439 .I procmail

440 without having a second, unused MDA, i.e.

441 .Pn slocal ,

442 installed.

443 That appears to be conceptionally the best solution.

444 Yet,

445 .Pn slocal

446 is not split off.

447 I defer the decision over

448 .Pn slocal

449 out of a need for deeper investigation.

450 In the meanwhile, it remains part of mmh

451 as its continued existence is not significant;

452 .Pn slocal

453 is unrelated to the rest of the project.

454

455

456

457 .H2 "Displaying Messages

458 .Id mhshow

459 .P

460 Since the very beginning, already in the first concept paper,

461 .Pn show

462 had been MH's message display program.

463 .Pn show

464 mapped message numbers and sequences to files and invoked

465 .Pn mhl

466 to have the files formatted.

467 With MIME, this approach was not sufficient anymore.

468 MIME messages can consist of multiple parts.

469 Some parts, like binary attachments or text content in foreign charsets,

470 are not directly displayable.

471 .Pn show 's

472 understanding of messages and

473 .Pn mhl 's

474 display capabilities could not cope with the task any longer.

475 .P

476 Instead of extending these tools, additional tools were written from

477 scratch and were added to the MH tool chest.

478 Doing so is encouraged by the tool chest approach.

479 Modular design is a great advantage for extending a system,

480 as new tools can be added without interfering with existing ones.

481 First, the new MIME features were added in form of the single program

482 .Pn mhn .

483 The command

484 .Cl "mhn -show 42

485 had then shown the message number

486 .Fn 42 ,

487 interpreting MIME.

488 With the 1.0 release of nmh in February 1999, Richard Coleman finished

489 the split of

490 .Pn mhn

491 into a set of specialized tools, which together covered the

492 multiple aspects of MIME.

493 One of them was

494 .Pn mhshow ,

495 which replaced

496 .Cl "mhn -show" .

497 It was capable of displaying MIME messages appropriately.

498 .P

499 From then on, two message display tools were part of nmh,

500 .Pn show

501 and

502 .Pn mhshow .

503 To ease the life of users,

504 .Pn show

505 was extended to automatically hand the job over to

506 .Pn mhshow

507 if displaying the message would be beyond

508 .Pn show 's

509 abilities.

510 In consequence, the user would simply invoke

511 .Pn show

512 (possibly through

513 .Pn next

514 or

515 .Pn prev )

516 and get the message printed with either

517 .Pn show

518 or

519 .Pn mhshow ,

520 whatever was more appropriate.

521 .P

522 Having two similar tools for basically the same task is redundancy.

523 Usually, users do not distinguish between

524 .Pn show

525 and

526 .Pn mhshow

527 in their daily mail reading.

528 Having two separate display programs was therefore unnecessary

529 from a user's point of view.

530 Besides, the development of both programs needed to be in sync,

531 to ensure that the programs behaved in a similar way,

532 because they were used like a single tool.

533 Different behavior would have surprised the user.

534 .P

535 Today, non-MIME messages are rather seen to be a special case of

536 MIME messages, although it is the other way round.

537 As

538 .Pn mhshow

539 already had been able to display non-MIME messages, it appeared natural

540 to drop

541 .Pn show

542 in favor of using

543 .Pn mhshow

544 exclusively

545 .Ci 4c1efddfd499300c7e74263e57d8aa137e84c853 .

546 Removing

547 .Pn show

548 is no loss in function, because

549 .Pn mhshow

550 covers it completely.

551 Yet, the old behavior of

552 .Pn show

553 can still be emulated with the simple command line:

554 .VS

555 mhl `mhpath c`

556 VE

557 .P

558 For convenience,

559 .Pn mhshow

560 was renamed to

561 .Pn show

562 after

563 .Pn show

564 was gone.

565 It is clear that such a rename may confuse future developers when

566 trying to understand the history.

567 Nevertheless, I consider the convenience on the user's side,

568 to outweigh the inconvenience for understanding the evolution

569 of the tools.

570 .P

571 To prepare for the transition,

572 .Pn mhshow

573 was reworked to behave more like

574 .Pn show

575 first (cf. Sec.

576 .Cf mhshow ).

577 Once the tools behaved more alike, the replacing appeared to be

578 even more natural.

579 Today, mmh's new

580 .Pn show

581 has become the one single message display program once again,

582 with the difference

583 that today it handles MIME messages as well as non-MIME messages.

584 The outcomes of the transition are one program less to maintain,

585 no second display program for users to deal with,

586 and less system complexity.

587 .P

588 Still, removing the old

589 .Pn show

590 hurts in one regard: It had been such a simple program.

591 Its lean elegance is missing from the new

592 .Pn show ,

593 but there is no alternative;

594 supporting MIME demands higher essential complexity.

595

596

597

598 .H2 "Configure Options

599 .P

600 Customization is a double-edged sword.

601 It allows better suiting setups, but not for free.

602 There is the cost of code complexity to be able to customize.

603 There is the cost of less tested setups, because there are

604 more possible setups and especially corner cases.

605 Steve Johnson confirms:

606 .[ [

607 eric raymond the art of unix programming

608 .], p. 233]

609 .QS

610 Unless it is done very carefully, the addition of an on/off

611 configuration option can lead to a need to double the amount of

612 testing. Since in practice one never does double the amount of testing,

613 the practical effect is to reduce the amount of testing that any

614 given configuration receives. Ten options leads to 1024 times as much

615 testing, and pretty soon you are talking real reliability problems.

616 .QE

617 .LP

618 Additionally, there is the cost of choice itself.

619 The code complexity directly affects the developers.

620 Less tested code affects both users and developers.

621 The problem of choice affects the users, for once by having to choose

622 but also by more complex interfaces that require more documentation.

623 Whenever options add few advantages but increase the complexity of the

624 system, they should be considered for removal.

625 I have reduced the number of project-specific configure options from

626 15 to 3.

627

628 .U3 "Mail Transfer Facilities

629 .P

630 With the removal of the mail transfer facilities 5 configure

631 options vanished:

632 .P

633 The switches

634 .Sw --with-tls

635 and

636 .Sw --with-cyrus-sasl

637 had activated the support for transfer encryption and authentication.

638 .\" XXX cf

639 .\" XXX gruende kurz wiederholen

640 They are not needed anymore.

641 .Ci fecd5d34f65597a4dfa16aeabea7d74b191532c3

642 .Ci 156d35f6425bea4c1ed3c4c79783dc613379c65b

643 .P

644 .\" XXX cf

645 .\" XXX ``For the same reason ...''

646 The configure switch

647 .Sw --enable-pop

648 activated the message retrieval facility.

649 Whereas the code area that had been conditionally compiled in

650 for TLS and SASL support was small,

651 the conditionally compiled code area for POP support was much larger.

652 The code base had only changed slightly on toggling TLS or SASL

653 support but it had changed much on toggling POP support.

654 The changes in the code base could hardly be overviewed.

655 By having POP support togglable, a second code base had been created,

656 one that needed to be tested.

657 This situation is basically similar for the conditional TLS and SASL

658 code, but there the changes are minor and can yet be overviewed.

659 Still, conditional compilation of a code base creates variations

660 of the original program.

661 More variations require more testing and maintenance work.

662 .P

663 Two other options only specified default configuration values:

664 .Sw --with-mts

665 defined the default transport service

666 .Ci f6aa95b724fd8c791164abe7ee5468bf5c34f226 .

667 With

668 .Sw --with-smtpservers

669 default SMTP servers could be specified

670 .Ci 128545e06224233b7e91fc4c83f8830252fe16c9 .

671 Both of them became irrelevant when the SMTP transport service was removed.

672 In mmh, all messages are handed over to

673 .Pn sendmail

674 for transportation.

675

676

677 .U3 "Backup Prefix

678 .P

679 The backup prefix is the string that was prepended to message

680 filenames to tag them as deleted.

681 By default it had been the comma character (`\fL,\fP').

682 In July 2000, Kimmo Suominen introduced

683 the configure option

684 .Sw --with-hash-backup

685 to change the default to the hash character `\f(CW#\fP'.

686 This choice was probably personal preference, but,

687 being related or not, words that start with the hash character

688 introduce a comment in the Unix shell.

689 Thus, the command line

690 .Cl "rm #13 #15

691 calls

692 .Pn rm

693 without arguments because the first hash character starts a comment

694 that reaches until the end of the line.

695 To delete the backup files,

696 .Cl "rm ./#13 ./#15"

697 needs to be used.

698 Thus, using the hash as backup prefix may be seen as a precaution

699 against backup loss.

700 .P

701 First, I removed the configure option but added the profile entry

702 .Pe Backup-Prefix ,

703 which allowed to specify an arbitrary string as backup prefix

704 .Ci 6c40d481d661d532dd527eaf34cebb6d3f8ed086 .

705 This change did not remove the choice but moved it to a location where

706 it suited better, in my eyes.

707 .P

708 Eventually however, the new trash folder concept

709 (cf. Sec.

710 .Cf trash-folder )

711 removed the need for the backup prefix completely.

712 .Ci 8edc5aaf86f9f77124664f6801bc6c6cdf258173

713 .Ci ca0b3e830b86700d9e5e31b1784de2bdcaf58fc5

714

715

716 .U3 "Editor and Pager

717 .P

718 The two configure options

719 .CW --with-editor=EDITOR

720 .CW --with-pager=PAGER

721 were used to specify the default editor and pager at configure time.

722 Doing so at configure time made sense in the eighties,

723 when the set of available editors and pagers varied much across

724 different systems.

725 Today, the situation is more homogeneous.

726 The programs

727 .Pn vi

728 and

729 .Pn more

730 can be expected to be available on every Unix system,

731 as they are specified by POSIX since two decades.

732 (The specifications for

733 .Pn vi

734 and

735 .Pn more

736 appeared in

737 .[

738 posix 1987

739 .]

740 and,

741 .[

742 posix 1992

743 .]

744 respectively.)

745 As a first step, these two tools were hard-coded as defaults

746 .Ci 5d43a99db70c12a673028c7758c20cbe3e13ef5f .

747 Not changed were the

748 .Pe editor

749 and

750 .Pe moreproc

751 profile entries, which allowed the user to override the system defaults.

752 Later, the concept was reworked again to respect the standard

753 environment variables

754 .Ev VISUAL

755 and

756 .Ev PAGER

757 if they are set.

758 Today, mmh determines the editor to use in the following order,

759 taking the first available and non-empty item

760 .Ci f85f4b7ae62e3d05a945dcd46ead51f0a2a89a9b :

761 .LI 1

762 Environment variable

763 .Ev MMHEDITOR

764 .LI 2

765 Profile entry

766 .Pe Editor

767 .LI 3

768 Environment variable

769 .Ev VISUAL

770 .LI 4

771 Environment variable

772 .Ev EDITOR

773 .LI 5

774 Command

775 .Pn vi .

776 .LP

777 The pager to use is determined in a similar order

778 .Ci 0c4214ea2aec6497d0d67b436bbee9bc1d225f1e :

779 .LI 1

780 Environment variable

781 .Ev MMHPAGER

782 .LI 2

783 Profile entry

784 .Pe Pager

785 (replaces

786 .Pe moreproc )

787 .LI 3

788 Environment variable

789 .Ev PAGER

790 .LI 4

791 Command

792 .Pn more .

793 .LP

794 By respecting the

795 .Ev VISUAL /\c

796 .Ev EDITOR

797 and

798 .Ev PAGER

799 environment variables,

800 the new behavior complies with the common style on Unix systems.

801 It is more uniform and clearer for users.

802

803

804 .U3 "ndbm

805 .P

806 .Pn slocal

807 used to depend on the database library

808 .I ndbm .

809 The database is used to store the

810 .Hd Message-ID

811 header field values of all messages delivered.

812 This enabled

813 .Pn slocal

814 to suppress delivering the same message to the same user twice.

815 This features was enabled by the

816 .Sw -suppressdup

817 switch.

818 .P

819 As a variety of versions of the database library exist,

820 .[

821 wolter unix incompat notes dbm

822 .]

823 complicated autoconf code was needed to detect them correctly.

824 Furthermore, the configure switches

825 .Sw --with-ndbm=ARG

826 and

827 .Sw --with-ndbmheader=ARG

828 were added to help with difficult setups that would

829 not be detected automatically or not correctly.

830 .P

831 By removing the suppress duplicates feature of

832 .Pn slocal ,

833 the dependency on

834 .I ndbm

835 vanished and 120 lines of complex autoconf code could be saved

836 .Ci ecd6d6a20cb7a1507e3a20d6c4cb3a1cf14c6bbf .

837 The change removed functionality but that is considered minor to the

838 improvement of dropping the dependency and the complex autoconf code.

839

840

841 .U3 "MH-E Support

842 .P

843 The configure option

844 .Sw --disable-mhe

845 was removed when the MH-E support was reworked.

846 MH-E is the Emacs front-end to MH.

847 It requires MH to provide minor additional functions.

848 The

849 .Sw --disable-mhe

850 configure option had switched off these extensions.

851 After removing the support for old versions of MH-E,

852 only the

853 .Sw -build

854 switches of

855 .Pn forw

856 and

857 .Pn repl

858 are left to be MH-E extensions.

859 They are now always built in because they add little code and complexity.

860 In consequence, the

861 .Sw --disable-mhe

862 configure option was removed

863 .Ci a7ce7b4a580d77b6c2c4d980812beb589aa4c643 .

864 Dropping the option also removed a variant of the code base

865 that would have needed to be tested.

866 This change was undertaken in January 2012 in nmh and

867 thereafter merged into mmh.

868

869

870 .U3 "Masquerading

871 .P

872 The configure option

873 .Sw --enable-masquerade

874 could take up to three arguments:

875 .Ar draft_from ,

876 .Ar mmailid ,

877 and

878 .Ar username_extension .

879 They activated different types of address masquerading.

880 All of them were implemented in the SMTP-speaking

881 .Pn post

882 command.

883 Address masquerading is an MTA's task and mmh does not cover

884 this field anymore.

885 Hence, true masquerading needs to be implemented in the external MTA.

886 .P

887 The

888 .I mmailid

889 masquerading type is the oldest one of the three and the only one

890 available in the original MH.

891 It provided a

892 .I username

893 to

894 .I fakeusername

895 mapping, based on the

896 .Fn passwd 's

897 GECOS field.

898 Nmh's man page

899 .Mp mh-tailor (5)

900 described the use case as being the following:

901 .QS

902 This is useful if you want the messages you send to always

903 appear to come from the name of an MTA alias rather than your

904 actual account name. For instance, many organizations set up

905 `First.Last' sendmail aliases for all users. If this is

906 the case, the GECOS field for each user should look like:

907 ``First [Middle] Last <First.Last>''

908 .QE

909 .P

910 As mmh sends outgoing mail via the local MTA only,

911 the best location to do such global rewrites is there.

912 Besides, the MTA is conceptionally the right location because it

913 does the reverse mapping for incoming mail (aliasing), too.

914 Furthermore, masquerading set up there is readily available for all

915 mail software on the system.

916 Hence, mmailid masquerading was removed.

917 .Ci 0836c8000ccb34b59410ef1c15b1b7feac70ce5f

918 .P

919 The

920 .I username_extension

921 masquerading type did not replace the username but would append a suffix,

922 specified by the

923 .Ev USERNAME_EXTENSION

924 environment variable, to it.

925 This provided support for the

926 .I user-extension

927 feature of qmail

928 .[ [

929 sill qmail handbook

930 .], p. 141]

931 and the similar

932 .I "plussed user

933 processing of Sendmail.

934 .[ [

935 sendmail costales

936 .], p. 476]

937 The decision to remove this username_extension masquerading

938 was motivated by the fact that

939 .Pn spost

940 had not supported it yet.

941 Username extensions can be used in mmh, but less convenient.

942 .\" XXX In the format file: %(getenv USERNAME_EXTENSION)

943 .Ci 2abae0bfd0ad5bf898461e50aa4b466d641f23d9

944 .P

945 The

946 .I draft_from

947 masquerading type instructed

948 .Pn post

949 to use the value of the

950 .Hd From

951 header field as SMTP envelope sender.

952 Sender addresses could be replaced completely.

953 Mmh offers a kind of masquerading similar in effect, but

954 with technical differences.

955 As mmh does not transfer messages itself, the local MTA has final control

956 over the sender's address.

957 Any masquerading mmh introduces may be reverted by the MTA.

958 In times of pedantic spam checking, an MTA will take care to use

959 sensible envelope sender addresses to keep its own reputation up.

960 Nonetheless, the MUA can set the

961 .Hd From

962 header field and thereby propose a sender address to the MTA.

963 The MTA may then decide to take that one or generate the canonical sender

964 address for use as envelope sender address.

965 .Ci b14ea6073f77b4359aaf3fddd0e105989db9

966 .P

967 In mmh, the MTA will always extract the recipient and sender from the

968 message header (\c

969 .Pn sendmail 's

970 .Sw -t

971 switch).

972 The

973 .Hd From

974 header field of the draft may be set arbitrary by the user.

975 If it is missing, the canonical sender address will be generated by the MTA.

976

977 .U3 "Remaining Options

978 .P

979 Two configure options remain in mmh.

980 One is the locking method to use:

981 .Sw --with-locking=[dot|fcntl|flock|lockf] .

982 The idea of removing all methods except the portable

983 .I "dot locking

984 and having that one as the default is appealing, but this change

985 requires deeper technical investigation into the topic.

986 The other option,

987 .Sw --enable-debug ,

988 compiles the programs with debugging symbols.

989 This option is likely to stay.

990

991

992

993

994 .H2 "Command Line Switches

995 .P

996 The command line switches of MH tools follow a style similar to

997 the X Window System style.

998 The switches consist of a single dash (`\fL-\fP') followed by a word.

999 For example

1000 .Cl -truncate .

1001 To ease typing, the word can be abbreviated, given the remaining

1002 prefix is unambiguous.

1003 If no other switch starts with the letter `t', then any of

1004 .Cl "-truncate" ,

1005 .Cl "-trunc" ,

1006 .Cl "-tr" ,

1007 and

1008 .Cl "-t

1009 is equal.

1010 As a result, switches can neither be grouped (as in

1011 .Cl "ls -ltr" )

1012 nor can switch arguments be appended directly to the switch (as in

1013 .Cl "sendmail -q30m" ).

1014 Many switches have negating counter-parts, which start with `no'.

1015 For example

1016 .Cl "-notruncate

1017 inverts the

1018 .Cl "-truncate

1019 switch.

1020 They exist to override the effect of default switches in the profile.

1021 Every program in mmh has two generic switches:

1022 .Sw -help ,

1023 to print a short message on how to use the program, and

1024 .Sw -Version

1025 (with capital `V'), to tell what version of mmh the program belongs to.

1026 .P

1027 Switches change the behavior of programs.

1028 Programs that do one thing in one way require no switches.

1029 In most cases, doing something in exactly one way is too limiting.

1030 If one task should be accomplished in various ways,

1031 switches are a good approach to alter the behavior of a program.

1032 Changing the behavior of programs provides flexibility and customization

1033 to users, but at the same time it complicates the code,

1034 the documentation, and the usage of the program.

1035 Therefore, the number of switches should be kept small.

1036 A small set of well-chosen switches is best.

1037 Usually, the number of switches increases over time.

1038 Already in 1985, Rose and Romine have identified this as a major

1039 problem of MH:

1040 .[ [

1041 rose romine real work

1042 .], p. 12]

1043 .QS

1044 A complaint often heard about systems which undergo substantial development

1045 by many people over a number of years, is that more and more options are

1046 introduced which add little to the functionality but greatly increase the

1047 amount of information a user needs to know in order to get useful work done.

1048 This is usually referred to as creeping featurism.

1049 .QP

1050 Unfortunately MH, having undergone six years of off-and-on development by

1051 ten or so well-meaning programmers (the present authors included),

1052 suffers mightily from this.

1053 .QE

1054 .P

1055 Being reluctant to adding new switches (or \fIoptions\fP,

1056 as Rose and Romine call them) is one part of a counter-action,

1057 the other part is removing hardly used switches.

1058 Nmh's tools have lots of switches already implemented.

1059 Hence, cleaning up by removing some of them was the more important part

1060 of the counter-action.

1061 Removing existing functionality is always difficult because it

1062 breaks programs that use these functions.

1063 Also, for every obsolete feature, there'll always be someone who still

1064 uses it and thus opposes its removal.

1065 This puts the developer into the position,

1066 where sensible improvements to style are regarded as destructive acts.

1067 Yet, living with the featurism is far worse, in my eyes, because

1068 future needs will demand adding further features,

1069 worsening the situation more and more.

1070 Rose and Romine added in a footnote,

1071 ``[...]

1072 .Pn send

1073 will no doubt acquire an endless number of switches in the years to come''

1074 .[ [

1075 rose romine real work

1076 .], p. 12].

1077 Although clearly humorous, the comment points to the nature of the problem.

1078 Refusing to add any new switches would encounter the problem at its root,

1079 but this is not practical.

1080 New needs will require new switches and it would be unwise to block

1081 them strictly.

1082 Nevertheless, removing obsolete switches still is an effective approach

1083 to deal with the problem.

1084 Working on an experimental branch without an established user base,

1085 eased my work because I did not offend users when I removed existing

1086 functions.

1087 .P

1088 Rose and Romine counted 24 visible and 9 more hidden switches for

1089 .Pn send .

1090 In nmh, they increased up to 32 visible and 12 hidden ones.

1091 At the time of writing, no more than 4 visible switches and 1 hidden switch

1092 have remained in mmh's

1093 .Pn send .

1094 These numbers include the two generic switches,

1095 .Sw -help

1096 and

1097 .Sw -Version .

1098 .P

1099 Hidden switches are ones not documented.

1100 In mmh, 12 tools have hidden switches.

1101 9 of them are

1102 .Sw -debug

1103 switches, the other 6 provide special interfaces for internal use.

1104 .P

1105 The following figure displays the number of switches for each of the tools

1106 that is available in both nmh and mmh.

1107 The tools are sorted by the number of switches they had in nmh.

1108 Both visible and hidden switches were counted,

1109 but not the generic help and version switches.

1110 Whereas in the beginning of the project, the average tool had 11 switches,

1111 now it has no more than 5 \(en only half as many.

1112 If the `no' switches and similar inverse variant are folded onto

1113 their counter-parts, the average tool had 8 switches in pre-mmh times and

1114 has 4 now.

1115 The total number of functional switches in mmh dropped from 465

1116 to 233.

1117

1118 .KS

1119 .in 1c

1120 .so input/switches.grap

1121 .KE

1122

1123 .P

1124 A part of the switches vanished after functions were removed.

1125 This was the case for network mail transfer, for instance.

1126 Sometimes, however, the work flow was the other way:

1127 I looked through the

1128 .Mp mh-chart (7)

1129 man page to identify the tools with apparently too many switches.

1130 Then I considered the benefit of each switch by examining

1131 the tool's man page and source code, aided by literature research

1132 and testing.

1133

1134

1135 .U3 "Draft Folder Facility

1136 .P

1137 A change early in the project was the complete transition from

1138 the single draft message to the draft folder facility

1139 .Ci 337338b404931f06f0db2119c9e145e8ca5a9860 .

1140 .\" XXX ref to section ...

1141 The draft folder facility was introduced in the mid-eighties, when

1142 Rose and Romine called it a ``relatively new feature''.

1143 .[

1144 rose romine real work

1145 .]

1146 Since then, the facility was included, inactive by default.

1147 By making it permanently active and by related rework of the tools, the

1148 .Sw -[no]draftfolder ,

1149 and

1150 .Sw -draftmessage

1151 switches could be removed from

1152 .Pn comp ,

1153 .Pn repl ,

1154 .Pn forw ,

1155 .Pn dist ,

1156 .Pn whatnow ,

1157 and

1158 .Pn send

1159 .Ci 337338b404931f06f0db2119c9e145e8ca5a9860 .

1160 The only flexibility lost with this change is having multiple

1161 draft folders within one profile.

1162 I consider this a theoretical problem only.

1163 At the same time, the

1164 .Sw -draft

1165 switch of

1166 .Pn anno ,

1167 .Pn refile ,

1168 and

1169 .Pn send

1170 was removed.

1171 The special treatment of \fIthe\fP draft message became irrelevant after

1172 the rework of the draft system

1173 (cf. Sec.

1174 .Cf draft-folder ).

1175 Furthermore,

1176 .Pn comp

1177 no longer needs a

1178 .Sw -file

1179 switch as the draft folder facility together with the

1180 .Sw -form

1181 switch are sufficient.

1182

1183

1184 .U3 "In Place Editing

1185 .P

1186 .Pn anno

1187 had the switches

1188 .Sw -[no]inplace

1189 to either annotate the message in place and thus preserve hard links,

1190 or annotate a copy to replace the original message.

1191 The latter approach broke hard links.

1192 Following the assumption that linked messages should truly be the

1193 same message and annotating it should not break the link, the

1194 .Sw -[no]inplace

1195 switches were removed and the previous default

1196 .Sw -inplace

1197 was made the definitive behavior

1198 .Ci c8195849d2e366c569271abb0f5f60f4ebf0b4d0 .

1199 The

1200 .Sw -[no]inplace

1201 switches of

1202 .Pn repl ,

1203 .Pn forw ,

1204 and

1205 .Pn dist

1206 could be removed, as well, as they were simply passed through to

1207 .Pn anno .

1208 .P

1209 .Pn burst

1210 also had

1211 .Sw -[no]inplace

1212 switches, but with a different meaning.

1213 With

1214 .Sw -inplace ,

1215 the digest had been replaced by the table of contents (i.e. the

1216 introduction text) and the burst messages were placed right

1217 after this message, renumbering all following messages.

1218 Also, any trailing text of the digest was lost, though,

1219 in practice, it usually consists of an end-of-digest marker only.

1220 Nonetheless, this behavior appeared less elegant than the

1221 .Sw -noinplace

1222 behavior, which already had been the default.

1223 Nmh's

1224 .Mp burst (1)

1225 man page reads:

1226 .QS

1227 If

1228 .Sw -noinplace

1229 is given, each digest is preserved, no table

1230 of contents is produced, and the messages contained within

1231 the digest are placed at the end of the folder. Other messages

1232 are not tampered with in any way.

1233 .QE

1234 .LP

1235 The decision to drop the

1236 .Sw -inplace

1237 behavior was supported by the code complexity and the possible data loss

1238 it caused.

1239 .Sw -noinplace

1240 was chosen to be the definitive behavior.

1241 .Ci 68a686adeb39223a5e1ad35e4a24890ec053679d

1242

1243

1244 .U3 "Forms and Format Strings

1245 .P

1246 Historically, the tools that had

1247 .Sw -form

1248 switches to supply a form file had

1249 .Sw -format

1250 switches as well to supply the contents of a form file as a string

1251 on the command line directly.

1252 In consequence, the following two lines equaled:

1253 .VS

1254 scan -form scan.mailx

1255 scan -format "`cat /path/to/scan.mailx`"

1256 VE

1257 The

1258 .Sw -format

1259 switches were dropped in favor for extending the

1260 .Sw -form

1261 switches

1262 .Ci f51956be123db66b00138f80464d06f030dbb88d .

1263 If their argument starts with an equal sign (`\fL=\fP'),

1264 then the rest of the argument is taken as a format string,

1265 otherwise the arguments is treated as the name of a format file.

1266 Thus, now the following two lines equal:

1267 .VS

1268 scan -form scan.mailx

1269 scan -form "=`cat /path/to/scan.mailx`"

1270 VE

1271 This rework removed the prefix collision between

1272 .Sw -form

1273 and

1274 .Sw -format .

1275 Typing `\fL-fo\fP' is sufficient to specify form file or format string.

1276 .P

1277 The different meaning of

1278 .Sw -format

1279 for

1280 .Pn forw

1281 and

1282 .Pn repl

1283 was removed in mmh.

1284 .Pn forw

1285 was completely switched to MIME-type forwarding, thus removing the

1286 .Sw -[no]format

1287 .Ci 6e271608b7b9c23771523f88d23a4d3593010cf1 .

1288 For

1289 .Pn repl ,

1290 the

1291 .Sw -[no]format

1292 switches were reworked to

1293 .Sw -[no]filter

1294 switches

1295 .Ci 67411b1f95d6ec987b4c732459e1ba8a8ac192c6 .

1296 The

1297 .Sw -format

1298 switches of

1299 .Pn send

1300 and

1301 .Pn post ,

1302 which had a third meaning, were removed likewise

1303 .Ci f3cb7cde0e6f10451b6848678d95860d512224b9 .

1304 Eventually, the ambiguity of the

1305 .Sw -format

1306 switches is resolved by not having such switches anymore in mmh.

1307

1308

1309 .U3 "MIME Tools

1310 .P

1311 The MIME tools, which once were part of

1312 .Pn mhn

1313 (whatever that stood for),

1314 had several switches that added little practical value to the programs.

1315 The

1316 .Sw -[no]realsize

1317 switches of

1318 .Pn mhbuild

1319 and

1320 .Pn mhlist

1321 were removed

1322 .Ci 8d8f1c3abc586c005c904e52c4adbfe694d2201c .

1323 Real size calculations are done always now because nmh's

1324 .Mp mhbuild (1)

1325 man page states that

1326 ``This provides an accurate count at the expense of a small delay''

1327 with the small delay not being noticable on modern systems.

1328 .P

1329 The

1330 .Sw -[no]check

1331 switches were removed together with the support for

1332 .Hd Content-MD5

1333 header fields [RFC\|1864]

1334 (cf. Sec.

1335 .Cf content-md5 )

1336 .Ci 31dc797eb5178970d68962ca8939da3fd9a8efda .

1337 .P

1338 The

1339 .Sw -[no]ebcdicsafe

1340 and

1341 .Sw -[no]rfc934mode

1342 switches of

1343 .Pn mhbuild

1344 were removed because they are considered obsolete

1345 .Ci 01a3480928da485b4d6109d36d751dfa71799d58

1346 .Ci 3363e2624dce0eb8164cf8b3f1ab385c8ff72e88 .

1347 .P

1348 Content caching of external MIME parts, activated with the

1349 .Sw -rcache

1350 and

1351 .Sw -wcache

1352 switches was completely removed

1353 .Ci d1fefd9f614e4dc3cda16da6c69133c1b2005269 .

1354 External MIME parts are rare today, having a caching facility

1355 for them appears to be unnecessary.

1356 .P

1357 In pre-MIME times,

1358 .Pn mhl

1359 had covered many tasks that are part of MIME handling today.

1360 Therefore,

1361 .Pn mhl

1362 could be simplified to a large extend, reducing the number of its

1363 switches from 21 to 6

1364 .Ci 350ad6d3542a07639213cf2a4fe524e829c1e7b6

1365 .Ci 0e46503be3c855bddaeae3843e1b659279c35d70 .

1366

1367

1368

1369

1370 .U3 "Header Printing

1371 .P

1372 .Pn folder 's

1373 data output is self-explaining enough that

1374 displaying the header line makes little sense.

1375 Hence, the

1376 .Sw -[no]header

1377 switch was removed and headers are never printed

1378 .Ci 601cc73d1fa05ce96faa728f036d6c51b91701c7 .

1379 .P

1380 In

1381 .Pn mhlist ,

1382 the

1383 .Sw -[no]header

1384 switches were removed, as well

1385 .Ci b24f96523aaf60e44e04a3ffb1d22e69a13a602f .

1386 In this case, the headers are printed always because the output

1387 is not self-explaining.

1388 .P

1389 .Pn scan

1390 also had

1391 .Sw -[no]header

1392 switches.

1393 Printing this header had been sensible until the introduction of

1394 format strings made it impossible to display column headings.

1395 Only the folder name and the current date remained to be printed.

1396 As this information can be perfectly generated with

1397 .Pn folder

1398 and

1399 .Pn date ,

1400 the switches were removed

1401 .Ci c477dc5d1d03fa6d9a8ab3dd3508c63cbddc044e .

1402 .P

1403 By removing all

1404 .Sw -header

1405 switches, the collision with

1406 .Sw -help

1407 on the first two letters was resolved.

1408 Currently,

1409 .Sw -h

1410 evaluates to

1411 .Sw -help

1412 for all tools of mmh.

1413

1414

1415 .U3 "Suppressing Edits or the Invocation of the WhatNow Shell

1416 .P

1417 The

1418 .Sw -noedit

1419 switch of

1420 .Pn comp ,

1421 .Pn repl ,

1422 .Pn forw ,

1423 .Pn dist ,

1424 and

1425 .Pn whatnow

1426 was removed and replaced by specifying

1427 .Sw -editor

1428 with an empty argument

1429 .Ci 75fca31a5b9d5c1a99c74ab14c94438d8852fba9 .

1430 (Specifying

1431 .Cl "-editor /bin/true

1432 is nearly the same. It differs only in setting the previous editor.)

1433 .P

1434 The more important change is the removal of the

1435 .Sw -nowhatnowproc

1436 switch

1437 .Ci ee4f43cf2ef0084ec698e4e87159a94c01940622 .

1438 This switch had once introduced an awkward behavior,

1439 as explained in nmh's man page for

1440 .Mp comp (1):

1441 .QS

1442 The

1443 .Sw -editor

1444 .Ar editor

1445 switch indicates the editor to use for

1446 the initial edit. Upon exiting from the editor,

1447 .Pn comp

1448 will invoke the

1449 .Pn whatnow

1450 program. See

1451 .Mp whatnow (1)

1452 for a discussion of available options.

1453 The invocation of this program can be

1454 inhibited by using the

1455 .Sw -nowhatnowproc

1456 switch. (In truth of fact, it is the

1457 .Pn whatnow

1458 program which starts the initial edit.

1459 Hence,

1460 .Sw -nowhatnowproc

1461 will prevent any edit from occurring.)

1462 .QE

1463 .P

1464 Effectively, the

1465 .Sw -nowhatnowproc

1466 switch caused only only a draft message to be created.

1467 As

1468 .Cl "-whatnowproc /bin/true

1469 does the same, the

1470 .Sw -nowhatnowproc

1471 switch was removed for being redundant.

1472

1473

1474

1475 .U3 "Various

1476 .BU

1477 With the removal of MMDF maildrop format support,

1478 .Pn packf

1479 and

1480 .Pn rcvpack

1481 no longer needed their

1482 .Sw -mbox

1483 and

1484 .Sw -mmdf

1485 switches.

1486 The behavior of

1487 .Sw -mbox

1488 is the sole behavior now

1489 .Ci 3916ab66ad5d183705ac12357621ea8661afd3c0 .

1490 Further rework in both tools made the

1491 .Sw -file

1492 switch unnecessary

1493 .Ci ca1023716d4c2ab890696f3e41fa0d94267a940e .

1494

1495 .BU

1496 Mmh's tools do no longer clear the screen (\c

1497 .Pn scan 's

1498 and

1499 .Pn mhl 's

1500 .Sw -[no]clear

1501 switches

1502 .Ci e57b17343dcb3ff373ef4dd089fbe778f0c7c270

1503 .Ci 943765e7ac5693ae177fd8d2b5a2440e53ce816e ).

1504 Neither does

1505 .Pn mhl

1506 ring the bell (\c

1507 .Sw -[no]bell

1508 .Ci e11983f44e59d8de236affa5b0d0d3067c192e24 )

1509 nor does it page the output itself (\c

1510 .Sw -length

1511 .Ci 5b9d883db0318ed2b84bb82dee880d7381f99188 ).

1512 .\" XXX Ref

1513 Generally, the pager to use is no longer specified with the

1514 .Sw -[no]moreproc

1515 command line switches for

1516 .Pn mhl

1517 and

1518 .Pn show /\c

1519 .Pn mhshow

1520 .Ci 39e87a75b5c2d3572ec72e717720b44af291e88a .

1521

1522 .BU

1523 In order to avoid prefix collisions among switch names, the

1524 .Sw -version

1525 switch was renamed to

1526 .Sw -Version

1527 (with capital `V')

1528 .Ci 32b2354dbaf4bf934936eb5b102a4a3d2fdd209a .

1529 Every program has the

1530 .Sw -version

1531 switch but its first three letters collided with the

1532 .Sw -verbose

1533 switch, present in many programs.

1534 The rename solved this problem once for all.

1535 Although this rename breaks a basic interface, having the

1536 .Sw -V

1537 abbreviation to display the version information, isn't all too bad.

1538

1539 .BU

1540 .Sw -[no]preserve

1541 of

1542 .Pn refile

1543 was removed

1544 .Ci 8edc5aaf86f9f77124664f6801bc6c6cdf258173

1545 because what use was it anyway?

1546 Quoting nmh's man page

1547 .Mp refile (1):

1548 .QS

1549 Normally when a message is refiled, for each destination

1550 folder it is assigned the number which is one above the current

1551 highest message number in that folder. Use of the

1552 .Sw -preserv

1553 [sic!] switch will override this message renaming, and try

1554 to preserve the number of the message. If a conflict for a

1555 particular folder occurs when using the

1556 .Sw -preserve

1557 switch, then

1558 .Pn refile

1559 will use the next available message number which

1560 is above the message number you wish to preserve.

1561 .QE

1562

1563 .BU

1564 The removal of the

1565 .Sw -[no]reverse

1566 switches of

1567 .Pn scan

1568 .Ci 8edc5aaf86f9f77124664f6801bc6c6cdf258173

1569 is a bug fix.

1570 This is supported by the comments

1571 ``\-[no]reverse under #ifdef BERK (I really HATE this)''

1572 by Rose and

1573 ``Lists messages in reverse order with the `\-reverse' switch.

1574 This should be considered a bug'' by Romine in the changelogs.

1575 The question remains why neither Rose nor Romine have fixed this

1576 bug in the eighties when they wrote these comments.

1577

1578

1579

1580

1581

1582

1583

1584 .\" --------------------------------------------------------------

1585 .H1 "Modernizing

1586 .P

1587 In the more than thirty years of MH's existence, its code base was

1588 increasingly extended.

1589 New features entered the project and became alternatives to the

1590 existing behavior.

1591 Relics from several decades have gathered in the code base

1592 but seldom obsolete features were dropped.

1593 This section describes the removing of old code

1594 and the modernizing of the default setup.

1595 It focuses on the functional aspect only;

1596 the non-functional aspects of code style are discussed in Sec.

1597 .Cf code-style .

1598

1599

1600 .H2 "Code Relics

1601 .P

1602 My position regarding the removal of obsolete code

1603 is much more revolutional than the nmh community appreciates.

1604 Working on an experimental version, I was able to quickly drop

1605 functionality that I considered ancient.

1606 The need for consensus with peers would have slowed this process down.

1607 Without the need to justify my decisions, I was able to rush forward.

1608 .P

1609 In December 2011, Paul Vixie motivated the nmh developers to just

1610 do the work:

1611 .[

1612 paul vixie edginess nmh-workers

1613 .]

1614 .QS

1615 let's stop walking on egg shells with this code base. there's no need to

1616 discuss whether to keep using vfork, just note in [sic!] passing, [...]

1617 we don't need a separate branch for removing vmh

1618 or ridding ourselves of #ifdef's or removing posix replacement functions

1619 or depending on pure ansi/posix ``libc''.

1620 .QP

1621 these things should each be a day or two of work and the ``main branch''

1622 should just be modern. [...]

1623 let's push forward, aggressively.

1624 .QE

1625 .LP

1626 I did so already in the months before.

1627 I pushed forward.

1628 I simply dropped the cruft.

1629 .P

1630 The decision to drop a feature was based on literature research and

1631 careful thinking, but whether having had contact with this particular

1632 feature within my own computer life served as a rule of thumb.

1633 I explained my reasons in the commit messages

1634 in the version control system.

1635 Hence, others can comprehend my view and argue for undoing the change

1636 if I have missed an important aspect.

1637 I was quick in dropping parts.

1638 I rather include falsely dropped parts again, than going at a slower pace.

1639 Mmh is experimental work; it requires tough decisions.

1640

1641

1642 .U3 "Process Forking

1643 .P

1644 Being a tool chest, MH creates many processes.

1645 In earlier times

1646 .Fu fork()

1647 had been an expensive system call, because the process's image needed

1648 to be completely duplicated at once.

1649 This expensive work was especially unnecessary in the commonly occurring

1650 case wherein the image is replaced by a call to

1651 .Fu exec()

1652 right after having forked the child process.

1653 The

1654 .Fu vfork()

1655 system call was invented to speed up this particular case.

1656 It completely omits the duplication of the image.

1657 On old systems this resulted in significant speed ups.

1658 Therefore MH used

1659 .Fu vfork()

1660 whenever possible.

1661 .P

1662 Modern memory management units support copy-on-write semantics, which make

1663 .Fu fork()

1664 almost as fast as

1665 .Fu vfork() .

1666 The man page of

1667 .Mp vfork (2)

1668 in FreeBSD 8.0 states:

1669 .QS

1670 This system call will be eliminated when proper system sharing mechanisms

1671 are implemented. Users should not depend on the memory sharing semantics

1672 of vfork() as it will, in that case, be made synonymous to fork(2).

1673 .QE

1674 .LP

1675 Vixie supports the removal with the note that ``the last

1676 system on which fork was so slow that an mh user would notice it, was

1677 Eunice. that was 1987''.

1678 .[

1679 nmh-workers vixie edginess

1680 .]

1681 I replaced all calls to

1682 .Fu vfork()

1683 with calls to

1684 .Fu fork()

1685 .Ci 40821f5c1316e9205a08375e7075909cc9968e7d .

1686 .P

1687 Related to the costs of

1688 .Fu fork()

1689 is the probability of its success.

1690 In the eighties, on heavy loaded systems, calls to

1691 .Fu fork()

1692 were prone to failure.

1693 Hence, many of the

1694 .Fu fork()

1695 calls in the code were wrapped into loops to retry the

1696 .Fu fork()

1697 several times, to increase the chances to succeed eventually.

1698 On modern systems, a failing

1699 .Fu fork()

1700 call is unusual.

1701 Hence, in the rare case when

1702 .Fu fork()

1703 fails, mmh programs simply abort

1704 .Ci 5fbf37ee68e018998ada61eeab73e035b26834b6 .

1705

1706

1707 .U3 "Header Fields

1708 .BU

1709 The

1710 .Hd Encrypted

1711 header field was introduced by RFC\|822,

1712 but already marked as legacy in RFC\|2822.

1713 Today, OpenPGP provides the basis for standardized exchange of encrypted

1714 messages [RFC\|4880, RFC\|3156].

1715 Hence, the support for

1716 .Hd Encrypted

1717 header fields is removed in mmh

1718 .Ci 064527f7b57ab050e5af13e15ad99aeeab125857 .

1719 .BU

1720 The native support for

1721 .Hd Face

1722 header fields has been removed, as well

1723 .Ci 8e5be81f784682822f5e868c1bf3c8624682bd23 .

1724 This feature is similar to the

1725 .Hd X-Face

1726 header field in its intent,

1727 but takes a different approach to store the image.

1728 Instead of encoding the image data directly into the header field,

1729 it contains the hostname and UDP port where the image

1730 date can be retrieved.

1731 There is even a third Face system,

1732 which is the successor of

1733 .Hd X-Face ,

1734 although it re-uses the

1735 .Hd Face

1736 header field name.

1737 It was invented in 2005 and supports colored PNG images.

1738 None of the Face systems described here is popular today.

1739 Hence, mmh has no direct support for them.

1740 .BU

1741 .Id content-md5

1742 The

1743 .Hd Content-MD5

1744 header field was introduced by RFC\|1864.

1745 It provides detection of data corruption during the transfer.

1746 But it can not ensure verbatim end-to-end delivery of the contents

1747 [RFC\|1864].

1748 The proper approach to verify content integrity in an

1749 end-to-end relationship is the use of digital signatures.

1750 .\" XXX (RFCs FIXME).

1751 On the other hand, transfer protocols should detect corruption during

1752 the transmission.

1753 The TCP includes a checksum field therefore.

1754 These two approaches in combinations render the

1755 .Hd Content-MD5

1756 header field superfluous.

1757 Not a single one out of 4\|200 messages from two decades

1758 in the nmh-workers mailing list archive

1759 .[

1760 nmh-workers mailing list archive website

1761 .]

1762 contains a

1763 .Hd Content-MD5

1764 header field.

1765 Neither did any of the 60\|000 messages in my personal mail storage.

1766 Removing the support for this header field

1767 .Ci 31dc797eb5178970d68962ca8939da3fd9a8efda ,

1768 removed the last place where MD5 computation was needed.

1769 Hence, the MD5 code could be removed as well.

1770 Over 500 lines of code vanished by this one change.

1771

1772

1773 .U3 "MMDF maildrop support

1774 .P

1775 This type of maildrop format is conceptionally similar to the mbox format,

1776 but uses a different message delimiter (`\fL\\1\\1\\1\\1\fP',

1777 commonly written as `\fL^A^A^A^A\fP', instead of `\fLFrom\0\fP').

1778 Mbox is the de-facto standard maildrop format on Unix,

1779 whereas the MMDF maildrop format is now forgotten.

1780 Mbox remains as the only packed mailbox format, supported in mmh.

1781 .P

1782 The simplifications within the code were moderate.

1783 Mainly, the reading and writing of MMDF mailbox files was removed.

1784 But also, switches of

1785 .Pn packf

1786 and

1787 .Pn rcvpack

1788 could be removed

1789 .Ci 3916ab66ad5d183705ac12357621ea8661afd3c0 .

1790 In the message parsing function

1791 .Fn sbr/m_getfld.c ,

1792 knowledge of MMDF packed mail boxes was removed

1793 .Ci 684ec30d81e1223a282764452f4902ed4ad1c754 .

1794 Further code structure simplifications may be possible there,

1795 because only one single packed mailbox format is left to be supported.

1796 I have not worked on them yet because

1797 .Fu m_getfld()

1798 is heavily optimized and thus dangerous to touch.

1799 The risk of damaging the intricate workings of the optimized code is

1800 too high.

1801

1802

1803 .U3 "Prompter's Control Keys

1804 .P

1805 The program

1806 .Pn prompter

1807 queries the user to fill in a message form.

1808 When used as

1809 .Cl "comp -editor prompter" ,

1810 the resulting behavior is similar to

1811 .Pn mailx .

1812 Apparently,

1813 .Pn prompter

1814 had not been touched lately.

1815 Otherwise it's hardly explainable why it

1816 still offered the switches

1817 .Sw -erase

1818 .Ar chr

1819 and

1820 .Sw -kill

1821 .Ar chr

1822 to name the characters for command line editing.

1823 The times when this had been necessary are long time gone.

1824 Today these things work out-of-the-box, and if not, are configured

1825 with the standard tool

1826 .Pn stty .

1827 The switches are removed now

1828 .Ci 0bd9750710cdbab80cfb4036dd87af20afe1552f .

1829

1830

1831 .U3 "Hardcopy Terminal Support

1832 .P

1833 More of a funny anecdote is a check for being connected to a

1834 hardcopy terminal.

1835 It remained in the code until spring 2012, when I finally removed it

1836 .Ci b7764c4a6b71d37918a97594d866258f154017ca .

1837 .P

1838 The check only prevented a pager to be placed between the printing

1839 program (\c

1840 .Pn mhl )

1841 and the terminal.

1842 In nmh, this could have been ensured statically with the

1843 .Sw -nomoreproc

1844 at the command line, too.

1845 In mmh, setting the profile entry

1846 .Pe Pager

1847 or the environment variable

1848 .Ev PAGER

1849 to

1850 .Pn cat

1851 is sufficient.

1852

1853

1854

1855

1856 .H2 "Attachments

1857 .P

1858 The mind model of email attachments is unrelated to MIME.

1859 Although the MIME RFCs [RFC\|2045\(en2049] define the technical

1860 requirements for having attachments, they do not mention the term.

1861 Instead of attachments, MIME talks about ``multi-part message bodies''

1862 [RFC\|2045], a more general concept.

1863 Multi-part messages are messages

1864 ``in which one or more different

1865 sets of data are combined in a single body''

1866 [RFC\|2046].

1867 MIME keeps its descriptions generic;

1868 it does not imply specific usage models.

1869 Today, one usage model is prevalent: attachments.

1870 The idea is having a main text document with files of arbitrary kind

1871 attached to it.

1872 In MIME terms, this is a multi-part message having a text part first

1873 and parts of arbitrary type following.

1874 .P

1875 MH's MIME support is a direct implementation of the RFCs.

1876 The perception of the topic described in the RFCs is clearly visible

1877 in MH's implementation.

1878 As a result,

1879 MH had all the MIME features but no idea of attachments.

1880 But users do not need all the MIME features,

1881 they want convenient attachment handling.

1882

1883

1884 .U3 "Composing MIME Messages

1885 .P

1886 In order to improve the situation on the message composing side,

1887 Jon Steinhart had added an attachment system to nmh in 2002

1888 .Ci 7480dbc14bc90f2d872d434205c0784704213252 .

1889 In the file

1890 .Fn docs/README-ATTACHMENTS ,

1891 he described his motivation to do so:

1892 .QS

1893 Although nmh contains the necessary functionality for MIME message

1894 handing [sic!], the interface to this functionality is pretty obtuse.

1895 There's no way that I'm ever going to convince my partner to write

1896 .Pn mhbuild

1897 composition files!

1898 .QE

1899 .LP

1900 With this change, the mind model of attachments entered nmh.

1901 In the same document:

1902 .QS

1903 These changes simplify the task of managing attachments on draft files.

1904 They allow attachments to be added, listed, and deleted.

1905 MIME messages are automatically created when drafts with attachments

1906 are sent.

1907 .QE

1908 .LP

1909 Unfortunately, the attachment system, like every new facilities in nmh,

1910 was inactive by default.

1911 .P

1912 During my time in Argentina, I tried to improve the attachment system.

1913 But, after long discussions my patch died as a proposal on the

1914 mailing list because of great opposition in the nmh community.

1915 .[

1916 nmh-workers attachment proposal

1917 .]

1918 In January 2012, I extended the patch and applied it to mmh

1919 .Ci 8ff284ff9167eff8f5349481529332d59ed913b1 .

1920 In mmh, the attachment system is active by default.

1921 Instead of command line switches, the

1922 .Pe Attachment-Header

1923 profile entry is used to specify

1924 the name of the attachment header field.

1925 It is pre-defined to

1926 .Hd Attach .

1927 .P

1928 To add an attachment to a draft, a header line needs to be added:

1929 .VS

1930 To: bob

1931 Subject: The file you wanted

1932 Attach: /path/to/the/file-bob-wanted

1933 --------

1934 Here it is.

1935 VE

1936 The header field can be added to the draft manually in the editor,

1937 or by using the `attach' command at the WhatNow prompt, or

1938 non-interactively with

1939 .Pn anno :

1940 .VS

1941 anno -append -nodate -component Attach -text /path/to/attachment

1942 VE

1943 Drafts with attachment headers are converted to MIME automatically by

1944 .Pn send .

1945 The conversion to MIME is invisible to the user.

1946 The draft stored in the draft folder is always in source form with

1947 attachment headers.

1948 If the MIMEification fails (e.g. because the file to attach

1949 is not accessible) the original draft is not changed.

1950 .P

1951 The attachment system handles the forwarding of messages, too.

1952 If the attachment header value starts with a plus character (`\fL+\fP'),

1953 like in

1954 .Cl "Attach: +bob 30 42" ,

1955 the given messages in the specified folder will be attached.

1956 This allowed to simplify

1957 .Pn forw

1958 .Ci f41f04cf4ceca7355232cf7413e59afafccc9550 .

1959 .P

1960 Closely related to attachments is non-ASCII text content,

1961 because it requires MIME as well.

1962 In nmh, the user needed to call `mime' at the WhatNow prompt

1963 to have the draft converted to MIME.

1964 This was necessary whenever the draft contained non-ASCII characters.

1965 If the user did not call `mime', a broken message would be sent.

1966 Therefore, the

1967 .Pe automimeproc

1968 profile entry could be specified to have the `mime' command invoked

1969 automatically each time.

1970 Unfortunately, this approach conflicted with the attachment system

1971 because the draft would already be in MIME format at the time

1972 when the attachment system wanted to MIMEify it.

1973 To use nmh's attachment system, `mime' must not be called at the

1974 WhatNow prompt and

1975 .Pe automimeproc

1976 must not be set in the profile.

1977 But then the case of non-ASCII text without attachment headers was

1978 not caught.

1979 All in all, the solution was complex and irritating.

1980 My patch from December 2010

1981 .[

1982 nmh-workers attachment proposal

1983 .]

1984 would have simplified the situation.

1985 .P

1986 Mmh's current solution is even more elaborate.

1987 Any necessary MIMEification is done automatically.

1988 There is no `mime' command at the WhatNow prompt anymore.

1989 The draft will be converted automatically to MIME when either an

1990 attachment header or non-ASCII text is present.

1991 Furthermore, the hash character (`\fL#\fP') is not special any more

1992 at line beginnings in the draft message.

1993 .\" XXX REF ?

1994 Users need not concern themselves with the whole topic at all.

1995 .P

1996 Although the new approach does not anymore support arbitrary MIME

1997 compositions directly, the full power of

1998 .Pn mhbuild

1999 can still be accessed.

2000 Given no attachment headers are included, users can create

2001 .Pn mhbuild

2002 composition drafts like in nmh.

2003 Then, at the WhatNow prompt, they can invoke

2004 .Cl "edit mhbuild

2005 to convert the draft to MIME.

2006 Because the resulting draft neither contains non-ASCII characters

2007 nor has it attachment headers, the attachment system will not touch it.

2008 .P

2009 The approach taken in mmh is tailored towards today's most common case:

2010 a text part, possibly with attachments.

2011 This case was simplified.

2012

2013

2014 .U3 "MIME Type Guessing

2015 .P

2016 From the programmer's point of view, the use of

2017 .Pn mhbuild

2018 composition drafts had one notable advantage over attachment headers:

2019 The user provides the appropriate MIME types for files to include.

2020 The new attachment system needs to find out the correct MIME type itself.

2021 This is a difficult task.

2022 Determining the correct MIME type of content is partly mechanical,

2023 partly intelligent work.

2024 Forcing the user to find out the correct MIME type,

2025 forces him to do partly mechanical work.

2026 Letting the computer do the work can lead to bad choices for difficult

2027 content.

2028 For mmh, the latter option was chosen to spare the user the work

2029 .Ci 3baec236a39c5c89a9bda8dbd988d643a21decc6 .

2030 .P

2031 Determining the MIME type by the suffix of the file name is a dumb

2032 approach, yet it is simple to implement and provides good results

2033 for the common cases.

2034 If no MIME type can be determined, text content is sent as `text/plain',

2035 anything else under the generic fall-back type `application/octet-stream'.

2036 Mmh implements this approach in the

2037 .Pn print-mimetype

2038 script

2039 .Ci 4b5944268ea0da7bb30598a27857304758ea9b44 .

2040 .P

2041 A far better, though less portable, approach is the use of

2042 .Pn file .

2043 This standard tool tries to determine the type of files.

2044 Unfortunately, its capabilities and accuracy varies from system to system.

2045 Additionally, its output was only intended for human beings,

2046 but not to be used by programs.

2047 Nevertheless, modern versions of GNU

2048 .Pn file ,

2049 which are prevalent on the popular GNU/Linux systems,

2050 provide MIME type output in machine-readable form.

2051 Although this solution is system-dependent,

2052 it solves the difficult problem well.

2053 On systems where GNU

2054 .Pn file ,

2055 version 5.04 or higher, is available it should be used.

2056 One needs to specify the following profile entry to do so:

2057 .VS

2058 Mime-Type-Query: file -b --mime

2059 VE

2060 .LP

2061 Other versions of

2062 .Pn file

2063 might possibly be usable with wrapper scripts that reformat the output.

2064 The diversity among

2065 .Pn file

2066 implementations is great; one needs to check the local variant.

2067 .P

2068 It is not possible in mmh to override the automatic MIME type guessing

2069 for a specific file.

2070 To do so, either the user would need to know in advance for which file

2071 the automatic guessing fails or the system would require interaction.

2072 I consider both cases impractical.

2073 The existing solution should be sufficient.

2074 If not, the user may always fall back to

2075 .Pn mhbuild

2076 composition drafts and bypass the attachment system.

2077

2078

2079 .U3 "Storing Attachments

2080 .P

2081 Extracting MIME parts of a message and storing them to disk is performed by

2082 .Pn mhstore .

2083 The program has two operation modes,

2084 .Sw -auto

2085 and

2086 .Sw -noauto .

2087 With the former one, each part is stored under the filename given in the

2088 MIME part's meta information, if available.

2089 This naming information is usually available for modern attachments.

2090 If no filename is available, this MIME part is stored as if

2091 .Sw -noauto

2092 would have been specified.

2093 In the

2094 .Sw -noauto

2095 mode, the parts are processed according to rules, defined by

2096 .Pe mhstore-store-*

2097 profile entries.

2098 These rules define generic filename templates for storing

2099 or commands to post-process the contents in arbitrary ways.

2100 If no matching rule is available the part is stored under a generic

2101 filename, built from message number, MIME part number, and MIME type.

2102 .P

2103 The

2104 .Sw -noauto

2105 mode had been the default in nmh because it was considered safe,

2106 in contrast to the

2107 .Sw -auto

2108 mode.

2109 In mmh,

2110 .Sw -auto

2111 is not dangerous anymore.

2112 Two changes were necessary:

2113 .LI 1

2114 Any directory path is removed from the proposed filename.

2115 Thus, the files are always stored in the expected directory.

2116 .Ci 41b6eadbcecf63c9a66aa5e582011987494abefb

2117 .LI 2

2118 Tar files are not extracted automatically any more.

2119 Thus, the rest of the file system will not be touched.

2120 .Ci 94c80042eae3383c812d9552089953f9846b1bb6

2121 .P

2122 In mmh, the result of

2123 .Cl "mhstore -auto

2124 can be foreseen from the output of

2125 .Cl "mhlist -verbose" .

2126 Although the

2127 .Sw -noauto

2128 mode is considered to be more powerful, it is less convenient and

2129 .Sw -auto

2130 is safe now.

2131 Additionally, storing attachments under their original name

2132 is intuitive.

2133 Hence,

2134 .Sw -auto

2135 serves better as the default option

2136 .Ci 3410b680416c49a7617491af38bc1929855a331d .

2137 .P

2138 Files are stored into the directory given by the

2139 .Pe Nmh-Storage

2140 profile entry, if set, or

2141 into the current working directory, otherwise.

2142 Storing to different directories is only possible with

2143 .Pe mhstore-store-*

2144 profile entries.

2145 .P

2146 Still existing files get overwritten silently in both modes.

2147 This can be considered a bug.

2148 Yet, each other behavior has its draw-backs, too.

2149 Refusing to replace files requires adding a

2150 .Sw -force

2151 switch.

2152 Users will likely need to invoke

2153 .Pn mhstore

2154 a second time with

2155 .Sw -force .

2156 Eventually, only the user can decide in the specific case.

2157 This requires interaction, which I like to avoid if possible.

2158 Appending a unique suffix to the filename is another bad option.

2159 For now, the behavior remains as it is.

2160 .P

2161 In mmh, only MIME parts of type message are special in

2162 .Pn mhstore 's

2163 .Sw -auto

2164 mode.

2165 Instead of storing message/rfc822 parts as files to disk,

2166 they are stored as messages into the current mail folder.

2167 The same applies to message/partial, although the parts are

2168 automatically reassembled beforehand.

2169 MIME parts of type message/external-body are not automatically retrieved

2170 anymore.

2171 Instead, information on how to retrieve them is output.

2172 Not supporting this rare case saved nearly one thousand lines of code

2173 .Ci 55e1d8c654ee0f7c45b9361ce34617983b454c32 .

2174 The MIME type `application/octet-stream; type=tar' is not special anymore.

2175 The automatically extracting of such MIME parts had been the

2176 dangerous part of the

2177 .Sw -auto

2178 mode

2179 .Ci 94c80042eae3383c812d9552089953f9846b1bb6 .

2180

2181

2182

2183 .U3 "Showing MIME Messages

2184 .P

2185 The program

2186 .Pn mhshow

2187 was written to display MIME messages.

2188 It implemented the conceptional view of the MIME RFCs.

2189 Nmh's

2190 .Pn mhshow

2191 handles each MIME part independently, presenting them separately

2192 to the user.

2193 This does not match today's understanding of email attachments,

2194 where displaying a message is seen to be a single, integrated operation.

2195 Today, email messages are expected to consist of a main text part

2196 plus possibly attachments.

2197 They are no more seen to be arbitrary MIME hierarchies with

2198 information on how to display the individual parts.

2199 I adjusted

2200 .Pn mhshow 's

2201 behavior to the modern view on the topic.

2202 .P

2203 One should note that this section completely ignores the original

2204 .Pn show

2205 program, because it was not capable to display MIME messages

2206 and is no longer part of mmh.

2207 .\" XXX ref to other section

2208 Although

2209 .Pn mhshow

2210 was renamed to

2211 .Pn show

2212 in mmh, this section uses the name

2213 .Pn mhshow ,

2214 in order to avoid confusion.

2215 .P

2216 In mmh, the basic idea is that

2217 .Pn mhshow

2218 should display a message in one single pager session.

2219 Therefore,

2220 .Pn mhshow

2221 invokes a pager session for all its output,

2222 whenever it prints to a terminal

2223 .Ci a4197ea6ffc5c1550e8b52d5a654bcaaaee04a4e .

2224 In consequence,

2225 .Pn mhl

2226 does no more invoke a pager

2227 .Ci 0e46503be3c855bddaeae3843e1b659279c35d70 .

2228 With

2229 .Pn mhshow

2230 replacing the original

2231 .Pn show ,

2232 the output of

2233 .Pn mhl

2234 no longer goes to the terminal directly, but through

2235 .Pn mhshow .

2236 Hence,

2237 .Pn mhl

2238 does not need to invoke a pager.

2239 The one and only job of

2240 .Pn mhl

2241 is to format messages or parts of them.

2242 The only place in mmh, where a pager is invoked is

2243 .Pn mhshow .

2244 .P

2245 In the intended setup, only text content is be displayed,

2246 in a single pager session.

2247 Non-text content needs to be converted to text by appropriate

2248 .Pe mhshow-show-*

2249 profile entries before, if this is possible and wanted.

2250 A common example for this are PDF files.

2251 In mmh, MIME parts are always displayed serially.

2252 The request to display the MIME type `multipart/parallel' in parallel

2253 is ignored.

2254 It is simply treated as `multipart/mixed'

2255 .Ci d0581ba306a7299113a346f9b4c46ce97bc4cef6 .

2256 This was already possible to requested with the, now removed,

2257 .Sw -serialonly

2258 switch of

2259 .Pn mhshow .

2260 As MIME parts are always processed exclusively, i.e. serially,

2261 the `\fL%e\fP' escape in

2262 .Pe mhshow-show-*

2263 profile entries became useless and was thus removed

2264 .Ci a20d405db09b7ccca74d3e8c57550883da49e1ae .

2265 .P

2266 Other kinds of attachments are ignored.

2267 With

2268 .Pe mhshow-show-*

2269 profile entries for them, they can be displayed serially along

2270 the message.

2271 For parallel display, the attachments need to be stored to disk first.

2272 .P

2273 To display text content in foreign charsets, they need to be converted

2274 to the native charset.

2275 Therefore,

2276 .Pe mhshow-charset-*

2277 profile entries were needed.

2278 In mmh, the conversion is performed automatically by piping the

2279 text through the

2280 .Pn iconv

2281 command, if necessary

2282 .Ci 2433122c20baccb10b70b49c04c6b0497b5b3b60 .

2283 Custom

2284 .Pe mhshow-show-*

2285 rules for textual content might need a

2286 .Cl "iconv -f %c %f |

2287 prefix to have the text converted to the native charset.

2288 .P

2289 Although the conversion of foreign charsets to the native one

2290 has improved, it is not consistent enough.

2291 Further work needs to be done and

2292 the basic concepts in this field need to be re-thought.

2293 Though, the default setup of mmh displays message in foreign charsets

2294 correctly without the need to configure anything.

2295

2296

2297 .ig

2298

2299 .P

2300 mhshow/mhstore: Removed support for retrieving message/external-body parts.

2301 These tools will not download the contents automatically anymore. Instead,

2302 they print the information needed to get the contents. If someone should

2303 really receive one of those rare message/external-body messages, he can

2304 do the job manually. We save nearly a thousand lines of code. That's worth

2305 it!

2306 (The profile entry `nmh-access-ftp' and sbr/ruserpass.c for reading

2307 ~/.netrc are gone now.)

2308 .Ci 55e1d8c654ee0f7c45b9361ce34617983b454c32

2309

2310 ..

2311

2312

2313

2314 .H2 "Signing and Encrypting

2315 .P

2316 Nmh offers no direct support for digital signatures and message encryption.

2317 This functionality needed to be added through third-party software.

2318 In mmh, the functionality is included because it

2319 is a part of modern email and is likely wanted by users of mmh.

2320 A fresh mmh installation supports signing and encrypting

2321 out-of-the-box.

2322 Therefore, Neil Rickert's

2323 .Pn mhsign

2324 and

2325 .Pn mhpgp

2326 scripts

2327 .[

2328 neil rickert mhsign mhpgp

2329 .]

2330 were included

2331 .Ci f45cdc98117a84f071759462c7ae212f4bc5ab2e

2332 .Ci 58cf09aa36e9f7f352a127158bbf1c5678bc6ed8 .

2333 The scripts fit well because they are lightweight and

2334 similar of style to the existing tools.

2335 Additionally, no licensing difficulties appeared

2336 as they are part of the public domain.

2337 .P

2338 .Pn mhsign

2339 handles the signing and encrypting part.

2340 It comprises about 250 lines of shell code and interfaces between

2341 .Pn gnupg

2342 and the MH system.

2343 It was meant to be invoked manually at the WhatNow prompt, but in mmh,

2344 .Pn send

2345 invokes

2346 .Pn mhsign

2347 automatically

2348 .Ci c7b5e1df086bcc37ff40163ee67571f076cf6683 .

2349 Special header fields were introduced to request this action.

2350 If a draft contains the

2351 .Hd Sign

2352 header field,

2353 .Pn send

2354 will initiate the signing.

2355 The signing key is either chosen automatically or it is specified by the

2356 .Pe Pgpkey

2357 profile entry.

2358 .Pn send

2359 always create signatures using the PGP/MIME standard [RFC\|4880],

2360 but by invoking

2361 .Pn mhsign

2362 manually, old-style non-MIME signatures can be created as well.

2363 To encrypt an outgoing message, the draft needs to contain an

2364 .Hd Enc

2365 header field.

2366 Public keys of all recipients are searched for in the gnupg keyring and

2367 in a file called

2368 .Fn pgpkeys ,

2369 which contains exceptions and overrides.

2370 Unless public keys are found for all recipients,

2371 .Pn mhsign

2372 will refuse to encrypt it.

2373 Currently, messages with hidden (BCC) recipients can not be encrypted.

2374 This work is pending because it requires a structurally more complex

2375 approach.

2376 .P

2377 .Pn mhpgp

2378 is the companion to

2379 .Pn mhsign .

2380 It verifies signatures and decrypts messages.

2381 Encrypted messages can be either temporarily decrypted and displayed

2382 or permanently decrypted and stored into the current folder.

2383 Currently,

2384 .Pn mhpgp

2385 needs to be invoked manually.

2386 The integration into

2387 .Pn show

2388 and

2389 .Pn mhstore

2390 to verify signatures and decrypt messages as needed

2391 is planned but not yet realized.

2392 .P

2393 Both scripts were written for nmh.

2394 Hence they needed to be adjust

2395 according to the differences between nmh and mmh.

2396 For instance, they use the backup prefix no longer.

2397 Furthermore, compatibility support for old PGP features was dropped.

2398 .P

2399 The integrated message signing and encrypting support is one of the

2400 most recent features in mmh.

2401 It has not had the time to mature.

2402 User feedback and personal experience need to be accumulated to

2403 direct the further development of the facility.

2404 Already it seems to be worthwhile to consider adding

2405 .Sw -[no]sign

2406 and

2407 .Sw -[no]enc

2408 switches to

2409 .Pn send ,

2410 to be able to override the corresponding header fields.

2411 A profile entry:

2412 .VS

2413 send: -sign

2414 VE

2415 would then activate signing for all outgoing messages.

2416 With the present approach, a

2417 .Hd Send

2418 header component needs to be added to each draft template

2419 to achieve the same result.

2420 Adding the switches would ease the work greatly and keep the

2421 template files clean.

2422

2423

2424

2425

2426 .H2 "Draft and Trash Folder

2427 .P

2428

2429 .U3 "Draft Folder

2430 .Id draft-folder

2431 .P

2432 In the beginning, MH had the concept of a draft message.

2433 This was a file named

2434 .Fn draft

2435 in the MH directory, which was treated special.

2436 On composing a message, this draft file was used.

2437 When starting to compose another message before the former one was sent,

2438 the user had to decide among:

2439 .LI 1

2440 Using the old draft to finish and send it before starting with a new one.

2441 .LI 2

2442 Discarding the old draft and replacing it with a new one.

2443 .LI 3

2444 Preserving the old draft by refiling it to a folder.

2445 .LP

2446 Working on multiple drafts was only possible in alternation.

2447 For that, the current draft needed to be refiled to a folder and

2448 another one re-used for editing.

2449 Working on multiple drafts at the same time was impossible.

2450 The usual approach of switching to a different MH context did not

2451 help anything.

2452 .P

2453 The draft folder facility exists to

2454 allow true parallel editing of drafts, in a straight forward way.

2455 It was introduced by Marshall T. Rose, already in 1984.

2456 Similar to other new features, the draft folder was inactive by default.

2457 Even in nmh, the highly useful draft folder was not available

2458 out-of-the-box.

2459 At least, Richard Coleman added the man page

2460 .Mp mh-draft (5)

2461 to better document the feature.

2462 .P

2463 Not using the draft folder facility has the single advantage of having

2464 the draft file at a static location.

2465 This is simple in simple cases but the concept does not scale for more

2466 complex cases.

2467 The concept of the draft message is too limited for the problem

2468 it tries to solve.

2469 Therefore the draft folder was introduced.

2470 It is the more powerful and more natural concept.

2471 The draft folder is a folder like any other folder in MH.

2472 Its messages can be listed like any other messages.

2473 A draft message is no longer a special case.

2474 Tools do not need special switches to work on the draft message.

2475 Hence corner cases were removed.

2476 .P

2477 The trivial part of the work was activating the draft folder with a

2478 default name.

2479 I chose the name

2480 .Fn +drafts ,

2481 for obvious reasons.

2482 In consequence, the command line switches

2483 .Sw -draftfolder

2484 and

2485 .Sw -draftmessage

2486 could be removed.

2487 More difficult but also more improving was updating the tools to the

2488 new concept.

2489 For nearly three decades, the tools needed to support two draft handling

2490 approaches.

2491 By fully switching to the draft folder, the tools could be

2492 simplified by dropping the awkward draft message handling code.

2493 .Sw -draft

2494 switches were removed because operating on a draft message is no longer

2495 special.

2496 It became indistinguishable to operating on any other message.

2497 .Ci 337338b404931f06f0db2119c9e145e8ca5a9860

2498 .P

2499 There is no more need to query the user for draft handling

2500 .Ci 2d48b455c303a807041c35e4248955f8bec59eeb .

2501 It is always possible to add another new draft.

2502 Refiling drafts is without difference to refiling other messages.

2503 All of these special cases are gone.

2504 Yet, one draft-related switch remained.

2505 .Pn comp

2506 still has

2507 .Sw -[no]use

2508 for switching between two modes:

2509 .LI 1

2510 Modifying an existing draft, with

2511 .Sw -use .

2512 .LI 2

2513 Composing a new draft, possibly taking some existing message as template,

2514 with

2515 .Sw -nouse ,

2516 the default.

2517 .LP

2518 In either case, the behavior of

2519 .Pn comp

2520 is deterministic.

2521 .P

2522 .Pn send

2523 now operates on the current message in the draft folder by default.

2524 As message and folder can both be overridden by specifying them on

2525 the command line, it is possible to send any message in the mail storage

2526 by simply specifying its number and folder.

2527 In contrast to the other tools,

2528 .Pn send

2529 takes the draft folder as its default folder.

2530 .P

2531 Dropping the draft message concept in favor for the draft folder concept,

2532 replaced special cases with regular cases.

2533 This simplified the source code of the tools, as well as the concepts.

2534 In mmh, draft management does not break with the MH concepts

2535 but applies them.

2536 .Cl "scan +drafts" ,

2537 for instance, is a truly natural request.

2538 .P

2539 Most of the work was already performed by Rose in the eighties.

2540 The original improvement of mmh is dropping the old draft message approach

2541 and thus simplifying the tools, the documentation,

2542 and the system as a whole.

2543 Although my part in the draft handling improvement was small,

2544 it was important.

2545

2546

2547 .U3 "Trash Folder

2548 .Id trash-folder

2549 .P

2550 Similar to the situation for drafts is the situation for removed messages.

2551 Historically, a message was ``deleted'' by prepending a specific

2552 \fIbackup prefix\fP, usually the comma character,

2553 to the file name.

2554 The specific file would then be ignored by MH because only files with

2555 names consisting of digits only are treated as messages.

2556 Although files remained in the file system,

2557 the messages were no longer visible in MH.

2558 To truly delete them, a maintenance job was needed.

2559 Usually a cron job was installed to delete them after a grace time.

2560 For instance:

2561 .VS

2562 find $HOME/Mail -type f -name ',*' -ctime +7 -delete

2563 VE

2564 In such a setup, the original message could be restored

2565 within the grace time interval by stripping the

2566 backup prefix from the file name \(en usually but not always.

2567 If the last message of a folder with six messages (\fL1-6\fP) was removed,

2568 message

2569 .Fn 6 ,

2570 became file

2571 .Fn ,6 .

2572 If then a new message entered the same folder, it would be named with

2573 the number one above the highest existing message number.

2574 In this case the message would be named

2575 .Fn 6 ,

2576 reusing the number.

2577 If this new message would be removed as well,

2578 then the backup of the former message becomes overwritten.

2579 Hence, the ability to restore removed messages did not only depend on

2580 the sweeping cron job but also on the removing of further messages.

2581 It is undesirable to have such obscure and complex mechanisms.

2582 The user should be given a small set of clear assertions, such as

2583 ``Removed files are restorable within a seven-day grace time.''

2584 With the addition ``... unless a message with the same name in the

2585 same folder is removed before.'' the statement becomes complex.

2586 A user will hardly be able to keep track of all removals to know

2587 if the assertion still holds true for a specific file.

2588 In practice, the real mechanism is unclear to the user.

2589 .P

2590 Furthermore, the backup files were scattered within the whole mail storage.

2591 This complicated managing them.

2592 It was possible with the help of

2593 .Pn find ,

2594 but everything is more convenient

2595 if the deleted messages are collected in one place.

2596 .P

2597 The profile entry

2598 .Pe rmmproc

2599 (previously named

2600 .Pe Delete-Prog )

2601 was introduced very early to improve the situation.

2602 It could be set to any command, which would be executed to remove

2603 the specified messages.

2604 This had overridden the default action, described above.

2605 Refiling the to-be-removed files to a trash folder was the usual example.

2606 Nmh's man page

2607 .Mp rmm (1)

2608 proposes to set the

2609 .Pe rmmproc

2610 to

2611 .Cl "refile +d

2612 to move messages to the trash folder

2613 .Fn +d

2614 instead of renaming them with the backup prefix.

2615 The man page additionally proposes the expunge command

2616 .Cl "rm `mhpath +d all`

2617 to empty the trash folder.

2618 .P

2619 Removing messages in such a way has advantages:

2620 .LI 1

2621 The mail storage is prevented from being cluttered with removed messages

2622 because they are all collected in one place.

2623 Existing and removed messages are thus separated more strictly.

2624 .LI 2

2625 No backup files are silently overwritten.

2626 .LI 3

2627 Most important, however, removed messages are kept in the MH domain.

2628 Messages in the trash folder can be listed like those in any other folder.

2629 Deleted messages can be displayed like any other messages.

2630 .Pn refile

2631 can restore deleted messages.

2632 All operations on deleted files are still covered by the MH tools.

2633 The trash folder is just like any other folder in the mail storage.

2634 .P

2635 Similar to the draft folder case, I dropped the old backup prefix approach

2636 in favor for replacing it by the better suiting trash folder system.

2637 Hence,

2638 .Pn rmm

2639 calls

2640 .Pn refile

2641 to move the to-be-removed message to the trash folder,

2642 .Fn +trash

2643 by default.

2644 To sweep it clean, the user can use

2645 .Cl "rmm -unlink +trash a" ,

2646 where the

2647 .Sw -unlink

2648 switch causes the files to be unlinked.

2649 .Ci 8edc5aaf86f9f77124664f6801bc6c6cdf258173

2650 .Ci ca0b3e830b86700d9e5e31b1784de2bdcaf58fc5

2651 .P

2652 Dropping the legacy approach and converting to the new approach

2653 completely, simplified the code base.

2654 The relationship between

2655 .Pn rmm

2656 and

2657 .Pn refile

2658 was inverted.

2659 In mmh,

2660 .Pn rmm

2661 invokes

2662 .Pn refile .

2663 That used to be the other way round.

2664 Yet, the relationship is simpler now.

2665 Loops, like described in nmh's man page for

2666 .Mp refile (1),

2667 can no longer occur:

2668 .QS

2669 Since

2670 .Pn refile

2671 uses your

2672 .Pe rmmproc

2673 to delete the message, the

2674 .Pe rmmproc

2675 must NOT call

2676 .Pn refile

2677 without specifying

2678 .Sw -normmproc

2679 or you will create an infinite loop.

2680 .QE

2681 .LP

2682 .Pn rmm

2683 either unlinks a message with

2684 .Fu unlink()

2685 or invokes

2686 .Pn refile

2687 to move it to the trash folder.

2688 .Pn refile

2689 does not invoke any tools.

2690 .P

2691 By generalizing the message removal in the way that it became covered

2692 by the MH concepts made the whole system more powerful.

2693

2694

2695

2696

2697

2698 .H2 "Modern Defaults

2699 .P

2700 Nmh has a bunch of convenience-improving features inactive by default,

2701 although one can expect every new user to want them active.

2702 The reason they are inactive by default is the wish to stay compatible

2703 with old versions.

2704 But what are old versions?

2705 Still, the highly useful draft folder facility has not been activated

2706 by default although it was introduced over twenty-five years ago.

2707 .[

2708 rose romine real work

2709 .]

2710 The community seems not to care.

2711 .P

2712 In nmh, new users are required to first build up

2713 a profile before they can access the modern features.

2714 Without an extensive profile, the setup is hardly usable

2715 for modern emailing.

2716 The point is not the customization of the setup,

2717 but the need to activate generally useful facilities.

2718 Yet, the real problem lies less in enabling the features,

2719 as this is straight forward as soon as one knows what he wants.

2720 The real problem is that new users need deep insight into the project

2721 to discover the available but inactive features.

2722 To give an example, I needed one year of using nmh

2723 before I became aware of the existence of the attachment system.

2724 One could argue that this fact disqualifies my reading of the

2725 documentation.

2726 If I would have installed nmh from source back then, I could agree.

2727 Yet, I had used a pre-packaged version and had expected that it would

2728 just work.

2729 Nevertheless, I had been convinced by the concepts of MH already

2730 and I am a software developer,

2731 still I required a lot of time to discover the cool features.

2732 How can we expect users to be even more advanced than me,

2733 just to enable them to use MH in a convenient and modern way?

2734 Unless they are strongly convinced of the concepts, they will fail.

2735 I have seen friends of me giving up disappointed

2736 before they truly used the system,

2737 although they had been motivated in the beginning.

2738 New users suffer hard enough to get used to the tool chest approach,

2739 we developers should spare them further inconveniences.

2740 .P

2741 Maintaining compatibility for its own sake is bad,

2742 because the code base will collect more and more compatibility code.

2743 Sticking to the compatibility code means remaining limited;

2744 whereas adjusting to the changes renders the compatibility unnecessary.

2745 Keeping unused alternatives in the code for longer than a short

2746 grace time is a bad choice as they likely

2747 gather bugs by not being constantly tested.

2748 Also, the increased code size and the greater number of conditions

2749 increase the maintenance costs.

2750 If any MH implementation would be the back-end of widespread

2751 email clients with large user bases, compatibility would be more

2752 important.

2753 Yet, it appears as if this is not the case.

2754 Hence, compatibility is hardly important for technical reasons.

2755 Its importance originates from personal reasons rather.

2756 Nmh's user base is small and old.

2757 Changing the interfaces causes inconvenience to long-term users of MH.

2758 It forces them to change their many years old MH configurations.

2759 I do understand this aspect, but by sticking to the old users,

2760 new users are kept from entering the world of MH.

2761 But the future lies in new users.

2762 In consequence, mmh invites new users by providing a convenient

2763 and modern setup, readily usable out-of-the-box.

2764 .P

2765 In mmh, all modern features are active by default and many previous

2766 approaches are removed or only accessible in a manual way.

2767 New default features include:

2768 .BU

2769 The attachment system (\c

2770 .Hd Attach )

2771 .Ci 8ff284ff9167eff8f5349481529332d59ed913b1 .

2772 .BU

2773 The draft folder facility (\c

2774 .Fn +drafts )

2775 .Ci 337338b404931f06f0db2119c9e145e8ca5a9860 .

2776 .BU

2777 The unseen sequence (`u')

2778 .Ci c2360569e1d8d3678e294eb7c1354cb8bf7501c1

2779 and the sequence negation prefix (`!')

2780 .Ci db74c2bd004b2dc9bf8086a6d8bf773ac051f3cc .

2781 .BU

2782 Quoting the original message in the reply

2783 .Ci 67411b1f95d6ec987b4c732459e1ba8a8ac192c6 .

2784 .BU

2785 Forwarding messages using MIME

2786 .Ci 6e271608b7b9c23771523f88d23a4d3593010cf1 .

2787 .LP

2788 An mmh setup with a profile that defines only the path to the

2789 mail storage, is already convenient to use.

2790 Again, Paul Vixie's supports the direction I took:

2791 ``the `main branch' should just be modern''.

2792 .[

2793 paul vixie edginess nmh-workers

2794 .]

2795

2796

2797

2798

2799

2800 .\" --------------------------------------------------------------

2801 .H1 "Styling

2802 .P

2803 Kernighan and Pike have emphasized the importance of style in the

2804 preface of \fPThe Practice of Programming\fP:

2805 .[ [

2806 kernighan pike practice of programming

2807 .], p. x]

2808 .QS

2809 Chapter 1 discusses programming style.

2810 Good style is so important to good programming that we have chosen

2811 to cover it first.

2812 .QE

2813 This section covers changes in mmh that were guided by the desire

2814 to improve on style.

2815 Many of them follow the advice given in the quoted book.

2816

2817

2818

2819

2820 .H2 "Code Style

2821 .Id code-style

2822 .P

2823 .U3 "Indentation Style

2824 .P

2825 Indentation styles are the holy cow of programming.

2826 Kernighan and Pike write:

2827 .[ [

2828 kernighan pike practice of programming

2829 .], p. 10]

2830 .QS

2831 Programmers have always argued about the layout of programs,

2832 but the specific style is much less important than its consistent

2833 application.

2834 Pick one style, preferably ours, use it consistently, and don't waste

2835 time arguing.

2836 .QE

2837 .P

2838 I agree that the constant application is most important,

2839 but I believe that some styles have advantages over others.

2840 For instance the indentation with tab characters only.

2841 The number of tabs corresponds to the nesting level \(en

2842 one tab, one level.

2843 Tab characters provide flexible visual appearance because developers

2844 can adjust their width as prefered.

2845 There is no more need to check for the correct mixture of

2846 tabs and spaces.

2847 Two simple rules ensure the integrity and flexibility of the visual

2848 appearance:

2849 .LI 1

2850 Leading whitespace must consist of tabs only.

2851 .LI 2

2852 All other whitespace should be spaces.

2853 .LP

2854 Although reformatting existing code should be avoided, I did it.

2855 I did not waste time arguing; I just reformatted the code.

2856 .Ci a485ed478abbd599d8c9aab48934e7a26733ecb1

2857

2858 .U3 "Comments

2859 .P

2860 Kernighan and Pike demand: ``Don't belabor the obvious''.

2861 .[ [

2862 kernighan pike practice of programming

2863 .], p. 23]

2864 Following the advice, I removed unnecessary comments.

2865 For instance, I removed all comments in the following code excerpt

2866 .Ci 426543622b377fc5d091455cba685e114b6df674 :

2867 .VS

2868 context_replace(curfolder, folder); /* update current folder */

2869 seq_setcur(mp, mp->lowsel); /* update current message */

2870 seq_save(mp); /* synchronize message sequences */

2871 folder_free(mp); /* free folder/message structure */

2872 context_save(); /* save the context file */

2873

2874 [...]

2875

2876 int c; /* current character */

2877 char *cp; /* miscellaneous character pointer */

2878

2879 [...]

2880

2881 /* NUL-terminate the field */

2882 *cp = '\0';

2883 VE

2884 .P

2885 The information in each of the comments was present in the code

2886 statements already, except for the NUL-termination, which became

2887 obvious from the context.

2888

2889

2890 .U3 "Names

2891 .P

2892 Regarding this topic, Kernighan and Pike suggest:

2893 ``Use active names for functions''.

2894 .[ [

2895 kernighan pike practice of programming

2896 .], p. 4]

2897 One application of this rule was the rename of

2898 .Fu check_charset()

2899 to

2900 .Fu is_native_charset()

2901 .Ci 8d77b48284c58c135a6b2787e721597346ab056d .

2902 The same change additionally fixed a violation of ``Be accurate'',

2903 .[ [

2904 kernighan pike practice of programming

2905 .], p. 4]

2906 as the code did not match the expectation the function suggested.

2907 It did not compare charset names but prefixes of them only.

2908 In case the native charset was `ISO-8859-1', then

2909 .VS

2910 check_charset("ISO-8859-11", strlen("ISO-8859-11"))

2911 VE

2912 had returned true although the upper halves of the code pages

2913 are different.

2914 .P

2915 More important than using active names is using descriptive names.

2916 .VS

2917 m_unknown(in); /* the MAGIC invocation... */

2918 VE

2919 Renaming the obscure

2920 .Fu m_unknown()

2921 function was a delightful event, although it made the code less funny

2922 .Ci 611d68d19204d7cbf5bd585391249cb5bafca846 .

2923 .P

2924 Magic numbers are generally considered bad style.

2925 Obviously, Kernighan and Pike agree:

2926 ``Give names to magic numbers''.

2927 .[ [

2928 kernighan pike practice of programming

2929 .], p. 19]

2930 .P

2931 The argument

2932 .CW outnum

2933 of the function

2934 .Fu scan()

2935 in

2936 .Fn uip/scansbr.c

2937 holds the number of the message to be created.

2938 As well it encodes program logic with negative numbers and zero.

2939 This led to obscure code.

2940 I clarified the code by introducing two variables that extracted

2941 the hidden information:

2942 .VS

2943 int incing = (outnum > 0);

2944 int ismbox = (outnum != 0);

2945 VE

2946 The readable names are thus used in conditions;

2947 the variable

2948 .CW outnum

2949 is used only to extract ordinary message numbers

2950 .Ci b8b075c77be7794f3ae9ff0e8cedb12b48fd139f .

2951 .P

2952 Through the clarity improvement of the change detours in the program

2953 logic of related code parts became apparent.

2954 The implementation was simplified.

2955 This possibility to improve had been invisible before

2956 .Ci aa60b0ab5e804f8befa890c0a6df0e3143ce0723 .

2957 .P

2958 The names just described were a first step, yet the situation

2959 was further improved by giving names to the magic values of

2960 .CW outnum :

2961 .VS

2962 #define SCN_MBOX (-1)

2963 #define SCN_FOLD 0

2964 VE

2965 The two variables were updated thereafter as well:

2966 .VS

2967 int incing = (outnum != SCN_MBOX && outnum != SCN_FOLD);

2968 int scanfolder = (outnum == SCN_FOLD);

2969 VE

2970 Furthermore,

2971 .CW ismbox

2972 was replaced by

2973 .CW scanfolder

2974 because that matched better to the program logic.

2975 .Ci 7ffb36d28e517a6f3a10272056fc127592ab1c19

2976

2977

2978

2979

2980 .H2 "Structural Rework

2981 .P

2982 Although the stylistic changes described already improve the

2983 readability of the source code, all of them were changes ``in the small''.

2984 Structural changes, in contrast, affect much larger code areas.

2985 They are more difficult to accomplish but lead to larger improvements,

2986 especially as they often influence the outer shape of the tools as well.

2987 .P

2988 At the end of their chapter on style,

2989 Kernighan and Pike ask: ``But why worry about style?''

2990 .[ [

2991 kernighan pike practice of programming

2992 .], p. 28].

2993 Following are two examples of structural rework that demonstrate

2994 why style is important in the first place.

2995

2996

2997 .U3 "Rework of \f(CWanno\fP

2998 .P

2999 Until 2002,

3000 .Pn anno

3001 had six functional command line switches:

3002 .Sw -component

3003 and

3004 .Sw -text ,

3005 each with an argument,

3006 and the two pairs of flags,

3007 .Sw -[no]date

3008 and

3009 .Sw -[no]inplace .

3010 Then Jon Steinhart introduced his attachment system.

3011 In need for more advanced annotation handling, he extended

3012 .Pn anno .

3013 He added five more switches:

3014 .Sw -draft ,

3015 .Sw -list ,

3016 .Sw -delete ,

3017 .Sw -append ,

3018 and

3019 .Sw -number ,

3020 the last one taking an argument

3021 .Ci 7480dbc14bc90f2d872d434205c0784704213252 .

3022 Later,

3023 .Sw -[no]preserve

3024 was added as well

3025 .Ci d9b1d57351d104d7ec1a5621f090657dcce8cb7f .

3026 Then, the Synopsis section of the man page

3027 .Mp anno (1)

3028 read:

3029 .VS

3030 anno [+folder] [msgs] [-component f(CIfieldfP] [-inplace | -noinplace]

3031 [-date | -nodate] [-draft] [-append] [-list] [-delete]

3032 [-number [f(CInumfP|fPallfP]] [-preserve | -nopreserve] [-version]

3033 [-help] [-text f(CIbodyfP]

3034 VE

3035 .LP

3036 The implementation followed the same structure.

3037 Problems became visible when

3038 .Cl "anno -list -number 42

3039 worked on the current message instead of on message number 42,

3040 and

3041 .Cl "anno -list -number l:5

3042 did not work on the last five messages but failed with the mysterious

3043 error message: ``anno: missing argument to -list''.

3044 Yet, the invocation matched the specification in the man page.

3045 There, the correct use of

3046 .Sw -number

3047 was defined as being

3048 .Cl "[-number [num|all]]

3049 and the textual description for the combination with

3050 .Sw -list

3051 read:

3052 .QS

3053 The

3054 .Sw -list

3055 option produces a listing of the field bodies for

3056 header fields with names matching the specified component,

3057 one per line. The listing is numbered, starting at 1, if the

3058 .Sw -number

3059 option is also used.

3060 .QE

3061 .LP

3062 The problem was manifold.

3063 Semantically, the argument to the

3064 .Sw -number

3065 switch is only necessary in combination with

3066 .Sw -delete ,

3067 but not with

3068 .Sw -list .

3069 The code, however, required a numeric argument in any case.

3070 If the argument was missing or non-numeric,

3071 .Pn anno

3072 aborted with an error message that additionally had an off-by-one error.

3073 It printed the name of the switch one before the concerned one.

3074 .P

3075 Trying to fix these problems on the surface would not have solved them.

3076 They originate from a discrepance between the

3077 structure of the problem and the structure implemented in the program.

3078 Such structural differences can only be solved by adjusting the

3079 structure of the implementation to the structure of the problem.

3080 .P

3081 Steinhart had added the new

3082 .Sw -list

3083 and

3084 .Sw -delete

3085 switches in a style similar to the other switches though

3086 they are of structural different type.

3087 Semantically,

3088 .Sw -list

3089 and

3090 .Sw -delete

3091 introduce operation modes.

3092 Historically,

3093 .Pn anno

3094 had only one operation mode: adding header fields.

3095 With the extension, two more modes were added:

3096 listing and deleting header fields.

3097 The structure of the code changes did not pay respect to this

3098 fundamental change.

3099 Neither the implementation nor the documentation did clearly

3100 declare the exclusive operation modes as such.

3101 Having identified the problem, I solved it by putting structure into

3102 .Pn anno

3103 and its documentation

3104 .Ci d54c8db8bdf01e8381890f7729bc0ef4a055ea11 .

3105 .P

3106 The difference is visible in both the code and the documentation.

3107 For instance in the following code excerpt:

3108 .VS

3109 int delete = -2; /* delete header element if set */

3110 int list = 0; /* list header elements if set */

3111 [...]

3112 case DELETESW: /* delete annotations */

3113 delete = 0;

3114 continue;

3115 case LISTSW: /* produce a listing */

3116 list = 1;

3117 continue;

3118 VE

3119 .LP

3120 which was replaced by:

3121 .VS

3122 static enum { MODE_ADD, MODE_DEL, MODE_LIST } mode = MODE_ADD;

3123 [...]

3124 case DELETESW: /* delete annotations */

3125 mode = MODE_DEL;

3126 continue;

3127 case LISTSW: /* produce a listing */

3128 mode = MODE_LIST;

3129 continue;

3130 VE

3131 .LP

3132 The replacement code does not only reflect the problem's structure better,

3133 it is easier to understand as well.

3134 The same applies to the documentation.

3135 The man page was completely reorganized to propagate the same structure.

3136 This is already visible in the Synopsis section:

3137 .VS

3138 anno [+folder] [msgs] [-component f(CIfieldfP] [-text fPbodyfP]

3139 [-append] [-date | -nodate] [-preserve | -nopreserve]

3140 [-Version] [-help]

3141

3142 anno -delete [+folder] [msgs] [-component fPfieldfP] [-text

3143 fPbodyfP] [-number fPnum fP| fPall fP] [-preserve | -nopreserve]

3144 [-Version] [-help]

3145

3146 anno -list [+folder] [msgs] [-component fPfieldfP] [-number]

3147 [-Version] [-help]

3148 VE

3149

3150

3151

3152 .U3 "Path Conversion

3153 .P

3154 Four kinds of path names can appear in MH:

3155 .LI 1

3156 Absolute Unix directory paths, like

3157 .Fn /etc/passwd .

3158 .LI 2

3159 Relative Unix directory paths, like

3160 .Fn ./foo/bar .

3161 .LI 3

3162 Absolute MH folder paths, like

3163 .Fn +projects/mmh .

3164 .LI 4

3165 Relative MH folder paths, like

3166 .Fn @subfolder .

3167 .LP

3168 Relative MH folder paths, are hardly documented

3169 although they are useful for large mail storages.

3170 The current mail folder is specified as `\c

3171 .Fn @ ',

3172 just like the current directory is specified as `\c

3173 .Fn . '.

3174 .P

3175 To allow MH tools to understand all four notations,

3176 they need to be able to convert between them.

3177 In nmh, these path name conversion functions were located in the files

3178 .Fn sbr/path.c

3179 (``return a pathname'') and

3180 .Fn sbr/m_maildir.c

3181 (``get the path for the mail directory'').

3182 The seven functions in the two files were documented with no more

3183 than two comments, which described obvious information.

3184 The signatures of the four exported functions did not explain their

3185 semantics:

3186 .LI 1

3187 .CW "char *path(char *, int);

3188 .LI 2

3189 .CW "char *pluspath(char *);

3190 .LI 3

3191 .CW "char *m_mailpath(char *);

3192 .LI 4

3193 .CW "char *m_maildir(char *);

3194 .P

3195 My investigations provided the following descriptions:

3196 .LI 1

3197 The second parameter of

3198 .Fu path()

3199 defines the type as which the path given in the first parameter should

3200 be treated.

3201 Directory paths are converted to absolute directory paths.

3202 Folder paths are converted to absolute folder paths.

3203 Folder paths must not include a leading `\fL@\fP' character.

3204 Leading plus characters are preserved.

3205 The result is a pointer to newly allocated memory.

3206 .LI 2

3207 .Fu pluspath()

3208 is a convenience-wrapper to

3209 .Fu path() ,

3210 to convert folder paths only.

3211 This function can not be used for directory paths.

3212 An empty string parameter causes a buffer overflow.

3213 .LI 3

3214 .Fu m_mailpath()

3215 converts directory paths to absolute directory paths.

3216 The characters `\fL+\fP' or `\fL@\fP' at the beginning of the path name are

3217 treated literal, i.e. as the first character of a relative directory path.

3218 Hence, this function can not be used for folder paths.

3219 In any case, the result is an absolute directory path,

3220 returned as a pointer to newly allocated memory.

3221 .LI 4

3222 .Fu m_maildir()

3223 returns the parameter unchanged if it is an absolute directory path

3224 or begins with the entry `\fL.\fP' or `\fL..\fP'.

3225 All other strings are prepended with the current working directory.

3226 Hence, this function can not be used for folder paths.

3227 The result is either an absolute directory path or a relative

3228 directory path, starting with dot or dot-dot.

3229 In contrast to the other functions, the result is a pointer to

3230 static memory.

3231 .P

3232 The situation was obscure, irritating, error-prone, and non-orthogonal.

3233 Additionally, no clear terminology was used to name the different

3234 kinds of path names.

3235 Sometimes, the names were even misleading, much as the first argument of

3236 .Fu m_mailpath() ,

3237 which was named

3238 .CW folder ,

3239 although

3240 .Fu m_mailpath()

3241 could not be used with MH folder arguments.

3242 .P

3243 I clarified the path name conversion by complete rework.

3244 First of all, the terminology needed to be defined.

3245 A path name is either in the Unix domain, then it is called

3246 \fIdirectory path\fP (\fIdirpath\fP for short) or it is in the MH domain,

3247 then it is called \fIfolder path\fP (\fIfolpath\fP for short).

3248 The two terms need to be used with strict distinction.

3249 Often a clear terminology indicates that the problem is understood.

3250 Second, I exploited the concept of path type indicators.

3251 By requiring every path name to start with a distinct type identifier,

3252 the conversion between the types could be fully automated.

3253 This allows the tools to accept paths of any type from the user.

3254 Therefore, it was necessary to require relative directory paths to be

3255 prefixed with a dot character.

3256 In consequence, the dot character could no longer be an alias for the

3257 current message.

3258 .Ci cff0e16925e7edbd25b8b9d6d4fbdf03e0e60c01

3259 Third, I created three new functions to replace the previous mess:

3260 .LI 1

3261 .Fu expandfol()

3262 converts folder paths to absolute folder paths.

3263 Directory paths are simply passed through.

3264 This function is to be used for folder paths only, thus the name.

3265 The result is a pointer to static memory.

3266 .LI 2

3267 .Fu expanddir()

3268 converts directory paths to absolute directory paths.

3269 Folder paths are treated as relative directory paths.

3270 This function is to be used for directory paths only, thus the name.

3271 The result is a pointer to static memory.

3272 .LI 3

3273 .Fu toabsdir()

3274 converts any type of path to an absolute directory path.

3275 This is the function of choice for path conversion.

3276 Absolute directory paths are the most general representation of a

3277 path name.

3278 The result is a pointer to static memory.

3279 .P

3280 The new functions have names that indicate their use.

3281 Two of the functions convert relative to absolute path names of the

3282 same type.

3283 The third function converts any path name type to the most general one,

3284 the absolute directory path.

3285 All of the functions return pointers to static memory.

3286 The file

3287 .Fn sbr/path.c

3288 contains the implementation of the functions;

3289 .Fn sbr/m_maildir.c

3290 was removed.

3291 .Ci d39e2c447b0d163a5a63f480b23d06edb7a73aa0

3292 .P

3293 Along with the path conversion rework, I also replaced

3294 .Fu getfolder(FDEF)

3295 with

3296 .Fu getdeffol()

3297 and

3298 .Fu getfolder(FCUR)

3299 with

3300 .Fu getcurfol() ,

3301 which only wraps

3302 .Fu expandfol(""@"")

3303 for convenience.

3304 This code was moved from

3305 .Fn sbr/getfolder.c

3306 into

3307 .Fn sbr/path.c

3308 as well.

3309 .Ci d39e2c447b0d163a5a63f480b23d06edb7a73aa0

3310 .P

3311 The related function

3312 .Fu etcpath()

3313 is now included in

3314 .Fn sbr/path.c ,

3315 too

3316 .Ci b4c29794c12099556151d93a860ee51badae2e35 .

3317 Previously, it had been located in

3318 .Fn config/config.c .

3319 .P

3320 Now,

3321 .Fn sbr/path.c

3322 contains all path handling code.

3323 Besides being less code, its readability is highly improved.

3324 The functions follow a common style and are well documented.

3325

3326

3327

3328

3329 .H2 "Profile Reading

3330 .P

3331 The MH profile contains the configuration of a user-specific MH setup.

3332 MH tools read the profile right after starting up

3333 because it contains the location of the user's mail storage

3334 and similar settings that influence the whole setup.

3335 Furthermore, the profile contains the default switches for the tools

3336 as well.

3337 The context file is read along with the profile.

3338 .P

3339 For historic reasons, some MH tools did not read the profile and context.

3340 Among them were

3341 .Pn post /\c

3342 .Pn spost ,

3343 .Pn mhmail ,

3344 and

3345 .Pn slocal .

3346 The reason why these tools ignored the profile were not clearly stated.

3347 During a discussion on the nmh-workers mailing list,

3348 David Levine posted an explanation, quoting John Romine:

3349 .[

3350 nmh-workers levine post profile

3351 .]

3352

3353 .QS

3354 I asked John Romine and here's what he had to say, which

3355 agrees and provides an example that convinces me:

3356 .QS

3357 My take on this is that

3358 .Pn post

3359 should not be called by users directly, and it doesn't read the

3360 .Fn .mh_profile

3361 (only front-end UI programs read the profile).

3362 .QP

3363 For example, there can be contexts where

3364 .Pn post

3365 is called by a helper program (like `\c

3366 .Pn mhmail ')

3367 which may be run by a non-MH user.

3368 We don't want this to prompt the user to create an MH profile, etc.

3369 .QP

3370 My suggestion would be to have

3371 .Pn send

3372 pass a (hidden) `\c

3373 .Sw -fileproc

3374 .Ar proc '

3375 option to

3376 .Pn post

3377 if needed.

3378 You could also

3379 use an environment variable (I think

3380 .Pn send /\c

3381 .Pn whatnow

3382 do this).

3383 .QE

3384 .sp \n(PDu

3385 I think that's the way to go.

3386 My personal preference is to use a command line option,

3387 not an environment variable.

3388 .QE

3389

3390 .P

3391 To solve the problem that

3392 .Pn post

3393 does not honor the

3394 .Pe fileproc

3395 profile entry,

3396 the community roughly agreed that a switch

3397 .Sw -fileproc

3398 should be added to

3399 .Pn post

3400 to be able to pass a different fileproc.

3401 I strongly disagree with this approach because it does not solve

3402 the problem; it only removes a single symptom.

3403 The actual problem is that

3404 .Pn post

3405 does not behave as expected,

3406 though all programs should behave as expected.

3407 Clear and general concepts are a precondition for this.

3408 Thus, there should be no separation into ``front-end UI programs''

3409 and ones that ``should not be called by users directly''.

3410 The real solution is having all MH tools read the profile.

3411 .P

3412 But the problem has a further aspect,

3413 which originates from

3414 .Pn mhmail

3415 mainly.

3416 .Pn mhmail

3417 was intended to be a replacement for

3418 .Pn mailx

3419 on systems with MH installations.

3420 In difference to

3421 .Pn mailx ,

3422 .Pn mhmail

3423 used MH's

3424 .Pn post

3425 to send the message.

3426 The idea was that using

3427 .Pn mhmail

3428 should not be influenced whether the user had

3429 MH set up for himself or not.

3430 Therefore

3431 .Pn mhmail

3432 had not read the profile.

3433 As

3434 .Pn mhmail

3435 used

3436 .Pn post ,

3437 .Pn post

3438 was not allowed to read the profile neither.

3439 This is the reason for the actual problem.

3440 Yet, this was not considered much of a problem because

3441 .Pn post

3442 was not intended to be used by users directly.

3443 To invoke

3444 .Pn post ,

3445 .Pn send

3446 was used an a front-end.

3447 .Pn send

3448 read the profile and passed all relevant values on the command line to

3449 .Pn post

3450 \(en an awkward solution.

3451 .P

3452 The important insight is that

3453 .Pn mhmail

3454 is a wolf in sheep's clothing.

3455 This alien tool broke the concepts because it was treated like

3456 a normal MH tool.

3457 Instead it should have been treated accordingly to its foreign style.

3458 .P

3459 The solution is not to prevent the tools from reading the profile but

3460 to instruct them to read a different profile.

3461 .Pn mhmail

3462 could have set up a well-defined profile and caused the following

3463 .Pn post

3464 to use this profile by exporting an environment variable.

3465 With this approach, no special cases would have been introduced

3466 and no surprises would have been caused.

3467 By writing a wrapper program to provide a clean temporary profile,

3468 the concept could have been generalized orthogonally to the whole

3469 MH tool chest.

3470 .P

3471 In mmh, the wish to have

3472 .Pn mhmail

3473 as a replacement for

3474 .Pn mailx

3475 is considered obsolete.

3476 Mmh's

3477 .Pn mhmail

3478 does no longer cover this use-case

3479 .Ci d36e56e695fe1c482c7920644bfbb6386ac9edb0 .

3480 Currently,

3481 .Pn mhmail

3482 is in a transition state

3483 .Ci 32d4f9daaa70519be3072479232ff7be0500d009 .

3484 It may become a front-end to

3485 .Pn comp ,

3486 which provides an alternative interface which can be more convenient

3487 in some cases.

3488 This would convert

3489 .Pn mhmail

3490 into an ordinary MH tool.

3491 If, however, this idea does not convince, then

3492 .Pn mhmail

3493 will be removed.

3494 .P

3495 In the mmh tool chest, every program reads the profile.

3496 (\c

3497 .Pn slocal

3498 is not considered part of the mmh tool chest (cf. Sec.

3499 .Cf slocal ).)

3500 Mmh has no

3501 .Pn post

3502 program, but it has

3503 .Pn spost ,

3504 which now does read the profile

3505 .Ci 3e017a7abbdf69bf0dff7a4073275961eda1ded8 .

3506 Following this change,

3507 .Pn send

3508 and

3509 .Pn spost

3510 can be considered for merging.

3511 Besides

3512 .Pn send ,

3513 .Pn spost

3514 is only invoked directly by the to-be-changed

3515 .Pn mhmail

3516 implementation and by

3517 .Pn rcvdist ,

3518 which requires rework anyway.

3519

3520 .P

3521 Jeffrey Honig quoted Marshall T. Rose explaining the decision that

3522 .Pn post

3523 ignores the profile:

3524 .[

3525 nmh-workers honig post profile

3526 .]

3527 .QS

3528 when you run mh commands in a script, you want all the defaults to be

3529 what the man page says.

3530 when you run a command by hand, then you want your own defaults...

3531 .QE

3532 .LP

3533 The explanation neither matches the problem concered exactly

3534 nor is the interpretation clear.

3535 If the described desire addresses the technical level,

3536 then it conflicts fundametally with the Unix philosophy,

3537 precisely because the indistinquishability of human and script

3538 input is the main reason for the huge software leverage in Unix.

3539 If, however, the described desire addresses the user's view,

3540 then different technical solutions are more appropriate.

3541 The two cases can be regarded simply as two different MH setups.

3542 Hence, mapping the problem of different behavior between interactive and

3543 automated use on the concept of switching between different profiles,

3544 marks it already solved.

3545

3546

3547

3548 .H2 "Standard Libraries

3549 .P

3550 MH is one decade older than the POSIX and ANSI C standards.

3551 Hence, MH included own implementations of functions

3552 that were neither standardized nor widely available, back then.

3553 Today, twenty years after POSIX and ANSI C were published,

3554 developers can expect that systems comply with these standards.

3555 In consequence, MH-specific replacements for standard functions

3556 can and should be dropped.

3557 Kernighan and Pike advise: ``Use standard libraries''.

3558 .[ [

3559 kernighan pike practice of programming

3560 .], p. 196]

3561 Actually, MH had followed this advice in history,

3562 but it had not adjusted to more recent changes in this field.

3563 The

3564 .Fu snprintf()

3565 function, for instance, was standardized with C99 and is available

3566 almost everywhere because of its high usefulness.

3567 Thus, the project's own implementation of

3568 .Fu snprintf()

3569 was dropped in March 2012 in favor for using the one of the

3570 standard library

3571 .Ci 0052f1024deb0a0a2fc2e5bacf93d45a5a9c9b32 .

3572 Such decisions limit the portability of mmh

3573 if systems do not support these standardized and widespread functions.

3574 This compromise is made because mmh focuses on the future.

3575 .P

3576 As I am still in my twenties, have no programming experience from

3577 past decades.

3578 I have not followed the evolution of C through time.

3579 I have not suffered from the the Unix wars.

3580 I have not longed for standardization.

3581 All my programming experience is from a time when ANSI C and POSIX

3582 were well established already.

3583 Thus, I needed to learn about the history in retrospective.

3584 I have only read a lot of books about the (good) old times.

3585 This put me in a difficult position when working with old code.

3586 I need to freshly acquire knowledge about old code constructs and ancient

3587 programming styles, whereas older programmers know these things by

3588 heart from their own experience.

3589 Being aware of the situation, I rather let people with more historic

3590 experience do the transition from ancient code constructs to

3591 standardized ones.

3592 Lyndon Nerenberg covered large parts of this task for the nmh project.

3593 He converted project-specific functions to POSIX replacements,

3594 also removing the conditionals compilation of now standardized features.

3595 Ken Hornstein and David Levine had their part in this work, as well.

3596 Often, I only pulled the changes over from nmh into mmh.

3597 These changes include many commits, among them:

3598 .Ci 768b5edd9623b7238e12ec8dfc409b82a1ed9e2d

3599 .Ci 0052f1024deb0a0a2fc2e5bacf93d45a5a9c9b32 .

3600 .P

3601 Nevertheless, I worked on the task as well, tidying up the

3602 \fIMH standard library\fP,

3603 .Fn libmh.a .

3604 It is located in the

3605 .Fn sbr

3606 (``subroutines'') directory in the source tree and

3607 includes functions that mmh tools usually need.

3608 Among them are MH-specific functions for profile, context, sequence,

3609 and folder handling, but as well

3610 MH-independent functions, such as auxiliary string functions,

3611 portability interfaces and error-checking wrappers for critical

3612 functions of the standard library.

3613 .BU

3614 I have replaced the

3615 .Fu atooi()

3616 function with calls to

3617 .Fu strtoul() ,

3618 setting the third parameter, the base, to eight.

3619 .Fu strtoul()

3620 is part of C89 and thus considered safe to use

3621 .Ci c490c51b3c0f8871b6953bd0c74551404f840a74 .

3622 .BU

3623 I did remove project-included fallback implementations of

3624 .Fu memmove()

3625 and

3626 .Fu strerror()

3627 .Ci b067ff5c465a5d243ce5a19e562085a9a1a97215 ,

3628 although Peter Maydell had re-included them into nmh in 2008

3629 to support SunOS 4.

3630 Nevertheless, these functions are part of ANSI C.

3631 Systems that do not even provide full ANSI C support should not

3632 put a load on mmh.

3633 .BU

3634 The

3635 .Fu copy()

3636 function copies the string in parameter one to the location in

3637 parameter two.

3638 In contrast to

3639 .Fu strcpy() ,

3640 it returns a pointer to the terminating null-byte in the destination area.

3641 The code was adjusted to replace

3642 .Fu copy()

3643 with

3644 .Fu strcpy() ,

3645 except within

3646 .Fu concat() ,

3647 where

3648 .Fu copy()

3649 was more convenient.

3650 Therefore, the definition of

3651 .Fu copy()

3652 was moved into the source file of

3653 .Fu concat()

3654 and its visibility it limited to that

3655 .Ci 552fd7253e5ee9e554c5c7a8248a6322aa4363bb .

3656 .BU

3657 The function

3658 .Fu r1bindex()

3659 had been a generalized version of

3660 .Fu basename()

3661 with minor differences.

3662 As all calls to

3663 .Fu r1bindex()

3664 had the slash (`\fL/\fP') as delimiter anyway,

3665 replacing

3666 .Fu r1bindex()

3667 with the more specific and better-named function

3668 .Fu basename()

3669 became desirable.

3670 Unfortunately, many of the 54 calls to

3671 .Fu r1bindex()

3672 depended on a special behavior,

3673 which differed from the POSIX specification for

3674 .Fu basename() .

3675 Hence,

3676 .Fu r1bindex()

3677 was kept but renamed to

3678 .Fu mhbasename() ,

3679 setting the delimiter to the slash

3680 .Ci 240013872c392fe644bd4f79382d9f5314b4ea60 .

3681 For possible uses of

3682 .Fu r1bindex()

3683 with a different delimiter,

3684 the ANSI C function

3685 .Fu strrchr()

3686 provides the core functionality.

3687 .BU

3688 The

3689 .Fu ssequal()

3690 function \(en apparently for ``substring equal'' \(en

3691 was renamed to

3692 .Fu isprefix() ,

3693 because this is what it actually checked

3694 .Ci c20b4fa14515c7ab388ce35411d89a7a92300711.

3695 Its source file had included both of the following comments, no joke.

3696 .in -\n(PIu

3697 .VS

3698 /*

3699 * THIS CODE DOES NOT WORK AS ADVERTISED.

3700 * It is actually checking if s1 is a PREFIX of s2.

3701 * All calls to this function need to be checked to see

3702 * if that needs to be changed. Prefix checking is cheaper, so

3703 * should be kept if it's sufficient.

3704 */

3705

3706 /*

3707 * Check if s1 is a substring of s2.

3708 * If yes, then return 1, else return 0.

3709 */

3710 VE

3711 .in +\n(PIu

3712 Eventually, the function was completely replaced with calls to

3713 .Fu strncmp()

3714 .Ci b0b1dd37ff515578cf7cba51625189eb34a196cb .

3715

3716

3717

3718

3719

3720 .H2 "User Data Locations

3721 .P

3722 In nmh, a personal setup consists of the MH profile and the MH directory.

3723 The profile is a file named

3724 .Fn \&.mh_profile

3725 in the user's home directory.

3726 It contains the static configuration.

3727 It also contains the location of the MH directory in the profile entry

3728 .Pe Path .

3729 The MH directory contains the mail storage and is the first

3730 place to search for form files, scan formats, and similar

3731 configuration files.

3732 The location of the MH directory can be chosen freely by the user.

3733 The usual name is a directory named

3734 .Fn Mail

3735 in the user's home directory.

3736 .P

3737 The way MH data is split between profile and MH directory is a legacy.

3738 It is only sensible in a situation where the profile is the only

3739 configuration file.

3740 Why else should the mail storage and the configuration files be intermixed?

3741 They are of different kind:

3742 One kind is the data to be operated on and the other kind is

3743 the configuration to change how tools operate.

3744 Splitting the configuration between the profile and the MH directory

3745 is inappropriate, as well.

3746 I improved the situation by breaking compatibility.

3747 .P

3748 In mmh, personal data is grouped by type.

3749 This results in two distinct parts:

3750 the mail storage and the configuration.

3751 The mail storage directory still contains all the messages,

3752 but, in exception of public sequences files, nothing else.

3753 In difference to nmh, the auxiliary configuration files are no longer

3754 located there.

3755 Therefore, the directory is no longer called the user's \fIMH directory\fP

3756 but the user's \fImail storage\fP.

3757 Its location is still user-chosen, with the default name

3758 .Fn Mail

3759 in the user's home directory.

3760 The configuration is grouped together in the hidden directory

3761 .Fn \&.mmh

3762 in the user's home directory.

3763 This \fImmh directory\fP contains the context file, personal forms,

3764 scan formats, and the like, but also the user's profile, now named

3765 .Fn profile .

3766 The path to the profile is no longer

3767 .Fn $HOME/.mh_profile

3768 but

3769 .Fn $HOME/.mmh/profile .

3770 (The alternative of having file

3771 .Fn $HOME/.mh_profile

3772 and a configuration directory

3773 .Fn $HOME/.mmh

3774 appeared to be inconsistent.)

3775 .P

3776 The approach chosen for mmh is consistent, simple, and familiar to

3777 Unix users.

3778 The main achievement of the change is the clear and sensible separation

3779 of the mail storage and the configuration.

3780 .Ci 7030d7edb099bff36ded7548bb5380f7acab4f9b

3781 .P

3782 As MH allows users to have multiple MH setups,

3783 it is necessary to switch the profile.

3784 The profile is the single entry point to access the rest of a

3785 personal MH setup.

3786 In nmh, the environment variable

3787 .Ev MH

3788 is used to specify a different profile.

3789 To operate in the same MH setup with a separate context, the

3790 .Ev MHCONTEXT

3791 environment variable is used.

3792 This allows having a separate current folder in each terminal at

3793 the same time, for instance.

3794 In mmh, three environment variables replace the two of nmh.

3795 .Ev MMH

3796 overrides the default location of the mmh directory (\c

3797 .Fn .mmh ).

3798 .Ev MMHP

3799 and

3800 .Ev MMHC

3801 override the paths to the profile and context file, respectively.

3802 This approach allows the set of personal configuration files to be chosen

3803 independently of the profile, context, and mail storage.

3804 The new approach has no functional disadvantages,

3805 as every setup I can imagine can be implemented with both approaches,

3806 possibly even easier with the new one.

3807 .Ci 7030d7edb099bff36ded7548bb5380f7acab4f9b

3808

3809

3810

3811

3812

3813 .H2 "Modularization

3814 .Id modularization

3815 .P

3816 The source code of the mmh tools is located in the

3817 .Fn uip

3818 (``user interface programs'') directory.

3819 Each tool has a source file with the name of the command.

3820 For example,

3821 .Pn rmm

3822 is built from

3823 .Fn uip/rmm.c .

3824 Some source files are used for multiple programs.

3825 For example

3826 .Fn uip/scansbr.c

3827 is used for both

3828 .Pn scan

3829 and

3830 .Pn inc .

3831 In nmh, 49 tools were built from 76 source files.

3832 This is a ratio of 1.6 source files per program.

3833 32 programs depended on multiple source files;

3834 17 programs depended on one source file only.

3835 In mmh, 39 tools are built from 51 source files.

3836 This is a ratio of 1.3 source files per program.

3837 18 programs depend on multiple source files;

3838 21 programs depend on one source file only.

3839 (These numbers and the ones in the following text ignore the MH library

3840 as well as shell scripts and multiple names for the same program.)

3841 .P

3842 Splitting the source code of a large program into multiple files can

3843 increase the readability of its source code,

3844 but most of the mmh tools are small and straight-forward programs.

3845 In exception of the MIME handling tools (i.e.

3846 .Pn mhbuild ,

3847 .Pn mhstore ,

3848 .Pn show ,

3849 etc.),

3850 .Pn pick

3851 is the only tool with more than one thousand lines of source code.

3852 Splitting programs with less than one thousand lines of code into

3853 multiple source files leads seldom to better readability.

3854 For such tools, splitting still makes sense

3855 when parts of the code are reused in other programs

3856 and the reused code fragment is (1) not general enough

3857 for including it in the MH library

3858 or (2) has dependencies on a library that only few programs need.

3859 .Fn uip/packsbr.c ,

3860 for instance, provides the core program logic for the

3861 .Pn packf

3862 and

3863 .Pn rcvpack

3864 programs.

3865 .Fn uip/packf.c

3866 and

3867 .Fn uip/rcvpack.c

3868 mainly wrap the core function appropriately.

3869 No other tools use the folder packing functions.

3870 As another example,

3871 .Fn uip/termsbr.c

3872 accesses terminal properties, which requires linking with the

3873 \fItermcap\fP or a \fIcurses\fP library.

3874 If

3875 .Fn uip/termsbr.c

3876 is included in the MH library, then every program needs to be linked

3877 with termcap or curses, although only few of the programs use

3878 the library.

3879 .P

3880 The task of MIME handling is complex enough that splitting its code

3881 into multiple source files improves the readability.

3882 The program

3883 .Pn mhstore ,

3884 for instance, is compiled out of seven source files with 2\|500

3885 lines of code in summary.

3886 The main code file

3887 .Fn uip/mhstore.c

3888 consists of 800 lines; the other 1\|700 lines are code reused in

3889 other MIME handling tools.

3890 It seems to be worthwhile to bundle the generic MIME handling code into

3891 a MH-MIME library, as a companion to the MH standard library.

3892 This is left to be done.

3893 .P

3894 The work already accomplished focussed on the non-MIME tools.

3895 The amount of code compiled into each program was reduced.

3896 This eases the understanding of the code base.

3897 In nmh,

3898 .Pn comp

3899 was built from six source files:

3900 .Fn comp.c ,

3901 .Fn whatnowproc.c ,

3902 .Fn whatnowsbr.c ,

3903 .Fn sendsbr.c ,

3904 .Fn annosbr.c ,

3905 and

3906 .Fn distsbr.c .

3907 In mmh, it builds from only two:

3908 .Fn comp.c

3909 and

3910 .Fn whatnowproc.c .

3911 In nmh's

3912 .Pn comp ,

3913 the core function of

3914 .Pn whatnow ,

3915 .Pn send ,

3916 and

3917 .Pn anno

3918 were all compiled into

3919 .Pn comp .

3920 This saved the need to execute these programs with

3921 the expensive system calls

3922 .Fu fork()

3923 and

3924 .Fu exec() .

3925 Whereas this approach improved the time performance,

3926 it interwove the source code.

3927 Core functionalities were not encapsulated into programs but into

3928 function, which were then wrapped by programs.

3929 For example,

3930 .Fn uip/annosbr.c

3931 included the function

3932 .Fu annotate() .

3933 Each program that wanted to annotate messages, included the source file

3934 .Fn uip/annosbr.c

3935 and called

3936 .Fu annotate() .

3937 Because the function

3938 .Fu annotate()

3939 was used like the tool

3940 .Pn anno ,

3941 it had seven parameters, reflecting the command line switches of the tool.

3942 When another pair of command line switches was added to

3943 .Pn anno ,

3944 a rather ugly hack was implemented to avoid adding another parameter

3945 to the function

3946 .Ci d9b1d57351d104d7ec1a5621f090657dcce8cb7f .

3947 .P

3948 In mmh, the relevant code of

3949 .Pn comp

3950 comprises the two files

3951 .Fn uip/comp.c

3952 and

3953 .Fn uip/whatnowproc.c ,

3954 together 210 lines of code,

3955 whereas in nmh,

3956 .Pn comp

3957 comprises six files with 2\|450 lines.

3958 Not all of the code in these six files is actually used by

3959 .Pn comp ,

3960 but the reader needed to read it all to know which parts are relevant.

3961 Understanding nmh's

3962 .Pn comp ,

3963 required understanding the inner workings of

3964 .Fn uip/annosbr.c

3965 first.

3966 To be sure to fully understand a program, its whole source code needs

3967 to be examined.

3968 Not doing so is a leap of faith, assuming that the developers

3969 have avoided obscure programming techniques.

3970 Here, it should be recalled that information passed in obscure ways

3971 through the program's source base, due to the aforementioned hack

3972 to save an additional parameter in nmh's

3973 .Pn anno .

3974 .P

3975 In mmh, understanding

3976 .Pn comp

3977 requires to read only 210 lines of code to read, whereas the amount

3978 is ten times more for nmh's

3979 .Pn comp .

3980 .P

3981 By separating the tools on the program-level,

3982 the boundaries are clearly visible, as the interfaces are calls to

3983 .Fu exec()

3984 rather than arbitrary function calls.

3985 Additionally, this kind of separation is more strict because

3986 it is technically enforced by the operating system;

3987 it can not be simply bypassed with global variables.

3988 Good separation simplifies the understanding of program code

3989 because the area influenced by any particular statement is small.

3990 As I have read a lot in nmh's code base during the last two years,

3991 I have learned about the easy and the difficult parts.

3992 In my observation, the understanding of code is enormously eased

3993 if the influenced area is small and clearly bounded.

3994 .P

3995 Yet, the real problem is another:

3996 Nmh violates the golden ``one tool, one job'' rule of the Unix philosophy.

3997 Understanding

3998 .Pn comp

3999 requires understanding

4000 .Fn uip/annosbr.c

4001 and

4002 .Fn uip/sendsbr.c

4003 because

4004 .Pn comp

4005 annotates and sends messages.

4006 In nmh, there surely exist the tools

4007 .Pn anno

4008 and

4009 .Pn send ,

4010 which cover these jobs,

4011 but

4012 .Pn comp

4013 and

4014 .Pn repl

4015 and

4016 .Pn forw

4017 and

4018 .Pn dist

4019 and

4020 .Pn whatnow

4021 and

4022 .Pn viamail

4023 \(en they all (!) \(en

4024 have the same annotating and sending functions included, once more.

4025 As a result,

4026 .Pn comp

4027 sends messages without using

4028 .Pn send .

4029 The situation is the same as if

4030 .Pn grep

4031 would page its output without using

4032 .Pn more

4033 just because both programs are part of the same code base.

4034 .P

4035 The clear separation on the surface of nmh

4036 \(en the tool chest approach \(en

4037 is violated on the level below.

4038 This violation is for the sake of time performance.

4039 Decades ago, sacrificing readability and conceptional beauty

4040 for speed might have been necessary to prevent MH from being

4041 unusably slow, but today this is not the case anymore.

4042 No longer should speed improvements that became unnecessary be kept.

4043 No longer should readability or conceptional beauty be sacrificed.

4044 No longer should the Unix philosophy's ``one tool, one job''

4045 guideline be violated.

4046 Therefore, mmh's

4047 .Pn comp

4048 no longer sends messages.

4049 .P

4050 In mmh, different jobs are divided among separate programs that

4051 invoke each other as needed.

4052 In consequence,

4053 .Pn comp

4054 invokes

4055 .Pn whatnow

4056 which thereafter invokes

4057 .Pn send

4058 .Ci 3df5ab3c116e6d4a2fb4bb5cc9dfc5f781825815

4059 .Ci c73c00bfccd22ec77e9593f47462aeca4a8cd9c0 .

4060 The clear separation on the surface is maintained on the level below.

4061 Human users and other tools use the same interface \(en

4062 annotations, for example, are made by invoking

4063 .Pn anno ,

4064 no matter if requested by programs or by human beings

4065 .Ci 469a4163c2a1a43731d412eaa5d9cae7d670c48b

4066 .Ci aed384169af5204b8002d06e7a22f89197963d2d

4067 .Ci 3caf9e298a8861729ca8b8a84f57022b6f3ea742 .

4068 The decrease of tools built from multiple source files and thus

4069 the decrease of

4070 .Fn uip/*sbr.c

4071 files confirm the improvement

4072 .Ci 9e6d91313f01c96b4058d6bf419a8ca9a207bc33

4073 .ci 81744a46ac9f845d6c2b9908074d269275178d2e

4074 .Ci f0f858069d21111f0dbea510044593f89c9b0829

4075 .Ci 0503a6e9be34f24858b55b555a5c948182b9f24b

4076 .Ci 27826f9353e0f0b04590b7d0f8f83e60462b90f0

4077 .Ci d1da1f94ce62160aebb30df4063ccbc53768656b

4078 .Ci c42222869e318fff5dec395eca3e776db3075455 .

4079 This is also visible in the complexity of the build dependency graphs:

4080

4081 .sp

4082 Nmh:

4083 .BP input/deps-nmh.eps .5i

4084 .EP

4085 .sp

4086 Mmh:

4087 .BP input/deps-mmh.eps .8i

4088 .EP

4089

4090 The figures display all program to source file relationships where

4091 programs (ellipses) are built from multiple source files (rectangles).

4092 The primary source file of each program is omited from the graph.