docs/master

annotate intro.roff @ 174:35fc40a5011d

We won't have an abstract.
author markus schnalke <meillo@marmaro.de>
date Tue, 10 Jul 2012 17:28:39 +0200
parents 346ff7e201f5
children eb6eeb10afd5
rev   line source
meillo@39 1 .RN 1
meillo@39 2
meillo@0 3 .H0 "Introduction
meillo@0 4 .P
meillo@53 5 MH is a set of mail handling tools with a common concept, similar to
meillo@53 6 the Unix tool chest, which is a set of file handling tools with a common
meillo@53 7 concept. \fInmh\fP is the currently most popular implementation of an
meillo@53 8 MH-like mail handling system.
meillo@53 9 This thesis describes an experimental version of nmh, named \fImmh\fP.
meillo@53 10 .P
meillo@32 11 This chapter introduces MH, its history, concepts and how it is used.
meillo@47 12 It describes nmh's code base and community to give the reader
meillo@106 13 a better understanding of the state of mmh when it started off.
meillo@47 14 Further more, this chapter outlines the mmh project itself,
meillo@47 15 describing the motivation for it and its goals.
meillo@8 16
meillo@0 17
meillo@28 18 .H1 "MH \(en the Mail Handler
meillo@0 19 .P
meillo@47 20 MH is a conceptual email system design and its concrete implementation.
meillo@47 21 Notably, MH had started as a design proposal at RAND Corporation,
meillo@159 22 .\" XXX ref to rand corp.
meillo@47 23 where the first implementation followed later.
meillo@47 24 In spirit, MH is similar to Unix, which
meillo@42 25 influenced the world more in being a set of system design concepts
meillo@32 26 than in being a specific software product.
meillo@47 27 The ideas behind Unix are summarized in the \fIUnix philosophy\fP.
meillo@42 28 MH follows this philosophy.
meillo@2 29
meillo@11 30 .U2 "History
meillo@2 31 .P
meillo@32 32 In 1977 at RAND Corporation, Norman Shapiro and Stockton Gaines
meillo@106 33 proposed the design
meillo@32 34 of a new mail handling system, called ``Mail Handler'' (MH),
meillo@32 35 to superseed RAND's old monolithic ``Mail System'' (MS).
meillo@27 36 Two years later, in 1979, Bruce Borden took the proposal and implemented a
meillo@32 37 prototype of MH.
meillo@106 38 Before the prototype's existence, the concept was
meillo@47 39 believed to be practically unusable.
meillo@106 40 But the prototype proved successful and replaced MS thereafter.
meillo@164 41 In replacing MS, MH grew to provide anything necessary for emailing.
meillo@2 42 .P
meillo@106 43 In the early eighties,
meillo@106 44 the University of California at Irvine (UCI) started to use MH.
meillo@106 45 Marshall T. Rose and John L. Romine then became the driving force.
meillo@57 46 They took over the development and pushed MH forward.
meillo@57 47 RAND had put the code into the public domain by then.
meillo@57 48 MH was developed at UCI at the time when the Internet appeared,
meillo@57 49 when UCB implemented the TCP/IP stack, and when Allman wrote Sendmail.
meillo@47 50 MH was extended as emailing became more featured.
meillo@32 51 The development of MH was closely related to the development of email
meillo@32 52 RFCs. In the advent of MIME, MH was the first implementation of this new
meillo@32 53 email standard.
meillo@2 54 .P
meillo@117 55 In the nineties, the Internet became popular and in December 1996,
meillo@47 56 Richard Coleman initiated the ``New Mail Handler'' (nmh) project.
meillo@57 57 Nmh is a fork of MH 6.8.3 and bases strongly on the
meillo@47 58 \fILBL changes\fP by Van Jacobson, Mike Karels and Craig Leres.
meillo@32 59 Colman intended to modernize MH and improve its portability and
meillo@32 60 MIME handling capabilities.
meillo@32 61 This should be done openly within the Internet community.
meillo@47 62 The development of MH at UCI stopped after the 6.8.4 release in
meillo@47 63 February 1996, soon after the development of nmh had started.
meillo@57 64 Today, nmh has almost completely replaced the original MH.
meillo@47 65 Some systems might still provide old MH, but mainly for historical reasons.
meillo@47 66 .P
meillo@171 67 In the last years, the changes in nmh were mostly maintenance work.
meillo@117 68 However, the development was revived in December 2011
meillo@57 69 and stayed busy since then.
meillo@0 70
meillo@11 71 .U2 "Concepts
meillo@0 72 .P
meillo@53 73 MH consists of a set of tools, each covering a specific task of
meillo@53 74 email handling, like composing a message, replying to a message,
meillo@53 75 refiling a message to a different folder, listing the messages in a folder.
meillo@47 76 All of the programs operate on a common mail storage.
meillo@42 77 .P
meillo@32 78 The mail storage consists of \fImail folders\fP (directories) and
meillo@32 79 \fPmessages\fP (regular files).
meillo@106 80 Each message is stored in a separate file in the format it was
meillo@47 81 received (i.e. transfer format).
meillo@47 82 The files are named with ascending numbers in each folder.
meillo@47 83 The specific format of the mail storage characterizes MH in the same way
meillo@106 84 as the format of the file system characterizes Unix.
meillo@42 85 .P
meillo@164 86 MH tools maintain a \fIcontext\fP, which includes for instance the
meillo@164 87 current mail folder.
meillo@32 88 Processes in Unix have a similar context, containing the current working
meillo@32 89 directory, for instance. In contrast, the process context is maintained
meillo@32 90 by the Unix kernel automatically, whereas MH tools need to maintain the MH
meillo@32 91 context themselves.
meillo@106 92 The user can have one MH context or multiple ones; he can even share it
meillo@106 93 with others.
meillo@42 94 .P
meillo@164 95 Messages are named by their numeric filename,
meillo@164 96 but they can have symbolic names, too.
meillo@164 97 These are either automatically updated
meillo@106 98 position names such as the next or the last message,
meillo@32 99 or user-settable group names for arbitrary sets of messages.
meillo@32 100 These names are called sequences.
meillo@47 101 Sequences can be bound to the containing folder or to the context.
meillo@2 102 .P
meillo@47 103 The user's \fIprofile\fP is a file that contains his MH configuration.
meillo@47 104 Default switches for the individual tools can be specified to
meillo@47 105 adjust them to the user's personal preferences.
meillo@164 106 Additionally, a single command can be linked under different names
meillo@164 107 with different default values easily.
meillo@51 108 Form templates for new messages or for replies are easily changeable,
meillo@47 109 and output is adjustable with format files.
meillo@47 110 Almost every part of the system can be adjusted to personal preference.
meillo@42 111 .P
meillo@51 112 The system is well scriptable and extensible.
meillo@47 113 New MH tools are built out of or on top of existing ones quickly.
meillo@51 114 Further more, MH encourages the user to tailor, extend and automate the system.
meillo@51 115 As the MH tool chest was modeled after the Unix tool chest, the
meillo@32 116 properties of the latter apply to the former as well.
meillo@8 117
meillo@102 118
meillo@102 119 .ig \"XXX
meillo@102 120
meillo@102 121 .P
meillo@102 122 To ease typing, the switches can be abbreviated as much as the remaining
meillo@102 123 prefix remains unambiguous.
meillo@102 124 If in our example no other switch would start with the letter `t', then
meillo@102 125 .Cl "-truncate" ,
meillo@102 126 .Cl "-trunc" ,
meillo@102 127 .Cl "-tr" ,
meillo@102 128 and
meillo@102 129 .Cl "-t
meillo@102 130 would all be the same.
meillo@102 131 As a result, switches can neither be grouped (as in
meillo@102 132 .Cl "ls -ltr" )
meillo@102 133 nor can switch arguments be appended directly to the switch (as in
meillo@102 134 .Cl "sendmail -q30m" ).
meillo@102 135 .P
meillo@102 136 Many switches have negating counter-parts, which start with `no'.
meillo@102 137 For example
meillo@102 138 .Cl "-notruncate
meillo@102 139 inverts the
meillo@102 140 .Cl "-truncate
meillo@102 141 switch.
meillo@102 142 They exist to undo the effect of default switches in the profile.
meillo@102 143 If the user has chosen to change the default behavior of some tool
meillo@102 144 by adding a default switch to the profile,
meillo@102 145 he can still undo this change in behavior by specifying the inverse
meillo@102 146 switch on the command line.
meillo@102 147
meillo@102 148 ..
meillo@102 149
meillo@102 150
meillo@54 151 .U2 "Using MH
meillo@53 152 .P
meillo@54 153 It is strongly recommended to have a look at the MH Book,
meillo@106 154 which offers a thorough introduction to using MH.
meillo@54 155 .[ [
meillo@54 156 peek mh book
meillo@54 157 .], Part II]
meillo@54 158 Rose and Romine provide a deeper and more technical
meillo@159 159 though slightly outdated introduction in only about two dozen pages.
meillo@54 160 .[
meillo@54 161 rose romine real work
meillo@54 162 .]
meillo@27 163 .P
meillo@53 164 Following is an example mail handling session.
meillo@53 165 It uses mmh but is mostly compatible with nmh and old MH.
meillo@106 166 Details might vary but the look and feel is the same.
meillo@82 167
meillo@83 168 .VF input/mh-session
meillo@27 169
meillo@27 170
meillo@131 171 .H1 "nmh
meillo@2 172 .P
meillo@49 173 In order to understand the condition, goals and dynamics of a project,
meillo@106 174 one needs to know the reasons behind them.
meillo@53 175 This section explains the background.
meillo@53 176 .P
meillo@106 177 MH predates the Internet; it comes from times before networking was universal,
meillo@49 178 it comes from times when emailing was small, short and simple.
meillo@106 179 Then it grew, spread and adapted to the changes email went through.
meillo@49 180 Its core-concepts, however, remained the same.
meillo@106 181 During the eighties, students at UCI actively worked on MH.
meillo@164 182 They added new features and optimized the code for the systems
meillo@164 183 popular at that time.
meillo@49 184 All this still was in times before POSIX and ANSI C.
meillo@49 185 As large parts of the code stem from this time, today's nmh source code
meillo@49 186 still contains many ancient parts.
meillo@51 187 BSD-specific code and constructs tailored for hardware of that time
meillo@49 188 are frequent.
meillo@2 189 .P
meillo@106 190 Nmh started about a decade after the POSIX and ANSI C standards were
meillo@49 191 established. A more modern coding style entered the code base, but still
meillo@49 192 a part of the developers came from ``the old days''. The developer
meillo@106 193 base became more diverse, thus broadening the range of different
meillo@106 194 coding styles.
meillo@49 195 Programming practices from different decades merged in the project.
meillo@51 196 As several peers added code, the system became more a conglomeration
meillo@51 197 of single tools rather than a homogeneous of-one-cast mail system.
meillo@49 198 Still, the existing basic concepts held it together.
meillo@8 199 They were mostly untouched throughout the years.
meillo@8 200 .P
meillo@106 201 Despite the separation of the tool chest approach at the surface
meillo@106 202 \(en a collection of small, separate programs \(en
meillo@171 203 on the source code level, it is much more interwoven.
meillo@49 204 Several separate components were compiled into one program
meillo@51 205 for efficiency reasons.
meillo@106 206 This led to intricate innards.
meillo@106 207 While clearly separated on the outside,
meillo@171 208 the programs turned out to be fairly interwoven inside.
meillo@106 209 .\" XXX FIXME rewrite...
meillo@171 210 .\" nicht zweimal ``interwoven''
meillo@106 211 .\" Unfortunately, the clear separation on the outside turned out to be
meillo@171 212 .\" fairly interwoven inside.
meillo@8 213 .P
meillo@106 214 The advent of MIME raised the complexity of email by a magnitude.
meillo@49 215 This is visible in nmh. The MIME-related parts are the most complex ones.
meillo@106 216 It is also visible that MIME support was added on top of the old MH core.
meillo@51 217 MH's tool chest style made this easily possible and encourages
meillo@106 218 such approaches, but unfortunately, it led to duplicated functions
meillo@49 219 and half-hearted implementation of the concepts.
meillo@49 220 .P
meillo@159 221 To provide backward-compatibility, it is a common understanding not to
meillo@49 222 change the default settings.
meillo@51 223 In consequence, the user needs to activate modern features explicitly
meillo@47 224 to be able to use them.
meillo@49 225 This puts a burden on new users, because out-of-the-box nmh remains
meillo@49 226 in the same ancient style.
meillo@171 227 If nmh is seen to be a back-end,
meillo@171 228 then this compatibility surely is important.
meillo@171 229 However, at the same time, new users have difficulties using nmh for
meillo@171 230 modern emailing.
meillo@173 231 The small but mature community around nmh needs little change
meillo@106 232 as they have had their convenient setups for decades.
meillo@159 233 .\" XXX Explain more
meillo@8 234
meillo@8 235
meillo@27 236 .H1 "mmh
meillo@28 237 .P
meillo@164 238 .\" XXX which version did I ``fork''?
meillo@49 239 I started to work on my experimental version in October 2011,
meillo@106 240 at a time when there had been no more than three commits to nmh
meillo@53 241 since the beginning of the year.
meillo@53 242 In December, when I announced my work in progress on the
meillo@53 243 nmh-workers mailing list,
meillo@42 244 .[
meillo@51 245 nmh-workers mmh announce December
meillo@42 246 .]
meillo@53 247 nmh's community became active, too.
meillo@49 248 This movement was heavily pushed by Paul Vixie's ``edginess'' comment.
meillo@42 249 .[
meillo@42 250 nmh-workers vixie edginess
meillo@42 251 .]
meillo@53 252 After long years of stagnation, nmh became actively developed again.
meillo@106 253 Hence, while I was working on mmh, the community was once more working
meillo@106 254 on nmh, in parallel.
meillo@164 255 .\" XXX interaction between nmh and mmh
meillo@28 256 .P
meillo@53 257 The name \fImmh\fP may stand for \fImodern mail handler\fP,
meillo@53 258 because the project tries to modernize nmh.
meillo@53 259 Personally however, I prefer to call mmh \fImeillo's mail handler\fP,
meillo@53 260 emphasizing that the project follows my visions and preferences.
meillo@42 261 (My login name is \fImeillo\fP.)
meillo@53 262 This project model was inspired by \fIdwm\fP,
meillo@159 263 .\" XXX Ref
meillo@42 264 which is Anselm Garbe's personal window manager \(en
meillo@42 265 targeted to satisfy Garbe's personal needs whenever conflicts appear.
meillo@53 266 Dwm had retained its lean elegance and its focused character, whereas
meillo@53 267 its community-driven predecessor \fIwmii\fP had grown fat over time.
meillo@159 268 .\" XXX ref
meillo@53 269 The development of mmh should remain focused.
meillo@27 270
meillo@45 271
meillo@27 272 .U2 "Motivation
meillo@27 273 .P
meillo@51 274 MH is the most important of very few command line tool chest email systems.
meillo@51 275 Tool chests are powerful because they can be perfectly automated and
meillo@53 276 extended. They allow arbitrary kinds of front-ends to be
meillo@53 277 implemented on top of them quickly and without internal knowledge.
meillo@106 278 Additionally, tool chests are easier to maintain than monolithic
meillo@43 279 programs.
meillo@53 280 As there are few tool chests for emailing and as MH-like ones are the most
meillo@106 281 popular among them, they should be developed further.
meillo@53 282 This keeps their
meillo@43 283 conceptional elegance and unique scripting qualities available to users.
meillo@106 284 Mmh creates a modern and convenient entry point to MH-like systems
meillo@53 285 for new and interested users.
meillo@43 286 .P
meillo@51 287 The mmh project is motivated by deficits of nmh and
meillo@45 288 my wish for general changes, combined
meillo@45 289 with the nmh community's reluctancy to change.
meillo@45 290 .P
meillo@106 291 At that time, nmh had not adjusted to modern emailing needs well enough.
meillo@45 292 The default setup was completely unusable for modern emailing.
meillo@45 293 Too much setup work was required.
meillo@45 294 Several modern features were already available but the community
meillo@106 295 did not want to have them as default.
meillo@106 296 Mmh is a way to change this.
meillo@45 297 .P
meillo@45 298 In my eyes, MH's concepts could be exploited even better and
meillo@45 299 the style of the tools could be improved. Both would simplify
meillo@45 300 and generalize the system, providing cleaner interfaces and more
meillo@53 301 software leverage at the same time.
meillo@106 302 Mmh is a way to demonstrate this.
meillo@45 303 .P
meillo@45 304 In providing several parts of an email system, nmh can hardly
meillo@45 305 compete with the large specialized projects that focus
meillo@45 306 on only one of the components.
meillo@45 307 The situation can be improved by concentrating the development power
meillo@51 308 on the most unique part of MH and letting the user pick his preferred
meillo@45 309 set of other mail components.
meillo@45 310 Today's pre-packaged software components encourage this model.
meillo@106 311 Mmh is a way to go for this approach.
meillo@45 312 .P
meillo@106 313 It is worthwhile to fork nmh for the development of mmh, because
meillo@43 314 the two projects focus on different goals and differ in fundamental questions.
meillo@106 315 The nmh community's reluctance regarding change conflicts
meillo@106 316 with my strong desire for it.
meillo@43 317 In developing a separate experimental version new approaches can
meillo@43 318 easily be tried out without the need to discuss changes beforehand.
meillo@43 319 In fact, revolutionary changes are hardly possible otherwise.
meillo@43 320 .P
meillo@117 321 The mmh project provides the basis on which the aforementioned
meillo@117 322 ideas can be implemented and demonstrated,
meillo@164 323 without the need to change the nmh project or its community.
meillo@43 324 Of course, the results of the mmh project shall improve nmh, in the end.
meillo@159 325 By no means it is my intent to work against the nmh project.
meillo@117 326
meillo@27 327
meillo@27 328 .U2 "Target Field
meillo@27 329 .P
meillo@45 330 Any effort needs to be targeted towards a specific goal
meillo@45 331 in order to be successful.
meillo@117 332 Following is a description of imagined typical mmh users.
meillo@117 333 Mmh should satisfy their needs.
meillo@53 334 .\" XXX Remove the next sentence?
meillo@48 335 Actually, as mmh is my personal version of MH, this is a description
meillo@48 336 of myself.
meillo@45 337 .P
meillo@117 338 The target users of mmh like Unix and its philosophy.
meillo@117 339 They appreciate to use programs that are conceptionally appealing.
meillo@117 340 They are familiar with the command line and enjoy its power.
meillo@117 341 They are at least capable of shell scripting and want to improve their
meillo@27 342 productivity by scripting the mail system.
meillo@169 343 They use modern email features, such as attachments,
meillo@169 344 non-ASCII text, digital signatures and message encryption in a natural way.
meillo@117 345 They are able to setup email system components besides mmh,
meillo@117 346 and actually like to have the choice to pick the ones they prefer.
meillo@164 347 They have a reasonably modern operating system that complies to the
meillo@164 348 POSIX and ANSI C standards.
meillo@27 349 .P
meillo@117 350 The typical users invoke mmh commands directly in an interactive
meillo@117 351 shell session, but they use them to automate mail handling tasks as well.
meillo@159 352 Likely, they run their mail setup on a server machine,
meillo@117 353 to which they connect via ssh.
meillo@117 354 They might also have local mmh installations on their workstations,
meillo@117 355 where they tend to work with mmh directly in the shell instead
meillo@117 356 of using graphical front-ends.
meillo@117 357 They definitely want to be flexible and thus be able to change
meillo@117 358 their setup to suit their needs.
meillo@8 359 .P
meillo@117 360 .\" XXX themself vs. themselves
meillo@117 361 Typical mmh users are programmers themselves.
meillo@117 362 They like to, occasionally, take the opportunity of Free Software to put
meillo@117 363 hands on and get involved in the software they use.
meillo@117 364 Hence, they like small and clean code bases and care for code quality.
meillo@117 365 In general, they believe that:
meillo@8 366 .BU
meillo@48 367 Elegance \(en i.e. simplicity, clarity and generality \(en
meillo@48 368 is most important.
meillo@8 369 .BU
meillo@48 370 Concepts are more important than the concrete implementation.
meillo@8 371 .BU
meillo@48 372 Code optimizations for anything but readability should be avoided
meillo@48 373 if possible.
meillo@8 374 .BU
meillo@45 375 Having a lot of choice is bad.
meillo@48 376 .BU
meillo@48 377 Removed code is debugged code.
meillo@8 378
meillo@48 379 .U2 "Goals
meillo@45 380 .P
meillo@45 381 The general goals for the mmh project are the following:
meillo@128 382 .IP "Streamlining
meillo@87 383 Mmh should be stripped down to its core, which is the user agent (MUA).
meillo@117 384 The feature set should be distilled to the indispensable ones,
meillo@171 385 effectively removing corner cases.
meillo@173 386 Parts that do not add to the main task of being a conceptionally
meillo@53 387 appealing MUA should be removed.
meillo@117 388 This includes the mail submission and mail retrieval facilities.
meillo@48 389 Choice should be reduced to the main options.
meillo@131 390 All tools should be tightly shaped.
meillo@48 391 .IP "Modernizing
meillo@48 392 Mmh's feature set needs to become more modern.
meillo@164 393 Better support for attachments, digital signatures and message encryption
meillo@164 394 should be added.
meillo@159 395 MIME support should be integrated deeper and more naturally.
meillo@48 396 The modern email features need to be readily available, out-of-the-box.
meillo@117 397 On the other hand,
meillo@117 398 bulletin board support and similar obsolete facilities can be dropped out.
meillo@131 399 Likewise, ancient technologies should not be supported any further.
meillo@131 400 The available concepts need to be expanded as far as possible.
meillo@131 401 A small set of concepts should recur consistently.
meillo@131 402 .IP "Styling
meillo@48 403 Mmh's source code needs to be updated to modern standards.
meillo@48 404 Standardized library functions should replace non-standard versions
meillo@48 405 whenever possible.
meillo@117 406 Code should be separated into distinct modules when feasible.
meillo@48 407 Time and space optimizations should to be replaced by
meillo@48 408 clear and readable code.
meillo@48 409 A uniform programming style should prevail.
meillo@117 410 The whole system should appear to be of-one-style;
meillo@117 411 it should feel like being cast as one.