docs/unix-phil
view unix-phil.ms @ 39:c87143793d82
a lot of rework in ch02
| author | meillo@marmaro.de |
|---|---|
| date | Thu, 08 Apr 2010 16:47:11 +0200 |
| parents | 3628e9649046 |
| children | 422679bdf384 |
line source
1 .\".if n .pl 1000i
2 .nr PS 11
3 .nr VS 13
4 .de XX
5 .pl 1v
6 ..
7 .em XX
8 .\".nr PI 0
9 .\".if t .nr PD .5v
10 .\".if n .nr PD 1v
11 .nr lu 0
12 .de CW
13 .nr PQ \\n(.f
14 .if t .ft CW
15 .ie ^\\$1^^ .if n .ul 999
16 .el .if n .ul 1
17 .if t .if !^\\$1^^ \&\\$1\f\\n(PQ\\$2
18 .if n .if \\n(.$=1 \&\\$1
19 .if n .if \\n(.$>1 \&\\$1\c
20 .if n .if \\n(.$>1 \&\\$2
21 ..
22 .ds [. \ [
23 .ds .] ]
24 .\"----------------------------------------
25 .TL
26 Why the Unix Philosophy still matters
27 .AU
28 markus schnalke <meillo@marmaro.de>
29 .AB
30 .ti \n(.iu
31 This paper explains the importance of the Unix Philosophy for software design.
32 Today, few software designers are aware of these concepts,
33 and thus a lot of modern software is more limited than necessary
34 and makes less use of software leverage than possible.
35 Knowing and following the guidelines of the Unix Philosophy makes software more valuable.
36 .AE
38 .\".if t .2C
40 .FS
41 .ps -1
42 This paper was prepared for the ``Software Analysis'' seminar at University Ulm.
43 Mentor was professor Schweiggert. 2010-04-05
44 .br
45 You may retrieve this document from
46 .CW \s-1http://marmaro.de/docs \ .
47 .FE
49 .NH 1
50 Introduction
51 .LP
52 Building a software is a process from an idea of the purpose of the software
53 to its release.
54 No matter \fIhow\fP the process is run, two things are common:
55 the initial idea and the release.
56 The process in between can be of any shape.
57 The the maintenance work after the release is ignored for the moment.
58 .PP
59 The process of building splits mainly in two parts:
60 the planning of what and how to build, and implementing the plan by writing code.
61 This paper focuses on the planning part \(en the designing of the software.
62 .PP
63 Software design is the plan of how the internals and externals of the software should look like,
64 based on the requirements.
65 This paper discusses the recommendations of the Unix Philosophy about software design.
66 .PP
67 The here discussed ideas can get applied by any development process.
68 The Unix Philosophy does recommend how the software development process should look like,
69 but this shall not be of matter here.
70 Similar, the question of how to write the code is out of focus.
71 .PP
72 The name ``Unix Philosophy'' was already mentioned several times, but it was not explained yet.
73 The Unix Philosophy is the essence of how the Unix operating system and its toolchest was designed.
74 It is no limited set of rules, but what people see to be common to typical Unix software.
75 Several people stated their view on the Unix Philosophy.
76 Best known are:
77 .IP \(bu
78 Doug McIlroy's summary: ``Write programs that do one thing and do it well.''
79 .[
80 %A M. D. McIlroy
81 %A E. N. Pinson
82 %A B. A. Taque
83 %T UNIX Time-Sharing System Forward
84 %J The Bell System Technical Journal
85 %D 1978
86 %V 57
87 %N 6
88 %P 1902
89 .]
90 .IP \(bu
91 Mike Gancarz' book ``The UNIX Philosophy''.
92 .[
93 %A Mike Gancarz
94 %T The UNIX Philosophy
95 %D 1995
96 %I Digital Press
97 .]
98 .IP \(bu
99 Eric S. Raymond's book ``The Art of UNIX Programming''.
100 .[
101 %A Eric S. Raymond
102 %T The Art of UNIX Programming
103 %D 2003
104 %I Addison-Wesley
105 %O .CW \s-1http://www.faqs.org/docs/artu/
106 .]
107 .LP
108 These different views on the Unix Philosophy have much in common.
109 Especially, the main concepts are similar for all of them.
110 But there are also points on which they differ.
111 This only underlines what the Unix Philosophy is:
112 A retrospective view on the main concepts of Unix software;
113 especially those that were successful and unique to Unix.
114 .\" really?
115 .PP
116 Before we will have a look at concrete concepts,
117 we discuss why software design is important
118 and what problems bad design introduces.
121 .NH 1
122 Importance of software design in general
123 .LP
124 The design of a software describes its internal and external shape,
125 meaning structure and interfaces.
126 It has nothing to do with visual appearance.
127 If we take a program as a car, then its color is of no matter.
128 Its design would be the car's size, its shape, the locations of doors,
129 the passenger/space ratio, the luggage capacity, and so forth.
130 .PP
131 Why should software get designed at all?
132 It is general knowledge, that even a bad plan is better than no plan.
133 Not designing software means programming without plan.
134 This will pretty sure lead to horrible results.
135 Horrible to use and horrible to maintain.
136 These two aspects are the visible ones.
137 Often invisible are the wasted possible gains.
138 Good software design can make these gains available.
139 .PP
140 A software's design deals with quality properties.
141 Good design leads to good quality, and quality is important.
142 Any car may be able to drive from A to B,
143 but it depends on the car's properties whether it is a good choice
144 for passenger transport or not.
145 It depends on its properties if it is a good choice
146 for a rough mountain area.
147 And it depends on its properties if the ride will be fun.
149 .PP
150 Requirements for a software are twofold:
151 functional and non-functional.
152 .IP \(bu
153 Functional requirements define directly the software's functions.
154 They are the reason why software gets written.
155 Someone has a problem and needs a tool to solve it.
156 Being able to solve the problem is the main functional goal.
157 It is the driving force behind all programming effort.
158 Functional requirements are easier to define and to verify.
159 .IP \(bu
160 Non-functional requirements are also called \fIquality\fP requirements.
161 The quality of a software are the properties that are not directly related to
162 the software's basic functions.
163 Tools of bad quality often solve the problems they were written for,
164 but introduce problems and difficulties for usage and development, later on.
165 Quality aspects are often overlooked at first sight,
166 and they are often difficult to define clearly and to verify.
167 .PP
168 Quality is of few matter when the software gets built initially,
169 but it is of matter for usage and maintenance of the software.
170 A short-sighted might see in developing a software mainly building something up.
171 But experience shows, that building the software the first time is
172 only a small amount of the overall work.
173 Bug fixing, extending, rebuilding of parts
174 \(en maintenance work, for short \(en
175 does soon take over the major part of the time spent on a software.
176 Not to forget the usage of the software.
177 These processes are highly influenced by the software's quality.
178 Thus, quality must not be neglected.
179 The problem with quality is that you hardly ``stumble over''
180 bad quality during the first build,
181 but this is the time when you should care about good quality most.
182 .PP
183 Software design is less the basic function of a software \(en
184 this requirement will get satisfied anyway.
185 Software design is more about quality aspects of the software.
186 Good design leads to good quality, bad design to bad quality.
187 The primary functions of the software will be affected modestly by bad quality,
188 but good quality can provide a lot of additional gain,
189 even at places where one never expected it.
190 .PP
191 The ISO/IEC 9126-1 standard, part 1,
192 .[
193 %I International Organization for Standardization
194 %T ISO Standard 9126: Software Engineering \(en Product Quality, part 1
195 %C Geneve
196 %D 2001
197 .]
198 defines the quality model as consisting out of:
199 .IP \(bu
200 .I Functionality
201 (suitability, accuracy, inter\%operability, security)
202 .IP \(bu
203 .I Reliability
204 (maturity, fault tolerance, recoverability)
205 .IP \(bu
206 .I Usability
207 (understandability, learnability, operability, attractiveness)
208 .IP \(bu
209 .I Efficiency
210 (time behavior, resource utilization)
211 .IP \(bu
212 .I Maintainability
213 (analyzability, changeability, stability, testability)
214 .IP \(bu
215 .I Portability
216 (adaptability, installability, co-existence, replaceability)
217 .LP
218 Good design can improve these properties of a software,
219 bad designed software probably suffers from not having them.
220 .PP
221 One further goal of software design is consistency.
222 Consistency eases understanding, working on, and using things.
223 Consistent internal structure and consistent interfaces to the outside
224 can be provided by good design.
225 .PP
226 Software should be well designed because good design avoids many
227 problems during the software's lifetime.
228 And software should be well designed because good design can offer
229 much additional gain.
230 Indeed, much effort should be spent into good design to make software more valuable.
231 The Unix Philosophy shows a way of how to design software well.
232 It offers guidelines to achieve good quality and high gain for the effort spent.
235 .NH 1
236 The Unix Philosophy
237 .LP
238 The origins of the Unix Philosophy were already introduced.
239 This chapter explains the philosophy, oriented on Gancarz,
240 and shows concrete examples of its application.
242 .NH 2
243 Pipes
244 .LP
245 Following are some examples to demonstrate how applied Unix Philosophy feels like.
246 Knowledge of using the Unix shell is assumed.
247 .PP
248 Counting the number of files in the current directory:
249 .DS I 2n
250 .CW
251 .ps -1
252 ls | wc -l
253 .DE
254 The
255 .CW ls
256 command lists all files in the current directory, one per line,
257 and
258 .CW "wc -l
259 counts the number of lines.
260 .PP
261 Counting the number of files that do not contain ``foo'' in their name:
262 .DS I 2n
263 .CW
264 .ps -1
265 ls | grep -v foo | wc -l
266 .DE
267 Here, the list of files is filtered by
268 .CW grep
269 to remove all that contain ``foo''.
270 The rest is the same as in the previous example.
271 .PP
272 Finding the five largest entries in the current directory.
273 .DS I 2n
274 .CW
275 .ps -1
276 du -s * | sort -nr | sed 5q
277 .DE
278 .CW "du -s *
279 returns the recursively summed sizes of all files
280 \(en no matter if they are regular files or directories.
281 .CW "sort -nr
282 sorts the list numerically in reverse order.
283 Finally,
284 .CW "sed 5q
285 quits after it has printed the fifth line.
286 .PP
287 The presented command lines are examples of what Unix people would use
288 to get the desired output.
289 There are also other ways to get the same output.
290 It's a user's decision which way to go.
291 .PP
292 The examples show that many tasks on a Unix system
293 are accomplished by combining several small programs.
294 The connection between the single programs is denoted by the pipe operator `|'.
295 .PP
296 Pipes, and their extensive and easy use, are one of the great
297 achievements of the Unix system.
298 Pipes between programs have been possible in earlier operating systems,
299 but it has never been a so central part of the concept.
300 When, in the early seventies, Doug McIlroy introduced pipes for the
301 Unix system,
302 ``it was this concept and notation for linking several programs together
303 that transformed Unix from a basic file-sharing system to an entirely new way of computing.''
304 .[
305 %T Unix: An Oral History
306 %O .CW \s-1http://www.princeton.edu/~hos/frs122/unixhist/finalhis.htm
307 .]
308 .PP
309 Being able to specify pipelines in an easy way is,
310 however, not enough by itself.
311 It is only one half.
312 The other is the design of the programs that are used in the pipeline.
313 They have to interfaces that allows them to be used in such a way.
315 .NH 2
316 Interface design
317 .LP
318 Unix is, first of all, simple \(en Everything is a file.
319 Files are sequences of bytes, without any special structure.
320 Programs should be filters, which read a stream of bytes from ``standard input'' (stdin)
321 and write a stream of bytes to ``standard output'' (stdout).
322 .PP
323 If the files \fIare\fP sequences of bytes,
324 and the programs \fIare\fP filters on byte streams,
325 then there is exactly one standardized data interface.
326 Thus it is possible to combine them in any desired way.
327 .PP
328 Even a handful of small programs will yield a large set of combinations,
329 and thus a large set of different functions.
330 This is leverage!
331 If the programs are orthogonal to each other \(en the best case \(en
332 then the set of different functions is greatest.
333 .PP
334 Programs might also have a separate control interface,
335 besides their data interface.
336 The control interface is often called ``user interface'',
337 because it is usually designed to be used by humans.
338 The Unix Philosophy discourages to assume the user to be human.
339 Interactive use of software is slow use of software,
340 because the program waits for user input most of the time.
341 Interactive software requires the user to be in front of the computer
342 all the time.
343 Interactive software occupy the user's attention while they are running.
344 .PP
345 Now we come back to the idea of using several small programs, combined,
346 to have a more specific function.
347 If these single tools would all be interactive,
348 how would the user control them?
349 It is not only a problem to control several programs at once if they run at the same time,
350 it also very inefficient to have to control each of the single programs
351 that are intended to work as one large program.
352 Hence, the Unix Philosophy discourages programs to demand interactive use.
353 The behavior of programs should be defined at invocation.
354 This is done by specifying arguments (``command line switches'') to the program call.
355 Gancarz discusses this topic as ``avoid captive user interfaces''.
356 .[
357 %A Mike Gancarz
358 %T The UNIX Philosophy
359 %I Digital Press
360 %D 1995
361 %P 88 ff.
362 .]
363 .PP
364 Non-interactive use is, during development, also an advantage for testing.
365 Testing of interactive programs is much more complicated,
366 than testing of non-interactive programs.
368 .NH 2
369 The toolchest approach
370 .LP
371 A toolchest is a set of tools.
372 Instead of having one big tool for all tasks, one has many small tools,
373 each for one task.
374 Difficult tasks are solved by combining several of the small, simple tools.
375 .PP
376 The Unix toolchest \fIis\fP a set of small, (mostly) non-interactive programs
377 that are filters on byte streams.
378 They are, to a large extend, unrelated in their function.
379 Hence, the Unix toolchest provides a large set of functions
380 that can be accessed by combining the programs in the desired way.
381 .PP
382 There are also advantages for developing small toolchest programs.
383 It is easier and less error-prone to write small programs.
384 It is also easier and less error-prone to write a large set of small programs,
385 than to write one large program with all the functionality included.
386 If the small programs are combinable, then they offer even a larger set
387 of functions than the single large program.
388 Hence, one gets two advantages out of writing small, combinable programs.
389 .PP
390 There are two drawbacks of the toolchest approach.
391 First, one simple, standardized, unidirectional interface has to be sufficient.
392 If one feels the need for more ``logic'' than a stream of bytes,
393 then a different approach might be of need.
394 But it is also possible, that he just can not imagine a design where
395 a stream of bytes is sufficient.
396 By becoming more familiar with the ``Unix style of thinking'',
397 developers will more often and easier find simple designs where
398 a stream of bytes is a sufficient interface.
399 .PP
400 The second drawback of a toolchest affects the users.
401 A toolchest is often more difficult to use for novices.
402 It is necessary to become familiar with each of the tools,
403 to be able to use the right one in a given situation.
404 Additionally, one needs to combine the tools in a senseful way on its own.
405 This is like a sharp knife \(en it is a powerful tool in the hand of a master,
406 but of no good value in the hand of an unskilled.
407 .PP
408 However, learning single, small tool of the toolchest is easier than
409 learning a complex tool.
410 The user will have a basic understanding of a yet unknown tool,
411 if the several tools of the toolchest have a common style.
412 He will be able to transfer knowledge over one tool to another.
413 .PP
414 Moreover, the second drawback can be removed easily by adding wrappers
415 around the single tools.
416 Novice users do not need to learn several tools if a professional wraps
417 the single commands into a more high-level script.
418 Note that the wrapper script still calls the small tools;
419 the wrapper script is just like a skin around.
420 No complexity is added this way,
421 but new programs can get created out of existing one with very low effort.
422 .PP
423 A wrapper script for finding the five largest entries in the current directory
424 could look like this:
425 .DS I 2n
426 .CW
427 .ps -1
428 #!/bin/sh
429 du -s * | sort -nr | sed 5q
430 .DE
431 The script itself is just a text file that calls the command line
432 a professional user would type in directly.
433 Making the program flexible on the number of entries it prints,
434 is easily possible:
435 .DS I 2n
436 .CW
437 .ps -1
438 #!/bin/sh
439 num=5
440 [ $# -eq 1 ] && num="$1"
441 du -sh * | sort -nr | sed "${num}q"
442 .DE
443 This script acts like the one before, when called without an argument.
444 But one can also specify a numerical argument to define the number of lines to print.
446 .NH 2
447 A powerful shell
448 .LP
449 It was already said, that the Unix shell provides the possibility to
450 combine small programs into large ones easily.
451 A powerful shell is a great feature in other ways, too.
452 .PP
453 For instance by including a scripting language.
454 The control statements are build into the shell.
455 The functions, however, are the normal programs, everyone can use on the system.
456 Thus, the programs are known, so learning to program in the shell is easy.
457 Using normal programs as functions in the shell programming language
458 is only possible because they are small and combinable tools in a toolchest style.
459 .PP
460 The Unix shell encourages to write small scripts out of other programs,
461 because it is so easy to do.
462 This is a great step towards automation.
463 It is wonderful if the effort to automate a task equals the effort
464 it takes to do it the second time by hand.
465 If it is so, then the user will be happy to automate everything he does more than once.
466 .PP
467 Small programs that do one job well, standardized interfaces between them,
468 a mechanism to combine parts to larger parts, and an easy way to automate tasks,
469 this will inevitably produce software leverage.
470 Getting multiple times the benefit of an investment is a great offer.
471 .PP
472 The shell also encourages rapid prototyping.
473 Many well known programs started as quickly hacked shell scripts,
474 and turned into ``real'' programs, written in C, later.
475 Building a prototype first is a way to avoid the biggest problems
476 in application development.
477 Fred Brooks writes in ``No Silver Bullet'':
478 .[
479 %A Frederick P. Brooks, Jr.
480 %T No Silver Bullet: Essence and Accidents of Software Engineering
481 %B Information Processing 1986, the Proceedings of the IFIP Tenth World Computing Conference
482 %E H.-J. Kugler
483 %D 1986
484 %P 1069\(en1076
485 %I Elsevier Science B.V.
486 %C Amsterdam, The Netherlands
487 .]
488 .QP
489 The hardest single part of building a software system is deciding precisely what to build.
490 No other part of the conceptual work is so difficult as establishing the detailed
491 technical requirements, [...].
492 No other part of the work so cripples the resulting system if done wrong.
493 No other part is more difficult to rectify later.
494 .PP
495 Writing a prototype is a great method to become familiar with the requirements
496 and to actually run into real problems.
497 Today, prototyping is often seen as a first step in building a software.
498 This is, of course, good.
499 However, the Unix Philosophy has an \fIadditional\fP perspective on prototyping:
500 After having built the prototype, one might notice, that the prototype is already
501 \fIgood enough\fP.
502 Hence, no reimplementation, in a more sophisticated programming language, might be of need,
503 for the moment.
504 Maybe later, it might be necessary to rewrite the software, but not now.
505 .PP
506 By delaying further work, one keeps the flexibility to react easily on
507 changing requirements.
508 Software parts that are not written will not miss the requirements.
510 .NH 2
511 Worse is better
512 .LP
513 The Unix Philosophy aims for the 80% solution;
514 others call it the ``Worse is better'' approach.
515 .PP
516 First, practical experience shows, that it is almost never possible to define the
517 requirements completely and correctly the first time.
518 Hence one should not try to; it will fail anyway.
519 Second, practical experience shows, that requirements change during time.
520 Hence it is best to delay requirement-based design decisions as long as possible.
521 Also, the software should be small and flexible as long as possible
522 to react on changing requirements.
523 Shell scripts, for example, are more easily adjusted as C programs.
524 Third, practical experience shows, that maintenance is hard work.
525 Hence, one should keep the amount of software as small as possible;
526 it should just fulfill the \fIcurrent\fP requirements.
527 Software parts that will be written later, do not need maintenance now.
528 .PP
529 Starting with a prototype in a scripting language has several advantages:
530 .IP \(bu
531 As the initial effort is low, one will likely start right away.
532 .IP \(bu
533 As working parts are available soon, the real requirements can get identified soon.
534 .IP \(bu
535 When a software is usable, it gets used, and thus tested.
536 Hence problems will be found at early stages of the development.
537 .IP \(bu
538 The prototype might be enough for the moment,
539 thus further work on the software can be delayed to a time
540 when one knows better about the requirements and problems,
541 than now.
542 .IP \(bu
543 Implementing now only the parts that are actually needed now,
544 requires fewer maintenance work.
545 .IP \(bu
546 If the global situation changes so that the software is not needed anymore,
547 then less effort was spent into the project, than it would have be
548 when a different approach had been used.
550 .NH 2
551 Upgrowth and survival of software
552 .LP
553 So far it was talked about \fIwriting\fP or \fIbuilding\fP software.
554 Although these are just verbs, they do imply a specific view on the work process
555 they describe.
556 The better verb, however, is to \fIgrow\fP.
557 .PP
558 Creating software in the sense of the Unix Philosophy is an incremental process.
559 It starts with a first prototype, which evolves as requirements change.
560 A quickly hacked shell script might become a large, sophisticated,
561 compiled program this way.
562 Its lifetime begins with the initial prototype and ends when the software is not used anymore.
563 While being alive it will get extended, rearranged, rebuilt (from scratch).
564 Growing software matches the view that ``software is never finished. It is only released.''
565 .[
566 %O FIXME
567 %A Mike Gancarz
568 %T The UNIX Philosophy
569 %P 26
570 .]
571 .PP
572 Software can be seen as being controlled by evolutionary processes.
573 Successful software is software that is used by many for a long time.
574 This implies that the software is needed, useful, and better than alternatives.
575 Darwin talks about: ``The survival of the fittest.''
576 .[
577 %O FIXME
578 %A Charles Darwin
579 .]
580 Transferred to software: The most successful software, is the fittest,
581 is the one that survives.
582 (This may be at the level of one creature, or at the level of one species.)
583 The fitness of software is affected mainly by four properties:
584 portability of code, portability of data, range of usability, and reusability of parts.
585 .\" .IP \(bu
586 .\" portability of code
587 .\" .IP \(bu
588 .\" portability of data
589 .\" .IP \(bu
590 .\" range of usability
591 .\" .IP \(bu
592 .\" reuseability of parts
593 .PP
594 (1)
595 .I "Portability of code
596 means, using high-level programming languages,
597 sticking to the standard,
598 and avoiding optimizations that introduce dependencies on specific hardware.
599 Hardware has a much lower lifetime than software.
600 By chaining software to a specific hardware,
601 the software's lifetime gets shortened to that of this hardware.
602 In contrast, software should be easy to port \(en
603 adaptation is the key to success.
604 .\" cf. practice of prog: ch08
605 .PP
606 (2)
607 .I "Portability of data
608 is best achieved by avoiding binary representations
609 to store data, because binary representations differ from machine to machine.
610 Textual representation is favored.
611 Historically, ASCII was the charset of choice.
612 In the future, UTF-8 might be the better choice, however.
613 Important is that it is a plain text representation in a
614 very common charset encoding.
615 Apart from being able to transfer data between machines,
616 readable data has the great advantage, that humans are able
617 to directly edit it with text editors and other tools from the Unix toolchest.
618 .\" gancarz tenet 5
619 .PP
620 (3)
621 A large
622 .I "range of usability
623 ensures good adaptation, and thus good survival.
624 It is a special distinction if a software becomes used in fields of action,
625 the original authors did never imagine.
626 Software that solves problems in a general way will likely be used
627 for all kinds of similar problems.
628 Being too specific limits the range of uses.
629 Requirements change through time, thus use cases change or even vanish.
630 A good example in this point is Allman's sendmail.
631 Allman identifies flexibility to be one major reason for sendmail's success:
632 .[
633 %O FIXME
634 %A Allman
635 %T sendmail
636 .]
637 .QP
638 Second, I limited myself to the routing function [...].
639 This was a departure from the dominant thought of the time, [...].
640 .QP
641 Third, the sendmail configuration file was flexible enough to adopt
642 to a rapidly changing world [...].
643 .LP
644 Successful software adopts itself to the changing world.
645 .PP
646 (4)
647 .I "Reuse of parts
648 is even one step further.
649 A software may completely lose its field of action,
650 but parts of which the software is build may be general and independent enough
651 to survive this death.
652 If software is build by combining small independent programs,
653 then there are parts readily available for reuse.
654 Who cares if the large program is a failure,
655 but parts of it become successful instead?
657 .NH 2
658 Summary
659 .LP
660 This chapter explained the central ideas of the Unix Philosophy.
661 For each of the ideas, it was exposed what advantages they introduce.
662 The Unix Philosophy are guidelines that help to write valuable software.
663 From the view point of a software developer or software designer,
664 the Unix Philosophy provides answers to many software design problem.
665 .PP
666 The various ideas of the Unix Philosophy are very interweaved
667 and can hardly be applied independently.
668 However, the probably most important messages are:
669 .I "``Do one thing well!''" ,
670 .I "``Keep it simple!''" ,
671 and
672 .I "``Use software leverage!''
676 .NH 1
677 Case study: \s-1MH\s0
678 .LP
679 The previous chapter introduced and explained the Unix Philosophy
680 from a general point of view.
681 The driving force were the guidelines; references to
682 existing software were given only sparsely.
683 In this and the next chapter, concrete software will be
684 the driving force in the discussion.
685 .PP
686 This first case study is about the mail user agents (\s-1MUA\s0)
687 \s-1MH\s0 (``mail handler'') and its descendent \fInmh\fP
688 (``new mail handler'').
689 \s-1MUA\s0s provide functions to read, compose, and organize mail,
690 but (ideally) not to transfer.
691 In this document, the name \s-1MH\s0 will be used for both of them.
692 A distinction will only be made if differences between
693 them are described.
696 .NH 2
697 Historical background
698 .LP
699 Electronic mail was available in Unix very early.
700 The first \s-1MUA\s0 on Unix was \f(CWmail\fP,
701 which was already present in the First Edition.
702 .[
703 %A Peter H. Salus
704 %T A Quarter Century of UNIX
705 %D 1994
706 %I Addison-Wesley
707 %P 41 f.
708 .]
709 It was a small program that either prints the user's mailbox file
710 or appends text to someone elses mailbox file,
711 depending on the command line arguments.
712 .[
713 %O http://cm.bell-labs.com/cm/cs/who/dmr/pdfs/man12.pdf
714 .]
715 It was a program that did one job well.
716 This job was emailing, which was very simple then.
717 .PP
718 Later, emailing became more powerful, and thus more complex.
719 The simple \f(CWmail\fP, which knew nothing of subjects,
720 independent handling of single messages,
721 and long-time storage of them, was not powerful enough anymore.
722 At Berkeley, Kurt Shoens wrote \fIMail\fP (with capital `M')
723 in 1978 to provide additional functions for emailing.
724 Mail was still one program, but now it was large and did
725 several jobs.
726 Its user interface is modeled after the one of \fIed\fP.
727 It is designed for humans, but is still scriptable.
728 \fImailx\fP is the adaptation of Berkeley Mail into System V.
729 .[
730 %A Gunnar Ritter
731 %O http://heirloom.sourceforge.net/mailx_history.html
732 .]
733 Elm, pine, mutt, and a whole bunch of graphical \s-1MUA\s0s
734 followed Mail's direction.
735 They are large, monolithic programs which include all emailing functions.
736 .PP
737 A different way was taken by the people of \s-1RAND\s0 Corporation.
738 In the beginning, they also had used a monolithic mail system,
739 called \s-1MS\s0 (for ``mail system'').
740 But in 1977, Stockton Gaines and Norman Shapiro
741 came up with a proposal of a new email system concept \(en
742 one that honors the Unix Philosophy.
743 The concept was implemented by Bruce Borden in 1978 and 1979.
744 This was the birth of \s-1MH\s0 \(en the ``mail handler''.
745 .PP
746 Since then, \s-1RAND\s0, the University of California at Irvine and
747 at Berkeley, and several others have contributed to the software.
748 However, it's core concepts remained the same.
749 In the late 90s, when development of \s-1MH\s0 slowed down,
750 Richard Coleman started with \fInmh\fP, the new mail handler.
751 His goal was to improve \s-1MH\s0, especially in regard of
752 the requirements of modern emailing.
753 Today, nmh is developed by various people on the Internet.
754 .[
755 %T RAND and the Information Evolution: A History in Essays and Vignettes
756 %A Willis H. Ware
757 %D 2008
758 %I The RAND Corporation
759 %P 128\(en137
760 %O .CW \s-1http://www.rand.org/pubs/corporate_pubs/CP537/
761 .]
762 .[
763 %T MH & xmh: Email for Users & Programmers
764 %A Jerry Peek
765 %D 1991, 1992, 1995
766 %I O'Reilly & Associates, Inc.
767 %P Appendix B
768 %O Also available online: \f(CW\s-2http://rand-mh.sourceforge.net/book/\fP
769 .]
771 .NH 2
772 Contrasts to monolithic mail systems
773 .LP
774 All \s-1MUA\s0s are monolithic, except \s-1MH\s0.
775 Although there might actually exist further, very little known,
776 toolchest \s-1MUA\s0s, this statement reflects the situation pretty well.
777 .PP
778 Monolithic \s-1MUA\s0s gather all their functions in one program.
779 In contrast, \s-1MH\s0 is a toolchest of many small tools \(en one for each job.
780 Following is a list of important programs of \s-1MH\s0's toolchest
781 and their function.
782 It gives a feeling of how the toolchest looks like.
783 .IP \(bu
784 .CW inc :
785 incorporate new mail (this is how mail enters the system)
786 .IP \(bu
787 .CW scan :
788 list messages in folder
789 .IP \(bu
790 .CW show :
791 show message
792 .IP \(bu
793 .CW next\fR/\fPprev :
794 show next/previous message
795 .IP \(bu
796 .CW folder :
797 change current folder
798 .IP \(bu
799 .CW refile :
800 refile message into folder
801 .IP \(bu
802 .CW rmm :
803 remove message
804 .IP \(bu
805 .CW comp :
806 compose a new message
807 .IP \(bu
808 .CW repl :
809 reply to a message
810 .IP \(bu
811 .CW forw :
812 forward a message
813 .IP \(bu
814 .CW send :
815 send a prepared message (this is how mail leaves the system)
816 .LP
817 \s-1MH\s0 has no special user interface like monolithic \s-1MUA\s0s have.
818 The user does not leave the shell to run \s-1MH\s0,
819 but he uses the various \s-1MH\s0 programs within the shell.
820 Using a monolithic program with a captive user interface
821 means ``entering'' the program, using it, and ``exiting'' the program.
822 Using toolchests like \s-1MH\s0 means running programs,
823 alone or in combination with others, even from other toolchests,
824 without leaving the shell.
826 .NH 2
827 Data storage
828 .LP
829 \s-1MH\s0's mail storage is a directory tree under the user's
830 \s-1MH\s0 directory (usually \f(CW$HOME/Mail\fP),
831 where mail folders are directories and mail messages are text files
832 within them.
833 Each mail folder contains a file \f(CW.mh_sequences\fP which lists
834 the public message sequences of that folder, for instance new messages.
835 Mail messages are text files located in a mail folder.
836 The files contain the messages as they were received.
837 They are numbered in ascending order in each folder.
838 .PP
839 This mailbox format is called ``\s-1MH\s0'' after the \s-1MUA\s0.
840 Alternatives are \fImbox\fP and \fImaildir\fP.
841 In the mbox format all messages are stored within one file.
842 This was a good solution in the early days, when messages
843 were only a few lines of text and were deleted soon.
844 Today, when single messages often include several megabytes
845 of attachments, it is a bad solution.
846 Another disadvantage of the mbox format is that it is
847 more difficult to write tools that work on mail messages,
848 because it is always necessary to first find and extract
849 the relevant message in the mbox file.
850 With the \s-1MH\s0 mailbox format,
851 each message is a self-standing item, by definition.
852 Also, the problem of concurrent access to one mailbox is
853 reduced to the problem of concurrent access to one message.
854 Maildir is generally similar to \s-1MH\s0's format,
855 but modified towards guaranteed reliability.
856 This involves some complexity, unfortunately.
857 .PP
858 Working with \s-1MH\s0's toolchest on mailboxes is much like
859 working with Unix' toolchest on directory trees:
860 \f(CWscan\fP is like \f(CWls\fP,
861 \f(CWshow\fP is like \f(CWcat\fP,
862 \f(CWfolder\fP is like \f(CWcd\fP and \f(CWpwd\fP,
863 \f(CWrefile\fP is like \f(CWmv\fP,
864 and \f(CWrmm\fP is like \f(CWrm\fP.
865 .PP
866 The context of tools in Unix consists mainly the current working directory,
867 the user identification, and the environment variables.
868 \s-1MH\s0 extends this context by two more items:
869 .IP \(bu
870 The current mail folder, which is similar to the current working directory.
871 For mail folders, \f(CWfolder\fP provides the corresponding functionality
872 of \f(CWcd\fP and \f(CWpwd\fP for directories.
873 .IP \(bu
874 Sequences, which are named sets of messages in a mail folder.
875 The current message, relative to a mail folder, is a special sequence.
876 It enables commands like \f(CWnext\fP and \f(CWprev\fP.
877 .LP
878 In contrast to Unix' context, which is chained to the shell session,
879 \s-1MH\s0's context is independent.
880 Usually there is one context for each user, but a user can have many
881 contexts.
882 Public sequences are an exception, as they belong to the mail folder.
883 .[
884 %O mh-profile(5) and mh-sequence(5)
885 .]
887 .NH 2
888 Discussion of the design
889 .LP
890 The following paragraphs discuss \s-1MH\s0 in regard to the tenets
891 of the Unix Philosophy which Gancarz identified.
893 .PP
894 .B "Small is beautiful
895 and
896 .B "do one thing well
897 are two design goals that are directly visible in \s-1MH\s0.
898 Gancarz actually presents \s-1MH\s0 as example under the headline
899 ``Making UNIX Do One Thing Well'':
900 .QP
901 [\s-1MH\s0] consists of a series of programs which
902 when combined give the user an enormous ability
903 to manipulate electronic mail messages.
904 A complex application, it shows that not only is it
905 possible to build large applications from smaller
906 components, but also that such designs are actually preferable.
907 .[
908 %A Mike Gancarz
909 %T unix-phil
910 %P 125
911 .]
912 .LP
913 The various small programs of \s-1MH\s0 were relatively easy
914 to write, because each of them is small, limited to one function,
915 and has clear boundaries.
916 For the same reasons, they are also good to maintain.
917 Further more, the system can easily get extended.
918 One only needs to put a new program into the toolchest.
919 This was done, for instance, when \s-1MIME\s0 support was added
920 (e.g. \f(CWmhbuild\fP).
921 Also, different programs can exist to do the basically same job
922 in different ways (e.g. in nmh: \f(CWshow\fP and \f(CWmhshow\fP).
923 If someone needs a mail system with some additionally
924 functions that are available nowhere yet, he best takes a
925 toolchest system like \s-1MH\s0 where he can add the
926 functionality with little work.
928 .PP
929 .B "Store data in flat text files
930 is followed by \s-1MH\s0.
931 This is not surprising, because email messages are already plain text.
932 \s-1MH\s0 stores the messages as it receives them,
933 thus any other tool that works on RFC 2822 mail messages can operate
934 on the messages in an \s-1MH\s0 mailbox.
935 All other files \s-1MH\s0 uses are plain text too.
936 It is therefore possible and encouraged to use the text processing
937 tools of Unix' toolchest to extend \s-1MH\s0's toolchest.
939 .PP
940 .B "Avoid captive user interfaces" .
941 \s-1MH\s0 is perfectly suited for non-interactive use.
942 It offers all functions directly and without captive user interfaces.
943 If, nonetheless, users want a graphical user interface,
944 they can have it with \fIxmh\fP or \fIexmh\fP, too.
945 These are graphical frontends for the \s-1MH\s0 toolchest.
946 This means, all email-related work is still done by \s-1MH\s0 tools,
947 but the frontend issues the appropriate calls when the user
948 clicks on buttons.
949 Providing easy-to-use user interfaces in form of frontends is a good
950 approach, because it does not limit the power of the backend itself.
951 The frontend will anyway only be able to make a subset of the
952 backend's power and flexibility available to the user.
953 But if it is a separate program,
954 then the missing parts can still be accessed at the backend directly.
955 If it is integrated, then this will hardly be possible.
956 Further more, it is possible to have different frontends to the same
957 backend.
959 .PP
960 .B "Choose portability over efficiency
961 and
962 .B "use shell scripts to increase leverage and portability" .
963 These two tenets are indirectly, but nicely, demonstrated by
964 Bolsky and Korn in their book about the Korn Shell.
965 .[
966 %T The KornShell: command and programming language
967 %A Morris I. Bolsky
968 %A David G. Korn
969 %I Prentice Hall
970 %D 1989
971 %P 254\(en290
972 %O \s-1ISBN\s0: 0-13-516972-0
973 .]
974 They demonstrated, in chapter 18 of the book, a basic implementation
975 of a subset of \s-1MH\s0 in ksh scripts.
976 Of course, this was just a demonstration, but a brilliant one.
977 It shows how quickly one can implement such a prototype with shell scripts,
978 and how readable they are.
979 The implementation in the scripting language may not be very fast,
980 but it can be fast enough though, and this is all that matters.
981 By having the code in an interpreted language, like the shell,
982 portability becomes a minor issue, if we assume the interpreter
983 to be widespread.
984 This demonstration also shows how easy it is to create single programs
985 of a toolchest software.
986 There are eight tools (two of them have multiple names) and 16 functions
987 with supporting code.
988 Each tool comprises between 12 and 38 lines of ksh,
989 in total about 200 lines.
990 The functions comprise between 3 and 78 lines of ksh,
991 in total about 450 lines.
992 Such small software is easy to write, easy to understand,
993 and thus easy to maintain.
994 A toolchest improves the possibility to only write some parts
995 and though create a working result.
996 Expanding the toolchest without global changes will likely be
997 possible, too.
999 .PP
1000 .B "Use software leverage to your advantage
1001 and the lesser tenet
1002 .B "allow the user to tailor the environment
1003 are ideally followed in the design of \s-1MH\s0.
1004 Tailoring the environment is heavily encouraged by the ability to
1005 directly define default options to programs.
1006 It is even possible to define different default options
1007 depending on the name under which the program was called.
1008 Software leverage is heavily encouraged by the ease it is to
1009 create shell scripts that run a specific command line,
1010 built of several \s-1MH\s0 programs.
1011 There is few software that so much wants users to tailor their
1012 environment and to leverage the use of the software, like \s-1MH\s0.
1013 Just to make one example:
1014 One might prefer a different listing format for the \f(CWscan\fP
1015 program.
1016 It is possible to take one of the distributed format files
1017 or to write one yourself.
1018 To use the format as default for \f(CWscan\fP, a single line,
1019 reading
1020 .DS
1021 .CW
1022 scan: -form FORMATFILE
1023 .DE
1024 must be added to \f(CW.mh_profile\fP.
1025 If one wants this different format as an additional command,
1026 instead of changing the default, he needs to create a link to
1027 \f(CWscan\fP, for instance titled \f(CWscan2\fP.
1028 The line in \f(CW.mh_profile\fP would then start with \f(CWscan2\fP,
1029 as the option should only be in effect when scan is called as
1030 \f(CWscan2\fP.
1032 .PP
1033 .B "Make every program a filter
1034 is hard to find in \s-1MH\s0.
1035 The reason therefore is that most of \s-1MH\s0's tools provide
1036 basic file system operations for the mailboxes.
1037 The reason is the same because of which
1038 \f(CWls\fP, \f(CWcp\fP, \f(CWmv\fP, and \f(CWrm\fP
1039 aren't filters neither.
1040 However, they build a basis on which filters can operate.
1041 \s-1MH\s0 does not provide many filters itself, but it is a basis
1042 to write filters for.
1043 An example would be a mail message text highlighter,
1044 that means a program that makes use of a color terminal to display
1045 header lines, quotations, and signatures in distinct colors.
1046 The author's version of this program, for instance,
1047 is a 25 line awk script.
1049 .PP
1050 .B "Build a prototype as soon as possible
1051 was again well followed by \s-1MH\s0.
1052 This tenet, of course, focuses on early development, which is
1053 long time ago for \s-1MH\s0.
1054 But without following this guideline at the very beginning,
1055 Bruce Borden may have not convinced the management of \s-1RAND\s0
1056 to ever create \s-1MH\s0.
1057 In Bruce' own words:
1058 .QP
1059 [...] but they [Stockton Gaines and Norm Shapiro] were not able
1060 to convince anyone that such a system would be fast enough to be usable.
1061 I proposed a very short project to prove the basic concepts,
1062 and my management agreed.
1063 Looking back, I realize that I had been very lucky with my first design.
1064 Without nearly enough design work,
1065 I built a working environment and some header files
1066 with key structures and wrote the first few \s-1MH\s0 commands:
1067 inc, show/next/prev, and comp.
1068 [...]
1069 With these three, I was able to convince people that the structure was viable.
1070 This took about three weeks.
1071 .[
1072 %O FIXME
1073 .]
1075 .NH 2
1076 Problems
1077 .LP
1078 \s-1MH\s0, for sure is not without problems.
1079 There are two main problems: one is technical, the other is about human behavior.
1080 .PP
1081 \s-1MH\s0 is old and email today is very different to email in the time
1082 when \s-1MH\s0 was designed.
1083 \s-1MH\s0 adopted to the changes pretty well, but it is limited.
1084 For example in development resources.
1085 \s-1MIME\s0 support and support for different character encodings
1086 is available, but only on a moderate level.
1087 More active developers could quickly improve there.
1088 It is also limited by design, which is the larger problem.
1089 \s-1IMAP\s0, for example, conflicts with \s-1MH\s0's design to a large extend.
1090 These design conflicts are not easily solvable.
1091 Possibly, they require a redesign.
1092 Maybe \s-1IMAP\s0 is too different to the classic mail model which \s-1MH\s0 covers,
1093 hence \s-1MH\s0 may never work well with \s-1IMAP\s0.
1094 .PP
1095 The other kind of problem is human habits.
1096 When in this world almost all \s-1MUA\s0s are monolithic,
1097 it is very difficult to convince people to use a toolbox style \s-1MUA\s0
1098 like \s-1MH\s0.
1099 The habits are so strong, that even people who understood the concept
1100 and advantages of \s-1MH\s0 do not like to switch,
1101 simply because \s-1MH\s0 is different.
1102 Unfortunately, the frontends to \s-1MH\s0, which could provide familiar look'n'feel,
1103 are quite outdated and thus not very appealing compared to the modern interfaces
1104 which monolithic \s-1MUA\s0s offer.
1106 .NH 2
1107 Summary \s-1MH\s0
1108 .LP
1109 \s-1MH\s0 is an \s-1MUA\s0 that follows the Unix Philosophy in its design
1110 and implementation.
1111 It consists of a toolchest of small tools, each of them does one job well.
1112 The tools are orthogonal to each other, to a large extend.
1113 However, for historical reasons, there also exist distinct tools
1114 that cover the same task.
1115 .PP
1116 The toolchest approach offers great flexibility to the user.
1117 He can use the complete power of the Unix shell with \s-1MH\s0.
1118 This makes \s-1MH\s0 a very powerful mail system.
1119 Extending and customizing \s-1MH\s0 is easy and encouraged, too.
1120 .PP
1121 Apart from the user's perspective, \s-1MH\s0 is development-friendly.
1122 Its overall design follows clear rules.
1123 The single tools do only one job, thus they are easy to understand,
1124 easy to write, and good to maintain.
1125 They are all independent and do not interfere with the others.
1126 Automated testing of their function is a straight forward task.
1127 .PP
1128 It is sad, that \s-1MH\s0's differentness is its largest problem,
1129 as its differentness is also its largest advantage.
1130 Unfortunately, for most people their habits are stronger
1131 than the attraction of the clear design and the power, \s-1MH\s0 offers.
1135 .NH 1
1136 Case study: uzbl
1137 .LP
1138 The last chapter took a look on the \s-1MUA\s0 \s-1MH\s0,
1139 this chapter is about uzbl, a web browser that adheres to the Unix Philosophy.
1140 ``uzbl'' is the \fIlolcat\fP's word for the English adjective ``usable''.
1141 It is pronounced the identical.
1143 .NH 2
1144 Historical background
1145 .LP
1146 Uzbl was started by Dieter Plaetinck in April 2009.
1147 The idea was born in a thread in the Arch Linux forum.
1148 .[
1149 %O http://bbs.archlinux.org/viewtopic.php?id=67463
1150 .]
1151 After some discussion about failures of well known web browsers,
1152 Plaetinck (alias Dieter@be) came up with a very sketchy proposal
1153 of how a better web browser could look like.
1154 To the question of another member, if Plaetinck would write that program,
1155 because it would sound fantastic, Plaetinck replied:
1156 ``Maybe, if I find the time ;-)''.
1157 .PP
1158 Fortunately, he found the time.
1159 One day later, the first prototype was out.
1160 One week later, uzbl had an own website.
1161 One month after the first code showed up,
1162 a mailing list was installed to coordinate and discuss further development.
1163 A wiki was set up to store documentation and scripts that showed up on the
1164 mailing list and elsewhere.
1165 .PP
1166 In the, now, one year of uzbl's existence, it was heavily developed in various branches.
1167 Plaetinck's task became more and more to only merge the best code from the
1168 different branches into his main branch, and to apply patches.
1169 About once a month, Plaetinck released a new version.
1170 In September 2009, he presented several forks of uzbl.
1171 Uzbl, actually, opened the field for a whole family of web browsers with similar shape.
1172 .PP
1173 In July 2009, \fILinux Weekly News\fP published an interview with Plaetinck about uzbl.
1174 In September 2009, the uzbl web browser was on \fISlashdot\fP.
1176 .NH 2
1177 Contrasts to other web browsers
1178 .LP
1179 Like most \s-1MUA\s0s are monolithic, but \s-1MH\s0 is a toolchest,
1180 most web browsers are monolithic, but uzbl is a frontend to a toolchest.
1181 .PP
1182 Today, uzbl is divided into uzbl-core and uzbl-browser.
1183 Uzbl-core is, how its name already indicates, the core of uzbl.
1184 It handles commands and events to interface other programs,
1185 and also displays webpages by using webkit as render engine.
1186 Uzbl-browser combines uzbl-core with a bunch of handler scripts, a status bar,
1187 an event manager, yanking, pasting, page searching, zooming, and more stuff,
1188 to form a ``complete'' web browser.
1189 In the following text, the term ``uzbl'' usually stands for uzbl-browser,
1190 so uzbl-core is included.
1191 .PP
1192 Unlike most other web browsers, uzbl is mainly the mediator between the
1193 various tools that cover single jobs of web browsing.
1194 Therefore, uzbl listens for commands on a named pipe (fifo), a Unix socket,
1195 and on stdin, and it writes events to a Unix socket and to stdout.
1196 The graphical rendering of the webpage is done by webkit, a web content engine.
1197 Uzbl-core is build around this library.
1198 Loading a webpage in a running uzbl instance requires only:
1199 .DS
1200 .CW
1201 echo 'uri http://example.org' >/path/to/uzbl-fifo
1202 .DE
1203 .PP
1204 Downloads, browsing history, bookmarks, and thelike are not provided
1205 by uzbl-core itself, as they are in other web browsers.
1206 Uzbl-browser also only provides, so called, handler scripts that wrap
1207 external applications which provide the actual functionality.
1208 For instance, \fIwget\fP is used to download files and uzbl-browser
1209 includes a script that calls wget with appropriate options in
1210 a prepared environment.
1211 .PP
1212 Modern web browsers are proud to have addons, plugins, and modules, instead.
1213 This is their effort to achieve similar goals.
1214 But instead of using existing, external programs, modern web browsers
1215 include these functions, although they might be loaded at runtime.
1217 .NH 2
1218 Discussion of the design
1219 .LP
1220 This section discusses uzbl in regard of the Unix Philosophy,
1221 as identified by Gancarz.
1223 .PP
1224 .B "Make each program do one thing well" .
1225 Uzbl tries to be a web browser and nothing else.
1226 The common definition of a web browser is, of course, highly influenced by
1227 existing implementations of web browsers, although they are degenerated.
1228 Web browsers should be programs to browse the web, and nothing more.
1229 This is the one thing they should do, as demanded by the Unix Philosophy.
1230 .PP
1231 Web browsers should, for instance, not manage downloads.
1232 This is the job download managers exist for.
1233 Download managers do primary care about being good in downloading files.
1234 Modern web browsers provide download management only as a secondary feature.
1235 How could they perform this job better, than programs that exist only for
1236 this very job?
1237 And how could anyone want less than the best download manager available?
1238 .PP
1239 A web browser's job is to let the user browse the web.
1240 This means, navigating through websites by following links.
1241 Rendering the \s-1HTML\s0 sources is a different job, too.
1242 It is covered by the webkit render engine, in uzbl's case.
1243 Audio and video content and files like PostScript, \s-1PDF\s0, and the like,
1244 are also not the job of a web browser.
1245 They should be handled by external applications \(en
1246 ones which's job is to handle such data.
1247 Uzbl strives to do it this way.
1248 .PP
1249 Remember Doug McIlroy:
1250 .I
1251 ``Write programs that do one thing and do it well.
1252 Write programs to work together.''
1253 .R
1254 .PP
1255 The lesser tenet
1256 .B "allow the user to tailor the environment
1257 matches good here.
1258 There was the question, how anyone could want anything less than the
1259 best program for the job.
1260 But as personal preferences matter much,
1261 it is probably more important to ask:
1262 How could anyone want something else than his preferred program for the job?
1263 .PP
1264 Usually users want one program for one job.
1265 Hence, whenever the task is, for instance, downloading,
1266 exactly one download manager should be used.
1267 More advanced users might want to have this download manager in this
1268 situation and that one in that situation.
1269 They should be able to configure it this way.
1270 With uzbl, one can use any download manager the user wants.
1271 To switch to a different one, only one line in a small handler script
1272 needs to be changed.
1273 Alternatively it would be possible to query an entry in a global file
1274 or an environment variable, which specifies the download manager to use,
1275 in the handler script.
1276 .PP
1277 As uzbl does neither have its own download manager nor depends on a
1278 specific one, thus uzbl's browsing abilities will not be lowered by having
1279 a bad download manager.
1280 Uzbl's download capabilities will just as good as the ones of the best
1281 download manager available on the system.
1282 Of course, this applies to all of the other supplementary tools, too.
1284 .PP
1285 .B "Use software leverage to your advantage" .
1286 Shell scripts are a good choice to extend uzbl.
1287 Uzbl is designed to be extended by external tools.
1288 These external tools are usually wrapped by small handler shell scripts.
1289 Shell scripts are the glue in this approach.
1290 They make the various parts fit together.
1291 .PP
1292 As an example, the history mechanism of uzbl shall be presented.
1293 Uzbl is configured to spawn a script to append an entry to the history
1294 whenever the event of a fully loaded page occurs.
1295 The script to append the entry to the history not much more than:
1296 .DS
1297 .CW
1298 #!/bin/sh
1299 file=/path/to/uzbl-history
1300 echo `date +'%Y-%m-%d %H:%M:%S'`" $6 $7" >> $file
1301 .DE
1302 \f(CW$6\fP and \f(CW$7\fP expand to the \s-1URL\s0 and the page title.
1303 For loading an entry, a key is bound to spawn a load from history script.
1304 The script reverses the history to have newer entries first,
1305 then displays \fIdmenu\fP to select an item,
1306 and afterwards writes the selected \s-1URL\s0 into uzbl's command input pipe.
1307 With error checking and corner cases removed, the script looks like this:
1308 .DS
1309 .CW
1310 #!/bin/sh
1311 file=/path/to/uzbl-history
1312 goto=`tac $file | dmenu | cut -d' ' -f 3`
1313 echo "uri $goto" > $4
1314 .DE
1315 \f(CW$4\fP expands to the path of the command input pipe of the current
1316 uzbl instance.
1318 .PP
1319 .B "Avoid captive user interfaces" .
1320 One could say, that uzbl, to a large extend, actually \fIis\fP
1321 a captive user interface.
1322 But the difference to most other web browsers is, that uzbl is only
1323 the captive user interface frontend and the core of the backend.
1324 Many parts of the backend are independent of uzbl.
1325 Some are distributed with uzbl, for some external programs, handler scripts
1326 are distributed, arbitrary additional functionality can be added if desired.
1327 .PP
1328 The frontend is captive \(en that is true.
1329 This is okay for the task of browsing the web, as this task is only relevant
1330 for humans.
1331 Automated programs would \fIcrawl\fP the web.
1332 That means, they read the source directly.
1333 The source includes all the semantics.
1334 The graphical representation is just for humans to transfer the semantics
1335 more intuitively.
1337 .PP
1338 .B "Make every program a filter" .
1339 Graphical web browsers are almost dead ends in the chain of information flow.
1340 Thus it is difficult to see what graphical web browsers should filter.
1341 Graphical web browsers exist almost only for interactive use by humans.
1342 The only case when one might want to automate the rendering function is
1343 to generate images of rendered webpages.
1345 .PP
1346 .B "Small is beautiful"
1347 is not easy to apply to a web browser, primary because modern web technology
1348 is very complex; hence the rendering task is very complex.
1349 Modern web browsers will always consist of many thousand lines of code,
1350 unfortunately.
1351 Using the toolchest approach and wrappers can split the browser into
1352 several small parts, tough.
1353 .PP
1354 Uzbl-core consists of about 3\,500 lines of C code.
1355 The distribution includes another 3\,500 lines of Shell and Python code,
1356 which are the handler scripts and plugins like a modal interface.
1357 Further more, uzbl uses functionality of external tools like
1358 \fIwget\fP and \fInetcat\fP.
1359 Up to this point, uzbl looks pretty neat and small.
1360 The ugly part of uzbl is the web content renderer, webkit.
1361 Webkit consists of roughly 400\,000 (!) lines of code.
1362 Unfortunately, small web render engines are not possible anymore
1363 because of the modern web.
1364 The problems section will explain this in more detail.
1366 .PP
1367 .B "Build a prototype as soon as possible" .
1368 Plaetinck made his code public, right from the beginning.
1369 Discussion and development was, and still is, open to everyone interested.
1370 Development versions of uzbl can be obtained very simply from the code
1371 repository.
1372 Within the first year of uzbl's existence, a new version was released
1373 more often than once a month.
1374 Different forks and branches arose.
1375 They introduced new features, which were tested for suitability.
1376 The experiences of using prototypes influenced further development.
1377 Actually, all development was community driven.
1378 Plaetinck says, three months after uzbl's birth:
1379 ``Right now I hardly code anything myself for Uzbl.
1380 I just merge in other people's code, ponder a lot, and lead the discussions.''
1381 .[
1382 %A FIXME
1383 %O http://lwn.net/Articles/341245/
1384 .]
1387 .NH 2
1388 Problems
1389 .LP
1390 Similar to \s-1MH\s0, uzbl, too suffers from being different.
1391 It is sad, but people use what they know.
1392 Fortunately, uzbl's user interface can look and feel very much the
1393 same as the one of the well known web browsers,
1394 hiding the internal differences.
1395 But uzbl has to provide this similar look and feel to be accepted
1396 as a ``normal'' browser by ``normal'' users.
1397 .PP
1398 The more important problem is the modern web.
1399 The modern web is simply broken.
1400 It has state in a state-less protocol,
1401 it misuses technologies,
1402 and it is helplessly overloaded.
1403 The result are web content render engines that must consist
1404 of hundreds of thousands lines of code.
1405 They also must combine and integrate many different technologies,
1406 only to make our modern web usable.
1407 Website to image converter are hardly possible to run without
1408 human interaction because of state in sessions, impossible
1409 deep-linking, and unautomatable technologies.
1410 .PP
1411 The web was misused to provide all kinds of imaginable wishes.
1412 Now web browsers, and eventually the users, suffer from it.
1415 .NH 2
1416 Summary uzbl
1417 .LP
1418 ``Uzbl is a browser that adheres to the Unix Philosophy'',
1419 that is how uzbl is seen by its authors.
1420 Indeed, uzbl follows the Unix Philosophy in many ways.
1421 It consists of independent parts that work together,
1422 its core is mainly a mediator which glues the parts together.
1423 .PP
1424 Software leverage can excellently be seen in uzbl.
1425 It makes use of external tools, separates independent tasks
1426 in independent parts, and glues them together with small
1427 handler scripts, around uzbl-core.
1428 .PP
1429 As uzbl, more or less, consists of a set of tools and a bit
1430 of glue, anyone can put the parts together and expand it
1431 in any desired way.
1432 Uzbl is very flexible and customizable.
1433 These properties make it valuable for advanced users,
1434 but may keep novice users from using it.
1435 .PP
1436 Uzbl's main problem is the modern web, that makes it hard
1437 to design a sane web browser.
1438 Despite this bad situation, uzbl does a fairly good job.
1441 .NH 1
1442 Final thoughts
1444 .NH 2
1445 Quick summary
1446 .LP
1447 good design
1448 .LP
1449 unix phil
1450 .LP
1451 case studies
1453 .NH 2
1454 Why people should choose
1455 .LP
1456 Make the right choice!
1458 .nr PI .5i
1459 .rm ]<
1460 .de ]<
1461 .LP
1462 .de FP
1463 .IP \\\\$1.
1464 \\..
1465 .rm FS FE
1466 ..
1467 .SH
1468 References
1469 .[
1470 $LIST$
1471 .]
1472 .wh -1p