docs/unix-phil
view unix-phil.ms @ 42:303e8f449e77
added TOC and improved visual appearance
| author | meillo@marmaro.de |
|---|---|
| date | Fri, 09 Apr 2010 13:31:51 +0200 |
| parents | 13ef7042fa28 |
| children | 46e34e433231 |
line source
1 .nr PS 11
2 .nr VS 13
3 .nr lu 0
4 .de CW
5 .nr PQ \\n(.f
6 .if t .ft CW
7 .ie ^\\$1^^ .if n .ul 999
8 .el .if n .ul 1
9 .if t .if !^\\$1^^ \&\\$1\f\\n(PQ\\$2
10 .if n .if \\n(.$=1 \&\\$1
11 .if n .if \\n(.$>1 \&\\$1\c
12 .if n .if \\n(.$>1 \&\\$2
13 ..
14 .ds [. \ [
15 .ds .] ]
17 .rn NH _N
18 .de NH
19 .if '\\$1'1' .sp 2v
20 .if '\\$1'1' .nr PS +2
21 ._N \\$1
22 .if '\\$1'1' .nr PS -2
23 ..
25 .\"----------------------------------------
27 .TL
28 .ps +4
29 Why the Unix Philosophy still matters
30 .AU
31 markus schnalke <meillo@marmaro.de>
32 .AB
33 .ti \n(.iu
34 This paper explains the importance of the Unix Philosophy for software design.
35 Today, few software designers are aware of these concepts,
36 and thus a lot of modern software is more limited than necessary
37 and makes less use of software leverage than possible.
38 Knowing and following the guidelines of the Unix Philosophy makes software more valuable.
39 .AE
41 .FS
42 .ps -1
43 This paper was prepared for the ``Software Analysis'' seminar at University Ulm.
44 Mentor was professor Schweiggert. 2010-04-05
45 .br
46 You may retrieve this document from
47 .CW \s-1http://marmaro.de/docs \ .
48 .FE
50 .NH 1
51 Introduction
52 .XS
53 \*(SN Introduction
54 .XE
55 .LP
56 The Unix Philosophy is the essence of how the Unix operating system,
57 especially its toolchest, was designed.
58 It is no limited set of fixed rules,
59 but a loose set of guidelines which tell how to write software that
60 suites well into Unix.
61 Actually, the Unix Philosophy describes what is common to typical Unix software.
62 The Wikipedia has an accurate definition:
63 .[
64 %A Wikipedia
65 %T Unix philosophy
66 %P Wikipedia, The Free Encyclopedia
67 %D 2010-03-21 17:20 UTC
68 %O .CW \s-1http://en.wikipedia.org/w/index.php?title=Unix_philosophy&oldid=351189719
69 .]
70 .QP
71 .ps -1
72 The \fIUnix philosophy\fP is a set of cultural norms and philosophical
73 approaches to developing software based on the experience of leading
74 developers of the Unix operating system.
75 .PP
76 As there is no single definition of the Unix Philosophy,
77 several people have stated their view on what it comprises.
78 Best known are:
79 .IP \(bu
80 Doug McIlroy's summary: ``Write programs that do one thing and do it well.''
81 .[
82 %A M. D. McIlroy
83 %A E. N. Pinson
84 %A B. A. Taque
85 %T UNIX Time-Sharing System Forward
86 %J The Bell System Technical Journal
87 %D 1978
88 %V 57
89 %N 6
90 %P 1902
91 .]
92 .IP \(bu
93 Mike Gancarz' book ``The UNIX Philosophy''.
94 .[
95 %A Mike Gancarz
96 %T The UNIX Philosophy
97 %D 1995
98 %I Digital Press
99 .]
100 .IP \(bu
101 Eric S. Raymond's book ``The Art of UNIX Programming''.
102 .[
103 %A Eric S. Raymond
104 %T The Art of UNIX Programming
105 %D 2003
106 %I Addison-Wesley
107 %O .CW \s-1http://www.faqs.org/docs/artu/
108 .]
109 .LP
110 These different views on the Unix Philosophy have much in common.
111 Especially, the main concepts are similar in all of them.
112 McIlroy's definition can surely be called the core of the Unix Philosophy,
113 but the fundamental idea behind it all, is ``small is beautiful''.
115 .PP
116 The Unix Philosophy tells how to design and write good software for Unix.
117 Many concepts described here base on facilities of Unix.
118 Other operating systems may not offer such facilities,
119 hence it may not be possible to design software in the way of the
120 Unix Philosophy for them.
121 .PP
122 The Unix Philosophy has an idea of how the process of software development
123 should look like, but large parts of the philosophy are quite independent
124 from the development process used.
125 However, one will soon recognize that some development processes work well
126 with the ideas of the Unix Philosophy and support them, while others are
127 at cross-purposes.
128 Kent Beck's books about Extreme Programming are valuable supplimental
129 resources.
130 .PP
131 The question of how to actually write code and how the code should looks
132 like internally, are out of focus here.
133 ``The Practice of Programming'' by Kernighan and Pike,
134 .[
135 %A Brian W. Kernighan
136 %A Rob Pike
137 %T The Practice of Programming
138 %I Addison-Wesley
139 %D 1999
140 .]
141 is a good book that covers this topic.
142 Its point of view matches to the one of this paper.
144 .NH 1
145 Importance of software design in general
146 .XS
147 .sp .5v
148 \*(SN Importance of software design in general
149 .XE
150 .LP
151 Software design is the planning of how the internal structure
152 and external interfaces of a software should look like.
153 It has nothing to do with visual appearance.
154 If we take a program as a car, then its color is of no matter.
155 Its design would be the car's size, its shape, the locations of doors,
156 the passenger/space ratio, the luggage capacity, and so forth.
157 .PP
158 Why should software get designed at all?
159 It is general knowledge, that even a bad plan is better than no plan.
160 Not designing software means programming without plan.
161 This will pretty sure lead to horrible results.
162 Horrible to use and horrible to maintain.
163 These two aspects are the visible ones.
164 Often invisible are the wasted possible gains.
165 Good software design can make these gains available.
166 .PP
167 A software's design deals with quality properties.
168 Good design leads to good quality, and quality is important.
169 Any car may be able to drive from A to B,
170 but it depends on the car's properties whether it is a good choice
171 for passenger transport or not.
172 It depends on its properties if it is a good choice
173 for a rough mountain area.
174 And it depends on its properties if the ride will be fun.
176 .PP
177 Requirements for a software are twofold:
178 functional and non-functional.
179 .IP \(bu
180 Functional requirements define directly the software's functions.
181 They are the reason why software gets written.
182 Someone has a problem and needs a tool to solve it.
183 Being able to solve the problem is the main functional goal.
184 It is the driving force behind all programming effort.
185 Functional requirements are easier to define and to verify.
186 .IP \(bu
187 Non-functional requirements are also called \fIquality\fP requirements.
188 The quality of a software are the properties that are not directly related to
189 the software's basic functions.
190 Tools of bad quality often solve the problems they were written for,
191 but introduce problems and difficulties for usage and development, later on.
192 Quality aspects are often overlooked at first sight,
193 and they are often difficult to define clearly and to verify.
194 .PP
195 Quality is of few matter when the software gets built initially,
196 but it is of matter for usage and maintenance of the software.
197 A short-sighted might see in developing a software mainly building something up.
198 But experience shows, that building the software the first time is
199 only a small amount of the overall work.
200 Bug fixing, extending, rebuilding of parts
201 \(en maintenance work, for short \(en
202 does soon take over the major part of the time spent on a software.
203 Not to forget the usage of the software.
204 These processes are highly influenced by the software's quality.
205 Thus, quality must not be neglected.
206 The problem with quality is that you hardly ``stumble over''
207 bad quality during the first build,
208 but this is the time when you should care about good quality most.
209 .PP
210 Software design is less the basic function of a software \(en
211 this requirement will get satisfied anyway.
212 Software design is more about quality aspects of the software.
213 Good design leads to good quality, bad design to bad quality.
214 The primary functions of the software will be affected modestly by bad quality,
215 but good quality can provide a lot of additional gain,
216 even at places where one never expected it.
217 .PP
218 The ISO/IEC 9126-1 standard, part 1,
219 .[
220 %I International Organization for Standardization
221 %T ISO Standard 9126: Software Engineering \(en Product Quality, part 1
222 %C Geneve
223 %D 2001
224 .]
225 defines the quality model as consisting out of:
226 .IP \(bu
227 .I Functionality
228 (suitability, accuracy, inter\%operability, security)
229 .IP \(bu
230 .I Reliability
231 (maturity, fault tolerance, recoverability)
232 .IP \(bu
233 .I Usability
234 (understandability, learnability, operability, attractiveness)
235 .IP \(bu
236 .I Efficiency
237 (time behavior, resource utilization)
238 .IP \(bu
239 .I Maintainability
240 (analyzability, changeability, stability, testability)
241 .IP \(bu
242 .I Portability
243 (adaptability, installability, co-existence, replaceability)
244 .LP
245 Good design can improve these properties of a software,
246 bad designed software probably suffers from not having them.
247 .PP
248 One further goal of software design is consistency.
249 Consistency eases understanding, working on, and using things.
250 Consistent internal structure and consistent interfaces to the outside
251 can be provided by good design.
252 .PP
253 Software should be well designed because good design avoids many
254 problems during the software's lifetime.
255 And software should be well designed because good design can offer
256 much additional gain.
257 Indeed, much effort should be spent into good design to make software more valuable.
258 The Unix Philosophy shows a way of how to design software well.
259 It offers guidelines to achieve good quality and high gain for the effort spent.
262 .NH 1
263 The Unix Philosophy
264 .XS
265 .sp .5v
266 \*(SN The Unix Philosophy
267 .XE
268 .LP
269 The origins of the Unix Philosophy were already introduced.
270 This chapter explains the philosophy, oriented on Gancarz,
271 and shows concrete examples of its application.
273 .NH 2
274 Pipes
275 .XS
276 \*(SN Pipes
277 .XE
278 .LP
279 Following are some examples to demonstrate how applied Unix Philosophy feels like.
280 Knowledge of using the Unix shell is assumed.
281 .PP
282 Counting the number of files in the current directory:
283 .DS
284 .CW
285 .ps -1
286 ls | wc -l
287 .DE
288 The
289 .CW ls
290 command lists all files in the current directory, one per line,
291 and
292 .CW "wc -l
293 counts the number of lines.
294 .PP
295 Counting the number of files that do not contain ``foo'' in their name:
296 .DS
297 .CW
298 .ps -1
299 ls | grep -v foo | wc -l
300 .DE
301 Here, the list of files is filtered by
302 .CW grep
303 to remove all that contain ``foo''.
304 The rest is the same as in the previous example.
305 .PP
306 Finding the five largest entries in the current directory.
307 .DS
308 .CW
309 .ps -1
310 du -s * | sort -nr | sed 5q
311 .DE
312 .CW "du -s *
313 returns the recursively summed sizes of all files
314 \(en no matter if they are regular files or directories.
315 .CW "sort -nr
316 sorts the list numerically in reverse order.
317 Finally,
318 .CW "sed 5q
319 quits after it has printed the fifth line.
320 .PP
321 The presented command lines are examples of what Unix people would use
322 to get the desired output.
323 There are also other ways to get the same output.
324 It's a user's decision which way to go.
325 .PP
326 The examples show that many tasks on a Unix system
327 are accomplished by combining several small programs.
328 The connection between the single programs is denoted by the pipe operator `|'.
329 .PP
330 Pipes, and their extensive and easy use, are one of the great
331 achievements of the Unix system.
332 Pipes between programs have been possible in earlier operating systems,
333 but it has never been a so central part of the concept.
334 When, in the early seventies, Doug McIlroy introduced pipes for the
335 Unix system,
336 ``it was this concept and notation for linking several programs together
337 that transformed Unix from a basic file-sharing system to an entirely new way of computing.''
338 .[
339 %T Unix: An Oral History
340 %O .CW \s-1http://www.princeton.edu/~hos/frs122/unixhist/finalhis.htm
341 .]
342 .PP
343 Being able to specify pipelines in an easy way is,
344 however, not enough by itself.
345 It is only one half.
346 The other is the design of the programs that are used in the pipeline.
347 They have to interfaces that allows them to be used in such a way.
349 .NH 2
350 Interface design
351 .XS
352 \*(SN Interface design
353 .XE
354 .LP
355 Unix is, first of all, simple \(en Everything is a file.
356 Files are sequences of bytes, without any special structure.
357 Programs should be filters, which read a stream of bytes from ``standard input'' (stdin)
358 and write a stream of bytes to ``standard output'' (stdout).
359 .PP
360 If the files \fIare\fP sequences of bytes,
361 and the programs \fIare\fP filters on byte streams,
362 then there is exactly one standardized data interface.
363 Thus it is possible to combine them in any desired way.
364 .PP
365 Even a handful of small programs will yield a large set of combinations,
366 and thus a large set of different functions.
367 This is leverage!
368 If the programs are orthogonal to each other \(en the best case \(en
369 then the set of different functions is greatest.
370 .PP
371 Programs might also have a separate control interface,
372 besides their data interface.
373 The control interface is often called ``user interface'',
374 because it is usually designed to be used by humans.
375 The Unix Philosophy discourages to assume the user to be human.
376 Interactive use of software is slow use of software,
377 because the program waits for user input most of the time.
378 Interactive software requires the user to be in front of the computer
379 all the time.
380 Interactive software occupy the user's attention while they are running.
381 .PP
382 Now we come back to the idea of using several small programs, combined,
383 to have a more specific function.
384 If these single tools would all be interactive,
385 how would the user control them?
386 It is not only a problem to control several programs at once if they run at the same time,
387 it also very inefficient to have to control each of the single programs
388 that are intended to work as one large program.
389 Hence, the Unix Philosophy discourages programs to demand interactive use.
390 The behavior of programs should be defined at invocation.
391 This is done by specifying arguments (``command line switches'') to the program call.
392 Gancarz discusses this topic as ``avoid captive user interfaces''.
393 .[
394 %A Mike Gancarz
395 %T The UNIX Philosophy
396 %I Digital Press
397 %D 1995
398 %P 88 ff.
399 .]
400 .PP
401 Non-interactive use is, during development, also an advantage for testing.
402 Testing of interactive programs is much more complicated,
403 than testing of non-interactive programs.
405 .NH 2
406 The toolchest approach
407 .XS
408 \*(SN The toolchest approach
409 .XE
410 .LP
411 A toolchest is a set of tools.
412 Instead of having one big tool for all tasks, one has many small tools,
413 each for one task.
414 Difficult tasks are solved by combining several of the small, simple tools.
415 .PP
416 The Unix toolchest \fIis\fP a set of small, (mostly) non-interactive programs
417 that are filters on byte streams.
418 They are, to a large extend, unrelated in their function.
419 Hence, the Unix toolchest provides a large set of functions
420 that can be accessed by combining the programs in the desired way.
421 .PP
422 There are also advantages for developing small toolchest programs.
423 It is easier and less error-prone to write small programs.
424 It is also easier and less error-prone to write a large set of small programs,
425 than to write one large program with all the functionality included.
426 If the small programs are combinable, then they offer even a larger set
427 of functions than the single large program.
428 Hence, one gets two advantages out of writing small, combinable programs.
429 .PP
430 There are two drawbacks of the toolchest approach.
431 First, one simple, standardized, unidirectional interface has to be sufficient.
432 If one feels the need for more ``logic'' than a stream of bytes,
433 then a different approach might be of need.
434 But it is also possible, that he just can not imagine a design where
435 a stream of bytes is sufficient.
436 By becoming more familiar with the ``Unix style of thinking'',
437 developers will more often and easier find simple designs where
438 a stream of bytes is a sufficient interface.
439 .PP
440 The second drawback of a toolchest affects the users.
441 A toolchest is often more difficult to use for novices.
442 It is necessary to become familiar with each of the tools,
443 to be able to use the right one in a given situation.
444 Additionally, one needs to combine the tools in a senseful way on its own.
445 This is like a sharp knife \(en it is a powerful tool in the hand of a master,
446 but of no good value in the hand of an unskilled.
447 .PP
448 However, learning single, small tool of the toolchest is easier than
449 learning a complex tool.
450 The user will have a basic understanding of a yet unknown tool,
451 if the several tools of the toolchest have a common style.
452 He will be able to transfer knowledge over one tool to another.
453 .PP
454 Moreover, the second drawback can be removed easily by adding wrappers
455 around the single tools.
456 Novice users do not need to learn several tools if a professional wraps
457 the single commands into a more high-level script.
458 Note that the wrapper script still calls the small tools;
459 the wrapper script is just like a skin around.
460 No complexity is added this way,
461 but new programs can get created out of existing one with very low effort.
462 .PP
463 A wrapper script for finding the five largest entries in the current directory
464 could look like this:
465 .DS
466 .CW
467 .ps -1
468 #!/bin/sh
469 du -s * | sort -nr | sed 5q
470 .DE
471 The script itself is just a text file that calls the command line
472 a professional user would type in directly.
473 Making the program flexible on the number of entries it prints,
474 is easily possible:
475 .DS
476 .CW
477 .ps -1
478 #!/bin/sh
479 num=5
480 [ $# -eq 1 ] && num="$1"
481 du -sh * | sort -nr | sed "${num}q"
482 .DE
483 This script acts like the one before, when called without an argument.
484 But one can also specify a numerical argument to define the number of lines to print.
486 .NH 2
487 A powerful shell
488 .XS
489 \*(SN A powerful shell
490 .XE
491 .LP
492 It was already said, that the Unix shell provides the possibility to
493 combine small programs into large ones easily.
494 A powerful shell is a great feature in other ways, too.
495 .PP
496 For instance by including a scripting language.
497 The control statements are build into the shell.
498 The functions, however, are the normal programs, everyone can use on the system.
499 Thus, the programs are known, so learning to program in the shell is easy.
500 Using normal programs as functions in the shell programming language
501 is only possible because they are small and combinable tools in a toolchest style.
502 .PP
503 The Unix shell encourages to write small scripts out of other programs,
504 because it is so easy to do.
505 This is a great step towards automation.
506 It is wonderful if the effort to automate a task equals the effort
507 it takes to do it the second time by hand.
508 If it is so, then the user will be happy to automate everything he does more than once.
509 .PP
510 Small programs that do one job well, standardized interfaces between them,
511 a mechanism to combine parts to larger parts, and an easy way to automate tasks,
512 this will inevitably produce software leverage.
513 Getting multiple times the benefit of an investment is a great offer.
514 .PP
515 The shell also encourages rapid prototyping.
516 Many well known programs started as quickly hacked shell scripts,
517 and turned into ``real'' programs, written in C, later.
518 Building a prototype first is a way to avoid the biggest problems
519 in application development.
520 Fred Brooks writes in ``No Silver Bullet'':
521 .[
522 %A Frederick P. Brooks, Jr.
523 %T No Silver Bullet: Essence and Accidents of Software Engineering
524 %B Information Processing 1986, the Proceedings of the IFIP Tenth World Computing Conference
525 %E H.-J. Kugler
526 %D 1986
527 %P 1069\(en1076
528 %I Elsevier Science B.V.
529 %C Amsterdam, The Netherlands
530 .]
531 .QP
532 .ps -1
533 The hardest single part of building a software system is deciding precisely what to build.
534 No other part of the conceptual work is so difficult as establishing the detailed
535 technical requirements, [...].
536 No other part of the work so cripples the resulting system if done wrong.
537 No other part is more difficult to rectify later.
538 .PP
539 Writing a prototype is a great method to become familiar with the requirements
540 and to actually run into real problems.
541 Today, prototyping is often seen as a first step in building a software.
542 This is, of course, good.
543 However, the Unix Philosophy has an \fIadditional\fP perspective on prototyping:
544 After having built the prototype, one might notice, that the prototype is already
545 \fIgood enough\fP.
546 Hence, no reimplementation, in a more sophisticated programming language, might be of need,
547 for the moment.
548 Maybe later, it might be necessary to rewrite the software, but not now.
549 .PP
550 By delaying further work, one keeps the flexibility to react easily on
551 changing requirements.
552 Software parts that are not written will not miss the requirements.
554 .NH 2
555 Worse is better
556 .XS
557 \*(SN Worse is better
558 .XE
559 .LP
560 The Unix Philosophy aims for the 80% solution;
561 others call it the ``Worse is better'' approach.
562 .PP
563 First, practical experience shows, that it is almost never possible to define the
564 requirements completely and correctly the first time.
565 Hence one should not try to; it will fail anyway.
566 Second, practical experience shows, that requirements change during time.
567 Hence it is best to delay requirement-based design decisions as long as possible.
568 Also, the software should be small and flexible as long as possible
569 to react on changing requirements.
570 Shell scripts, for example, are more easily adjusted as C programs.
571 Third, practical experience shows, that maintenance is hard work.
572 Hence, one should keep the amount of software as small as possible;
573 it should just fulfill the \fIcurrent\fP requirements.
574 Software parts that will be written later, do not need maintenance now.
575 .PP
576 Starting with a prototype in a scripting language has several advantages:
577 .IP \(bu
578 As the initial effort is low, one will likely start right away.
579 .IP \(bu
580 As working parts are available soon, the real requirements can get identified soon.
581 .IP \(bu
582 When a software is usable, it gets used, and thus tested.
583 Hence problems will be found at early stages of the development.
584 .IP \(bu
585 The prototype might be enough for the moment,
586 thus further work on the software can be delayed to a time
587 when one knows better about the requirements and problems,
588 than now.
589 .IP \(bu
590 Implementing now only the parts that are actually needed now,
591 requires fewer maintenance work.
592 .IP \(bu
593 If the global situation changes so that the software is not needed anymore,
594 then less effort was spent into the project, than it would have be
595 when a different approach had been used.
597 .NH 2
598 Upgrowth and survival of software
599 .XS
600 \*(SN Upgrowth and survival of software
601 .XE
602 .LP
603 So far it was talked about \fIwriting\fP or \fIbuilding\fP software.
604 Although these are just verbs, they do imply a specific view on the work process
605 they describe.
606 The better verb, however, is to \fIgrow\fP.
607 .PP
608 Creating software in the sense of the Unix Philosophy is an incremental process.
609 It starts with a first prototype, which evolves as requirements change.
610 A quickly hacked shell script might become a large, sophisticated,
611 compiled program this way.
612 Its lifetime begins with the initial prototype and ends when the software is not used anymore.
613 While being alive it will get extended, rearranged, rebuilt (from scratch).
614 Growing software matches the view that ``software is never finished. It is only released.''
615 .[
616 %O FIXME
617 %A Mike Gancarz
618 %T The UNIX Philosophy
619 %P 26
620 .]
621 .PP
622 Software can be seen as being controlled by evolutionary processes.
623 Successful software is software that is used by many for a long time.
624 This implies that the software is needed, useful, and better than alternatives.
625 Darwin talks about: ``The survival of the fittest.''
626 .[
627 %O FIXME
628 %A Charles Darwin
629 .]
630 Transferred to software: The most successful software, is the fittest,
631 is the one that survives.
632 (This may be at the level of one creature, or at the level of one species.)
633 The fitness of software is affected mainly by four properties:
634 portability of code, portability of data, range of usability, and reusability of parts.
635 .\" .IP \(bu
636 .\" portability of code
637 .\" .IP \(bu
638 .\" portability of data
639 .\" .IP \(bu
640 .\" range of usability
641 .\" .IP \(bu
642 .\" reuseability of parts
643 .PP
644 (1)
645 .I "Portability of code
646 means, using high-level programming languages,
647 sticking to the standard,
648 and avoiding optimizations that introduce dependencies on specific hardware.
649 Hardware has a much lower lifetime than software.
650 By chaining software to a specific hardware,
651 the software's lifetime gets shortened to that of this hardware.
652 In contrast, software should be easy to port \(en
653 adaptation is the key to success.
654 .\" cf. practice of prog: ch08
655 .PP
656 (2)
657 .I "Portability of data
658 is best achieved by avoiding binary representations
659 to store data, because binary representations differ from machine to machine.
660 Textual representation is favored.
661 Historically, ASCII was the charset of choice.
662 In the future, UTF-8 might be the better choice, however.
663 Important is that it is a plain text representation in a
664 very common charset encoding.
665 Apart from being able to transfer data between machines,
666 readable data has the great advantage, that humans are able
667 to directly edit it with text editors and other tools from the Unix toolchest.
668 .\" gancarz tenet 5
669 .PP
670 (3)
671 A large
672 .I "range of usability
673 ensures good adaptation, and thus good survival.
674 It is a special distinction if a software becomes used in fields of action,
675 the original authors did never imagine.
676 Software that solves problems in a general way will likely be used
677 for all kinds of similar problems.
678 Being too specific limits the range of uses.
679 Requirements change through time, thus use cases change or even vanish.
680 A good example in this point is Allman's sendmail.
681 Allman identifies flexibility to be one major reason for sendmail's success:
682 .[
683 %O FIXME
684 %A Allman
685 %T sendmail
686 .]
687 .QP
688 .ps -1
689 Second, I limited myself to the routing function [...].
690 This was a departure from the dominant thought of the time, [...].
691 .QP
692 .ps -1
693 Third, the sendmail configuration file was flexible enough to adopt
694 to a rapidly changing world [...].
695 .LP
696 Successful software adopts itself to the changing world.
697 .PP
698 (4)
699 .I "Reuse of parts
700 is even one step further.
701 A software may completely lose its field of action,
702 but parts of which the software is build may be general and independent enough
703 to survive this death.
704 If software is build by combining small independent programs,
705 then there are parts readily available for reuse.
706 Who cares if the large program is a failure,
707 but parts of it become successful instead?
709 .NH 2
710 Summary
711 .XS
712 \*(SN Summary
713 .XE
714 .LP
715 This chapter explained the central ideas of the Unix Philosophy.
716 For each of the ideas, it was exposed what advantages they introduce.
717 The Unix Philosophy are guidelines that help to write valuable software.
718 From the view point of a software developer or software designer,
719 the Unix Philosophy provides answers to many software design problem.
720 .PP
721 The various ideas of the Unix Philosophy are very interweaved
722 and can hardly be applied independently.
723 However, the probably most important messages are:
724 .I "``Do one thing well!''" ,
725 .I "``Keep it simple!''" ,
726 and
727 .I "``Use software leverage!''
731 .NH 1
732 Case study: \s-1MH\s0
733 .XS
734 .sp .5v
735 \*(SN Case study: \s-1MH\s0
736 .XE
737 .LP
738 The previous chapter introduced and explained the Unix Philosophy
739 from a general point of view.
740 The driving force were the guidelines; references to
741 existing software were given only sparsely.
742 In this and the next chapter, concrete software will be
743 the driving force in the discussion.
744 .PP
745 This first case study is about the mail user agents (\s-1MUA\s0)
746 \s-1MH\s0 (``mail handler'') and its descendent \fInmh\fP
747 (``new mail handler'').
748 \s-1MUA\s0s provide functions to read, compose, and organize mail,
749 but (ideally) not to transfer.
750 In this document, the name \s-1MH\s0 will be used for both of them.
751 A distinction will only be made if differences between
752 them are described.
755 .NH 2
756 Historical background
757 .XS
758 \*(SN Historical background
759 .XE
760 .LP
761 Electronic mail was available in Unix very early.
762 The first \s-1MUA\s0 on Unix was \f(CWmail\fP,
763 which was already present in the First Edition.
764 .[
765 %A Peter H. Salus
766 %T A Quarter Century of UNIX
767 %D 1994
768 %I Addison-Wesley
769 %P 41 f.
770 .]
771 It was a small program that either prints the user's mailbox file
772 or appends text to someone elses mailbox file,
773 depending on the command line arguments.
774 .[
775 %O http://cm.bell-labs.com/cm/cs/who/dmr/pdfs/man12.pdf
776 .]
777 It was a program that did one job well.
778 This job was emailing, which was very simple then.
779 .PP
780 Later, emailing became more powerful, and thus more complex.
781 The simple \f(CWmail\fP, which knew nothing of subjects,
782 independent handling of single messages,
783 and long-time storage of them, was not powerful enough anymore.
784 At Berkeley, Kurt Shoens wrote \fIMail\fP (with capital `M')
785 in 1978 to provide additional functions for emailing.
786 Mail was still one program, but now it was large and did
787 several jobs.
788 Its user interface is modeled after the one of \fIed\fP.
789 It is designed for humans, but is still scriptable.
790 \fImailx\fP is the adaptation of Berkeley Mail into System V.
791 .[
792 %A Gunnar Ritter
793 %O http://heirloom.sourceforge.net/mailx_history.html
794 .]
795 Elm, pine, mutt, and a whole bunch of graphical \s-1MUA\s0s
796 followed Mail's direction.
797 They are large, monolithic programs which include all emailing functions.
798 .PP
799 A different way was taken by the people of \s-1RAND\s0 Corporation.
800 In the beginning, they also had used a monolithic mail system,
801 called \s-1MS\s0 (for ``mail system'').
802 But in 1977, Stockton Gaines and Norman Shapiro
803 came up with a proposal of a new email system concept \(en
804 one that honors the Unix Philosophy.
805 The concept was implemented by Bruce Borden in 1978 and 1979.
806 This was the birth of \s-1MH\s0 \(en the ``mail handler''.
807 .PP
808 Since then, \s-1RAND\s0, the University of California at Irvine and
809 at Berkeley, and several others have contributed to the software.
810 However, it's core concepts remained the same.
811 In the late 90s, when development of \s-1MH\s0 slowed down,
812 Richard Coleman started with \fInmh\fP, the new mail handler.
813 His goal was to improve \s-1MH\s0, especially in regard of
814 the requirements of modern emailing.
815 Today, nmh is developed by various people on the Internet.
816 .[
817 %T RAND and the Information Evolution: A History in Essays and Vignettes
818 %A Willis H. Ware
819 %D 2008
820 %I The RAND Corporation
821 %P 128\(en137
822 %O .CW \s-1http://www.rand.org/pubs/corporate_pubs/CP537/
823 .]
824 .[
825 %T MH & xmh: Email for Users & Programmers
826 %A Jerry Peek
827 %D 1991, 1992, 1995
828 %I O'Reilly & Associates, Inc.
829 %P Appendix B
830 %O Also available online: \f(CW\s-2http://rand-mh.sourceforge.net/book/\fP
831 .]
833 .NH 2
834 Contrasts to monolithic mail systems
835 .XS
836 \*(SN Contrasts to monolithic mail systems
837 .XE
838 .LP
839 All \s-1MUA\s0s are monolithic, except \s-1MH\s0.
840 Although there might actually exist further, very little known,
841 toolchest \s-1MUA\s0s, this statement reflects the situation pretty well.
842 .PP
843 Monolithic \s-1MUA\s0s gather all their functions in one program.
844 In contrast, \s-1MH\s0 is a toolchest of many small tools \(en one for each job.
845 Following is a list of important programs of \s-1MH\s0's toolchest
846 and their function.
847 It gives a feeling of how the toolchest looks like.
848 .IP \(bu
849 .CW inc :
850 incorporate new mail (this is how mail enters the system)
851 .IP \(bu
852 .CW scan :
853 list messages in folder
854 .IP \(bu
855 .CW show :
856 show message
857 .IP \(bu
858 .CW next\fR/\fPprev :
859 show next/previous message
860 .IP \(bu
861 .CW folder :
862 change current folder
863 .IP \(bu
864 .CW refile :
865 refile message into folder
866 .IP \(bu
867 .CW rmm :
868 remove message
869 .IP \(bu
870 .CW comp :
871 compose a new message
872 .IP \(bu
873 .CW repl :
874 reply to a message
875 .IP \(bu
876 .CW forw :
877 forward a message
878 .IP \(bu
879 .CW send :
880 send a prepared message (this is how mail leaves the system)
881 .LP
882 \s-1MH\s0 has no special user interface like monolithic \s-1MUA\s0s have.
883 The user does not leave the shell to run \s-1MH\s0,
884 but he uses the various \s-1MH\s0 programs within the shell.
885 Using a monolithic program with a captive user interface
886 means ``entering'' the program, using it, and ``exiting'' the program.
887 Using toolchests like \s-1MH\s0 means running programs,
888 alone or in combination with others, even from other toolchests,
889 without leaving the shell.
891 .NH 2
892 Data storage
893 .XS
894 \*(SN Data storage
895 .XE
896 .LP
897 \s-1MH\s0's mail storage is a directory tree under the user's
898 \s-1MH\s0 directory (usually \f(CW$HOME/Mail\fP),
899 where mail folders are directories and mail messages are text files
900 within them.
901 Each mail folder contains a file \f(CW.mh_sequences\fP which lists
902 the public message sequences of that folder, for instance new messages.
903 Mail messages are text files located in a mail folder.
904 The files contain the messages as they were received.
905 They are numbered in ascending order in each folder.
906 .PP
907 This mailbox format is called ``\s-1MH\s0'' after the \s-1MUA\s0.
908 Alternatives are \fImbox\fP and \fImaildir\fP.
909 In the mbox format all messages are stored within one file.
910 This was a good solution in the early days, when messages
911 were only a few lines of text and were deleted soon.
912 Today, when single messages often include several megabytes
913 of attachments, it is a bad solution.
914 Another disadvantage of the mbox format is that it is
915 more difficult to write tools that work on mail messages,
916 because it is always necessary to first find and extract
917 the relevant message in the mbox file.
918 With the \s-1MH\s0 mailbox format,
919 each message is a self-standing item, by definition.
920 Also, the problem of concurrent access to one mailbox is
921 reduced to the problem of concurrent access to one message.
922 Maildir is generally similar to \s-1MH\s0's format,
923 but modified towards guaranteed reliability.
924 This involves some complexity, unfortunately.
925 .PP
926 Working with \s-1MH\s0's toolchest on mailboxes is much like
927 working with Unix' toolchest on directory trees:
928 \f(CWscan\fP is like \f(CWls\fP,
929 \f(CWshow\fP is like \f(CWcat\fP,
930 \f(CWfolder\fP is like \f(CWcd\fP and \f(CWpwd\fP,
931 \f(CWrefile\fP is like \f(CWmv\fP,
932 and \f(CWrmm\fP is like \f(CWrm\fP.
933 .PP
934 The context of tools in Unix consists mainly the current working directory,
935 the user identification, and the environment variables.
936 \s-1MH\s0 extends this context by two more items:
937 .IP \(bu
938 The current mail folder, which is similar to the current working directory.
939 For mail folders, \f(CWfolder\fP provides the corresponding functionality
940 of \f(CWcd\fP and \f(CWpwd\fP for directories.
941 .IP \(bu
942 Sequences, which are named sets of messages in a mail folder.
943 The current message, relative to a mail folder, is a special sequence.
944 It enables commands like \f(CWnext\fP and \f(CWprev\fP.
945 .LP
946 In contrast to Unix' context, which is chained to the shell session,
947 \s-1MH\s0's context is independent.
948 Usually there is one context for each user, but a user can have many
949 contexts.
950 Public sequences are an exception, as they belong to the mail folder.
951 .[
952 %O mh-profile(5) and mh-sequence(5)
953 .]
955 .NH 2
956 Discussion of the design
957 .XS
958 \*(SN Discussion of the design
959 .XE
960 .LP
961 The following paragraphs discuss \s-1MH\s0 in regard to the tenets
962 of the Unix Philosophy which Gancarz identified.
964 .PP
965 .B "Small is beautiful
966 and
967 .B "do one thing well
968 are two design goals that are directly visible in \s-1MH\s0.
969 Gancarz actually presents \s-1MH\s0 as example under the headline
970 ``Making UNIX Do One Thing Well'':
971 .[
972 %A Mike Gancarz
973 %T unix-phil
974 %P 125
975 .]
976 .QP
977 .ps -1
978 [\s-1MH\s0] consists of a series of programs which
979 when combined give the user an enormous ability
980 to manipulate electronic mail messages.
981 A complex application, it shows that not only is it
982 possible to build large applications from smaller
983 components, but also that such designs are actually preferable.
984 .LP
985 The various small programs of \s-1MH\s0 were relatively easy
986 to write, because each of them is small, limited to one function,
987 and has clear boundaries.
988 For the same reasons, they are also good to maintain.
989 Further more, the system can easily get extended.
990 One only needs to put a new program into the toolchest.
991 This was done, for instance, when \s-1MIME\s0 support was added
992 (e.g. \f(CWmhbuild\fP).
993 Also, different programs can exist to do the basically same job
994 in different ways (e.g. in nmh: \f(CWshow\fP and \f(CWmhshow\fP).
995 If someone needs a mail system with some additionally
996 functions that are available nowhere yet, he best takes a
997 toolchest system like \s-1MH\s0 where he can add the
998 functionality with little work.
1000 .PP
1001 .B "Store data in flat text files
1002 is followed by \s-1MH\s0.
1003 This is not surprising, because email messages are already plain text.
1004 \s-1MH\s0 stores the messages as it receives them,
1005 thus any other tool that works on RFC 2822 mail messages can operate
1006 on the messages in an \s-1MH\s0 mailbox.
1007 All other files \s-1MH\s0 uses are plain text too.
1008 It is therefore possible and encouraged to use the text processing
1009 tools of Unix' toolchest to extend \s-1MH\s0's toolchest.
1011 .PP
1012 .B "Avoid captive user interfaces" .
1013 \s-1MH\s0 is perfectly suited for non-interactive use.
1014 It offers all functions directly and without captive user interfaces.
1015 If, nonetheless, users want a graphical user interface,
1016 they can have it with \fIxmh\fP or \fIexmh\fP, too.
1017 These are graphical frontends for the \s-1MH\s0 toolchest.
1018 This means, all email-related work is still done by \s-1MH\s0 tools,
1019 but the frontend issues the appropriate calls when the user
1020 clicks on buttons.
1021 Providing easy-to-use user interfaces in form of frontends is a good
1022 approach, because it does not limit the power of the backend itself.
1023 The frontend will anyway only be able to make a subset of the
1024 backend's power and flexibility available to the user.
1025 But if it is a separate program,
1026 then the missing parts can still be accessed at the backend directly.
1027 If it is integrated, then this will hardly be possible.
1028 Further more, it is possible to have different frontends to the same
1029 backend.
1031 .PP
1032 .B "Choose portability over efficiency
1033 and
1034 .B "use shell scripts to increase leverage and portability" .
1035 These two tenets are indirectly, but nicely, demonstrated by
1036 Bolsky and Korn in their book about the Korn Shell.
1037 .[
1038 %T The KornShell: command and programming language
1039 %A Morris I. Bolsky
1040 %A David G. Korn
1041 %I Prentice Hall
1042 %D 1989
1043 %P 254\(en290
1044 %O \s-1ISBN\s0: 0-13-516972-0
1045 .]
1046 They demonstrated, in chapter 18 of the book, a basic implementation
1047 of a subset of \s-1MH\s0 in ksh scripts.
1048 Of course, this was just a demonstration, but a brilliant one.
1049 It shows how quickly one can implement such a prototype with shell scripts,
1050 and how readable they are.
1051 The implementation in the scripting language may not be very fast,
1052 but it can be fast enough though, and this is all that matters.
1053 By having the code in an interpreted language, like the shell,
1054 portability becomes a minor issue, if we assume the interpreter
1055 to be widespread.
1056 This demonstration also shows how easy it is to create single programs
1057 of a toolchest software.
1058 There are eight tools (two of them have multiple names) and 16 functions
1059 with supporting code.
1060 Each tool comprises between 12 and 38 lines of ksh,
1061 in total about 200 lines.
1062 The functions comprise between 3 and 78 lines of ksh,
1063 in total about 450 lines.
1064 Such small software is easy to write, easy to understand,
1065 and thus easy to maintain.
1066 A toolchest improves the possibility to only write some parts
1067 and though create a working result.
1068 Expanding the toolchest without global changes will likely be
1069 possible, too.
1071 .PP
1072 .B "Use software leverage to your advantage
1073 and the lesser tenet
1074 .B "allow the user to tailor the environment
1075 are ideally followed in the design of \s-1MH\s0.
1076 Tailoring the environment is heavily encouraged by the ability to
1077 directly define default options to programs.
1078 It is even possible to define different default options
1079 depending on the name under which the program was called.
1080 Software leverage is heavily encouraged by the ease it is to
1081 create shell scripts that run a specific command line,
1082 built of several \s-1MH\s0 programs.
1083 There is few software that so much wants users to tailor their
1084 environment and to leverage the use of the software, like \s-1MH\s0.
1085 Just to make one example:
1086 One might prefer a different listing format for the \f(CWscan\fP
1087 program.
1088 It is possible to take one of the distributed format files
1089 or to write one yourself.
1090 To use the format as default for \f(CWscan\fP, a single line,
1091 reading
1092 .DS
1093 .CW
1094 scan: -form FORMATFILE
1095 .DE
1096 must be added to \f(CW.mh_profile\fP.
1097 If one wants this different format as an additional command,
1098 instead of changing the default, he needs to create a link to
1099 \f(CWscan\fP, for instance titled \f(CWscan2\fP.
1100 The line in \f(CW.mh_profile\fP would then start with \f(CWscan2\fP,
1101 as the option should only be in effect when scan is called as
1102 \f(CWscan2\fP.
1104 .PP
1105 .B "Make every program a filter
1106 is hard to find in \s-1MH\s0.
1107 The reason therefore is that most of \s-1MH\s0's tools provide
1108 basic file system operations for the mailboxes.
1109 The reason is the same because of which
1110 \f(CWls\fP, \f(CWcp\fP, \f(CWmv\fP, and \f(CWrm\fP
1111 aren't filters neither.
1112 However, they build a basis on which filters can operate.
1113 \s-1MH\s0 does not provide many filters itself, but it is a basis
1114 to write filters for.
1115 An example would be a mail message text highlighter,
1116 that means a program that makes use of a color terminal to display
1117 header lines, quotations, and signatures in distinct colors.
1118 The author's version of this program, for instance,
1119 is a 25 line awk script.
1121 .PP
1122 .B "Build a prototype as soon as possible
1123 was again well followed by \s-1MH\s0.
1124 This tenet, of course, focuses on early development, which is
1125 long time ago for \s-1MH\s0.
1126 But without following this guideline at the very beginning,
1127 Bruce Borden may have not convinced the management of \s-1RAND\s0
1128 to ever create \s-1MH\s0.
1129 In Bruce' own words:
1130 .[
1131 %O FIXME
1132 .]
1133 .QP
1134 .ps -1
1135 [...] but they [Stockton Gaines and Norm Shapiro] were not able
1136 to convince anyone that such a system would be fast enough to be usable.
1137 I proposed a very short project to prove the basic concepts,
1138 and my management agreed.
1139 Looking back, I realize that I had been very lucky with my first design.
1140 Without nearly enough design work,
1141 I built a working environment and some header files
1142 with key structures and wrote the first few \s-1MH\s0 commands:
1143 inc, show/next/prev, and comp.
1144 [...]
1145 With these three, I was able to convince people that the structure was viable.
1146 This took about three weeks.
1148 .NH 2
1149 Problems
1150 .XS
1151 \*(SN Problems
1152 .XE
1153 .LP
1154 \s-1MH\s0, for sure is not without problems.
1155 There are two main problems: one is technical, the other is about human behavior.
1156 .PP
1157 \s-1MH\s0 is old and email today is very different to email in the time
1158 when \s-1MH\s0 was designed.
1159 \s-1MH\s0 adopted to the changes pretty well, but it is limited.
1160 For example in development resources.
1161 \s-1MIME\s0 support and support for different character encodings
1162 is available, but only on a moderate level.
1163 More active developers could quickly improve there.
1164 It is also limited by design, which is the larger problem.
1165 \s-1IMAP\s0, for example, conflicts with \s-1MH\s0's design to a large extend.
1166 These design conflicts are not easily solvable.
1167 Possibly, they require a redesign.
1168 Maybe \s-1IMAP\s0 is too different to the classic mail model which \s-1MH\s0 covers,
1169 hence \s-1MH\s0 may never work well with \s-1IMAP\s0.
1170 .PP
1171 The other kind of problem is human habits.
1172 When in this world almost all \s-1MUA\s0s are monolithic,
1173 it is very difficult to convince people to use a toolbox style \s-1MUA\s0
1174 like \s-1MH\s0.
1175 The habits are so strong, that even people who understood the concept
1176 and advantages of \s-1MH\s0 do not like to switch,
1177 simply because \s-1MH\s0 is different.
1178 Unfortunately, the frontends to \s-1MH\s0, which could provide familiar look'n'feel,
1179 are quite outdated and thus not very appealing compared to the modern interfaces
1180 which monolithic \s-1MUA\s0s offer.
1182 .NH 2
1183 Summary \s-1MH\s0
1184 .XS
1185 \*(SN Summary \s-1MH\s0
1186 .XE
1187 .LP
1188 \s-1MH\s0 is an \s-1MUA\s0 that follows the Unix Philosophy in its design
1189 and implementation.
1190 It consists of a toolchest of small tools, each of them does one job well.
1191 The tools are orthogonal to each other, to a large extend.
1192 However, for historical reasons, there also exist distinct tools
1193 that cover the same task.
1194 .PP
1195 The toolchest approach offers great flexibility to the user.
1196 He can use the complete power of the Unix shell with \s-1MH\s0.
1197 This makes \s-1MH\s0 a very powerful mail system.
1198 Extending and customizing \s-1MH\s0 is easy and encouraged, too.
1199 .PP
1200 Apart from the user's perspective, \s-1MH\s0 is development-friendly.
1201 Its overall design follows clear rules.
1202 The single tools do only one job, thus they are easy to understand,
1203 easy to write, and good to maintain.
1204 They are all independent and do not interfere with the others.
1205 Automated testing of their function is a straight forward task.
1206 .PP
1207 It is sad, that \s-1MH\s0's differentness is its largest problem,
1208 as its differentness is also its largest advantage.
1209 Unfortunately, for most people their habits are stronger
1210 than the attraction of the clear design and the power, \s-1MH\s0 offers.
1214 .NH 1
1215 Case study: uzbl
1216 .XS
1217 .sp .5v
1218 \*(SN Case study: uzbl
1219 .XE
1220 .LP
1221 The last chapter took a look on the \s-1MUA\s0 \s-1MH\s0,
1222 this chapter is about uzbl, a web browser that adheres to the Unix Philosophy.
1223 ``uzbl'' is the \fIlolcat\fP's word for the English adjective ``usable''.
1224 It is pronounced the identical.
1226 .NH 2
1227 Historical background
1228 .XS
1229 \*(SN Historical background
1230 .XE
1231 .LP
1232 Uzbl was started by Dieter Plaetinck in April 2009.
1233 The idea was born in a thread in the Arch Linux forum.
1234 .[
1235 %O http://bbs.archlinux.org/viewtopic.php?id=67463
1236 .]
1237 After some discussion about failures of well known web browsers,
1238 Plaetinck (alias Dieter@be) came up with a very sketchy proposal
1239 of how a better web browser could look like.
1240 To the question of another member, if Plaetinck would write that program,
1241 because it would sound fantastic, Plaetinck replied:
1242 ``Maybe, if I find the time ;-)''.
1243 .PP
1244 Fortunately, he found the time.
1245 One day later, the first prototype was out.
1246 One week later, uzbl had an own website.
1247 One month after the first code showed up,
1248 a mailing list was installed to coordinate and discuss further development.
1249 A wiki was set up to store documentation and scripts that showed up on the
1250 mailing list and elsewhere.
1251 .PP
1252 In the, now, one year of uzbl's existence, it was heavily developed in various branches.
1253 Plaetinck's task became more and more to only merge the best code from the
1254 different branches into his main branch, and to apply patches.
1255 About once a month, Plaetinck released a new version.
1256 In September 2009, he presented several forks of uzbl.
1257 Uzbl, actually, opened the field for a whole family of web browsers with similar shape.
1258 .PP
1259 In July 2009, \fILinux Weekly News\fP published an interview with Plaetinck about uzbl.
1260 In September 2009, the uzbl web browser was on \fISlashdot\fP.
1262 .NH 2
1263 Contrasts to other web browsers
1264 .XS
1265 \*(SN Contrasts to other web browsers
1266 .XE
1267 .LP
1268 Like most \s-1MUA\s0s are monolithic, but \s-1MH\s0 is a toolchest,
1269 most web browsers are monolithic, but uzbl is a frontend to a toolchest.
1270 .PP
1271 Today, uzbl is divided into uzbl-core and uzbl-browser.
1272 Uzbl-core is, how its name already indicates, the core of uzbl.
1273 It handles commands and events to interface other programs,
1274 and also displays webpages by using webkit as render engine.
1275 Uzbl-browser combines uzbl-core with a bunch of handler scripts, a status bar,
1276 an event manager, yanking, pasting, page searching, zooming, and more stuff,
1277 to form a ``complete'' web browser.
1278 In the following text, the term ``uzbl'' usually stands for uzbl-browser,
1279 so uzbl-core is included.
1280 .PP
1281 Unlike most other web browsers, uzbl is mainly the mediator between the
1282 various tools that cover single jobs of web browsing.
1283 Therefore, uzbl listens for commands on a named pipe (fifo), a Unix socket,
1284 and on stdin, and it writes events to a Unix socket and to stdout.
1285 The graphical rendering of the webpage is done by webkit, a web content engine.
1286 Uzbl-core is build around this library.
1287 Loading a webpage in a running uzbl instance requires only:
1288 .DS
1289 .CW
1290 echo 'uri http://example.org' >/path/to/uzbl-fifo
1291 .DE
1292 .PP
1293 Downloads, browsing history, bookmarks, and thelike are not provided
1294 by uzbl-core itself, as they are in other web browsers.
1295 Uzbl-browser also only provides, so called, handler scripts that wrap
1296 external applications which provide the actual functionality.
1297 For instance, \fIwget\fP is used to download files and uzbl-browser
1298 includes a script that calls wget with appropriate options in
1299 a prepared environment.
1300 .PP
1301 Modern web browsers are proud to have addons, plugins, and modules, instead.
1302 This is their effort to achieve similar goals.
1303 But instead of using existing, external programs, modern web browsers
1304 include these functions, although they might be loaded at runtime.
1306 .NH 2
1307 Discussion of the design
1308 .XS
1309 \*(SN Discussion of the design
1310 .XE
1311 .LP
1312 This section discusses uzbl in regard of the Unix Philosophy,
1313 as identified by Gancarz.
1315 .PP
1316 .B "Make each program do one thing well" .
1317 Uzbl tries to be a web browser and nothing else.
1318 The common definition of a web browser is, of course, highly influenced by
1319 existing implementations of web browsers, although they are degenerated.
1320 Web browsers should be programs to browse the web, and nothing more.
1321 This is the one thing they should do, as demanded by the Unix Philosophy.
1322 .PP
1323 Web browsers should, for instance, not manage downloads.
1324 This is the job download managers exist for.
1325 Download managers do primary care about being good in downloading files.
1326 Modern web browsers provide download management only as a secondary feature.
1327 How could they perform this job better, than programs that exist only for
1328 this very job?
1329 And how could anyone want less than the best download manager available?
1330 .PP
1331 A web browser's job is to let the user browse the web.
1332 This means, navigating through websites by following links.
1333 Rendering the \s-1HTML\s0 sources is a different job, too.
1334 It is covered by the webkit render engine, in uzbl's case.
1335 Audio and video content and files like PostScript, \s-1PDF\s0, and the like,
1336 are also not the job of a web browser.
1337 They should be handled by external applications \(en
1338 ones which's job is to handle such data.
1339 Uzbl strives to do it this way.
1340 .PP
1341 Remember Doug McIlroy:
1342 .I
1343 ``Write programs that do one thing and do it well.
1344 Write programs to work together.''
1345 .R
1346 .PP
1347 The lesser tenet
1348 .B "allow the user to tailor the environment
1349 matches good here.
1350 There was the question, how anyone could want anything less than the
1351 best program for the job.
1352 But as personal preferences matter much,
1353 it is probably more important to ask:
1354 How could anyone want something else than his preferred program for the job?
1355 .PP
1356 Usually users want one program for one job.
1357 Hence, whenever the task is, for instance, downloading,
1358 exactly one download manager should be used.
1359 More advanced users might want to have this download manager in this
1360 situation and that one in that situation.
1361 They should be able to configure it this way.
1362 With uzbl, one can use any download manager the user wants.
1363 To switch to a different one, only one line in a small handler script
1364 needs to be changed.
1365 Alternatively it would be possible to query an entry in a global file
1366 or an environment variable, which specifies the download manager to use,
1367 in the handler script.
1368 .PP
1369 As uzbl does neither have its own download manager nor depends on a
1370 specific one, thus uzbl's browsing abilities will not be lowered by having
1371 a bad download manager.
1372 Uzbl's download capabilities will just as good as the ones of the best
1373 download manager available on the system.
1374 Of course, this applies to all of the other supplementary tools, too.
1376 .PP
1377 .B "Use software leverage to your advantage" .
1378 Shell scripts are a good choice to extend uzbl.
1379 Uzbl is designed to be extended by external tools.
1380 These external tools are usually wrapped by small handler shell scripts.
1381 Shell scripts are the glue in this approach.
1382 They make the various parts fit together.
1383 .PP
1384 As an example, the history mechanism of uzbl shall be presented.
1385 Uzbl is configured to spawn a script to append an entry to the history
1386 whenever the event of a fully loaded page occurs.
1387 The script to append the entry to the history not much more than:
1388 .DS
1389 .CW
1390 #!/bin/sh
1391 file=/path/to/uzbl-history
1392 echo `date +'%Y-%m-%d %H:%M:%S'`" $6 $7" >> $file
1393 .DE
1394 \f(CW$6\fP and \f(CW$7\fP expand to the \s-1URL\s0 and the page title.
1395 For loading an entry, a key is bound to spawn a load from history script.
1396 The script reverses the history to have newer entries first,
1397 then displays \fIdmenu\fP to select an item,
1398 and afterwards writes the selected \s-1URL\s0 into uzbl's command input pipe.
1399 With error checking and corner cases removed, the script looks like this:
1400 .DS
1401 .CW
1402 #!/bin/sh
1403 file=/path/to/uzbl-history
1404 goto=`tac $file | dmenu | cut -d' ' -f 3`
1405 echo "uri $goto" > $4
1406 .DE
1407 \f(CW$4\fP expands to the path of the command input pipe of the current
1408 uzbl instance.
1410 .PP
1411 .B "Avoid captive user interfaces" .
1412 One could say, that uzbl, to a large extend, actually \fIis\fP
1413 a captive user interface.
1414 But the difference to most other web browsers is, that uzbl is only
1415 the captive user interface frontend and the core of the backend.
1416 Many parts of the backend are independent of uzbl.
1417 Some are distributed with uzbl, for some external programs, handler scripts
1418 are distributed, arbitrary additional functionality can be added if desired.
1419 .PP
1420 The frontend is captive \(en that is true.
1421 This is okay for the task of browsing the web, as this task is only relevant
1422 for humans.
1423 Automated programs would \fIcrawl\fP the web.
1424 That means, they read the source directly.
1425 The source includes all the semantics.
1426 The graphical representation is just for humans to transfer the semantics
1427 more intuitively.
1429 .PP
1430 .B "Make every program a filter" .
1431 Graphical web browsers are almost dead ends in the chain of information flow.
1432 Thus it is difficult to see what graphical web browsers should filter.
1433 Graphical web browsers exist almost only for interactive use by humans.
1434 The only case when one might want to automate the rendering function is
1435 to generate images of rendered webpages.
1437 .PP
1438 .B "Small is beautiful"
1439 is not easy to apply to a web browser, primary because modern web technology
1440 is very complex; hence the rendering task is very complex.
1441 Modern web browsers will always consist of many thousand lines of code,
1442 unfortunately.
1443 Using the toolchest approach and wrappers can split the browser into
1444 several small parts, tough.
1445 .PP
1446 Uzbl-core consists of about 3\,500 lines of C code.
1447 The distribution includes another 3\,500 lines of Shell and Python code,
1448 which are the handler scripts and plugins like a modal interface.
1449 Further more, uzbl uses functionality of external tools like
1450 \fIwget\fP and \fInetcat\fP.
1451 Up to this point, uzbl looks pretty neat and small.
1452 The ugly part of uzbl is the web content renderer, webkit.
1453 Webkit consists of roughly 400\,000 (!) lines of code.
1454 Unfortunately, small web render engines are not possible anymore
1455 because of the modern web.
1456 The problems section will explain this in more detail.
1458 .PP
1459 .B "Build a prototype as soon as possible" .
1460 Plaetinck made his code public, right from the beginning.
1461 Discussion and development was, and still is, open to everyone interested.
1462 Development versions of uzbl can be obtained very simply from the code
1463 repository.
1464 Within the first year of uzbl's existence, a new version was released
1465 more often than once a month.
1466 Different forks and branches arose.
1467 They introduced new features, which were tested for suitability.
1468 The experiences of using prototypes influenced further development.
1469 Actually, all development was community driven.
1470 Plaetinck says, three months after uzbl's birth:
1471 ``Right now I hardly code anything myself for Uzbl.
1472 I just merge in other people's code, ponder a lot, and lead the discussions.''
1473 .[
1474 %A FIXME
1475 %O http://lwn.net/Articles/341245/
1476 .]
1479 .NH 2
1480 Problems
1481 .XS
1482 \*(SN Problems
1483 .XE
1484 .LP
1485 Similar to \s-1MH\s0, uzbl, too suffers from being different.
1486 It is sad, but people use what they know.
1487 Fortunately, uzbl's user interface can look and feel very much the
1488 same as the one of the well known web browsers,
1489 hiding the internal differences.
1490 But uzbl has to provide this similar look and feel to be accepted
1491 as a ``normal'' browser by ``normal'' users.
1492 .PP
1493 The more important problem is the modern web.
1494 The modern web is simply broken.
1495 It has state in a state-less protocol,
1496 it misuses technologies,
1497 and it is helplessly overloaded.
1498 The result are web content render engines that must consist
1499 of hundreds of thousands lines of code.
1500 They also must combine and integrate many different technologies,
1501 only to make our modern web usable.
1502 Website to image converter are hardly possible to run without
1503 human interaction because of state in sessions, impossible
1504 deep-linking, and unautomatable technologies.
1505 .PP
1506 The web was misused to provide all kinds of imaginable wishes.
1507 Now web browsers, and eventually the users, suffer from it.
1510 .NH 2
1511 Summary uzbl
1512 .XS
1513 \*(SN Summary uzbl
1514 .XE
1515 .LP
1516 ``Uzbl is a browser that adheres to the Unix Philosophy'',
1517 that is how uzbl is seen by its authors.
1518 Indeed, uzbl follows the Unix Philosophy in many ways.
1519 It consists of independent parts that work together,
1520 its core is mainly a mediator which glues the parts together.
1521 .PP
1522 Software leverage can excellently be seen in uzbl.
1523 It makes use of external tools, separates independent tasks
1524 in independent parts, and glues them together with small
1525 handler scripts, around uzbl-core.
1526 .PP
1527 As uzbl, more or less, consists of a set of tools and a bit
1528 of glue, anyone can put the parts together and expand it
1529 in any desired way.
1530 Uzbl is very flexible and customizable.
1531 These properties make it valuable for advanced users,
1532 but may keep novice users from using it.
1533 .PP
1534 Uzbl's main problem is the modern web, that makes it hard
1535 to design a sane web browser.
1536 Despite this bad situation, uzbl does a fairly good job.
1539 .NH 1
1540 Final thoughts
1541 .XS
1542 .sp .5v
1543 \*(SN Final thoughts
1544 .XE
1546 .NH 2
1547 Quick summary
1548 .XS
1549 \*(SN Quick summary
1550 .XE
1551 .LP
1552 good design
1553 .LP
1554 unix phil
1555 .LP
1556 case studies
1558 .NH 2
1559 Why people should choose
1560 .XS
1561 \*(SN Why people should choose
1562 .XE
1563 .LP
1564 Make the right choice!
1566 .nr PI .3i
1567 .rm ]<
1568 .de ]<
1569 .LP
1570 .de FP
1571 .IP \\\\$1.
1572 \\..
1573 .rm FS FE
1574 ..
1575 .ds CH "
1576 .bp
1577 .rs
1578 .sp .3i
1579 .TL
1580 References
1581 .LP
1582 .XS
1583 .sp .5v
1584 References
1585 .XE
1586 .sp 2v
1587 .nr PS -1
1588 .nr VS -1
1589 .[
1590 $LIST$
1591 .]
1592 .\".wh -1p
1593 .bp
1594 .PX