docs/unix-phil
view unix-phil.ms @ 44:46e34e433231
moved bibliography to external file
| author | meillo@marmaro.de |
|---|---|
| date | Sat, 10 Apr 2010 15:00:38 +0200 |
| parents | 303e8f449e77 |
| children | ade392f024aa |
line source
1 .nr PS 11
2 .nr VS 13
3 .nr lu 0
4 .de CW
5 .nr PQ \\n(.f
6 .if t .ft CW
7 .ie ^\\$1^^ .if n .ul 999
8 .el .if n .ul 1
9 .if t .if !^\\$1^^ \&\\$1\f\\n(PQ\\$2
10 .if n .if \\n(.$=1 \&\\$1
11 .if n .if \\n(.$>1 \&\\$1\c
12 .if n .if \\n(.$>1 \&\\$2
13 ..
14 .ds [. \ [
15 .ds .] ]
17 .rn NH _N
18 .de NH
19 .if '\\$1'1' .sp 2v
20 .if '\\$1'1' .nr PS +2
21 ._N \\$1
22 .if '\\$1'1' .nr PS -2
23 ..
25 .\"----------------------------------------
27 .TL
28 .ps +4
29 Why the Unix Philosophy still matters
30 .AU
31 markus schnalke <meillo@marmaro.de>
32 .AB
33 .ti \n(.iu
34 This paper explains the importance of the Unix Philosophy for software design.
35 Today, few software designers are aware of these concepts,
36 and thus a lot of modern software is more limited than necessary
37 and makes less use of software leverage than possible.
38 Knowing and following the guidelines of the Unix Philosophy makes software more valuable.
39 .AE
41 .FS
42 .ps -1
43 This paper was prepared for the ``Software Analysis'' seminar at University Ulm.
44 Mentor was professor Schweiggert. 2010-04-05
45 .br
46 You may retrieve this document from
47 .CW \s-1http://marmaro.de/docs \ .
48 .FE
50 .NH 1
51 Introduction
52 .XS
53 \*(SN Introduction
54 .XE
55 .LP
56 The Unix Philosophy is the essence of how the Unix operating system,
57 especially its toolchest, was designed.
58 It is no limited set of fixed rules,
59 but a loose set of guidelines which tell how to write software that
60 suites well into Unix.
61 Actually, the Unix Philosophy describes what is common to typical Unix software.
62 The Wikipedia has an accurate definition:
63 .[
64 wikipedia
65 unix philosophy
66 .]
67 .QP
68 .ps -1
69 The \fIUnix philosophy\fP is a set of cultural norms and philosophical
70 approaches to developing software based on the experience of leading
71 developers of the Unix operating system.
72 .PP
73 As there is no single definition of the Unix Philosophy,
74 several people have stated their view on what it comprises.
75 Best known are:
76 .IP \(bu
77 Doug McIlroy's summary: ``Write programs that do one thing and do it well.''
78 .[
79 mahoney
80 oral history
81 .]
82 .IP \(bu
83 Mike Gancarz' book ``The UNIX Philosophy''.
84 .[
85 gancarz
86 unix philosophy
87 .]
88 .IP \(bu
89 Eric S. Raymond's book ``The Art of UNIX Programming''.
90 .[
91 raymond
92 art of unix programming
93 .]
94 .LP
95 These different views on the Unix Philosophy have much in common.
96 Especially, the main concepts are similar in all of them.
97 McIlroy's definition can surely be called the core of the Unix Philosophy,
98 but the fundamental idea behind it all, is ``small is beautiful''.
100 .PP
101 The Unix Philosophy tells how to design and write good software for Unix.
102 Many concepts described here base on facilities of Unix.
103 Other operating systems may not offer such facilities,
104 hence it may not be possible to design software in the way of the
105 Unix Philosophy for them.
106 .PP
107 The Unix Philosophy has an idea of how the process of software development
108 should look like, but large parts of the philosophy are quite independent
109 from the development process used.
110 However, one will soon recognize that some development processes work well
111 with the ideas of the Unix Philosophy and support them, while others are
112 at cross-purposes.
113 Kent Beck's books about Extreme Programming are valuable supplimental
114 resources.
115 .PP
116 The question of how to actually write code and how the code should looks
117 like internally, are out of focus here.
118 ``The Practice of Programming'' by Kernighan and Pike,
119 .[
120 kernighan pike
121 practice of programming
122 .]
123 is a good book that covers this topic.
124 Its point of view matches to the one of this paper.
126 .NH 1
127 Importance of software design in general
128 .XS
129 .sp .5v
130 \*(SN Importance of software design in general
131 .XE
132 .LP
133 Software design is the planning of how the internal structure
134 and external interfaces of a software should look like.
135 It has nothing to do with visual appearance.
136 If we take a program as a car, then its color is of no matter.
137 Its design would be the car's size, its shape, the locations of doors,
138 the passenger/space ratio, the luggage capacity, and so forth.
139 .PP
140 Why should software get designed at all?
141 It is general knowledge, that even a bad plan is better than no plan.
142 Not designing software means programming without plan.
143 This will pretty sure lead to horrible results.
144 Horrible to use and horrible to maintain.
145 These two aspects are the visible ones.
146 Often invisible are the wasted possible gains.
147 Good software design can make these gains available.
148 .PP
149 A software's design deals with quality properties.
150 Good design leads to good quality, and quality is important.
151 Any car may be able to drive from A to B,
152 but it depends on the car's properties whether it is a good choice
153 for passenger transport or not.
154 It depends on its properties if it is a good choice
155 for a rough mountain area.
156 And it depends on its properties if the ride will be fun.
158 .PP
159 Requirements for a software are twofold:
160 functional and non-functional.
161 .IP \(bu
162 Functional requirements define directly the software's functions.
163 They are the reason why software gets written.
164 Someone has a problem and needs a tool to solve it.
165 Being able to solve the problem is the main functional goal.
166 It is the driving force behind all programming effort.
167 Functional requirements are easier to define and to verify.
168 .IP \(bu
169 Non-functional requirements are also called \fIquality\fP requirements.
170 The quality of a software are the properties that are not directly related to
171 the software's basic functions.
172 Tools of bad quality often solve the problems they were written for,
173 but introduce problems and difficulties for usage and development, later on.
174 Quality aspects are often overlooked at first sight,
175 and they are often difficult to define clearly and to verify.
176 .PP
177 Quality is of few matter when the software gets built initially,
178 but it is of matter for usage and maintenance of the software.
179 A short-sighted might see in developing a software mainly building something up.
180 But experience shows, that building the software the first time is
181 only a small amount of the overall work.
182 Bug fixing, extending, rebuilding of parts
183 \(en maintenance work, for short \(en
184 does soon take over the major part of the time spent on a software.
185 Not to forget the usage of the software.
186 These processes are highly influenced by the software's quality.
187 Thus, quality must not be neglected.
188 The problem with quality is that you hardly ``stumble over''
189 bad quality during the first build,
190 but this is the time when you should care about good quality most.
191 .PP
192 Software design is less the basic function of a software \(en
193 this requirement will get satisfied anyway.
194 Software design is more about quality aspects of the software.
195 Good design leads to good quality, bad design to bad quality.
196 The primary functions of the software will be affected modestly by bad quality,
197 but good quality can provide a lot of additional gain,
198 even at places where one never expected it.
199 .PP
200 The ISO/IEC 9126-1 standard, part 1,
201 .[
202 iso product quality
203 .]
204 defines the quality model as consisting out of:
205 .IP \(bu
206 .I Functionality
207 (suitability, accuracy, inter\%operability, security)
208 .IP \(bu
209 .I Reliability
210 (maturity, fault tolerance, recoverability)
211 .IP \(bu
212 .I Usability
213 (understandability, learnability, operability, attractiveness)
214 .IP \(bu
215 .I Efficiency
216 (time behavior, resource utilization)
217 .IP \(bu
218 .I Maintainability
219 (analyzability, changeability, stability, testability)
220 .IP \(bu
221 .I Portability
222 (adaptability, installability, co-existence, replaceability)
223 .LP
224 Good design can improve these properties of a software,
225 bad designed software probably suffers from not having them.
226 .PP
227 One further goal of software design is consistency.
228 Consistency eases understanding, working on, and using things.
229 Consistent internal structure and consistent interfaces to the outside
230 can be provided by good design.
231 .PP
232 Software should be well designed because good design avoids many
233 problems during the software's lifetime.
234 And software should be well designed because good design can offer
235 much additional gain.
236 Indeed, much effort should be spent into good design to make software more valuable.
237 The Unix Philosophy shows a way of how to design software well.
238 It offers guidelines to achieve good quality and high gain for the effort spent.
241 .NH 1
242 The Unix Philosophy
243 .XS
244 .sp .5v
245 \*(SN The Unix Philosophy
246 .XE
247 .LP
248 The origins of the Unix Philosophy were already introduced.
249 This chapter explains the philosophy, oriented on Gancarz,
250 and shows concrete examples of its application.
252 .NH 2
253 Pipes
254 .XS
255 \*(SN Pipes
256 .XE
257 .LP
258 Following are some examples to demonstrate how applied Unix Philosophy feels like.
259 Knowledge of using the Unix shell is assumed.
260 .PP
261 Counting the number of files in the current directory:
262 .DS
263 .CW
264 .ps -1
265 ls | wc -l
266 .DE
267 The
268 .CW ls
269 command lists all files in the current directory, one per line,
270 and
271 .CW "wc -l
272 counts the number of lines.
273 .PP
274 Counting the number of files that do not contain ``foo'' in their name:
275 .DS
276 .CW
277 .ps -1
278 ls | grep -v foo | wc -l
279 .DE
280 Here, the list of files is filtered by
281 .CW grep
282 to remove all that contain ``foo''.
283 The rest is the same as in the previous example.
284 .PP
285 Finding the five largest entries in the current directory.
286 .DS
287 .CW
288 .ps -1
289 du -s * | sort -nr | sed 5q
290 .DE
291 .CW "du -s *
292 returns the recursively summed sizes of all files
293 \(en no matter if they are regular files or directories.
294 .CW "sort -nr
295 sorts the list numerically in reverse order.
296 Finally,
297 .CW "sed 5q
298 quits after it has printed the fifth line.
299 .PP
300 The presented command lines are examples of what Unix people would use
301 to get the desired output.
302 There are also other ways to get the same output.
303 It's a user's decision which way to go.
304 .PP
305 The examples show that many tasks on a Unix system
306 are accomplished by combining several small programs.
307 The connection between the single programs is denoted by the pipe operator `|'.
308 .PP
309 Pipes, and their extensive and easy use, are one of the great
310 achievements of the Unix system.
311 Pipes between programs have been possible in earlier operating systems,
312 but it has never been a so central part of the concept.
313 When, in the early seventies, Doug McIlroy introduced pipes for the
314 Unix system,
315 ``it was this concept and notation for linking several programs together
316 that transformed Unix from a basic file-sharing system to an entirely new way of computing.''
317 .[
318 aughenbaugh
319 unix oral history
320 .]
321 .PP
322 Being able to specify pipelines in an easy way is,
323 however, not enough by itself.
324 It is only one half.
325 The other is the design of the programs that are used in the pipeline.
326 They have to interfaces that allows them to be used in such a way.
328 .NH 2
329 Interface design
330 .XS
331 \*(SN Interface design
332 .XE
333 .LP
334 Unix is, first of all, simple \(en Everything is a file.
335 Files are sequences of bytes, without any special structure.
336 Programs should be filters, which read a stream of bytes from ``standard input'' (stdin)
337 and write a stream of bytes to ``standard output'' (stdout).
338 .PP
339 If the files \fIare\fP sequences of bytes,
340 and the programs \fIare\fP filters on byte streams,
341 then there is exactly one standardized data interface.
342 Thus it is possible to combine them in any desired way.
343 .PP
344 Even a handful of small programs will yield a large set of combinations,
345 and thus a large set of different functions.
346 This is leverage!
347 If the programs are orthogonal to each other \(en the best case \(en
348 then the set of different functions is greatest.
349 .PP
350 Programs might also have a separate control interface,
351 besides their data interface.
352 The control interface is often called ``user interface'',
353 because it is usually designed to be used by humans.
354 The Unix Philosophy discourages to assume the user to be human.
355 Interactive use of software is slow use of software,
356 because the program waits for user input most of the time.
357 Interactive software requires the user to be in front of the computer
358 all the time.
359 Interactive software occupy the user's attention while they are running.
360 .PP
361 Now we come back to the idea of using several small programs, combined,
362 to have a more specific function.
363 If these single tools would all be interactive,
364 how would the user control them?
365 It is not only a problem to control several programs at once if they run at the same time,
366 it also very inefficient to have to control each of the single programs
367 that are intended to work as one large program.
368 Hence, the Unix Philosophy discourages programs to demand interactive use.
369 The behavior of programs should be defined at invocation.
370 This is done by specifying arguments (``command line switches'') to the program call.
371 Gancarz discusses this topic as ``avoid captive user interfaces''.
372 .[
373 gancarz unix philosophy
374 %P 88 ff.
375 .]
376 .ds _p ", p. 88 ff.
377 .PP
378 Non-interactive use is, during development, also an advantage for testing.
379 Testing of interactive programs is much more complicated,
380 than testing of non-interactive programs.
382 .NH 2
383 The toolchest approach
384 .XS
385 \*(SN The toolchest approach
386 .XE
387 .LP
388 A toolchest is a set of tools.
389 Instead of having one big tool for all tasks, one has many small tools,
390 each for one task.
391 Difficult tasks are solved by combining several of the small, simple tools.
392 .PP
393 The Unix toolchest \fIis\fP a set of small, (mostly) non-interactive programs
394 that are filters on byte streams.
395 They are, to a large extend, unrelated in their function.
396 Hence, the Unix toolchest provides a large set of functions
397 that can be accessed by combining the programs in the desired way.
398 .PP
399 There are also advantages for developing small toolchest programs.
400 It is easier and less error-prone to write small programs.
401 It is also easier and less error-prone to write a large set of small programs,
402 than to write one large program with all the functionality included.
403 If the small programs are combinable, then they offer even a larger set
404 of functions than the single large program.
405 Hence, one gets two advantages out of writing small, combinable programs.
406 .PP
407 There are two drawbacks of the toolchest approach.
408 First, one simple, standardized, unidirectional interface has to be sufficient.
409 If one feels the need for more ``logic'' than a stream of bytes,
410 then a different approach might be of need.
411 But it is also possible, that he just can not imagine a design where
412 a stream of bytes is sufficient.
413 By becoming more familiar with the ``Unix style of thinking'',
414 developers will more often and easier find simple designs where
415 a stream of bytes is a sufficient interface.
416 .PP
417 The second drawback of a toolchest affects the users.
418 A toolchest is often more difficult to use for novices.
419 It is necessary to become familiar with each of the tools,
420 to be able to use the right one in a given situation.
421 Additionally, one needs to combine the tools in a senseful way on its own.
422 This is like a sharp knife \(en it is a powerful tool in the hand of a master,
423 but of no good value in the hand of an unskilled.
424 .PP
425 However, learning single, small tool of the toolchest is easier than
426 learning a complex tool.
427 The user will have a basic understanding of a yet unknown tool,
428 if the several tools of the toolchest have a common style.
429 He will be able to transfer knowledge over one tool to another.
430 .PP
431 Moreover, the second drawback can be removed easily by adding wrappers
432 around the single tools.
433 Novice users do not need to learn several tools if a professional wraps
434 the single commands into a more high-level script.
435 Note that the wrapper script still calls the small tools;
436 the wrapper script is just like a skin around.
437 No complexity is added this way,
438 but new programs can get created out of existing one with very low effort.
439 .PP
440 A wrapper script for finding the five largest entries in the current directory
441 could look like this:
442 .DS
443 .CW
444 .ps -1
445 #!/bin/sh
446 du -s * | sort -nr | sed 5q
447 .DE
448 The script itself is just a text file that calls the command line
449 a professional user would type in directly.
450 Making the program flexible on the number of entries it prints,
451 is easily possible:
452 .DS
453 .CW
454 .ps -1
455 #!/bin/sh
456 num=5
457 [ $# -eq 1 ] && num="$1"
458 du -sh * | sort -nr | sed "${num}q"
459 .DE
460 This script acts like the one before, when called without an argument.
461 But one can also specify a numerical argument to define the number of lines to print.
463 .NH 2
464 A powerful shell
465 .XS
466 \*(SN A powerful shell
467 .XE
468 .LP
469 It was already said, that the Unix shell provides the possibility to
470 combine small programs into large ones easily.
471 A powerful shell is a great feature in other ways, too.
472 .PP
473 For instance by including a scripting language.
474 The control statements are build into the shell.
475 The functions, however, are the normal programs, everyone can use on the system.
476 Thus, the programs are known, so learning to program in the shell is easy.
477 Using normal programs as functions in the shell programming language
478 is only possible because they are small and combinable tools in a toolchest style.
479 .PP
480 The Unix shell encourages to write small scripts out of other programs,
481 because it is so easy to do.
482 This is a great step towards automation.
483 It is wonderful if the effort to automate a task equals the effort
484 it takes to do it the second time by hand.
485 If it is so, then the user will be happy to automate everything he does more than once.
486 .PP
487 Small programs that do one job well, standardized interfaces between them,
488 a mechanism to combine parts to larger parts, and an easy way to automate tasks,
489 this will inevitably produce software leverage.
490 Getting multiple times the benefit of an investment is a great offer.
491 .PP
492 The shell also encourages rapid prototyping.
493 Many well known programs started as quickly hacked shell scripts,
494 and turned into ``real'' programs, written in C, later.
495 Building a prototype first is a way to avoid the biggest problems
496 in application development.
497 Fred Brooks writes in ``No Silver Bullet'':
498 .[
499 brooks
500 no silver bullet
501 .]
502 .QP
503 .ps -1
504 The hardest single part of building a software system is deciding precisely what to build.
505 No other part of the conceptual work is so difficult as establishing the detailed
506 technical requirements, [...].
507 No other part of the work so cripples the resulting system if done wrong.
508 No other part is more difficult to rectify later.
509 .PP
510 Writing a prototype is a great method to become familiar with the requirements
511 and to actually run into real problems.
512 Today, prototyping is often seen as a first step in building a software.
513 This is, of course, good.
514 However, the Unix Philosophy has an \fIadditional\fP perspective on prototyping:
515 After having built the prototype, one might notice, that the prototype is already
516 \fIgood enough\fP.
517 Hence, no reimplementation, in a more sophisticated programming language, might be of need,
518 for the moment.
519 Maybe later, it might be necessary to rewrite the software, but not now.
520 .PP
521 By delaying further work, one keeps the flexibility to react easily on
522 changing requirements.
523 Software parts that are not written will not miss the requirements.
525 .NH 2
526 Worse is better
527 .XS
528 \*(SN Worse is better
529 .XE
530 .LP
531 The Unix Philosophy aims for the 80% solution;
532 others call it the ``Worse is better'' approach.
533 .PP
534 First, practical experience shows, that it is almost never possible to define the
535 requirements completely and correctly the first time.
536 Hence one should not try to; it will fail anyway.
537 Second, practical experience shows, that requirements change during time.
538 Hence it is best to delay requirement-based design decisions as long as possible.
539 Also, the software should be small and flexible as long as possible
540 to react on changing requirements.
541 Shell scripts, for example, are more easily adjusted as C programs.
542 Third, practical experience shows, that maintenance is hard work.
543 Hence, one should keep the amount of software as small as possible;
544 it should just fulfill the \fIcurrent\fP requirements.
545 Software parts that will be written later, do not need maintenance now.
546 .PP
547 Starting with a prototype in a scripting language has several advantages:
548 .IP \(bu
549 As the initial effort is low, one will likely start right away.
550 .IP \(bu
551 As working parts are available soon, the real requirements can get identified soon.
552 .IP \(bu
553 When a software is usable, it gets used, and thus tested.
554 Hence problems will be found at early stages of the development.
555 .IP \(bu
556 The prototype might be enough for the moment,
557 thus further work on the software can be delayed to a time
558 when one knows better about the requirements and problems,
559 than now.
560 .IP \(bu
561 Implementing now only the parts that are actually needed now,
562 requires fewer maintenance work.
563 .IP \(bu
564 If the global situation changes so that the software is not needed anymore,
565 then less effort was spent into the project, than it would have be
566 when a different approach had been used.
568 .NH 2
569 Upgrowth and survival of software
570 .XS
571 \*(SN Upgrowth and survival of software
572 .XE
573 .LP
574 So far it was talked about \fIwriting\fP or \fIbuilding\fP software.
575 Although these are just verbs, they do imply a specific view on the work process
576 they describe.
577 The better verb, however, is to \fIgrow\fP.
578 .PP
579 Creating software in the sense of the Unix Philosophy is an incremental process.
580 It starts with a first prototype, which evolves as requirements change.
581 A quickly hacked shell script might become a large, sophisticated,
582 compiled program this way.
583 Its lifetime begins with the initial prototype and ends when the software is not used anymore.
584 While being alive it will get extended, rearranged, rebuilt (from scratch).
585 Growing software matches the view that ``software is never finished. It is only released.''
586 .[
587 gancarz
588 unix philosophy
589 %P 26
590 .]
591 .PP
592 Software can be seen as being controlled by evolutionary processes.
593 Successful software is software that is used by many for a long time.
594 This implies that the software is needed, useful, and better than alternatives.
595 Darwin talks about: ``The survival of the fittest.''
596 .[
597 darwin
598 origin of species
599 .]
600 Transferred to software: The most successful software, is the fittest,
601 is the one that survives.
602 (This may be at the level of one creature, or at the level of one species.)
603 The fitness of software is affected mainly by four properties:
604 portability of code, portability of data, range of usability, and reusability of parts.
605 .\" .IP \(bu
606 .\" portability of code
607 .\" .IP \(bu
608 .\" portability of data
609 .\" .IP \(bu
610 .\" range of usability
611 .\" .IP \(bu
612 .\" reuseability of parts
613 .PP
614 (1)
615 .I "Portability of code
616 means, using high-level programming languages,
617 sticking to the standard,
618 and avoiding optimizations that introduce dependencies on specific hardware.
619 Hardware has a much lower lifetime than software.
620 By chaining software to a specific hardware,
621 the software's lifetime gets shortened to that of this hardware.
622 In contrast, software should be easy to port \(en
623 adaptation is the key to success.
624 .\" cf. practice of prog: ch08
625 .PP
626 (2)
627 .I "Portability of data
628 is best achieved by avoiding binary representations
629 to store data, because binary representations differ from machine to machine.
630 Textual representation is favored.
631 Historically, ASCII was the charset of choice.
632 In the future, UTF-8 might be the better choice, however.
633 Important is that it is a plain text representation in a
634 very common charset encoding.
635 Apart from being able to transfer data between machines,
636 readable data has the great advantage, that humans are able
637 to directly edit it with text editors and other tools from the Unix toolchest.
638 .\" gancarz tenet 5
639 .PP
640 (3)
641 A large
642 .I "range of usability
643 ensures good adaptation, and thus good survival.
644 It is a special distinction if a software becomes used in fields of action,
645 the original authors did never imagine.
646 Software that solves problems in a general way will likely be used
647 for all kinds of similar problems.
648 Being too specific limits the range of uses.
649 Requirements change through time, thus use cases change or even vanish.
650 A good example in this point is Allman's sendmail.
651 Allman identifies flexibility to be one major reason for sendmail's success:
652 .[
653 allman
654 sendmail
655 .]
656 .QP
657 .ps -1
658 Second, I limited myself to the routing function [...].
659 This was a departure from the dominant thought of the time, [...].
660 .QP
661 .ps -1
662 Third, the sendmail configuration file was flexible enough to adopt
663 to a rapidly changing world [...].
664 .LP
665 Successful software adopts itself to the changing world.
666 .PP
667 (4)
668 .I "Reuse of parts
669 is even one step further.
670 A software may completely lose its field of action,
671 but parts of which the software is build may be general and independent enough
672 to survive this death.
673 If software is build by combining small independent programs,
674 then there are parts readily available for reuse.
675 Who cares if the large program is a failure,
676 but parts of it become successful instead?
678 .NH 2
679 Summary
680 .XS
681 \*(SN Summary
682 .XE
683 .LP
684 This chapter explained the central ideas of the Unix Philosophy.
685 For each of the ideas, it was exposed what advantages they introduce.
686 The Unix Philosophy are guidelines that help to write valuable software.
687 From the view point of a software developer or software designer,
688 the Unix Philosophy provides answers to many software design problem.
689 .PP
690 The various ideas of the Unix Philosophy are very interweaved
691 and can hardly be applied independently.
692 However, the probably most important messages are:
693 .I "``Do one thing well!''" ,
694 .I "``Keep it simple!''" ,
695 and
696 .I "``Use software leverage!''
700 .NH 1
701 Case study: \s-1MH\s0
702 .XS
703 .sp .5v
704 \*(SN Case study: \s-1MH\s0
705 .XE
706 .LP
707 The previous chapter introduced and explained the Unix Philosophy
708 from a general point of view.
709 The driving force were the guidelines; references to
710 existing software were given only sparsely.
711 In this and the next chapter, concrete software will be
712 the driving force in the discussion.
713 .PP
714 This first case study is about the mail user agents (\s-1MUA\s0)
715 \s-1MH\s0 (``mail handler'') and its descendent \fInmh\fP
716 (``new mail handler'').
717 \s-1MUA\s0s provide functions to read, compose, and organize mail,
718 but (ideally) not to transfer.
719 In this document, the name \s-1MH\s0 will be used for both of them.
720 A distinction will only be made if differences between
721 them are described.
724 .NH 2
725 Historical background
726 .XS
727 \*(SN Historical background
728 .XE
729 .LP
730 Electronic mail was available in Unix very early.
731 The first \s-1MUA\s0 on Unix was \f(CWmail\fP,
732 which was already present in the First Edition.
733 .[
734 salus
735 quarter century of unix
736 %P 41 f.
737 .]
738 It was a small program that either prints the user's mailbox file
739 or appends text to someone elses mailbox file,
740 depending on the command line arguments.
741 .[
742 manual mail(1)
743 .]
744 It was a program that did one job well.
745 This job was emailing, which was very simple then.
746 .PP
747 Later, emailing became more powerful, and thus more complex.
748 The simple \f(CWmail\fP, which knew nothing of subjects,
749 independent handling of single messages,
750 and long-time storage of them, was not powerful enough anymore.
751 At Berkeley, Kurt Shoens wrote \fIMail\fP (with capital `M')
752 in 1978 to provide additional functions for emailing.
753 Mail was still one program, but now it was large and did
754 several jobs.
755 Its user interface is modeled after the one of \fIed\fP.
756 It is designed for humans, but is still scriptable.
757 \fImailx\fP is the adaptation of Berkeley Mail into System V.
758 .[
759 ritter
760 mailx history
761 .]
762 Elm, pine, mutt, and a whole bunch of graphical \s-1MUA\s0s
763 followed Mail's direction.
764 They are large, monolithic programs which include all emailing functions.
765 .PP
766 A different way was taken by the people of \s-1RAND\s0 Corporation.
767 In the beginning, they also had used a monolithic mail system,
768 called \s-1MS\s0 (for ``mail system'').
769 But in 1977, Stockton Gaines and Norman Shapiro
770 came up with a proposal of a new email system concept \(en
771 one that honors the Unix Philosophy.
772 The concept was implemented by Bruce Borden in 1978 and 1979.
773 This was the birth of \s-1MH\s0 \(en the ``mail handler''.
774 .PP
775 Since then, \s-1RAND\s0, the University of California at Irvine and
776 at Berkeley, and several others have contributed to the software.
777 However, it's core concepts remained the same.
778 In the late 90s, when development of \s-1MH\s0 slowed down,
779 Richard Coleman started with \fInmh\fP, the new mail handler.
780 His goal was to improve \s-1MH\s0, especially in regard of
781 the requirements of modern emailing.
782 Today, nmh is developed by various people on the Internet.
783 .[
784 ware
785 rand history
786 %P 128-137
787 .]
788 .[
789 peek
790 mh
791 .]
793 .NH 2
794 Contrasts to monolithic mail systems
795 .XS
796 \*(SN Contrasts to monolithic mail systems
797 .XE
798 .LP
799 All \s-1MUA\s0s are monolithic, except \s-1MH\s0.
800 Although there might actually exist further, very little known,
801 toolchest \s-1MUA\s0s, this statement reflects the situation pretty well.
802 .PP
803 Monolithic \s-1MUA\s0s gather all their functions in one program.
804 In contrast, \s-1MH\s0 is a toolchest of many small tools \(en one for each job.
805 Following is a list of important programs of \s-1MH\s0's toolchest
806 and their function.
807 It gives a feeling of how the toolchest looks like.
808 .IP \(bu
809 .CW inc :
810 incorporate new mail (this is how mail enters the system)
811 .IP \(bu
812 .CW scan :
813 list messages in folder
814 .IP \(bu
815 .CW show :
816 show message
817 .IP \(bu
818 .CW next\fR/\fPprev :
819 show next/previous message
820 .IP \(bu
821 .CW folder :
822 change current folder
823 .IP \(bu
824 .CW refile :
825 refile message into folder
826 .IP \(bu
827 .CW rmm :
828 remove message
829 .IP \(bu
830 .CW comp :
831 compose a new message
832 .IP \(bu
833 .CW repl :
834 reply to a message
835 .IP \(bu
836 .CW forw :
837 forward a message
838 .IP \(bu
839 .CW send :
840 send a prepared message (this is how mail leaves the system)
841 .LP
842 \s-1MH\s0 has no special user interface like monolithic \s-1MUA\s0s have.
843 The user does not leave the shell to run \s-1MH\s0,
844 but he uses the various \s-1MH\s0 programs within the shell.
845 Using a monolithic program with a captive user interface
846 means ``entering'' the program, using it, and ``exiting'' the program.
847 Using toolchests like \s-1MH\s0 means running programs,
848 alone or in combination with others, even from other toolchests,
849 without leaving the shell.
851 .NH 2
852 Data storage
853 .XS
854 \*(SN Data storage
855 .XE
856 .LP
857 \s-1MH\s0's mail storage is a directory tree under the user's
858 \s-1MH\s0 directory (usually \f(CW$HOME/Mail\fP),
859 where mail folders are directories and mail messages are text files
860 within them.
861 Each mail folder contains a file \f(CW.mh_sequences\fP which lists
862 the public message sequences of that folder, for instance new messages.
863 Mail messages are text files located in a mail folder.
864 The files contain the messages as they were received.
865 They are numbered in ascending order in each folder.
866 .PP
867 This mailbox format is called ``\s-1MH\s0'' after the \s-1MUA\s0.
868 Alternatives are \fImbox\fP and \fImaildir\fP.
869 In the mbox format all messages are stored within one file.
870 This was a good solution in the early days, when messages
871 were only a few lines of text and were deleted soon.
872 Today, when single messages often include several megabytes
873 of attachments, it is a bad solution.
874 Another disadvantage of the mbox format is that it is
875 more difficult to write tools that work on mail messages,
876 because it is always necessary to first find and extract
877 the relevant message in the mbox file.
878 With the \s-1MH\s0 mailbox format,
879 each message is a self-standing item, by definition.
880 Also, the problem of concurrent access to one mailbox is
881 reduced to the problem of concurrent access to one message.
882 Maildir is generally similar to \s-1MH\s0's format,
883 but modified towards guaranteed reliability.
884 This involves some complexity, unfortunately.
885 .PP
886 Working with \s-1MH\s0's toolchest on mailboxes is much like
887 working with Unix' toolchest on directory trees:
888 \f(CWscan\fP is like \f(CWls\fP,
889 \f(CWshow\fP is like \f(CWcat\fP,
890 \f(CWfolder\fP is like \f(CWcd\fP and \f(CWpwd\fP,
891 \f(CWrefile\fP is like \f(CWmv\fP,
892 and \f(CWrmm\fP is like \f(CWrm\fP.
893 .PP
894 The context of tools in Unix consists mainly the current working directory,
895 the user identification, and the environment variables.
896 \s-1MH\s0 extends this context by two more items:
897 .IP \(bu
898 The current mail folder, which is similar to the current working directory.
899 For mail folders, \f(CWfolder\fP provides the corresponding functionality
900 of \f(CWcd\fP and \f(CWpwd\fP for directories.
901 .IP \(bu
902 Sequences, which are named sets of messages in a mail folder.
903 The current message, relative to a mail folder, is a special sequence.
904 It enables commands like \f(CWnext\fP and \f(CWprev\fP.
905 .LP
906 In contrast to Unix' context, which is chained to the shell session,
907 \s-1MH\s0's context is independent.
908 Usually there is one context for each user, but a user can have many
909 contexts.
910 Public sequences are an exception, as they belong to the mail folder.
911 .[
912 man page mh-profile mh-sequence
913 .]
915 .NH 2
916 Discussion of the design
917 .XS
918 \*(SN Discussion of the design
919 .XE
920 .LP
921 The following paragraphs discuss \s-1MH\s0 in regard to the tenets
922 of the Unix Philosophy which Gancarz identified.
924 .PP
925 .B "Small is beautiful
926 and
927 .B "do one thing well
928 are two design goals that are directly visible in \s-1MH\s0.
929 Gancarz actually presents \s-1MH\s0 as example under the headline
930 ``Making UNIX Do One Thing Well'':
931 .[
932 gancarz
933 unix philosophy
934 %P 125
935 .]
936 .QP
937 .ps -1
938 [\s-1MH\s0] consists of a series of programs which
939 when combined give the user an enormous ability
940 to manipulate electronic mail messages.
941 A complex application, it shows that not only is it
942 possible to build large applications from smaller
943 components, but also that such designs are actually preferable.
944 .LP
945 The various small programs of \s-1MH\s0 were relatively easy
946 to write, because each of them is small, limited to one function,
947 and has clear boundaries.
948 For the same reasons, they are also good to maintain.
949 Further more, the system can easily get extended.
950 One only needs to put a new program into the toolchest.
951 This was done, for instance, when \s-1MIME\s0 support was added
952 (e.g. \f(CWmhbuild\fP).
953 Also, different programs can exist to do the basically same job
954 in different ways (e.g. in nmh: \f(CWshow\fP and \f(CWmhshow\fP).
955 If someone needs a mail system with some additionally
956 functions that are available nowhere yet, he best takes a
957 toolchest system like \s-1MH\s0 where he can add the
958 functionality with little work.
960 .PP
961 .B "Store data in flat text files
962 is followed by \s-1MH\s0.
963 This is not surprising, because email messages are already plain text.
964 \s-1MH\s0 stores the messages as it receives them,
965 thus any other tool that works on RFC 2822 mail messages can operate
966 on the messages in an \s-1MH\s0 mailbox.
967 All other files \s-1MH\s0 uses are plain text too.
968 It is therefore possible and encouraged to use the text processing
969 tools of Unix' toolchest to extend \s-1MH\s0's toolchest.
971 .PP
972 .B "Avoid captive user interfaces" .
973 \s-1MH\s0 is perfectly suited for non-interactive use.
974 It offers all functions directly and without captive user interfaces.
975 If, nonetheless, users want a graphical user interface,
976 they can have it with \fIxmh\fP or \fIexmh\fP, too.
977 These are graphical frontends for the \s-1MH\s0 toolchest.
978 This means, all email-related work is still done by \s-1MH\s0 tools,
979 but the frontend issues the appropriate calls when the user
980 clicks on buttons.
981 Providing easy-to-use user interfaces in form of frontends is a good
982 approach, because it does not limit the power of the backend itself.
983 The frontend will anyway only be able to make a subset of the
984 backend's power and flexibility available to the user.
985 But if it is a separate program,
986 then the missing parts can still be accessed at the backend directly.
987 If it is integrated, then this will hardly be possible.
988 Further more, it is possible to have different frontends to the same
989 backend.
991 .PP
992 .B "Choose portability over efficiency
993 and
994 .B "use shell scripts to increase leverage and portability" .
995 These two tenets are indirectly, but nicely, demonstrated by
996 Bolsky and Korn in their book about the Korn Shell.
997 .[
998 bolsky korn
999 korn shell
1000 .]
1001 They demonstrated, in chapter 18 of the book, a basic implementation
1002 of a subset of \s-1MH\s0 in ksh scripts.
1003 Of course, this was just a demonstration, but a brilliant one.
1004 It shows how quickly one can implement such a prototype with shell scripts,
1005 and how readable they are.
1006 The implementation in the scripting language may not be very fast,
1007 but it can be fast enough though, and this is all that matters.
1008 By having the code in an interpreted language, like the shell,
1009 portability becomes a minor issue, if we assume the interpreter
1010 to be widespread.
1011 This demonstration also shows how easy it is to create single programs
1012 of a toolchest software.
1013 There are eight tools (two of them have multiple names) and 16 functions
1014 with supporting code.
1015 Each tool comprises between 12 and 38 lines of ksh,
1016 in total about 200 lines.
1017 The functions comprise between 3 and 78 lines of ksh,
1018 in total about 450 lines.
1019 Such small software is easy to write, easy to understand,
1020 and thus easy to maintain.
1021 A toolchest improves the possibility to only write some parts
1022 and though create a working result.
1023 Expanding the toolchest without global changes will likely be
1024 possible, too.
1026 .PP
1027 .B "Use software leverage to your advantage
1028 and the lesser tenet
1029 .B "allow the user to tailor the environment
1030 are ideally followed in the design of \s-1MH\s0.
1031 Tailoring the environment is heavily encouraged by the ability to
1032 directly define default options to programs.
1033 It is even possible to define different default options
1034 depending on the name under which the program was called.
1035 Software leverage is heavily encouraged by the ease it is to
1036 create shell scripts that run a specific command line,
1037 built of several \s-1MH\s0 programs.
1038 There is few software that so much wants users to tailor their
1039 environment and to leverage the use of the software, like \s-1MH\s0.
1040 Just to make one example:
1041 One might prefer a different listing format for the \f(CWscan\fP
1042 program.
1043 It is possible to take one of the distributed format files
1044 or to write one yourself.
1045 To use the format as default for \f(CWscan\fP, a single line,
1046 reading
1047 .DS
1048 .CW
1049 scan: -form FORMATFILE
1050 .DE
1051 must be added to \f(CW.mh_profile\fP.
1052 If one wants this different format as an additional command,
1053 instead of changing the default, he needs to create a link to
1054 \f(CWscan\fP, for instance titled \f(CWscan2\fP.
1055 The line in \f(CW.mh_profile\fP would then start with \f(CWscan2\fP,
1056 as the option should only be in effect when scan is called as
1057 \f(CWscan2\fP.
1059 .PP
1060 .B "Make every program a filter
1061 is hard to find in \s-1MH\s0.
1062 The reason therefore is that most of \s-1MH\s0's tools provide
1063 basic file system operations for the mailboxes.
1064 The reason is the same because of which
1065 \f(CWls\fP, \f(CWcp\fP, \f(CWmv\fP, and \f(CWrm\fP
1066 aren't filters neither.
1067 However, they build a basis on which filters can operate.
1068 \s-1MH\s0 does not provide many filters itself, but it is a basis
1069 to write filters for.
1070 An example would be a mail message text highlighter,
1071 that means a program that makes use of a color terminal to display
1072 header lines, quotations, and signatures in distinct colors.
1073 The author's version of this program, for instance,
1074 is a 25 line awk script.
1076 .PP
1077 .B "Build a prototype as soon as possible
1078 was again well followed by \s-1MH\s0.
1079 This tenet, of course, focuses on early development, which is
1080 long time ago for \s-1MH\s0.
1081 But without following this guideline at the very beginning,
1082 Bruce Borden may have not convinced the management of \s-1RAND\s0
1083 to ever create \s-1MH\s0.
1084 In Bruce' own words:
1085 .[
1086 ware rand history
1087 %P 132
1088 .]
1089 .QP
1090 .ps -1
1091 [...] but they [Stockton Gaines and Norm Shapiro] were not able
1092 to convince anyone that such a system would be fast enough to be usable.
1093 I proposed a very short project to prove the basic concepts,
1094 and my management agreed.
1095 Looking back, I realize that I had been very lucky with my first design.
1096 Without nearly enough design work,
1097 I built a working environment and some header files
1098 with key structures and wrote the first few \s-1MH\s0 commands:
1099 inc, show/next/prev, and comp.
1100 [...]
1101 With these three, I was able to convince people that the structure was viable.
1102 This took about three weeks.
1104 .NH 2
1105 Problems
1106 .XS
1107 \*(SN Problems
1108 .XE
1109 .LP
1110 \s-1MH\s0, for sure is not without problems.
1111 There are two main problems: one is technical, the other is about human behavior.
1112 .PP
1113 \s-1MH\s0 is old and email today is very different to email in the time
1114 when \s-1MH\s0 was designed.
1115 \s-1MH\s0 adopted to the changes pretty well, but it is limited.
1116 For example in development resources.
1117 \s-1MIME\s0 support and support for different character encodings
1118 is available, but only on a moderate level.
1119 More active developers could quickly improve there.
1120 It is also limited by design, which is the larger problem.
1121 \s-1IMAP\s0, for example, conflicts with \s-1MH\s0's design to a large extend.
1122 These design conflicts are not easily solvable.
1123 Possibly, they require a redesign.
1124 Maybe \s-1IMAP\s0 is too different to the classic mail model which \s-1MH\s0 covers,
1125 hence \s-1MH\s0 may never work well with \s-1IMAP\s0.
1126 .PP
1127 The other kind of problem is human habits.
1128 When in this world almost all \s-1MUA\s0s are monolithic,
1129 it is very difficult to convince people to use a toolbox style \s-1MUA\s0
1130 like \s-1MH\s0.
1131 The habits are so strong, that even people who understood the concept
1132 and advantages of \s-1MH\s0 do not like to switch,
1133 simply because \s-1MH\s0 is different.
1134 Unfortunately, the frontends to \s-1MH\s0, which could provide familiar look'n'feel,
1135 are quite outdated and thus not very appealing compared to the modern interfaces
1136 which monolithic \s-1MUA\s0s offer.
1138 .NH 2
1139 Summary \s-1MH\s0
1140 .XS
1141 \*(SN Summary \s-1MH\s0
1142 .XE
1143 .LP
1144 \s-1MH\s0 is an \s-1MUA\s0 that follows the Unix Philosophy in its design
1145 and implementation.
1146 It consists of a toolchest of small tools, each of them does one job well.
1147 The tools are orthogonal to each other, to a large extend.
1148 However, for historical reasons, there also exist distinct tools
1149 that cover the same task.
1150 .PP
1151 The toolchest approach offers great flexibility to the user.
1152 He can use the complete power of the Unix shell with \s-1MH\s0.
1153 This makes \s-1MH\s0 a very powerful mail system.
1154 Extending and customizing \s-1MH\s0 is easy and encouraged, too.
1155 .PP
1156 Apart from the user's perspective, \s-1MH\s0 is development-friendly.
1157 Its overall design follows clear rules.
1158 The single tools do only one job, thus they are easy to understand,
1159 easy to write, and good to maintain.
1160 They are all independent and do not interfere with the others.
1161 Automated testing of their function is a straight forward task.
1162 .PP
1163 It is sad, that \s-1MH\s0's differentness is its largest problem,
1164 as its differentness is also its largest advantage.
1165 Unfortunately, for most people their habits are stronger
1166 than the attraction of the clear design and the power, \s-1MH\s0 offers.
1170 .NH 1
1171 Case study: uzbl
1172 .XS
1173 .sp .5v
1174 \*(SN Case study: uzbl
1175 .XE
1176 .LP
1177 The last chapter took a look on the \s-1MUA\s0 \s-1MH\s0,
1178 this chapter is about uzbl, a web browser that adheres to the Unix Philosophy.
1179 ``uzbl'' is the \fIlolcat\fP's word for the English adjective ``usable''.
1180 It is pronounced the identical.
1182 .NH 2
1183 Historical background
1184 .XS
1185 \*(SN Historical background
1186 .XE
1187 .LP
1188 Uzbl was started by Dieter Plaetinck in April 2009.
1189 The idea was born in a thread in the Arch Linux Forums.
1190 .[
1191 arch linux forums
1192 browser
1193 .]
1194 After some discussion about failures of well known web browsers,
1195 Plaetinck (alias Dieter@be) came up with a very sketchy proposal
1196 of how a better web browser could look like.
1197 To the question of another member, if Plaetinck would write that program,
1198 because it would sound fantastic, Plaetinck replied:
1199 ``Maybe, if I find the time ;-)''.
1200 .PP
1201 Fortunately, he found the time.
1202 One day later, the first prototype was out.
1203 One week later, uzbl had an own website.
1204 One month after the first code showed up,
1205 a mailing list was installed to coordinate and discuss further development.
1206 A wiki was set up to store documentation and scripts that showed up on the
1207 mailing list and elsewhere.
1208 .PP
1209 In the, now, one year of uzbl's existence, it was heavily developed in various branches.
1210 Plaetinck's task became more and more to only merge the best code from the
1211 different branches into his main branch, and to apply patches.
1212 About once a month, Plaetinck released a new version.
1213 In September 2009, he presented several forks of uzbl.
1214 Uzbl, actually, opened the field for a whole family of web browsers with similar shape.
1215 .PP
1216 In July 2009, \fILinux Weekly News\fP published an interview with Plaetinck about uzbl.
1217 In September 2009, the uzbl web browser was on \fISlashdot\fP.
1219 .NH 2
1220 Contrasts to other web browsers
1221 .XS
1222 \*(SN Contrasts to other web browsers
1223 .XE
1224 .LP
1225 Like most \s-1MUA\s0s are monolithic, but \s-1MH\s0 is a toolchest,
1226 most web browsers are monolithic, but uzbl is a frontend to a toolchest.
1227 .PP
1228 Today, uzbl is divided into uzbl-core and uzbl-browser.
1229 Uzbl-core is, how its name already indicates, the core of uzbl.
1230 It handles commands and events to interface other programs,
1231 and also displays webpages by using webkit as render engine.
1232 Uzbl-browser combines uzbl-core with a bunch of handler scripts, a status bar,
1233 an event manager, yanking, pasting, page searching, zooming, and more stuff,
1234 to form a ``complete'' web browser.
1235 In the following text, the term ``uzbl'' usually stands for uzbl-browser,
1236 so uzbl-core is included.
1237 .PP
1238 Unlike most other web browsers, uzbl is mainly the mediator between the
1239 various tools that cover single jobs of web browsing.
1240 Therefore, uzbl listens for commands on a named pipe (fifo), a Unix socket,
1241 and on stdin, and it writes events to a Unix socket and to stdout.
1242 The graphical rendering of the webpage is done by webkit, a web content engine.
1243 Uzbl-core is build around this library.
1244 Loading a webpage in a running uzbl instance requires only:
1245 .DS
1246 .CW
1247 echo 'uri http://example.org' >/path/to/uzbl-fifo
1248 .DE
1249 .PP
1250 Downloads, browsing history, bookmarks, and thelike are not provided
1251 by uzbl-core itself, as they are in other web browsers.
1252 Uzbl-browser also only provides, so called, handler scripts that wrap
1253 external applications which provide the actual functionality.
1254 For instance, \fIwget\fP is used to download files and uzbl-browser
1255 includes a script that calls wget with appropriate options in
1256 a prepared environment.
1257 .PP
1258 Modern web browsers are proud to have addons, plugins, and modules, instead.
1259 This is their effort to achieve similar goals.
1260 But instead of using existing, external programs, modern web browsers
1261 include these functions, although they might be loaded at runtime.
1263 .NH 2
1264 Discussion of the design
1265 .XS
1266 \*(SN Discussion of the design
1267 .XE
1268 .LP
1269 This section discusses uzbl in regard of the Unix Philosophy,
1270 as identified by Gancarz.
1272 .PP
1273 .B "Make each program do one thing well" .
1274 Uzbl tries to be a web browser and nothing else.
1275 The common definition of a web browser is, of course, highly influenced by
1276 existing implementations of web browsers, although they are degenerated.
1277 Web browsers should be programs to browse the web, and nothing more.
1278 This is the one thing they should do, as demanded by the Unix Philosophy.
1279 .PP
1280 Web browsers should, for instance, not manage downloads.
1281 This is the job download managers exist for.
1282 Download managers do primary care about being good in downloading files.
1283 Modern web browsers provide download management only as a secondary feature.
1284 How could they perform this job better, than programs that exist only for
1285 this very job?
1286 And how could anyone want less than the best download manager available?
1287 .PP
1288 A web browser's job is to let the user browse the web.
1289 This means, navigating through websites by following links.
1290 Rendering the \s-1HTML\s0 sources is a different job, too.
1291 It is covered by the webkit render engine, in uzbl's case.
1292 Audio and video content and files like PostScript, \s-1PDF\s0, and the like,
1293 are also not the job of a web browser.
1294 They should be handled by external applications \(en
1295 ones which's job is to handle such data.
1296 Uzbl strives to do it this way.
1297 .PP
1298 Remember Doug McIlroy:
1299 .I
1300 ``Write programs that do one thing and do it well.
1301 Write programs to work together.''
1302 .R
1303 .PP
1304 The lesser tenet
1305 .B "allow the user to tailor the environment
1306 matches good here.
1307 There was the question, how anyone could want anything less than the
1308 best program for the job.
1309 But as personal preferences matter much,
1310 it is probably more important to ask:
1311 How could anyone want something else than his preferred program for the job?
1312 .PP
1313 Usually users want one program for one job.
1314 Hence, whenever the task is, for instance, downloading,
1315 exactly one download manager should be used.
1316 More advanced users might want to have this download manager in this
1317 situation and that one in that situation.
1318 They should be able to configure it this way.
1319 With uzbl, one can use any download manager the user wants.
1320 To switch to a different one, only one line in a small handler script
1321 needs to be changed.
1322 Alternatively it would be possible to query an entry in a global file
1323 or an environment variable, which specifies the download manager to use,
1324 in the handler script.
1325 .PP
1326 As uzbl does neither have its own download manager nor depends on a
1327 specific one, thus uzbl's browsing abilities will not be lowered by having
1328 a bad download manager.
1329 Uzbl's download capabilities will just as good as the ones of the best
1330 download manager available on the system.
1331 Of course, this applies to all of the other supplementary tools, too.
1333 .PP
1334 .B "Use software leverage to your advantage" .
1335 Shell scripts are a good choice to extend uzbl.
1336 Uzbl is designed to be extended by external tools.
1337 These external tools are usually wrapped by small handler shell scripts.
1338 Shell scripts are the glue in this approach.
1339 They make the various parts fit together.
1340 .PP
1341 As an example, the history mechanism of uzbl shall be presented.
1342 Uzbl is configured to spawn a script to append an entry to the history
1343 whenever the event of a fully loaded page occurs.
1344 The script to append the entry to the history not much more than:
1345 .DS
1346 .CW
1347 #!/bin/sh
1348 file=/path/to/uzbl-history
1349 echo `date +'%Y-%m-%d %H:%M:%S'`" $6 $7" >> $file
1350 .DE
1351 \f(CW$6\fP and \f(CW$7\fP expand to the \s-1URL\s0 and the page title.
1352 For loading an entry, a key is bound to spawn a load from history script.
1353 The script reverses the history to have newer entries first,
1354 then displays \fIdmenu\fP to select an item,
1355 and afterwards writes the selected \s-1URL\s0 into uzbl's command input pipe.
1356 With error checking and corner cases removed, the script looks like this:
1357 .DS
1358 .CW
1359 #!/bin/sh
1360 file=/path/to/uzbl-history
1361 goto=`tac $file | dmenu | cut -d' ' -f 3`
1362 echo "uri $goto" > $4
1363 .DE
1364 \f(CW$4\fP expands to the path of the command input pipe of the current
1365 uzbl instance.
1367 .PP
1368 .B "Avoid captive user interfaces" .
1369 One could say, that uzbl, to a large extend, actually \fIis\fP
1370 a captive user interface.
1371 But the difference to most other web browsers is, that uzbl is only
1372 the captive user interface frontend and the core of the backend.
1373 Many parts of the backend are independent of uzbl.
1374 Some are distributed with uzbl, for some external programs, handler scripts
1375 are distributed, arbitrary additional functionality can be added if desired.
1376 .PP
1377 The frontend is captive \(en that is true.
1378 This is okay for the task of browsing the web, as this task is only relevant
1379 for humans.
1380 Automated programs would \fIcrawl\fP the web.
1381 That means, they read the source directly.
1382 The source includes all the semantics.
1383 The graphical representation is just for humans to transfer the semantics
1384 more intuitively.
1386 .PP
1387 .B "Make every program a filter" .
1388 Graphical web browsers are almost dead ends in the chain of information flow.
1389 Thus it is difficult to see what graphical web browsers should filter.
1390 Graphical web browsers exist almost only for interactive use by humans.
1391 The only case when one might want to automate the rendering function is
1392 to generate images of rendered webpages.
1394 .PP
1395 .B "Small is beautiful"
1396 is not easy to apply to a web browser, primary because modern web technology
1397 is very complex; hence the rendering task is very complex.
1398 Modern web browsers will always consist of many thousand lines of code,
1399 unfortunately.
1400 Using the toolchest approach and wrappers can split the browser into
1401 several small parts, tough.
1402 .PP
1403 Uzbl-core consists of about 3\,500 lines of C code.
1404 The distribution includes another 3\,500 lines of Shell and Python code,
1405 which are the handler scripts and plugins like a modal interface.
1406 Further more, uzbl uses functionality of external tools like
1407 \fIwget\fP and \fInetcat\fP.
1408 Up to this point, uzbl looks pretty neat and small.
1409 The ugly part of uzbl is the web content renderer, webkit.
1410 Webkit consists of roughly 400\,000 (!) lines of code.
1411 Unfortunately, small web render engines are not possible anymore
1412 because of the modern web.
1413 The problems section will explain this in more detail.
1415 .PP
1416 .B "Build a prototype as soon as possible" .
1417 Plaetinck made his code public, right from the beginning.
1418 Discussion and development was, and still is, open to everyone interested.
1419 Development versions of uzbl can be obtained very simply from the code
1420 repository.
1421 Within the first year of uzbl's existence, a new version was released
1422 more often than once a month.
1423 Different forks and branches arose.
1424 They introduced new features, which were tested for suitability.
1425 The experiences of using prototypes influenced further development.
1426 Actually, all development was community driven.
1427 Plaetinck says, three months after uzbl's birth:
1428 ``Right now I hardly code anything myself for Uzbl.
1429 I just merge in other people's code, ponder a lot, and lead the discussions.''
1430 .[
1431 lwn
1432 uzbl
1433 .]
1436 .NH 2
1437 Problems
1438 .XS
1439 \*(SN Problems
1440 .XE
1441 .LP
1442 Similar to \s-1MH\s0, uzbl, too suffers from being different.
1443 It is sad, but people use what they know.
1444 Fortunately, uzbl's user interface can look and feel very much the
1445 same as the one of the well known web browsers,
1446 hiding the internal differences.
1447 But uzbl has to provide this similar look and feel to be accepted
1448 as a ``normal'' browser by ``normal'' users.
1449 .PP
1450 The more important problem is the modern web.
1451 The modern web is simply broken.
1452 It has state in a state-less protocol,
1453 it misuses technologies,
1454 and it is helplessly overloaded.
1455 The result are web content render engines that must consist
1456 of hundreds of thousands lines of code.
1457 They also must combine and integrate many different technologies,
1458 only to make our modern web usable.
1459 Website to image converter are hardly possible to run without
1460 human interaction because of state in sessions, impossible
1461 deep-linking, and unautomatable technologies.
1462 .PP
1463 The web was misused to provide all kinds of imaginable wishes.
1464 Now web browsers, and eventually the users, suffer from it.
1467 .NH 2
1468 Summary uzbl
1469 .XS
1470 \*(SN Summary uzbl
1471 .XE
1472 .LP
1473 ``Uzbl is a browser that adheres to the Unix Philosophy'',
1474 that is how uzbl is seen by its authors.
1475 Indeed, uzbl follows the Unix Philosophy in many ways.
1476 It consists of independent parts that work together,
1477 its core is mainly a mediator which glues the parts together.
1478 .PP
1479 Software leverage can excellently be seen in uzbl.
1480 It makes use of external tools, separates independent tasks
1481 in independent parts, and glues them together with small
1482 handler scripts, around uzbl-core.
1483 .PP
1484 As uzbl, more or less, consists of a set of tools and a bit
1485 of glue, anyone can put the parts together and expand it
1486 in any desired way.
1487 Uzbl is very flexible and customizable.
1488 These properties make it valuable for advanced users,
1489 but may keep novice users from using it.
1490 .PP
1491 Uzbl's main problem is the modern web, that makes it hard
1492 to design a sane web browser.
1493 Despite this bad situation, uzbl does a fairly good job.
1496 .NH 1
1497 Final thoughts
1498 .XS
1499 .sp .5v
1500 \*(SN Final thoughts
1501 .XE
1503 .NH 2
1504 Quick summary
1505 .XS
1506 \*(SN Quick summary
1507 .XE
1508 .LP
1509 good design
1510 .LP
1511 unix phil
1512 .LP
1513 case studies
1515 .NH 2
1516 Why people should choose
1517 .XS
1518 \*(SN Why people should choose
1519 .XE
1520 .LP
1521 Make the right choice!
1523 .nr PI .3i
1524 .rm ]<
1525 .de ]<
1526 .LP
1527 .de FP
1528 .IP \\\\$1.
1529 \\..
1530 .rm FS FE
1531 ..
1532 .ds CH "
1533 .bp
1534 .rs
1535 .sp .3i
1536 .TL
1537 References
1538 .LP
1539 .XS
1540 .sp .5v
1541 References
1542 .XE
1543 .sp 2v
1544 .nr PS -1
1545 .nr VS -1
1546 .[
1547 $LIST$
1548 .]
1549 .\".wh -1p
1550 .bp
1551 .PX