docs/unix-phil
view unix-phil.ms @ 45:ade392f024aa
a lot of rework throughout the whole document
| author | meillo@marmaro.de |
|---|---|
| date | Mon, 12 Apr 2010 09:33:01 +0200 |
| parents | 46e34e433231 |
| children | 98a1446744c7 |
line source
1 .nr PS 11
2 .nr VS 13
3 .nr lu 0
4 .de CW
5 .nr PQ \\n(.f
6 .if t .ft CW
7 .ie ^\\$1^^ .if n .ul 999
8 .el .if n .ul 1
9 .if t .if !^\\$1^^ \&\\$1\f\\n(PQ\\$2
10 .if n .if \\n(.$=1 \&\\$1
11 .if n .if \\n(.$>1 \&\\$1\c
12 .if n .if \\n(.$>1 \&\\$2
13 ..
14 .ds [. \ [
15 .ds .] ]
17 .rn NH _N
18 .de NH
19 .if '\\$1'1' .sp 2v
20 .if '\\$1'1' .nr PS +2
21 ._N \\$1
22 .if '\\$1'1' .nr PS -2
23 ..
25 .am QP
26 .ps -1
27 ..
29 .\"----------------------------------------
31 .TL
32 .ps +4
33 Why the Unix Philosophy still matters
34 .AU
35 markus schnalke <meillo@marmaro.de>
36 .AB
37 .ti \n(.iu
38 This paper explains the importance of the Unix Philosophy for software design.
39 Today, few software designers are aware of these concepts,
40 and thus a lot of modern software is more limited than necessary
41 and makes less use of software leverage than possible.
42 Knowing and following the guidelines of the Unix Philosophy makes software more valuable.
43 .AE
45 .FS
46 .ps -1
47 This paper was prepared for the ``Software Analysis'' seminar at University Ulm.
48 Mentor was professor Schweiggert. 2010-04-05
49 .br
50 You may retrieve this document from
51 .CW \s-1http://marmaro.de/docs \ .
52 .FE
54 .NH 1
55 Introduction
56 .XS
57 \*(SN Introduction
58 .XE
59 .LP
60 The Unix Philosophy is the essence of how the Unix operating system,
61 especially its toolchest, was designed.
62 It is no limited set of fixed rules,
63 but a loose set of guidelines which tell how to write software that
64 suites well into Unix.
65 Actually, the Unix Philosophy describes what is common to typical Unix software.
66 The Wikipedia has an accurate definition:
67 .[
68 wikipedia
69 unix philosophy
70 .]
71 .QP
72 The \fIUnix philosophy\fP is a set of cultural norms and philosophical
73 approaches to developing software based on the experience of leading
74 developers of the Unix operating system.
75 .PP
76 As there is no single definition of the Unix Philosophy,
77 several people have stated their view on what it comprises.
78 Best known are:
79 .IP \(bu
80 Doug McIlroy's summary: ``Write programs that do one thing and do it well.''
81 .[
82 mahoney
83 oral history
84 .]
85 .IP \(bu
86 Mike Gancarz' book ``The UNIX Philosophy''.
87 .[
88 gancarz
89 unix philosophy
90 .]
91 .IP \(bu
92 Eric S. Raymond's book ``The Art of UNIX Programming''.
93 .[
94 raymond
95 art of unix programming
96 .]
97 .LP
98 These different views on the Unix Philosophy have much in common.
99 Especially, the main concepts are similar in all of them.
100 McIlroy's definition can surely be called the core of the Unix Philosophy,
101 but the fundamental idea behind it all, is ``small is beautiful''.
103 .PP
104 The Unix Philosophy explains how to design good software for Unix.
105 Many concepts described here, base on facilities of Unix.
106 Other operating systems may not offer such facilities,
107 hence it may not be possible to design software in the way of the
108 Unix Philosophy for them.
109 .PP
110 The Unix Philosophy has an idea of how the process of software development
111 should look like, but large parts of the philosophy are quite independent
112 from a concrete development process.
113 However, one will soon recognize that some development processes work well
114 with the ideas of the Unix Philosophy and support them, while others are
115 at cross-purposes.
116 Kent Beck's books about Extreme Programming are valuable supplemental
117 resources on this topic.
118 .PP
119 The question of how to actually write code and how the code should looks
120 like in detail, are out of focus here.
121 ``The Practice of Programming'' by Kernighan and Pike,
122 .[
123 kernighan pike
124 practice of programming
125 .]
126 is a good book that covers this topic.
127 Its point of view matches to the one of this paper.
129 .NH 1
130 Importance of software design in general
131 .XS
132 .sp .5v
133 \*(SN Importance of software design in general
134 .XE
135 .LP
136 Software design is the planning of how the internal structure
137 and external interfaces of a software should look like.
138 It has nothing to do with visual appearance.
139 If we take a program as a car, then its color is of no matter.
140 Its design would be the car's size, its shape, the locations of doors,
141 the passenger/space ratio, the available controls and instruments,
142 and so forth.
143 .PP
144 Why should software get designed at all?
145 It is general knowledge, that even a bad plan is better than no plan.
146 Not designing software means programming without plan.
147 This will pretty sure lead to horrible results.
148 Software that is horrible to use and horrible to maintain.
149 These two aspects are the visible ones.
150 Often invisible though, are the wasted possible gains.
151 Good software design can make these gains available.
152 .PP
153 A software's design deals with quality properties.
154 Good design leads to good quality, and quality is important.
155 Any car may be able to drive from A to B,
156 but it depends on the car's properties whether it is a good choice
157 for passenger transport or not.
158 It depends on its properties if it is a good choice
159 for a rough mountain area.
160 And it depends on its properties if the ride will be fun.
162 .PP
163 Requirements for a software are twofold:
164 functional and non-functional.
165 .IP \(bu
166 Functional requirements define directly the software's functions.
167 They are the reason why software gets written.
168 Someone has a problem and needs a tool to solve it.
169 Being able to solve the problem is the main functional goal.
170 It is the driving force behind all programming effort.
171 Functional requirements are easier to define and to verify.
172 .IP \(bu
173 Non-functional requirements are called \fIquality\fP requirements, too.
174 The quality of a software are the properties that are not directly related to
175 the software's basic functions.
176 Tools of bad quality often do solve the problems they were written for,
177 but introduce problems and difficulties for usage and development, later on.
178 Quality aspects are often overlooked at first sight,
179 and are often difficult to define clearly and to verify.
180 .PP
181 Quality is hardly interesting when the software gets built initially,
182 but it has a high impact on usability and maintenance of the software, later.
183 A short-sighted might see in developing a software, mainly building something up.
184 But experience shows, that building the software the first time is
185 only a small amount of the overall work.
186 Bug fixing, extending, rebuilding of parts \(en maintenance work \(en
187 does soon take over the major part of the time spent on a software.
188 And of course, the time spent actually using the software.
189 These processes are highly influenced by the software's quality.
190 Thus, quality must not be neglected.
191 However, the problem with quality is that you hardly ``stumble over''
192 bad quality during the first build,
193 although this is the time when you should care about good quality most.
194 .PP
195 Software design has little to do with the basic function of a software \(en
196 this requirement will get satisfied anyway.
197 Software design is more about quality aspects of the software.
198 Good design leads to good quality, bad design to bad quality.
199 The primary functions of the software will be affected modestly by bad quality,
200 but good quality can provide a lot of additional gain,
201 even at places where one never expected it.
202 .PP
203 The ISO/IEC\|9126-1 standard, part\|1,
204 .[
205 iso product quality
206 .]
207 defines the quality model as consisting out of:
208 .IP \(bu
209 .I Functionality
210 (suitability, accuracy, inter\%operability, security)
211 .IP \(bu
212 .I Reliability
213 (maturity, fault tolerance, recoverability)
214 .IP \(bu
215 .I Usability
216 (understandability, learnability, operability, attractiveness)
217 .IP \(bu
218 .I Efficiency
219 (time behavior, resource utilization)
220 .IP \(bu
221 .I Maintainability
222 (analyzability, changeability, stability, testability)
223 .IP \(bu
224 .I Portability
225 (adaptability, installability, co-existence, replaceability)
226 .LP
227 Good design can improve these properties of a software,
228 bad designed software likely suffers in these points.
229 .PP
230 One further goal of software design is consistency.
231 Consistency eases understanding, working on, and using things.
232 Consistent internal structure and consistent interfaces to the outside
233 can be provided by good design.
234 .PP
235 Software should be well designed because good design avoids many
236 problems during a software's lifetime.
237 And software should be well designed because good design can offer
238 much additional gain.
239 Indeed, much effort should be spent into good design to make software more valuable.
240 The Unix Philosophy shows a way of how to design software well.
241 It offers guidelines to achieve good quality and high gain for the effort spent.
244 .NH 1
245 The Unix Philosophy
246 .XS
247 .sp .5v
248 \*(SN The Unix Philosophy
249 .XE
250 .LP
251 The origins of the Unix Philosophy were already introduced.
252 This chapter explains the philosophy, oriented on Gancarz,
253 and shows concrete examples of its application.
255 .NH 2
256 Pipes
257 .XS
258 \*(SN Pipes
259 .XE
260 .LP
261 Following are some examples to demonstrate how applied Unix Philosophy feels like.
262 Knowledge of using the Unix shell is assumed.
263 .PP
264 Counting the number of files in the current directory:
265 .DS
266 .CW
267 .ps -1
268 ls | wc -l
269 .DE
270 The
271 .CW ls
272 command lists all files in the current directory, one per line,
273 and
274 .CW "wc -l
275 counts the number of lines.
276 .PP
277 Counting the number of files that do not contain ``foo'' in their name:
278 .DS
279 .CW
280 .ps -1
281 ls | grep -v foo | wc -l
282 .DE
283 Here, the list of files is filtered by
284 .CW grep
285 to remove all lines that contain ``foo''.
286 The rest equals the previous example.
287 .PP
288 Finding the five largest entries in the current directory.
289 .DS
290 .CW
291 .ps -1
292 du -s * | sort -nr | sed 5q
293 .DE
294 .CW "du -s *
295 returns the recursively summed sizes of all files in the current directory
296 \(en no matter if they are regular files or directories.
297 .CW "sort -nr
298 sorts the list numerically in reverse order (descending).
299 Finally,
300 .CW "sed 5q
301 quits after it has printed the fifth line.
302 .PP
303 The presented command lines are examples of what Unix people would use
304 to get the desired output.
305 There are also other ways to get the same output.
306 It's a user's decision which way to go.
307 .PP
308 The examples show that many tasks on a Unix system
309 are accomplished by combining several small programs.
310 The connection between the single programs is denoted by the pipe operator `|'.
311 .PP
312 Pipes, and their extensive and easy use, are one of the great
313 achievements of the Unix system.
314 Pipes between programs have been possible in earlier operating systems,
315 but it has never been a so central part of the concept.
316 When, in the early seventies, Doug McIlroy introduced pipes into the
317 Unix system,
318 ``it was this concept and notation for linking several programs together
319 that transformed Unix from a basic file-sharing system to an entirely new way of computing.''
320 .[
321 aughenbaugh
322 unix oral history
323 .]
324 .PP
325 Being able to specify pipelines in an easy way is,
326 however, not enough by itself.
327 It is only one half.
328 The other is the design of the programs that are used in the pipeline.
329 They need interfaces that allow them to be used in such a way.
331 .NH 2
332 Interface design
333 .XS
334 \*(SN Interface design
335 .XE
336 .LP
337 Unix is, first of all, simple \(en Everything is a file.
338 Files are sequences of bytes, without any special structure.
339 Programs should be filters, which read a stream of bytes from standard input (stdin)
340 and write a stream of bytes to standard output (stdout).
341 If the files \fIare\fP sequences of bytes,
342 and the programs \fIare\fP filters on byte streams,
343 then there is exactly one data interface.
344 Hence it is possible to combine programs in any desired way.
345 .PP
346 Even a handful of small programs yields a large set of combinations,
347 and thus a large set of different functions.
348 This is leverage!
349 If the programs are orthogonal to each other \(en the best case \(en
350 then the set of different functions is greatest.
351 .PP
352 Programs can also have a separate control interface,
353 besides their data interface.
354 The control interface is often called ``user interface'',
355 because it is usually designed to be used by humans.
356 The Unix Philosophy discourages to assume the user to be human.
357 Interactive use of software is slow use of software,
358 because the program waits for user input most of the time.
359 Interactive software requires the user to be in front of the computer.
360 Interactive software occupy the user's attention while they are running.
361 .PP
362 Now to come back to the idea of combining several small programs,
363 to have a more specific function.
364 If these single tools would all be interactive,
365 how would the user control them?
366 It is not only a problem to control several programs at once,
367 if they run at the same time,
368 it also very inefficient to have to control each of the single programs
369 that are intended to act as one large program.
370 Hence, the Unix Philosophy discourages programs to demand interactive use.
371 The behavior of programs should be defined at invocation.
372 This is done by specifying arguments to the program call
373 (command line switches).
374 Gancarz discusses this topic as ``avoid captive user interfaces''.
375 .[
376 gancarz unix philosophy
377 %P 88 ff.
378 .]
379 .ds _p ", p. 88 ff.
380 .PP
381 Non-interactive use is, during development, also an advantage for testing.
382 Testing of interactive programs is much more complicated,
383 than testing of non-interactive programs.
385 .NH 2
386 The toolchest approach
387 .XS
388 \*(SN The toolchest approach
389 .XE
390 .LP
391 A toolchest is a set of tools.
392 Instead of having one big tool for all tasks, one has many small tools,
393 each for one task.
394 Difficult tasks are solved by combining several of the small, simple tools.
395 .PP
396 The Unix toolchest \fIis\fP a set of small, (mostly) non-interactive programs
397 that are filters on byte streams.
398 They are, to a large extend, unrelated in their function.
399 Hence, the Unix toolchest provides a large set of functions
400 that can be accessed by combining the programs in the desired way.
401 .PP
402 There are also advantages for developing small toolchest programs.
403 It is easier and less error-prone to write small programs.
404 It is also easier and less error-prone to write a large set of small programs,
405 than to write one large program with all the functionality included.
406 If the small programs are combinable, then they offer even a larger set
407 of functions than the single large program.
408 Hence, one gets two advantages out of writing small, combinable programs:
409 They are easier to write and they offer a greater set of functions through
410 combination.
411 .PP
412 But there are also two main drawbacks of the toolchest approach.
413 First, one simple, standardized interface has to be sufficient.
414 If one feels the need for more ``logic'' than a stream of bytes,
415 then a different approach might be of need.
416 But it is also possible, that he just can not imagine a design where
417 a stream of bytes is sufficient.
418 By becoming more familiar with the ``Unix style of thinking'',
419 developers will more often and easier find simple designs where
420 a stream of bytes is a sufficient interface.
421 .PP
422 The second drawback of a toolchest affects the users.
423 A toolchest is often more difficult to use.
424 It is necessary to become familiar with each of the tools,
425 to be able to use the right one in a given situation.
426 Additionally, one needs to combine the tools in a senseful way himself.
427 This is like a sharp knife \(en it is a powerful tool in the hand of a
428 master, but of no good value in the hand of an unskilled.
429 However, learning single, small tools of a toolchest is easier than
430 learning a complex tool.
431 And the user will already have a basic understanding of a yet unknown tool,
432 if the tools of a toolchest have a common, consistent style.
433 He will be able to transfer knowledge over from one tool to another.
434 .PP
435 Moreover, the second drawback can be removed to a large extend
436 by adding wrappers around the basic tools.
437 Novice users do not need to learn several tools, if a professional wraps
438 complete command lines into a higher-level script.
439 Note that the wrapper script still calls the small tools;
440 it is just like a skin around them.
441 No complexity is added this way.
442 But new programs can get created out of existing one with very low effort.
443 .PP
444 A wrapper script for finding the five largest entries in the current directory
445 could look like this:
446 .DS
447 .CW
448 .ps -1
449 #!/bin/sh
450 du -s * | sort -nr | sed 5q
451 .DE
452 The script itself is just a text file that calls the command line,
453 which a professional user would type in directly.
454 It is probably worth to make the program flexible on the number of
455 entries it prints:
456 .DS
457 .CW
458 .ps -1
459 #!/bin/sh
460 num=5
461 [ $# -eq 1 ] && num="$1"
462 du -sh * | sort -nr | sed "${num}q"
463 .DE
464 This script acts like the one before, when called without an argument.
465 But one can also specify a numerical argument to define the number of lines to print.
466 One can surely imagine even more flexible versions, however,
467 they will still relay on the external programs,
468 which do the actual work.
470 .NH 2
471 A powerful shell
472 .XS
473 \*(SN A powerful shell
474 .XE
475 .LP
476 The Unix shell provides the possibility to combine small programs into large ones.
477 But a powerful shell is a great feature in other ways, too.
478 For instance by being scriptable.
479 Control statements are build into the shell.
480 The functions, however, are the normal programs of the system.
481 Thus, as the programs are already known,
482 learning to program in the shell becomes easy.
483 Using normal programs as functions in the shell programming language
484 is only possible because they are small and combinable tools in a toolchest style.
485 .PP
486 The Unix shell encourages to write small scripts,
487 by combining existing programs, because it is so easy to do.
488 This is a great step towards automation.
489 It is wonderful if the effort to automate a task equals the effort
490 to do the task a second time by hand.
491 If this holds,
492 then the user will be happy to automate everything he does more than once.
493 .PP
494 Small programs that do one job well, standardized interfaces between them,
495 a mechanism to combine parts to larger parts, and an easy way to automate tasks,
496 this will inevitably produce software leverage.
497 Getting multiple times the benefit of an investment is a great offer.
498 .PP
499 The shell also encourages rapid prototyping.
500 Many well known programs started as quickly hacked shell scripts,
501 and turned into ``real'' programs, written in C, later.
502 Building a prototype first, is a way to avoid the biggest problems
503 in application development.
504 Fred Brooks explains in ``No Silver Bullet'':
505 .[
506 brooks
507 no silver bullet
508 .]
509 .QP
510 The hardest single part of building a software system is deciding precisely what to build.
511 No other part of the conceptual work is so difficult as establishing the detailed
512 technical requirements, [...].
513 No other part of the work so cripples the resulting system if done wrong.
514 No other part is more difficult to rectify later.
515 .PP
516 Writing a prototype is a great method for becoming familiar with the requirements
517 and to run into real problems early.
518 .PP
519 Prototyping is often seen as a first step in building a software.
520 This is, of course, good.
521 However, the Unix Philosophy has an \fIadditional\fP perspective on prototyping:
522 After having built the prototype, one might notice, that the prototype is already
523 \fIgood enough\fP.
524 Hence, no reimplementation, in a more sophisticated programming language,
525 might be of need, at least for the moment.
526 Maybe later, it might be necessary to rewrite the software, but not now.
527 By delaying further work, one keeps the flexibility to react on
528 changing requirements.
529 Software parts that are not written will not miss the requirements.
531 .NH 2
532 Worse is better
533 .XS
534 \*(SN Worse is better
535 .XE
536 .LP
537 The Unix Philosophy aims for the 90% solution;
538 others call it the ``Worse is better'' approach.
539 Practical experience shows, that:
540 .PP
541 (1) It is almost never possible to define the
542 requirements completely and correctly the first time.
543 Hence one should not try to; one will fail anyway.
544 .PP
545 (2) Requirements change during time.
546 Hence it is best to delay requirement-based design decisions as long as possible.
547 The software should be small and flexible as long as possible
548 to react on changing requirements.
549 Shell scripts, for example, are more easily adjusted as C programs.
550 .PP
551 (3) Maintenance work is hard work.
552 Hence, one should keep the amount of code as small as possible;
553 it should just fulfill the \fIcurrent\fP requirements.
554 Software parts that will be written in future,
555 do not need maintenance till then.
556 .PP
557 Starting with a prototype in a scripting language has several advantages:
558 .IP \(bu
559 As the initial effort is low, one will likely start right away.
560 .IP \(bu
561 As working parts are available soon, the real requirements can get identified soon.
562 .IP \(bu
563 When a software is usable and valuable, it gets used, and thus tested.
564 Hence problems will be found at early stages of the development.
565 .IP \(bu
566 The prototype might be enough for the moment,
567 thus further work on the software can get delayed to a time
568 when one knows better about the requirements and problems,
569 than now.
570 .IP \(bu
571 Implementing now only the parts that are actually needed at the moment,
572 introduces fewer programming and maintenance work.
573 .IP \(bu
574 If the global situation changes so that the software is not needed anymore,
575 then less effort was spent into the project, than it would have be
576 when a different approach had been used.
578 .NH 2
579 Upgrowth and survival of software
580 .XS
581 \*(SN Upgrowth and survival of software
582 .XE
583 .LP
584 So far it was talked about \fIwriting\fP or \fIbuilding\fP software.
585 Although these are just verbs, they do imply a specific view on the work process
586 they describe.
587 The better verb, however, is to \fIgrow\fP.
588 Creating software in the sense of the Unix Philosophy is an incremental process.
589 It starts with a first prototype, which evolves as requirements change.
590 A quickly hacked shell script might become a large, sophisticated,
591 compiled program this way.
592 Its lifetime begins with the initial prototype and ends when the software is not used anymore.
593 While being alive it will get extended, rearranged, rebuilt.
594 Growing software matches the view that ``software is never finished. It is only released.''
595 .[
596 gancarz
597 unix philosophy
598 %P 26
599 .]
600 .PP
601 Software can be seen as being controlled by evolutionary processes.
602 Successful software is software that is used by many for a long time.
603 This implies that the software is needed, useful, and better than alternatives.
604 Darwin talks about: ``The survival of the fittest.''
605 .[
606 darwin
607 origin of species
608 .]
609 Transferred to software: The most successful software, is the fittest,
610 is the one that survives.
611 (This may be at the level of one creature, or at the level of one species.)
612 The fitness of software is affected mainly by four properties:
613 portability of code, portability of data, range of usability, and reusability of parts.
614 .\" .IP \(bu
615 .\" portability of code
616 .\" .IP \(bu
617 .\" portability of data
618 .\" .IP \(bu
619 .\" range of usability
620 .\" .IP \(bu
621 .\" reuseability of parts
622 .PP
623 (1)
624 .I "Portability of code
625 means, using high-level programming languages,
626 sticking to the standard,
627 and avoiding optimizations that introduce dependencies on specific hardware.
628 Hardware has a much lower lifetime than software.
629 By chaining software to a specific hardware,
630 the software's lifetime gets shortened to that of this hardware.
631 In contrast, software should be easy to port \(en
632 adaptation is the key to success.
633 .\" cf. practice of prog: ch08
634 .PP
635 (2)
636 .I "Portability of data
637 is best achieved by avoiding binary representations
638 to store data, because binary representations differ from machine to machine.
639 Textual representation is favored.
640 Historically, \s-1ASCII\s0 was the charset of choice.
641 For the future, \s-1UTF\s0-8 might be the better choice.
642 Important is that it is a plain text representation in a
643 very common charset encoding.
644 Apart from being able to transfer data between machines,
645 readable data has the great advantage, that humans are able to directly
646 read and edit it with text editors and other tools from the Unix toolchest.
647 .\" gancarz tenet 5
648 .PP
649 (3)
650 A large
651 .I "range of usability
652 ensures good adaptation, and thus good survival.
653 It is a special distinction if a software becomes used in fields of action,
654 the original authors did never imagine.
655 Software that solves problems in a general way will likely be used
656 for many kinds of similar problems.
657 Being too specific limits the range of usability.
658 Requirements change through time, thus use cases change or even vanish.
659 As a good example in this point,
660 Allman identifies flexibility to be one major reason for sendmail's success:
661 .[
662 allman
663 sendmail
664 .]
665 .QP
666 Second, I limited myself to the routing function [...].
667 This was a departure from the dominant thought of the time, [...].
668 .QP
669 Third, the sendmail configuration file was flexible enough to adapt
670 to a rapidly changing world [...].
671 .LP
672 Successful software adapts itself to the changing world.
673 .PP
674 (4)
675 .I "Reuse of parts
676 is even one step further.
677 A software may completely lose its field of action,
678 but parts of which the software is build may be general and independent enough
679 to survive this death.
680 If software is build by combining small independent programs,
681 then these parts are readily available for reuse.
682 Who cares if the large program is a failure,
683 but parts of it become successful instead?
685 .NH 2
686 Summary
687 .XS
688 \*(SN Summary
689 .XE
690 .LP
691 This chapter explained central ideas of the Unix Philosophy.
692 For each of the ideas, the advantages they introduce were explained.
693 The Unix Philosophy are guidelines that help to write more valuable software.
694 From the view point of a software developer or software designer,
695 the Unix Philosophy provides answers to many software design problem.
696 .PP
697 The various ideas of the Unix Philosophy are very interweaved
698 and can hardly be applied independently.
699 However, the probably most important messages are:
700 .I "``Keep it simple!''" ,
701 .I "``Do one thing well!''" ,
702 and
703 .I "``Use software leverage!''
707 .NH 1
708 Case study: \s-1MH\s0
709 .XS
710 .sp .5v
711 \*(SN Case study: \s-1MH\s0
712 .XE
713 .LP
714 The previous chapter introduced and explained the Unix Philosophy
715 from a general point of view.
716 The driving force were the guidelines; references to
717 existing software were given only sparsely.
718 In this and the next chapter, concrete software will be
719 the driving force in the discussion.
720 .PP
721 This first case study is about the mail user agents (\s-1MUA\s0)
722 \s-1MH\s0 (``mail handler'') and its descendent \fInmh\fP
723 (``new mail handler'').
724 \s-1MUA\s0s provide functions to read, compose, and organize mail,
725 but (ideally) not to transfer it.
726 In this document, the name \s-1MH\s0 will be used to include nmh.
727 A distinction will only be made if differences between
728 \s-1MH\s0 and nmh are described.
731 .NH 2
732 Historical background
733 .XS
734 \*(SN Historical background
735 .XE
736 .LP
737 Electronic mail was available in Unix very early.
738 The first \s-1MUA\s0 on Unix was \f(CWmail\fP,
739 which was already present in the First Edition.
740 .[
741 salus
742 quarter century of unix
743 %P 41 f.
744 .]
745 It was a small program that either printed the user's mailbox file
746 or appended text to someone elses mailbox file,
747 depending on the command line arguments.
748 .[
749 manual mail(1)
750 .]
751 It was a program that did one job well.
752 This job was emailing, which was very simple then.
753 .PP
754 Later, emailing became more powerful, and thus more complex.
755 The simple \f(CWmail\fP, which knew nothing of subjects,
756 independent handling of single messages,
757 and long-time email storage, was not powerful enough anymore.
758 In 1978 at Berkeley, Kurt Shoens wrote \fIMail\fP (with capital `M')
759 to provide additional functions for emailing.
760 Mail was still one program, but now it was large and did
761 several jobs.
762 Its user interface is modeled after the one of \fIed\fP.
763 It is designed for humans, but is still scriptable.
764 \fImailx\fP is the adaptation of Berkeley Mail into System V.
765 .[
766 ritter
767 mailx history
768 .]
769 Elm, pine, mutt, and a whole bunch of graphical \s-1MUA\s0s
770 followed Mail's direction.
771 They are large, monolithic programs which include all emailing functions.
772 .PP
773 A different way was taken by the people of \s-1RAND\s0 Corporation.
774 In the beginning, they also had used a monolithic mail system,
775 called \s-1MS\s0 (for ``mail system'').
776 But in 1977, Stockton Gaines and Norman Shapiro
777 came up with a proposal of a new email system concept \(en
778 one that honored the Unix Philosophy.
779 The concept was implemented by Bruce Borden in 1978 and 1979.
780 This was the birth of \s-1MH\s0 \(en the ``mail handler''.
781 .PP
782 Since then, \s-1RAND\s0, the University of California at Irvine and
783 at Berkeley, and several others have contributed to the software.
784 However, it's core concepts remained the same.
785 In the late 90s, when development of \s-1MH\s0 slowed down,
786 Richard Coleman started with \fInmh\fP, the new mail handler.
787 His goal was to improve \s-1MH\s0 especially in regard of
788 the requirements of modern emailing.
789 Today, nmh is developed by various people on the Internet.
790 .[
791 ware
792 rand history
793 %P 128-137
794 .]
795 .[
796 peek
797 mh
798 .]
800 .NH 2
801 Contrasts to monolithic mail systems
802 .XS
803 \*(SN Contrasts to monolithic mail systems
804 .XE
805 .LP
806 All \s-1MUA\s0s are monolithic, except \s-1MH\s0.
807 Although there might actually exist further, very little known,
808 toolchest \s-1MUA\s0s, this statement reflects the situation pretty well.
809 .PP
810 Monolithic \s-1MUA\s0s gather all their functions in one program.
811 In contrast, \s-1MH\s0 is a toolchest of many small tools \(en one for each job.
812 Following is a list of important programs of \s-1MH\s0's toolchest
813 and their function.
814 It gives a feeling of how the toolchest looks like.
815 .IP \(bu
816 .CW inc :
817 incorporate new mail (this is how mail enters the system)
818 .IP \(bu
819 .CW scan :
820 list messages in folder
821 .IP \(bu
822 .CW show :
823 show message
824 .IP \(bu
825 .CW next\fR/\fPprev :
826 show next/previous message
827 .IP \(bu
828 .CW folder :
829 change current folder
830 .IP \(bu
831 .CW refile :
832 refile message into different folder
833 .IP \(bu
834 .CW rmm :
835 remove message
836 .IP \(bu
837 .CW comp :
838 compose new message
839 .IP \(bu
840 .CW repl :
841 reply to message
842 .IP \(bu
843 .CW forw :
844 forward message
845 .IP \(bu
846 .CW send :
847 send prepared message (this is how mail leaves the system)
848 .LP
849 \s-1MH\s0 has no special user interface like monolithic \s-1MUA\s0s have.
850 The user does not leave the shell to run \s-1MH\s0,
851 instead he uses the various \s-1MH\s0 programs within the shell.
852 Using a monolithic program with a captive user interface
853 means ``entering'' the program, using it, and ``exiting'' the program.
854 Using toolchests like \s-1MH\s0 means running programs,
855 alone or in combination with others, also from other toolchests,
856 without leaving the shell.
858 .NH 2
859 Data storage
860 .XS
861 \*(SN Data storage
862 .XE
863 .LP
864 \s-1MH\s0's mail storage is a directory tree under the user's
865 \s-1MH\s0 directory (usually \f(CW$HOME/Mail\fP),
866 where mail folders are directories and mail messages are text files
867 within them.
868 Each mail folder contains a file \f(CW.mh_sequences\fP which lists
869 the public message sequences of that folder,
870 for instance the \fIunseen\fP sequence for new messages.
871 Mail messages are text files located in a mail folder.
872 The files contain the messages as they were received.
873 They are named by ascending numbers in each folder.
874 .PP
875 This mailbox format is called ``\s-1MH\s0'' after the \s-1MUA\s0.
876 Alternatives are \fImbox\fP and \fImaildir\fP.
877 In the mbox format all messages are stored within one file.
878 This was a good solution in the early days, when messages
879 were only a few lines of text and were deleted soon.
880 Today, when single messages often include several megabytes
881 of attachments, it is a bad solution.
882 Another disadvantage of the mbox format is that it is
883 more difficult to write tools that work on mail messages,
884 because it is always necessary to first find and extract
885 the relevant message in the mbox file.
886 With the \s-1MH\s0 mailbox format, each message is a separate file.
887 Also, the problem of concurrent access to one mailbox is
888 reduced to the problem of concurrent access to one message.
889 The maildir format is generally similar to the \s-1MH\s0 format,
890 but modified towards guaranteed reliability.
891 This involves some complexity, unfortunately.
892 .PP
893 Working with \s-1MH\s0's toolchest on mailboxes is much like
894 working with Unix' toolchest on directory trees:
895 \f(CWscan\fP is like \f(CWls\fP,
896 \f(CWshow\fP is like \f(CWcat\fP,
897 \f(CWfolder\fP is like \f(CWcd\fP and \f(CWpwd\fP,
898 \f(CWrefile\fP is like \f(CWmv\fP,
899 and \f(CWrmm\fP is like \f(CWrm\fP.
900 .PP
901 \s-1MH\s0 extends the context of processes in Unix by two more items,
902 for its tools:
903 .IP \(bu
904 The current mail folder, which is similar to the current working directory.
905 For mail folders, \f(CWfolder\fP provides the corresponding functionality
906 of \f(CWcd\fP and \f(CWpwd\fP for directories.
907 .IP \(bu
908 Sequences, which are named sets of messages in a mail folder.
909 The current message, relative to a mail folder, is a special sequence.
910 It enables commands like \f(CWnext\fP and \f(CWprev\fP.
911 .LP
912 In contrast to Unix' context, which is maintained by the kernel,
913 \s-1MH\s0's context must be maintained by the tools themselves.
914 Usually there is one context per user, which resides in his
915 \f(CWcontext\fP file in the \s-1MH\s0 directory,
916 but a user can have several contexts, too.
917 Public sequences are an exception, as they belong to a mail folder,
918 and reside in the \f(CW.mh_sequences\fP file there.
919 .[
920 man page mh-profile mh-sequence
921 .]
923 .NH 2
924 Discussion of the design
925 .XS
926 \*(SN Discussion of the design
927 .XE
928 .LP
929 This section discusses \s-1MH\s0 in regard to the tenets
930 of the Unix Philosophy that Gancarz identified.
932 .PP
933 .B "Small is beautiful
934 and
935 .B "do one thing well
936 are two design goals that are directly visible in \s-1MH\s0.
937 Gancarz actually presents \s-1MH\s0 in his book as example under the
938 headline ``Making \s-1UNIX\s0 Do One Thing Well'':
939 .[
940 gancarz
941 unix philosophy
942 %P 125
943 .]
944 .QP
945 [\s-1MH\s0] consists of a series of programs which
946 when combined give the user an enormous ability
947 to manipulate electronic mail messages.
948 A complex application, it shows that not only is it
949 possible to build large applications from smaller
950 components, but also that such designs are actually preferable.
951 .LP
952 The various programs of \s-1MH\s0 were relatively easy to write,
953 because each of them is small, limited to one function,
954 and has clear boundaries.
955 For the same reasons, they are also good to maintain.
956 Further more, the system can easily get extended.
957 One only needs to put a new program into the toolchest.
958 This was done, for instance, when \s-1MIME\s0 support was added
959 (e.g. \f(CWmhbuild\fP).
960 Also, different programs can exist to do the basically same job
961 in different ways (e.g. in nmh: \f(CWshow\fP and \f(CWmhshow\fP).
962 .PP
963 If someone needs a mail system with some additionally
964 functions that are not available anywhere yet, he best expands a
965 toolchest system like \s-1MH\s0.
966 There he can add new functionality by simply adding additional
967 programs to the toolchest.
968 There he does not risk to break existing functionality by doing so.
970 .PP
971 .B "Store data in flat text files
972 is followed by \s-1MH\s0.
973 This is not surprising, because email messages are already plain text.
974 \s-1MH\s0 stores the messages as it receives them,
975 thus any other tool that works on \s-1RFC\s0\|2822 mail messages can operate
976 on the messages in an \s-1MH\s0 mailbox.
977 All other files \s-1MH\s0 uses are plain text, too.
978 It is therefore possible and encouraged to use the text processing
979 tools of Unix' toolchest to extend \s-1MH\s0's toolchest.
981 .PP
982 .B "Avoid captive user interfaces" .
983 \s-1MH\s0 is perfectly suited for non-interactive use.
984 It offers all functions directly and without captive user interfaces.
985 If, nonetheless, users want a graphical user interface,
986 they can have it with \fIxmh\fP or \fIexmh\fP.
987 These are graphical frontends for the \s-1MH\s0 toolchest.
988 This means, all email-related work is still done by \s-1MH\s0 tools,
989 but the frontend calls the appropriate commands when the user
990 clicks on buttons.
991 .PP
992 Providing easy-to-use user interfaces in form of frontends is a good
993 approach, because it does not limit the power of the backend itself.
994 The frontend will anyway only be able to make a subset of the
995 backend's power and flexibility available to the user.
996 But if it is a separate program,
997 then the missing parts can still be accessed at the backend directly.
998 If it is integrated, then this will hardly be possible.
999 An additional advantage is the possibility to have different frontends
1000 to the same backend.
1002 .PP
1003 .B "Choose portability over efficiency
1004 and
1005 .B "use shell scripts to increase leverage and portability" .
1006 These two tenets are indirectly, but nicely, demonstrated by
1007 Bolsky and Korn in their book about the Korn Shell.
1008 .[
1009 bolsky korn
1010 korn shell
1011 .]
1012 Chapter\|18 of the book shows a basic implementation
1013 of a subset of \s-1MH\s0 in ksh scripts.
1014 Of course, this is just a demonstration, but a brilliant one.
1015 It shows how quickly one can implement such a prototype with shell scripts,
1016 and how readable they are.
1017 The implementation in the scripting language may not be very fast,
1018 but it can be fast enough though, and this is all that matters.
1019 By having the code in an interpreted language, like the shell,
1020 portability becomes a minor issue, if we assume the interpreter
1021 to be widespread.
1022 .PP
1023 This demonstration also shows how easy it is to create single programs
1024 of a toolchest software.
1025 Eight tools (two of them have multiple names) and 16 functions
1026 with supporting code are presented to the reader.
1027 The tools comprise less than 40 lines of ksh each,
1028 in total about 200 lines.
1029 The functions comprise less than 80 lines of ksh each,
1030 in total about 450 lines.
1031 Such small software is easy to write, easy to understand,
1032 and thus easy to maintain.
1033 A toolchest improves the possibility to only write some parts
1034 and though create a working result.
1035 Expanding the toolchest, even without global changes,
1036 will likely be possible.
1038 .PP
1039 .B "Use software leverage to your advantage
1040 and the lesser tenet
1041 .B "allow the user to tailor the environment
1042 are ideally followed in the design of \s-1MH\s0.
1043 Tailoring the environment is heavily encouraged by the ability to
1044 directly define default options to programs.
1045 It is even possible to define different default options
1046 depending on the name under which a program is called.
1047 Software leverage is heavily encouraged by the ease of
1048 creating shell scripts that run a specific command line,
1049 built of several \s-1MH\s0 programs.
1050 There is few software that so much wants users to tailor their
1051 environment and to leverage the use of the software, like \s-1MH\s0.
1052 .PP
1053 Just to make one example:
1054 One might prefer a different listing format for the \f(CWscan\fP
1055 program.
1056 It is possible to take one of the distributed format files
1057 or to write one yourself.
1058 To use the format as default for \f(CWscan\fP, a single line,
1059 reading
1060 .DS
1061 .CW
1062 scan: -form FORMATFILE
1063 .DE
1064 must be added to \f(CW.mh_profile\fP.
1065 If one wants this different format as an additional command,
1066 instead of changing the default, he needs to create a link to
1067 \f(CWscan\fP, for instance titled \f(CWscan2\fP.
1068 The line in \f(CW.mh_profile\fP would then start with \f(CWscan2\fP,
1069 as the option should only be in effect for a program that is called as
1070 \f(CWscan2\fP.
1072 .PP
1073 .B "Make every program a filter
1074 is hard to find in \s-1MH\s0.
1075 The reason therefore is that most of \s-1MH\s0's tools provide
1076 basic file system operations for mailboxes.
1077 It is the same reason because of which \f(CWls\fP, \f(CWcp\fP, \f(CWmv\fP,
1078 and \f(CWrm\fP aren't filters neither.
1079 \s-1MH\s0 does not provide many filters itself, but it is a basis
1080 to write filters for.
1081 An example would be a mail text highlighter,
1082 that means a program that makes use of a color terminal to display
1083 header lines, quotations, and signatures in distinct colors.
1084 The author's version of such a program is an awk script with 25 lines.
1086 .PP
1087 .B "Build a prototype as soon as possible
1088 was again well followed by \s-1MH\s0.
1089 This tenet, of course, focuses on early development, which is
1090 long time ago for \s-1MH\s0.
1091 But without following this guideline at the very beginning,
1092 Bruce Borden may have not convinced the management of \s-1RAND\s0
1093 to ever create \s-1MH\s0.
1094 In Bruce' own words:
1095 .[
1096 ware rand history
1097 %P 132
1098 .]
1099 .QP
1100 [...] but [Stockton Gaines and Norm Shapiro] were not able
1101 to convince anyone that such a system would be fast enough to be usable.
1102 I proposed a very short project to prove the basic concepts,
1103 and my management agreed.
1104 Looking back, I realize that I had been very lucky with my first design.
1105 Without nearly enough design work,
1106 I built a working environment and some header files
1107 with key structures and wrote the first few \s-1MH\s0 commands:
1108 inc, show/next/prev, and comp.
1109 [...]
1110 With these three, I was able to convince people that the structure was viable.
1111 This took about three weeks.
1113 .NH 2
1114 Problems
1115 .XS
1116 \*(SN Problems
1117 .XE
1118 .LP
1119 \s-1MH\s0 is not without problems.
1120 There are two main problems: one is technical, the other is about human behavior.
1121 .PP
1122 \s-1MH\s0 is old and email today is very different to email in the time
1123 when \s-1MH\s0 was designed.
1124 \s-1MH\s0 adapted to the changes pretty well, but it is limited, though.
1125 \s-1MIME\s0 support and support for different character encodings
1126 is available, but only on a moderate level.
1127 This comes from limited development resources.
1128 More active developers could quickly change this.
1129 But \s-1MH\s0 is also limited by design, which is the larger problem.
1130 \s-1IMAP\s0, for example, conflicts with \s-1MH\s0's design to a large extend.
1131 These design conflicts are not easily solvable.
1132 Possibly, they require a redesign.
1133 \s-1IMAP\s0 may be too different to the classic mail model,
1134 which \s-1MH\s0 covers, so that \s-1MH\s0 may never support it well.
1135 .PP
1136 The other kind of problem are human habits.
1137 In this world, where almost all \s-1MUA\s0s are monolithic,
1138 it is very difficult to convince people to use a toolbox style \s-1MUA\s0
1139 like \s-1MH\s0.
1140 The habits are so strong, that even people who understand the concept
1141 and advantages of \s-1MH\s0 do not like to switch,
1142 simply because \s-1MH\s0 is different.
1143 Unfortunately, the frontends to \s-1MH\s0, which could provide familiar look'n'feel,
1144 are quite outdated and thus not very appealing, compared to the modern interfaces
1145 of many monolithic \s-1MUA\s0s.
1147 .NH 2
1148 Summary \s-1MH\s0
1149 .XS
1150 \*(SN Summary \s-1MH\s0
1151 .XE
1152 .LP
1153 \s-1MH\s0 is an \s-1MUA\s0 that follows the Unix Philosophy in its design.
1154 It consists of a toolchest of small tools, each of them does one job well.
1155 The toolchest approach offers great flexibility to the user.
1156 It is possible to utilize the complete power of the Unix shell with \s-1MH\s0.
1157 This makes \s-1MH\s0 a very powerful mail system.
1158 Extending and customizing \s-1MH\s0 is easy and encouraged.
1159 .PP
1160 Apart from the user's perspective, \s-1MH\s0 is development-friendly.
1161 Its overall design follows clear rules.
1162 The single tools do only one job, thus they are easy to understand,
1163 easy to write, and good to maintain.
1164 They are all independent and do not interfere with the others.
1165 Automated testing of their function is a straight forward task.
1166 .PP
1167 It is sad, that \s-1MH\s0's differentness is its largest problem,
1168 as its differentness is also its largest advantage.
1169 Unfortunately, for most people their habits are stronger
1170 than the attraction of the clear design and the power, \s-1MH\s0 offers.
1174 .NH 1
1175 Case study: uzbl
1176 .XS
1177 .sp .5v
1178 \*(SN Case study: uzbl
1179 .XE
1180 .LP
1181 The last chapter took a look on the \s-1MUA\s0 \s-1MH\s0,
1182 which is an old and established software.
1183 This chapter covers uzbl, a fresh new project.
1184 Uzbl is a web browser that adheres to the Unix Philosophy.
1185 Its name comes from the \fILolspeak\fP word for ``usable'';
1186 it is pronounced identical.
1188 .NH 2
1189 Historical background
1190 .XS
1191 \*(SN Historical background
1192 .XE
1193 .LP
1194 Uzbl was started by Dieter Plaetinck in April 2009.
1195 The idea was born in a thread in the Arch Linux Forums.
1196 .[
1197 arch linux forums
1198 browser
1199 .]
1200 After some discussion about failures of well known web browsers,
1201 Plaetinck (alias Dieter@be) came up with a very sketchy proposal
1202 of how a better web browser could look like.
1203 To the question of another member, if Plaetinck would write that program,
1204 because it would sound fantastic, Plaetinck replied:
1205 ``Maybe, if I find the time ;-)''.
1206 .PP
1207 Fortunately, he found the time.
1208 One day later, the first prototype was out.
1209 One week later, uzbl had an own website.
1210 One month after the first code showed up,
1211 a mailing list was installed to coordinate and discuss further development.
1212 Then a wiki followed to store documentation and scripts that showed up on the
1213 mailing list and elsewhere.
1214 .PP
1215 In the, now, one year of uzbl's existence, it was heavily developed on various branches.
1216 Plaetinck's task became more and more to only merge the best code from the
1217 different branches into his main branch, and to apply patches.
1218 About once a month, Plaetinck released a new version.
1219 In September 2009, he presented several forks of uzbl.
1220 Uzbl, actually, opened the field for a whole family of web browsers with similar shape.
1221 .PP
1222 In July 2009, \fILinux Weekly News\fP published an interview with Plaetinck about uzbl.
1223 In September 2009, the uzbl web browser was on \fISlashdot\fP.
1225 .NH 2
1226 Contrasts to other web browsers
1227 .XS
1228 \*(SN Contrasts to other web browsers
1229 .XE
1230 .LP
1231 Like most \s-1MUA\s0s are monolithic, but \s-1MH\s0 is a toolchest,
1232 most web browsers are monolithic, but uzbl is a frontend to a toolchest.
1233 .PP
1234 Today, uzbl is divided into uzbl-core and uzbl-browser.
1235 Uzbl-core is, how its name already indicates, the core of uzbl.
1236 It handles commands and events to interface other programs,
1237 and also displays webpages by using \fIwebkit\fP as render engine.
1238 Uzbl-browser combines uzbl-core with a bunch of handler scripts, a status bar,
1239 an event manager, yanking, pasting, page searching, zooming, and more stuff,
1240 to form a ``complete'' web browser.
1241 In the following text, the term ``uzbl'' usually stands for uzbl-browser,
1242 so uzbl-core is included.
1243 .PP
1244 Unlike most other web browsers, uzbl is mainly the mediator between the
1245 various tools that cover single jobs.
1246 Therefore, uzbl listens for commands on a named pipe (fifo), a Unix socket,
1247 and on stdin, and it writes events to a Unix socket and to stdout.
1248 Loading a webpage in a running uzbl instance requires only:
1249 .DS
1250 .CW
1251 echo 'uri http://example.org' >/path/to/uzbl-fifo
1252 .DE
1253 The graphical rendering of the webpage is done by webkit,
1254 a web content engine.
1255 Uzbl-core is built around libwebkit.
1256 .PP
1257 Downloads, browsing history, bookmarks, and the like are not provided
1258 by the core itself, like they are in other web browsers.
1259 Uzbl-browser also only provides, so called, handler scripts that wrap
1260 external applications which provide the actual functionality.
1261 For instance, \fIwget\fP is used to download files and uzbl-browser
1262 includes a script that calls wget with appropriate options in
1263 a prepared environment.
1264 .PP
1265 Modern web browsers are proud to have addons, plugins, and modules, instead.
1266 This is their effort to achieve similar goals.
1267 But instead of using existing, external programs, modern web browsers
1268 include these functions.
1270 .NH 2
1271 Discussion of the design
1272 .XS
1273 \*(SN Discussion of the design
1274 .XE
1275 .LP
1276 This section discusses uzbl in regard of the Unix Philosophy,
1277 as identified by Gancarz.
1279 .PP
1280 .B "Make each program do one thing well" .
1281 Uzbl tries to be a web browser and nothing else.
1282 The common definition of a web browser is, of course, highly influenced by
1283 existing implementations of web browsers, although they are degenerated.
1284 Web browsers should be programs to browse the web, and nothing more.
1285 This is the one thing they should do.
1286 .PP
1287 Web browsers should not, for instance, manage downloads.
1288 This is the job download managers exist for.
1289 Download managers do primary care about being good in downloading files.
1290 Modern web browsers provide download management only as a secondary feature.
1291 How could they do this job better, than programs that exist only for
1292 this very job?
1293 And how could anyone want less than the best download manager available?
1294 .PP
1295 A web browser's job is to let the user browse the web.
1296 This means, navigating through websites by following links.
1297 Rendering the \s-1HTML\s0 sources is a different job, too.
1298 It is covered by the webkit render engine, in uzbl's case.
1299 Audio and video content and files like PostScript, \s-1PDF\s0, and the like,
1300 are also not the job of a web browser.
1301 They should be handled by external applications \(en
1302 ones which's job is to handle such data.
1303 Uzbl strives to do it this way.
1304 .PP
1305 Remember Doug McIlroy:
1306 .I
1307 ``Write programs that do one thing and do it well.
1308 Write programs to work together.''
1309 .R
1310 .PP
1311 The lesser tenet
1312 .B "allow the user to tailor the environment
1313 matches good here.
1314 There was the question, how anyone could want anything less than the
1315 best program for the job.
1316 But as personal preferences matter, it is probably more important to ask:
1317 How could anyone want something else than his preferred program for the job?
1318 .PP
1319 Usually users want one program for a specific job.
1320 Hence, whenever the task is, for instance, downloading,
1321 the same download manager should be used.
1322 More advanced users might want to have this download manager in this
1323 situation and that one in that situation.
1324 They should be able to configure it this way.
1325 With uzbl, one can use any download manager the user wants.
1326 To switch to a different one, only one line in a small handler script
1327 needs to be changed.
1328 Alternatively it would be possible to query the program to use by
1329 reading a global file or an environment variable, in the handler script.
1330 .PP
1331 Uzbl does neither have its own download manager nor depends on a
1332 specific one, hence uzbl's browsing abilities will not be lowered by having
1333 a bad download manager.
1334 Uzbl's download capabilities will be just as good as the ones of the best
1335 download manager available on the system.
1336 Of course, this applies to all of the other supplementary tools, too.
1338 .PP
1339 .B "Use software leverage to your advantage" .
1340 Uzbl is designed to be extended by external tools.
1341 These external tools are usually wrapped by small handler shell scripts.
1342 Shell scripts are the glue in this approach.
1343 They make the various parts fit together.
1344 .PP
1345 The history mechanism of uzbl shall be presented as an example.
1346 Uzbl is configured to spawn a script to append an entry to the history
1347 whenever the event of a fully loaded page occurs.
1348 The script to append the entry to the history is not much more than:
1349 .DS
1350 .CW
1351 #!/bin/sh
1352 file=/path/to/uzbl-history
1353 echo `date +'%Y-%m-%d %H:%M:%S'`" $6 $7" >> $file
1354 .DE
1355 \f(CW$6\fP and \f(CW$7\fP expand to the \s-1URL\s0 and the page title.
1356 .PP
1357 For loading an entry, a key is bound to spawn a load-from-history script.
1358 The script reverses the history to have newer entries first,
1359 then displays \fIdmenu\fP to let the user select an item,
1360 and afterwards writes the selected \s-1URL\s0 into uzbl's command input pipe.
1361 With error checking and corner case handling removed,
1362 the script looks like this:
1363 .DS
1364 .CW
1365 #!/bin/sh
1366 file=/path/to/uzbl-history
1367 goto=`tac $file | dmenu | cut -d' ' -f 3`
1368 echo "uri $goto" > $4
1369 .DE
1370 \f(CW$4\fP expands to the path of the command input pipe of the current
1371 uzbl instance.
1373 .PP
1374 .B "Avoid captive user interfaces" .
1375 One could say, that uzbl, to a large extend, actually \fIis\fP
1376 a captive user interface.
1377 But the difference to most other web browsers is, that uzbl is only
1378 the captive user interface frontend (and the core of the backend).
1379 Many parts of the backend are independent of uzbl.
1380 Some are distributed with uzbl, for some external programs,
1381 handler scripts are distributed,
1382 but arbitrary additional functionality can be added if desired.
1383 .PP
1384 The frontend is captive \(en that is true.
1385 This is okay for the task of browsing the web, as this task is only relevant
1386 for humans.
1387 Automated programs would \fIcrawl\fP the web.
1388 That means, they read the source directly.
1389 The source includes all the semantics.
1390 The graphical representation is just for humans to transfer the semantics
1391 more intuitively.
1393 .PP
1394 .B "Make every program a filter" .
1395 Graphical web browsers are almost dead ends in the chain of information flow.
1396 Thus it is difficult to see what graphical web browsers should filter.
1397 Graphical web browsers exist almost only to be interactively used by humans.
1398 The only case when one might want to automate the rendering function is
1399 to generate images of rendered webpages.
1401 .PP
1402 .B "Small is beautiful"
1403 is not easy to apply to a web browser, because modern web technology
1404 is very complex, hence the rendering task is very complex.
1405 Modern web browsers have to consist of many thousand lines of code,
1406 unfortunately.
1407 Using the toolchest approach and wrappers can split the browser into
1408 several small parts, tough.
1409 .PP
1410 As of March 2010, uzbl-core consists of about 3\,500 lines of C code.
1411 The distribution includes another 3\,500 lines of Shell and Python code,
1412 which are the handler scripts and plugins like a modal interface.
1413 Further more, uzbl uses functionality of external tools like
1414 \fIwget\fP and \fInetcat\fP.
1415 Up to this point, uzbl looks pretty neat and small.
1416 The ugly part of uzbl is the web content renderer, webkit.
1417 Webkit consists of roughly 400\,000 (!) lines of code.
1418 Unfortunately, small web render engines are not possible anymore
1419 because of the modern web.
1421 .PP
1422 .B "Build a prototype as soon as possible" .
1423 Plaetinck made his code public, right from the beginning.
1424 Discussion and development was, and still is, open to everyone interested.
1425 Development versions of uzbl can be obtained very simply from the code
1426 repository.
1427 Within the first year of uzbl's existence, a new version was released
1428 more often than once a month.
1429 Different forks and branches arose.
1430 They introduced new features, which were tested for suitability
1431 for the main branch.
1432 The experiences of using prototypes influenced further development.
1433 Actually, all development was community driven.
1434 Plaetinck says, three months after uzbl's birth:
1435 ``Right now I hardly code anything myself for Uzbl.
1436 I just merge in other people's code, ponder a lot, and lead the discussions.''
1437 .[
1438 lwn
1439 uzbl
1440 .]
1443 .NH 2
1444 Problems
1445 .XS
1446 \*(SN Problems
1447 .XE
1448 .LP
1449 Similar to \s-1MH\s0, uzbl, too suffers from being different.
1450 It is sad, but people use what they know.
1451 Fortunately, uzbl's user interface can look and feel very much the
1452 same as the one of the well known web browsers,
1453 hiding the internal differences.
1454 But uzbl has to provide this similar look and feel to be accepted
1455 as a ``normal'' browser by ``normal'' users.
1456 .PP
1457 Though, the more important problem is the modern web.
1458 The modern web is simply broken.
1459 It has state in a state-less protocol,
1460 it misuses technologies,
1461 and it is helplessly overloaded.
1462 The result are web content render engines that must consist
1463 of hundreds of thousands lines of code.
1464 They also must combine and integrate many different technologies,
1465 only to make our modern web accessible.
1466 Website to image converter are hardly possible to run without
1467 human interaction because of state in sessions, impossible
1468 deep-linking, and unautomatable technologies.
1469 .PP
1470 The web was misused to provide all kinds of imaginable wishes.
1471 Now web browsers, and eventually the users, suffer from it.
1474 .NH 2
1475 Summary uzbl
1476 .XS
1477 \*(SN Summary uzbl
1478 .XE
1479 .LP
1480 ``Uzbl is a browser that adheres to the Unix Philosophy'',
1481 that is how uzbl is seen by its authors.
1482 Indeed, uzbl follows the Unix Philosophy in many ways.
1483 It consists of independent parts that work together,
1484 while its core is mainly a mediator which glues the parts together.
1485 .PP
1486 Software leverage can excellently be seen in uzbl.
1487 External tools are used, independent tasks are separated
1488 in independent parts and glued together with small handler scripts.
1489 .PP
1490 As uzbl, more or less, consists of a set of tools and a bit
1491 of glue, anyone can put the parts together and expand it
1492 in any desired way.
1493 Uzbl is very flexible and customizable.
1494 These properties make it valuable for advanced users,
1495 but may keep novice users from using it.
1496 .PP
1497 But uzbl's main problem is the modern web, that makes it hard
1498 to design a sane web browser.
1499 Despite this bad situation, uzbl does a fairly good job.
1502 .NH 1
1503 Final thoughts
1504 .XS
1505 .sp .5v
1506 \*(SN Final thoughts
1507 .XE
1509 .NH 2
1510 Quick summary
1511 .XS
1512 \*(SN Quick summary
1513 .XE
1514 .LP
1515 good design
1516 .LP
1517 unix phil
1518 .LP
1519 case studies
1521 .NH 2
1522 Why people should choose
1523 .XS
1524 \*(SN Why people should choose
1525 .XE
1526 .LP
1527 Make the right choice!
1529 .nr PI .3i
1530 .rm ]<
1531 .de ]<
1532 .LP
1533 .de FP
1534 .IP \\\\$1.
1535 \\..
1536 .rm FS FE
1537 ..
1538 .ds CH "
1539 .bp
1540 .rs
1541 .sp .3i
1542 .TL
1543 References
1544 .LP
1545 .XS
1546 .sp .5v
1547 References
1548 .XE
1549 .sp 2v
1550 .nr PS -1
1551 .nr VS -1
1552 .[
1553 $LIST$
1554 .]
1555 .\".wh -1p
1556 .bp
1557 .PX