docs/unix-phil
view unix-phil.ms @ 47:b6ae4a8ab1d3
improved references and some minor design issues
| author | meillo@marmaro.de |
|---|---|
| date | Mon, 12 Apr 2010 14:11:39 +0200 |
| parents | 98a1446744c7 |
| children | 40caeb9e9b25 |
line source
1 .nr PS 11
2 .nr VS 13
3 .nr lu 0
5 .de CW
6 .nr PQ \\n(.f
7 .if t .ft CW
8 .ie ^\\$1^^ .if n .ul 999
9 .el .if n .ul 1
10 .if t .if !^\\$1^^ \&\\$1\f\\n(PQ\\$2
11 .if n .if \\n(.$=1 \&\\$1
12 .if n .if \\n(.$>1 \&\\$1\c
13 .if n .if \\n(.$>1 \&\\$2
14 ..
16 .ds [. \ [
17 .ds .] ]
19 .rn NH _N
20 .de NH
21 .if '\\$1'1' .sp 2v
22 .if '\\$1'1' .nr PS +2
23 ._N \\$1
24 .if '\\$1'1' .nr PS -2
25 ..
27 .am QP
28 .ps -1
29 ..
31 .am DS
32 .CW
33 .ps -1
34 ..
36 .\"----------------------------------------
38 .TL
39 .ps +4
40 Why the Unix Philosophy still matters
41 .AU
42 markus schnalke <meillo@marmaro.de>
43 .AB
44 .ti \n(.iu
45 This paper explains the importance of the Unix Philosophy for software design.
46 Today, few software designers are aware of these concepts,
47 and thus a lot of modern software is more limited than necessary
48 and makes less use of software leverage than possible.
49 Knowing and following the guidelines of the Unix Philosophy makes software more valuable.
50 .AE
52 .FS
53 .ps -1
54 This paper was prepared for the ``Software Analysis'' seminar at University Ulm.
55 Mentor was professor Franz Schweiggert.
56 Handed in on 2010-04-12.
57 You may retrieve this document from
58 .CW \s-1http://marmaro.de/docs \ .
59 .FE
61 .NH 1
62 Introduction
63 .XS
64 .sp .5v
65 .B
66 \*(SN Introduction
67 .XE
68 .LP
69 The Unix Philosophy is the essence of how the Unix operating system,
70 especially its toolchest, was designed.
71 It is no limited set of fixed rules,
72 but a loose set of guidelines which tell how to write software that
73 suites well into Unix.
74 Actually, the Unix Philosophy describes what is common to typical Unix software.
75 The Wikipedia has an accurate definition:
76 .[
77 wikipedia
78 unix philosophy
79 .]
80 .QP
81 The \fIUnix philosophy\fP is a set of cultural norms and philosophical
82 approaches to developing software based on the experience of leading
83 developers of the Unix operating system.
84 .PP
85 As there is no single definition of the Unix Philosophy,
86 several people have stated their view on what it comprises.
87 Best known are:
88 .IP \(bu
89 Doug McIlroy's summary: ``Write programs that do one thing and do it well.''
90 .[
91 mahoney
92 oral history
93 .]
94 .IP \(bu
95 Mike Gancarz' book ``The UNIX Philosophy''.
96 .[
97 gancarz
98 unix philosophy
99 .]
100 .IP \(bu
101 Eric S. Raymond's book ``The Art of UNIX Programming''.
102 .[
103 raymond
104 art of unix programming
105 .]
106 .LP
107 These different views on the Unix Philosophy have much in common.
108 Especially, the main concepts are similar in all of them.
109 McIlroy's definition can surely be called the core of the Unix Philosophy,
110 but the fundamental idea behind it all, is ``small is beautiful''.
112 .PP
113 The Unix Philosophy explains how to design good software for Unix.
114 Many concepts described here, base on facilities of Unix.
115 Other operating systems may not offer such facilities,
116 hence it may not be possible to design software in the way of the
117 Unix Philosophy for them.
118 .PP
119 The Unix Philosophy has an idea of how the process of software development
120 should look like, but large parts of the philosophy are quite independent
121 from a concrete development process.
122 However, one will soon recognize that some development processes work well
123 with the ideas of the Unix Philosophy and support them, while others are
124 at cross-purposes.
125 Kent Beck's books about Extreme Programming are valuable supplemental
126 resources on this topic.
127 .PP
128 The question of how to actually write code and how the code should looks
129 like in detail, are out of focus here.
130 ``The Practice of Programming'' by Kernighan and Pike,
131 .[
132 kernighan pike
133 practice of programming
134 .]
135 is a good book that covers this topic.
136 Its point of view matches to the one of this paper.
138 .NH 1
139 Importance of software design in general
140 .XS
141 .sp .5v
142 .B
143 \*(SN Importance of software design in general
144 .XE
145 .LP
146 Software design is the planning of how the internal structure
147 and external interfaces of a software should look like.
148 It has nothing to do with visual appearance.
149 If we take a program as a car, then its color is of no matter.
150 Its design would be the car's size, its shape, the locations of doors,
151 the passenger/space ratio, the available controls and instruments,
152 and so forth.
153 .PP
154 Why should software get designed at all?
155 It is general knowledge, that even a bad plan is better than no plan.
156 Not designing software means programming without plan.
157 This will pretty sure lead to horrible results.
158 Software that is horrible to use and horrible to maintain.
159 These two aspects are the visible ones.
160 Often invisible though, are the wasted possible gains.
161 Good software design can make these gains available.
162 .PP
163 A software's design deals with quality properties.
164 Good design leads to good quality, and quality is important.
165 Any car may be able to drive from A to B,
166 but it depends on the car's properties whether it is a good choice
167 for passenger transport or not.
168 It depends on its properties if it is a good choice
169 for a rough mountain area.
170 And it depends on its properties if the ride will be fun.
172 .PP
173 Requirements for a software are twofold:
174 functional and non-functional.
175 .IP \(bu
176 Functional requirements define directly the software's functions.
177 They are the reason why software gets written.
178 Someone has a problem and needs a tool to solve it.
179 Being able to solve the problem is the main functional goal.
180 It is the driving force behind all programming effort.
181 Functional requirements are easier to define and to verify.
182 .IP \(bu
183 Non-functional requirements are called \fIquality\fP requirements, too.
184 The quality of a software are the properties that are not directly related to
185 the software's basic functions.
186 Tools of bad quality often do solve the problems they were written for,
187 but introduce problems and difficulties for usage and development, later on.
188 Quality aspects are often overlooked at first sight,
189 and are often difficult to define clearly and to verify.
190 .PP
191 Quality is hardly interesting when the software gets built initially,
192 but it has a high impact on usability and maintenance of the software, later.
193 A short-sighted might see in developing a software, mainly building something up.
194 But experience shows, that building the software the first time is
195 only a small amount of the overall work.
196 Bug fixing, extending, rebuilding of parts \(en maintenance work \(en
197 does soon take over the major part of the time spent on a software.
198 And of course, the time spent actually using the software.
199 These processes are highly influenced by the software's quality.
200 Thus, quality must not be neglected.
201 However, the problem with quality is that you hardly ``stumble over''
202 bad quality during the first build,
203 although this is the time when you should care about good quality most.
204 .PP
205 Software design has little to do with the basic function of a software \(en
206 this requirement will get satisfied anyway.
207 Software design is more about quality aspects of the software.
208 Good design leads to good quality, bad design to bad quality.
209 The primary functions of the software will be affected modestly by bad quality,
210 but good quality can provide a lot of additional gain,
211 even at places where one never expected it.
212 .PP
213 The ISO/IEC\|9126-1 standard, part\|1,
214 .[
215 iso product quality
216 .]
217 defines the quality model as consisting out of:
218 .IP \(bu
219 .I Functionality
220 (suitability, accuracy, inter\%operability, security)
221 .IP \(bu
222 .I Reliability
223 (maturity, fault tolerance, recoverability)
224 .IP \(bu
225 .I Usability
226 (understandability, learnability, operability, attractiveness)
227 .IP \(bu
228 .I Efficiency
229 (time behavior, resource utilization)
230 .IP \(bu
231 .I Maintainability
232 (analyzability, changeability, stability, testability)
233 .IP \(bu
234 .I Portability
235 (adaptability, installability, co-existence, replaceability)
236 .LP
237 Good design can improve these properties of a software,
238 bad designed software likely suffers in these points.
239 .PP
240 One further goal of software design is consistency.
241 Consistency eases understanding, working on, and using things.
242 Consistent internal structure and consistent interfaces to the outside
243 can be provided by good design.
244 .PP
245 Software should be well designed because good design avoids many
246 problems during a software's lifetime.
247 And software should be well designed because good design can offer
248 much additional gain.
249 Indeed, much effort should be spent into good design to make software more valuable.
250 The Unix Philosophy shows a way of how to design software well.
251 It offers guidelines to achieve good quality and high gain for the effort spent.
254 .NH 1
255 The Unix Philosophy
256 .XS
257 .sp .5v
258 .B
259 \*(SN The Unix Philosophy
260 .XE
261 .LP
262 The origins of the Unix Philosophy were already introduced.
263 This chapter explains the philosophy, oriented on Gancarz,
264 and shows concrete examples of its application.
266 .NH 2
267 Pipes
268 .XS
269 \*(SN Pipes
270 .XE
271 .LP
272 Following are some examples to demonstrate how applied Unix Philosophy feels like.
273 Knowledge of using the Unix shell is assumed.
274 .PP
275 Counting the number of files in the current directory:
276 .DS
277 ls | wc -l
278 .DE
279 The
280 .CW ls
281 command lists all files in the current directory, one per line,
282 and
283 .CW "wc -l
284 counts the number of lines.
285 .PP
286 Counting the number of files that do not contain ``foo'' in their name:
287 .DS
288 ls | grep -v foo | wc -l
289 .DE
290 Here, the list of files is filtered by
291 .CW grep
292 to remove all lines that contain ``foo''.
293 The rest equals the previous example.
294 .PP
295 Finding the five largest entries in the current directory.
296 .DS
297 du -s * | sort -nr | sed 5q
298 .DE
299 .CW "du -s *
300 returns the recursively summed sizes of all files in the current directory
301 \(en no matter if they are regular files or directories.
302 .CW "sort -nr
303 sorts the list numerically in reverse order (descending).
304 Finally,
305 .CW "sed 5q
306 quits after it has printed the fifth line.
307 .PP
308 The presented command lines are examples of what Unix people would use
309 to get the desired output.
310 There are also other ways to get the same output.
311 It's a user's decision which way to go.
312 .PP
313 The examples show that many tasks on a Unix system
314 are accomplished by combining several small programs.
315 The connection between the single programs is denoted by the pipe operator `|'.
316 .PP
317 Pipes, and their extensive and easy use, are one of the great
318 achievements of the Unix system.
319 Pipes between programs have been possible in earlier operating systems,
320 but it has never been a so central part of the concept.
321 When, in the early seventies, Doug McIlroy introduced pipes into the
322 Unix system,
323 ``it was this concept and notation for linking several programs together
324 that transformed Unix from a basic file-sharing system to an entirely new way of computing.''
325 .[
326 aughenbaugh
327 unix oral history
328 .]
329 .PP
330 Being able to specify pipelines in an easy way is,
331 however, not enough by itself.
332 It is only one half.
333 The other is the design of the programs that are used in the pipeline.
334 They need interfaces that allow them to be used in such a way.
336 .NH 2
337 Interface design
338 .XS
339 \*(SN Interface design
340 .XE
341 .LP
342 Unix is, first of all, simple \(en Everything is a file.
343 Files are sequences of bytes, without any special structure.
344 Programs should be filters, which read a stream of bytes from standard input (stdin)
345 and write a stream of bytes to standard output (stdout).
346 If the files \fIare\fP sequences of bytes,
347 and the programs \fIare\fP filters on byte streams,
348 then there is exactly one data interface.
349 Hence it is possible to combine programs in any desired way.
350 .PP
351 Even a handful of small programs yields a large set of combinations,
352 and thus a large set of different functions.
353 This is leverage!
354 If the programs are orthogonal to each other \(en the best case \(en
355 then the set of different functions is greatest.
356 .PP
357 Programs can also have a separate control interface,
358 besides their data interface.
359 The control interface is often called ``user interface'',
360 because it is usually designed to be used by humans.
361 The Unix Philosophy discourages to assume the user to be human.
362 Interactive use of software is slow use of software,
363 because the program waits for user input most of the time.
364 Interactive software requires the user to be in front of the computer.
365 Interactive software occupy the user's attention while they are running.
366 .PP
367 Now to come back to the idea of combining several small programs,
368 to have a more specific function.
369 If these single tools would all be interactive,
370 how would the user control them?
371 It is not only a problem to control several programs at once,
372 if they run at the same time,
373 it also very inefficient to have to control each of the single programs
374 that are intended to act as one large program.
375 Hence, the Unix Philosophy discourages programs to demand interactive use.
376 The behavior of programs should be defined at invocation.
377 This is done by specifying arguments to the program call
378 (command line switches).
379 Gancarz discusses this topic as ``avoid captive user interfaces''.
380 .[ [
381 gancarz unix philosophy
382 .], page 88 ff.]
383 .PP
384 Non-interactive use is, during development, also an advantage for testing.
385 Testing of interactive programs is much more complicated,
386 than testing of non-interactive programs.
388 .NH 2
389 The toolchest approach
390 .XS
391 \*(SN The toolchest approach
392 .XE
393 .LP
394 A toolchest is a set of tools.
395 Instead of having one big tool for all tasks, one has many small tools,
396 each for one task.
397 Difficult tasks are solved by combining several of the small, simple tools.
398 .PP
399 The Unix toolchest \fIis\fP a set of small, (mostly) non-interactive programs
400 that are filters on byte streams.
401 They are, to a large extend, unrelated in their function.
402 Hence, the Unix toolchest provides a large set of functions
403 that can be accessed by combining the programs in the desired way.
404 .PP
405 There are also advantages for developing small toolchest programs.
406 It is easier and less error-prone to write small programs.
407 It is also easier and less error-prone to write a large set of small programs,
408 than to write one large program with all the functionality included.
409 If the small programs are combinable, then they offer even a larger set
410 of functions than the single large program.
411 Hence, one gets two advantages out of writing small, combinable programs:
412 They are easier to write and they offer a greater set of functions through
413 combination.
414 .PP
415 But there are also two main drawbacks of the toolchest approach.
416 First, one simple, standardized interface has to be sufficient.
417 If one feels the need for more ``logic'' than a stream of bytes,
418 then a different approach might be of need.
419 But it is also possible, that he just can not imagine a design where
420 a stream of bytes is sufficient.
421 By becoming more familiar with the ``Unix style of thinking'',
422 developers will more often and easier find simple designs where
423 a stream of bytes is a sufficient interface.
424 .PP
425 The second drawback of a toolchest affects the users.
426 A toolchest is often more difficult to use.
427 It is necessary to become familiar with each of the tools,
428 to be able to use the right one in a given situation.
429 Additionally, one needs to combine the tools in a senseful way himself.
430 This is like a sharp knife \(en it is a powerful tool in the hand of a
431 master, but of no good value in the hand of an unskilled.
432 However, learning single, small tools of a toolchest is easier than
433 learning a complex tool.
434 And the user will already have a basic understanding of a yet unknown tool,
435 if the tools of a toolchest have a common, consistent style.
436 He will be able to transfer knowledge over from one tool to another.
437 .PP
438 Moreover, the second drawback can be removed to a large extend
439 by adding wrappers around the basic tools.
440 Novice users do not need to learn several tools, if a professional wraps
441 complete command lines into a higher-level script.
442 Note that the wrapper script still calls the small tools;
443 it is just like a skin around them.
444 No complexity is added this way.
445 But new programs can get created out of existing one with very low effort.
446 .PP
447 A wrapper script for finding the five largest entries in the current directory
448 could look like this:
449 .DS
450 #!/bin/sh
451 du -s * | sort -nr | sed 5q
452 .DE
453 The script itself is just a text file that calls the command line,
454 which a professional user would type in directly.
455 It is probably worth to make the program flexible on the number of
456 entries it prints:
457 .DS
458 #!/bin/sh
459 num=5
460 [ $# -eq 1 ] && num="$1"
461 du -sh * | sort -nr | sed "${num}q"
462 .DE
463 This script acts like the one before, when called without an argument.
464 But one can also specify a numerical argument to define the number of lines to print.
465 One can surely imagine even more flexible versions, however,
466 they will still relay on the external programs,
467 which do the actual work.
469 .NH 2
470 A powerful shell
471 .XS
472 \*(SN A powerful shell
473 .XE
474 .LP
475 The Unix shell provides the possibility to combine small programs into large ones.
476 But a powerful shell is a great feature in other ways, too.
477 For instance by being scriptable.
478 Control statements are build into the shell.
479 The functions, however, are the normal programs of the system.
480 Thus, as the programs are already known,
481 learning to program in the shell becomes easy.
482 Using normal programs as functions in the shell programming language
483 is only possible because they are small and combinable tools in a toolchest style.
484 .PP
485 The Unix shell encourages to write small scripts,
486 by combining existing programs, because it is so easy to do.
487 This is a great step towards automation.
488 It is wonderful if the effort to automate a task equals the effort
489 to do the task a second time by hand.
490 If this holds,
491 then the user will be happy to automate everything he does more than once.
492 .PP
493 Small programs that do one job well, standardized interfaces between them,
494 a mechanism to combine parts to larger parts, and an easy way to automate tasks,
495 this will inevitably produce software leverage.
496 Getting multiple times the benefit of an investment is a great offer.
497 .PP
498 The shell also encourages rapid prototyping.
499 Many well known programs started as quickly hacked shell scripts,
500 and turned into ``real'' programs, written in C, later.
501 Building a prototype first, is a way to avoid the biggest problems
502 in application development.
503 Fred Brooks explains in ``No Silver Bullet'':
504 .[
505 brooks
506 no silver bullet
507 .]
508 .QP
509 The hardest single part of building a software system is deciding precisely what to build.
510 No other part of the conceptual work is so difficult as establishing the detailed
511 technical requirements, [...].
512 No other part of the work so cripples the resulting system if done wrong.
513 No other part is more difficult to rectify later.
514 .PP
515 Writing a prototype is a great method for becoming familiar with the requirements
516 and to run into real problems early.
517 .[ [
518 gancarz
519 unix philosophy
520 .], page 28 f.]
521 .PP
522 Prototyping is often seen as a first step in building a software.
523 This is, of course, good.
524 However, the Unix Philosophy has an \fIadditional\fP perspective on prototyping:
525 After having built the prototype, one might notice, that the prototype is already
526 \fIgood enough\fP.
527 Hence, no reimplementation, in a more sophisticated programming language,
528 might be of need, at least for the moment.
529 Maybe later, it might be necessary to rewrite the software, but not now.
530 By delaying further work, one keeps the flexibility to react on
531 changing requirements.
532 Software parts that are not written will not miss the requirements.
534 .NH 2
535 Worse is better
536 .XS
537 \*(SN Worse is better
538 .XE
539 .LP
540 The Unix Philosophy aims for the 90% solution;
541 others call it the ``Worse is better'' approach.
542 Experience from real life projects shows:
543 .PP
544 (1) It is almost never possible to define the
545 requirements completely and correctly the first time.
546 Hence one should not try to; one will fail anyway.
547 .PP
548 (2) Requirements change during time.
549 Hence it is best to delay requirement-based design decisions as long as possible.
550 The software should be small and flexible as long as possible
551 to react on changing requirements.
552 Shell scripts, for example, are more easily adjusted as C programs.
553 .PP
554 (3) Maintenance work is hard work.
555 Hence, one should keep the amount of code as small as possible;
556 it should just fulfill the \fIcurrent\fP requirements.
557 Software parts that will be written in future,
558 do not need maintenance till then.
559 .PP
560 See Brooks' ``The Mythical Man-Month'' for reference.
561 .[ [
562 brooks
563 mythical man-month
564 .], page 115 ff.]
565 .PP
566 Starting with a prototype in a scripting language has several advantages:
567 .IP \(bu
568 As the initial effort is low, one will likely start right away.
569 .IP \(bu
570 As working parts are available soon, the real requirements can get identified soon.
571 .IP \(bu
572 When a software is usable and valuable, it gets used, and thus tested.
573 Hence problems will be found at early stages of the development.
574 .IP \(bu
575 The prototype might be enough for the moment,
576 thus further work on the software can get delayed to a time
577 when one knows better about the requirements and problems,
578 than now.
579 .IP \(bu
580 Implementing now only the parts that are actually needed at the moment,
581 introduces fewer programming and maintenance work.
582 .IP \(bu
583 If the global situation changes so that the software is not needed anymore,
584 then less effort was spent into the project, than it would have be
585 when a different approach had been used.
587 .NH 2
588 Upgrowth and survival of software
589 .XS
590 \*(SN Upgrowth and survival of software
591 .XE
592 .LP
593 So far it was talked about \fIwriting\fP or \fIbuilding\fP software.
594 Although these are just verbs, they do imply a specific view on the work process
595 they describe.
596 The better verb, however, is to \fIgrow\fP.
597 Creating software in the sense of the Unix Philosophy is an incremental process.
598 It starts with a first prototype, which evolves as requirements change.
599 A quickly hacked shell script might become a large, sophisticated,
600 compiled program this way.
601 Its lifetime begins with the initial prototype and ends when the software is not used anymore.
602 While being alive it will get extended, rearranged, rebuilt.
603 Growing software matches the view that ``software is never finished. It is only released.''
604 .[ [
605 gancarz
606 unix philosophy
607 .], page 26]
608 .PP
609 Software can be seen as being controlled by evolutionary processes.
610 Successful software is software that is used by many for a long time.
611 This implies that the software is needed, useful, and better than alternatives.
612 Darwin talks about: ``The survival of the fittest.''
613 .[
614 darwin
615 origin of species
616 .]
617 Transferred to software: The most successful software, is the fittest,
618 is the one that survives.
619 (This may be at the level of one creature, or at the level of one species.)
620 The fitness of software is affected mainly by four properties:
621 portability of code, portability of data, range of usability, and reusability of parts.
622 .PP
623 (1)
624 .I "Portability of code
625 means, using high-level programming languages,
626 sticking to the standard,
627 .[ [
628 kernighan pike
629 practice of programming
630 .], chapter\|8]
631 and avoiding optimizations that introduce dependencies on specific hardware.
632 Hardware has a much lower lifetime than software.
633 By chaining software to a specific hardware,
634 the software's lifetime gets shortened to that of this hardware.
635 In contrast, software should be easy to port \(en
636 adaptation is the key to success.
637 .PP
638 (2)
639 .I "Portability of data
640 is best achieved by avoiding binary representations
641 to store data, because binary representations differ from machine to machine.
642 Textual representation is favored.
643 Historically, \s-1ASCII\s0 was the charset of choice.
644 For the future, \s-1UTF\s0-8 might be the better choice.
645 Important is that it is a plain text representation in a
646 very common charset encoding.
647 Apart from being able to transfer data between machines,
648 readable data has the great advantage, that humans are able to directly
649 read and edit it with text editors and other tools from the Unix toolchest.
650 .[ [
651 gancarz
652 unix philosophy
653 .], page 56 ff.]
654 .PP
655 (3)
656 A large
657 .I "range of usability
658 ensures good adaptation, and thus good survival.
659 It is a special distinction if a software becomes used in fields of action,
660 the original authors did never imagine.
661 Software that solves problems in a general way will likely be used
662 for many kinds of similar problems.
663 Being too specific limits the range of usability.
664 Requirements change through time, thus use cases change or even vanish.
665 As a good example in this point,
666 Allman identifies flexibility to be one major reason for sendmail's success:
667 .[
668 allman
669 sendmail
670 .]
671 .QP
672 Second, I limited myself to the routing function [...].
673 This was a departure from the dominant thought of the time, [...].
674 .QP
675 Third, the sendmail configuration file was flexible enough to adapt
676 to a rapidly changing world [...].
677 .LP
678 Successful software adapts itself to the changing world.
679 .PP
680 (4)
681 .I "Reuse of parts
682 is even one step further.
683 A software may completely lose its field of action,
684 but parts of which the software is build may be general and independent enough
685 to survive this death.
686 If software is build by combining small independent programs,
687 then these parts are readily available for reuse.
688 Who cares if the large program is a failure,
689 but parts of it become successful instead?
691 .NH 2
692 Summary
693 .XS
694 \*(SN Summary
695 .XE
696 .LP
697 This chapter explained central ideas of the Unix Philosophy.
698 For each of the ideas, the advantages they introduce were explained.
699 The Unix Philosophy are guidelines that help to write more valuable software.
700 From the view point of a software developer or software designer,
701 the Unix Philosophy provides answers to many software design problem.
702 .PP
703 The various ideas of the Unix Philosophy are very interweaved
704 and can hardly be applied independently.
705 However, the probably most important messages are:
706 .I "``Keep it simple!''" ,
707 .I "``Do one thing well!''" ,
708 and
709 .I "``Use software leverage!''
713 .NH 1
714 Case study: \s-1MH\s0
715 .XS
716 .sp .5v
717 .B
718 \*(SN Case study: \s-1MH\s0
719 .XE
720 .LP
721 The previous chapter introduced and explained the Unix Philosophy
722 from a general point of view.
723 The driving force were the guidelines; references to
724 existing software were given only sparsely.
725 In this and the next chapter, concrete software will be
726 the driving force in the discussion.
727 .PP
728 This first case study is about the mail user agents (\s-1MUA\s0)
729 \s-1MH\s0 (``mail handler'') and its descendent \fInmh\fP
730 (``new mail handler'').
731 .[
732 nmh website
733 .]
734 \s-1MUA\s0s provide functions to read, compose, and organize mail,
735 but (ideally) not to transfer it.
736 In this document, the name \s-1MH\s0 will be used to include nmh.
737 A distinction will only be made if differences between
738 \s-1MH\s0 and nmh are described.
741 .NH 2
742 Historical background
743 .XS
744 \*(SN Historical background
745 .XE
746 .LP
747 Electronic mail was available in Unix very early.
748 The first \s-1MUA\s0 on Unix was \f(CWmail\fP,
749 which was already present in the First Edition.
750 .[ [
751 salus
752 quarter century of unix
753 .], page 41 f.]
754 It was a small program that either printed the user's mailbox file
755 or appended text to someone elses mailbox file,
756 depending on the command line arguments.
757 .[
758 manual mail(1)
759 .]
760 It was a program that did one job well.
761 This job was emailing, which was very simple then.
762 .PP
763 Later, emailing became more powerful, and thus more complex.
764 The simple \f(CWmail\fP, which knew nothing of subjects,
765 independent handling of single messages,
766 and long-time email storage, was not powerful enough anymore.
767 In 1978 at Berkeley, Kurt Shoens wrote \fIMail\fP (with capital `M')
768 to provide additional functions for emailing.
769 Mail was still one program, but now it was large and did
770 several jobs.
771 Its user interface is modeled after the one of \fIed\fP.
772 It is designed for humans, but is still scriptable.
773 \fImailx\fP is the adaptation of Berkeley Mail into System V.
774 .[
775 ritter
776 mailx history
777 .]
778 Elm, pine, mutt, and a whole bunch of graphical \s-1MUA\s0s
779 followed Mail's direction.
780 They are large, monolithic programs which include all emailing functions.
781 .PP
782 A different way was taken by the people of \s-1RAND\s0 Corporation.
783 In the beginning, they also had used a monolithic mail system,
784 called \s-1MS\s0 (for ``mail system'').
785 But in 1977, Stockton Gaines and Norman Shapiro
786 came up with a proposal of a new email system concept \(en
787 one that honored the Unix Philosophy.
788 The concept was implemented by Bruce Borden in 1978 and 1979.
789 This was the birth of \s-1MH\s0 \(en the ``mail handler''.
790 .PP
791 Since then, \s-1RAND\s0, the University of California at Irvine and
792 at Berkeley, and several others have contributed to the software.
793 However, it's core concepts remained the same.
794 In the late 90s, when development of \s-1MH\s0 slowed down,
795 Richard Coleman started with \fInmh\fP, the new mail handler.
796 His goal was to improve \s-1MH\s0 especially in regard of
797 the requirements of modern emailing.
798 Today, nmh is developed by various people on the Internet.
799 .[
800 ware
801 rand history
802 .]
803 .[
804 peek
805 mh
806 .]
808 .NH 2
809 Contrasts to monolithic mail systems
810 .XS
811 \*(SN Contrasts to monolithic mail systems
812 .XE
813 .LP
814 All \s-1MUA\s0s are monolithic, except \s-1MH\s0.
815 Although there might actually exist further, very little known,
816 toolchest \s-1MUA\s0s, this statement reflects the situation pretty well.
817 .PP
818 Monolithic \s-1MUA\s0s gather all their functions in one program.
819 In contrast, \s-1MH\s0 is a toolchest of many small tools \(en one for each job.
820 Following is a list of important programs of \s-1MH\s0's toolchest
821 and their function.
822 It gives a feeling of how the toolchest looks like.
823 .IP \(bu
824 .CW inc :
825 incorporate new mail (this is how mail enters the system)
826 .IP \(bu
827 .CW scan :
828 list messages in folder
829 .IP \(bu
830 .CW show :
831 show message
832 .IP \(bu
833 .CW next\fR/\fPprev :
834 show next/previous message
835 .IP \(bu
836 .CW folder :
837 change current folder
838 .IP \(bu
839 .CW refile :
840 refile message into different folder
841 .IP \(bu
842 .CW rmm :
843 remove message
844 .IP \(bu
845 .CW comp :
846 compose new message
847 .IP \(bu
848 .CW repl :
849 reply to message
850 .IP \(bu
851 .CW forw :
852 forward message
853 .IP \(bu
854 .CW send :
855 send prepared message (this is how mail leaves the system)
856 .LP
857 \s-1MH\s0 has no special user interface like monolithic \s-1MUA\s0s have.
858 The user does not leave the shell to run \s-1MH\s0,
859 instead he uses the various \s-1MH\s0 programs within the shell.
860 Using a monolithic program with a captive user interface
861 means ``entering'' the program, using it, and ``exiting'' the program.
862 Using toolchests like \s-1MH\s0 means running programs,
863 alone or in combination with others, also from other toolchests,
864 without leaving the shell.
866 .NH 2
867 Data storage
868 .XS
869 \*(SN Data storage
870 .XE
871 .LP
872 \s-1MH\s0's mail storage is a directory tree under the user's
873 \s-1MH\s0 directory (usually \f(CW$HOME/Mail\fP),
874 where mail folders are directories and mail messages are text files
875 within them.
876 Each mail folder contains a file \f(CW.mh_sequences\fP which lists
877 the public message sequences of that folder,
878 for instance the \fIunseen\fP sequence for new messages.
879 Mail messages are text files located in a mail folder.
880 The files contain the messages as they were received.
881 They are named by ascending numbers in each folder.
882 .PP
883 This mailbox format is called ``\s-1MH\s0'' after the \s-1MUA\s0.
884 Alternatives are \fImbox\fP and \fImaildir\fP.
885 In the mbox format all messages are stored within one file.
886 This was a good solution in the early days, when messages
887 were only a few lines of text and were deleted soon.
888 Today, when single messages often include several megabytes
889 of attachments, it is a bad solution.
890 Another disadvantage of the mbox format is that it is
891 more difficult to write tools that work on mail messages,
892 because it is always necessary to first find and extract
893 the relevant message in the mbox file.
894 With the \s-1MH\s0 mailbox format, each message is a separate file.
895 Also, the problem of concurrent access to one mailbox is
896 reduced to the problem of concurrent access to one message.
897 The maildir format is generally similar to the \s-1MH\s0 format,
898 but modified towards guaranteed reliability.
899 This involves some complexity, unfortunately.
900 .PP
901 Working with \s-1MH\s0's toolchest on mailboxes is much like
902 working with Unix' toolchest on directory trees:
903 \f(CWscan\fP is like \f(CWls\fP,
904 \f(CWshow\fP is like \f(CWcat\fP,
905 \f(CWfolder\fP is like \f(CWcd\fP and \f(CWpwd\fP,
906 \f(CWrefile\fP is like \f(CWmv\fP,
907 and \f(CWrmm\fP is like \f(CWrm\fP.
908 .PP
909 \s-1MH\s0 extends the context of processes in Unix by two more items,
910 for its tools:
911 .IP \(bu
912 The current mail folder, which is similar to the current working directory.
913 For mail folders, \f(CWfolder\fP provides the corresponding functionality
914 of \f(CWcd\fP and \f(CWpwd\fP for directories.
915 .IP \(bu
916 Sequences, which are named sets of messages in a mail folder.
917 The current message, relative to a mail folder, is a special sequence.
918 It enables commands like \f(CWnext\fP and \f(CWprev\fP.
919 .LP
920 In contrast to Unix' context, which is maintained by the kernel,
921 \s-1MH\s0's context must be maintained by the tools themselves.
922 Usually there is one context per user, which resides in his
923 \f(CWcontext\fP file in the \s-1MH\s0 directory,
924 but a user can have several contexts, too.
925 Public sequences are an exception, as they belong to a mail folder,
926 and reside in the \f(CW.mh_sequences\fP file there.
927 .[
928 man page mh-profile mh-sequence
929 .]
931 .NH 2
932 Discussion of the design
933 .XS
934 \*(SN Discussion of the design
935 .XE
936 .LP
937 This section discusses \s-1MH\s0 in regard to the tenets
938 of the Unix Philosophy that Gancarz identified.
940 .PP
941 .B "Small is beautiful
942 and
943 .B "do one thing well
944 are two design goals that are directly visible in \s-1MH\s0.
945 Gancarz actually presents \s-1MH\s0 in his book as example under the
946 headline ``Making \s-1UNIX\s0 Do One Thing Well'':
947 .[ [
948 gancarz
949 unix philosophy
950 .], page 125 ff.]
951 .QP
952 [\s-1MH\s0] consists of a series of programs which
953 when combined give the user an enormous ability
954 to manipulate electronic mail messages.
955 A complex application, it shows that not only is it
956 possible to build large applications from smaller
957 components, but also that such designs are actually preferable.
958 .LP
959 The various programs of \s-1MH\s0 were relatively easy to write,
960 because each of them is small, limited to one function,
961 and has clear boundaries.
962 For the same reasons, they are also good to maintain.
963 Further more, the system can easily get extended.
964 One only needs to put a new program into the toolchest.
965 This was done, for instance, when \s-1MIME\s0 support was added
966 (e.g. \f(CWmhbuild\fP).
967 Also, different programs can exist to do the basically same job
968 in different ways (e.g. in nmh: \f(CWshow\fP and \f(CWmhshow\fP).
969 .PP
970 If someone needs a mail system with some additionally
971 functions that are not available anywhere yet, he best expands a
972 toolchest system like \s-1MH\s0.
973 There he can add new functionality by simply adding additional
974 programs to the toolchest.
975 There he does not risk to break existing functionality by doing so.
977 .PP
978 .B "Store data in flat text files
979 is followed by \s-1MH\s0.
980 This is not surprising, because email messages are already plain text.
981 \s-1MH\s0 stores the messages as it receives them,
982 thus any other tool that works on \s-1RFC\s0\|2822 mail messages can operate
983 on the messages in an \s-1MH\s0 mailbox.
984 All other files \s-1MH\s0 uses are plain text, too.
985 It is therefore possible and encouraged to use the text processing
986 tools of Unix' toolchest to extend \s-1MH\s0's toolchest.
988 .PP
989 .B "Avoid captive user interfaces" .
990 \s-1MH\s0 is perfectly suited for non-interactive use.
991 It offers all functions directly and without captive user interfaces.
992 If, nonetheless, users want a graphical user interface,
993 they can have it with \fIxmh\fP or \fIexmh\fP.
994 These are graphical frontends for the \s-1MH\s0 toolchest.
995 This means, all email-related work is still done by \s-1MH\s0 tools,
996 but the frontend calls the appropriate commands when the user
997 clicks on buttons.
998 .PP
999 Providing easy-to-use user interfaces in form of frontends is a good
1000 approach, because it does not limit the power of the backend itself.
1001 The frontend will anyway only be able to make a subset of the
1002 backend's power and flexibility available to the user.
1003 But if it is a separate program,
1004 then the missing parts can still be accessed at the backend directly.
1005 If it is integrated, then this will hardly be possible.
1006 An additional advantage is the possibility to have different frontends
1007 to the same backend.
1009 .PP
1010 .B "Choose portability over efficiency
1011 and
1012 .B "use shell scripts to increase leverage and portability" .
1013 These two tenets are indirectly, but nicely, demonstrated by
1014 Bolsky and Korn in their book about the Korn Shell.
1015 .[
1016 bolsky korn
1017 korn shell
1018 .]
1019 Chapter\|18 of the book shows a basic implementation
1020 of a subset of \s-1MH\s0 in ksh scripts.
1021 Of course, this is just a demonstration, but a brilliant one.
1022 It shows how quickly one can implement such a prototype with shell scripts,
1023 and how readable they are.
1024 The implementation in the scripting language may not be very fast,
1025 but it can be fast enough though, and this is all that matters.
1026 By having the code in an interpreted language, like the shell,
1027 portability becomes a minor issue, if we assume the interpreter
1028 to be widespread.
1029 .PP
1030 This demonstration also shows how easy it is to create single programs
1031 of a toolchest software.
1032 Eight tools (two of them have multiple names) and 16 functions
1033 with supporting code are presented to the reader.
1034 The tools comprise less than 40 lines of ksh each,
1035 in total about 200 lines.
1036 The functions comprise less than 80 lines of ksh each,
1037 in total about 450 lines.
1038 Such small software is easy to write, easy to understand,
1039 and thus easy to maintain.
1040 A toolchest improves the possibility to only write some parts
1041 and though create a working result.
1042 Expanding the toolchest, even without global changes,
1043 will likely be possible.
1045 .PP
1046 .B "Use software leverage to your advantage
1047 and the lesser tenet
1048 .B "allow the user to tailor the environment
1049 are ideally followed in the design of \s-1MH\s0.
1050 Tailoring the environment is heavily encouraged by the ability to
1051 directly define default options to programs.
1052 It is even possible to define different default options
1053 depending on the name under which a program is called.
1054 Software leverage is heavily encouraged by the ease of
1055 creating shell scripts that run a specific command line,
1056 built of several \s-1MH\s0 programs.
1057 There is few software that so much wants users to tailor their
1058 environment and to leverage the use of the software, like \s-1MH\s0.
1059 .PP
1060 Just to make one example:
1061 One might prefer a different listing format for the \f(CWscan\fP
1062 program.
1063 It is possible to take one of the distributed format files
1064 or to write one yourself.
1065 To use the format as default for \f(CWscan\fP, a single line,
1066 reading
1067 .DS
1068 scan: -form FORMATFILE
1069 .DE
1070 must be added to \f(CW.mh_profile\fP.
1071 If one wants this different format as an additional command,
1072 instead of changing the default, he needs to create a link to
1073 \f(CWscan\fP, for instance titled \f(CWscan2\fP.
1074 The line in \f(CW.mh_profile\fP would then start with \f(CWscan2\fP,
1075 as the option should only be in effect for a program that is called as
1076 \f(CWscan2\fP.
1078 .PP
1079 .B "Make every program a filter
1080 is hard to find in \s-1MH\s0.
1081 The reason therefore is that most of \s-1MH\s0's tools provide
1082 basic file system operations for mailboxes.
1083 It is the same reason because of which \f(CWls\fP, \f(CWcp\fP, \f(CWmv\fP,
1084 and \f(CWrm\fP aren't filters neither.
1085 \s-1MH\s0 does not provide many filters itself, but it is a basis
1086 to write filters for.
1087 An example would be a mail text highlighter,
1088 that means a program that makes use of a color terminal to display
1089 header lines, quotations, and signatures in distinct colors.
1090 The author's version of such a program is an awk script with 25 lines.
1092 .PP
1093 .B "Build a prototype as soon as possible
1094 was again well followed by \s-1MH\s0.
1095 This tenet, of course, focuses on early development, which is
1096 long time ago for \s-1MH\s0.
1097 But without following this guideline at the very beginning,
1098 Bruce Borden may have not convinced the management of \s-1RAND\s0
1099 to ever create \s-1MH\s0.
1100 In Bruce' own words:
1101 .[ [
1102 ware rand history
1103 .], page 132]
1104 .QP
1105 [...] but [Stockton Gaines and Norm Shapiro] were not able
1106 to convince anyone that such a system would be fast enough to be usable.
1107 I proposed a very short project to prove the basic concepts,
1108 and my management agreed.
1109 Looking back, I realize that I had been very lucky with my first design.
1110 Without nearly enough design work,
1111 I built a working environment and some header files
1112 with key structures and wrote the first few \s-1MH\s0 commands:
1113 inc, show/next/prev, and comp.
1114 [...]
1115 With these three, I was able to convince people that the structure was viable.
1116 This took about three weeks.
1118 .NH 2
1119 Problems
1120 .XS
1121 \*(SN Problems
1122 .XE
1123 .LP
1124 \s-1MH\s0 is not without problems.
1125 There are two main problems: one is technical, the other is about human behavior.
1126 .PP
1127 \s-1MH\s0 is old and email today is very different to email in the time
1128 when \s-1MH\s0 was designed.
1129 \s-1MH\s0 adapted to the changes pretty well, but it is limited, though.
1130 \s-1MIME\s0 support and support for different character encodings
1131 is available, but only on a moderate level.
1132 This comes from limited development resources.
1133 More active developers could quickly change this.
1134 But \s-1MH\s0 is also limited by design, which is the larger problem.
1135 \s-1IMAP\s0, for example, conflicts with \s-1MH\s0's design to a large extend.
1136 These design conflicts are not easily solvable.
1137 Possibly, they require a redesign.
1138 \s-1IMAP\s0 may be too different to the classic mail model,
1139 which \s-1MH\s0 covers, so that \s-1MH\s0 may never support it well.
1140 .PP
1141 The other kind of problem are human habits.
1142 In this world, where almost all \s-1MUA\s0s are monolithic,
1143 it is very difficult to convince people to use a toolbox style \s-1MUA\s0
1144 like \s-1MH\s0.
1145 The habits are so strong, that even people who understand the concept
1146 and advantages of \s-1MH\s0 do not like to switch,
1147 simply because \s-1MH\s0 is different.
1148 Unfortunately, the frontends to \s-1MH\s0, which could provide familiar look'n'feel,
1149 are quite outdated and thus not very appealing, compared to the modern interfaces
1150 of many monolithic \s-1MUA\s0s.
1152 .NH 2
1153 Summary \s-1MH\s0
1154 .XS
1155 \*(SN Summary \s-1MH\s0
1156 .XE
1157 .LP
1158 \s-1MH\s0 is an \s-1MUA\s0 that follows the Unix Philosophy in its design.
1159 It consists of a toolchest of small tools, each of them does one job well.
1160 The toolchest approach offers great flexibility to the user.
1161 It is possible to utilize the complete power of the Unix shell with \s-1MH\s0.
1162 This makes \s-1MH\s0 a very powerful mail system.
1163 Extending and customizing \s-1MH\s0 is easy and encouraged.
1164 .PP
1165 Apart from the user's perspective, \s-1MH\s0 is development-friendly.
1166 Its overall design follows clear rules.
1167 The single tools do only one job, thus they are easy to understand,
1168 easy to write, and good to maintain.
1169 They are all independent and do not interfere with the others.
1170 Automated testing of their function is a straight forward task.
1171 .PP
1172 It is sad, that \s-1MH\s0's differentness is its largest problem,
1173 as its differentness is also its largest advantage.
1174 Unfortunately, for most people their habits are stronger
1175 than the attraction of the clear design and the power, \s-1MH\s0 offers.
1179 .NH 1
1180 Case study: uzbl
1181 .XS
1182 .sp .5v
1183 .B
1184 \*(SN Case study: uzbl
1185 .XE
1186 .LP
1187 The last chapter took a look on the \s-1MUA\s0 \s-1MH\s0,
1188 which is an old and established software.
1189 This chapter covers uzbl, a fresh new project.
1190 Uzbl is a web browser that adheres to the Unix Philosophy.
1191 Its name comes from the \fILolspeak\fP word for ``usable'';
1192 it is pronounced identical.
1194 .NH 2
1195 Historical background
1196 .XS
1197 \*(SN Historical background
1198 .XE
1199 .LP
1200 Uzbl was started by Dieter Plaetinck in April 2009.
1201 The idea was born in a thread in the Arch Linux Forums.
1202 .[
1203 arch linux forums
1204 browser
1205 .]
1206 After some discussion about failures of well known web browsers,
1207 Plaetinck (alias Dieter@be) came up with a very sketchy proposal
1208 of how a better web browser could look like.
1209 To the question of another member, if Plaetinck would write that program,
1210 because it would sound fantastic, Plaetinck replied:
1211 ``Maybe, if I find the time ;-)''.
1212 .PP
1213 Fortunately, he found the time.
1214 One day later, the first prototype was out.
1215 One week later, uzbl had an own website.
1216 .[
1217 uzbl website
1218 .]
1219 One month after the first code showed up,
1220 a mailing list was installed to coordinate and discuss further development,
1221 and a wiki was added to store documentation and scripts that showed up on the
1222 mailing list and elsewhere.
1223 .PP
1224 In the, now, one year of uzbl's existence, it was heavily developed on various branches.
1225 Plaetinck's task became more and more to only merge the best code from the
1226 different branches into his main branch, and to apply patches.
1227 .[
1228 lwn uzbl
1229 .]
1230 About once a month, Plaetinck released a new version.
1231 In September 2009, he presented several forks of uzbl.
1232 .[ [
1233 uzbl website
1234 .], news archive]
1235 Uzbl, actually, opened the field for a whole family of web browsers with similar shape.
1236 .PP
1237 In July 2009, \fILinux Weekly News\fP published an interview with Plaetinck about uzbl.
1238 .[
1239 lwn uzbl
1240 .]
1241 In September 2009, the uzbl web browser was on \fISlashdot\fP.
1242 .[
1243 slashdot uzbl
1244 .]
1246 .NH 2
1247 Contrasts to other web browsers
1248 .XS
1249 \*(SN Contrasts to other web browsers
1250 .XE
1251 .LP
1252 Like most \s-1MUA\s0s are monolithic, but \s-1MH\s0 is a toolchest,
1253 most web browsers are monolithic, but uzbl is a frontend to a toolchest.
1254 .PP
1255 Today, uzbl is divided into uzbl-core and uzbl-browser.
1256 Uzbl-core is, how its name already indicates, the core of uzbl.
1257 It handles commands and events to interface other programs,
1258 and also displays webpages by using \fIwebkit\fP as render engine.
1259 Uzbl-browser combines uzbl-core with a bunch of handler scripts, a status bar,
1260 an event manager, yanking, pasting, page searching, zooming, and more stuff,
1261 to form a ``complete'' web browser.
1262 In the following text, the term ``uzbl'' usually stands for uzbl-browser,
1263 so uzbl-core is included.
1264 .PP
1265 Unlike most other web browsers, uzbl is mainly the mediator between the
1266 various tools that cover single jobs.
1267 Therefore, uzbl listens for commands on a named pipe (fifo), a Unix socket,
1268 and on stdin, and it writes events to a Unix socket and to stdout.
1269 Loading a webpage in a running uzbl instance requires only:
1270 .DS
1271 echo 'uri http://example.org' >/path/to/uzbl-fifo
1272 .DE
1273 The graphical rendering of the webpage is done by webkit,
1274 a web content engine.
1275 Uzbl-core is built around libwebkit.
1276 .PP
1277 Downloads, browsing history, bookmarks, and the like are not provided
1278 by the core itself, like they are in other web browsers.
1279 Uzbl-browser also only provides, so called, handler scripts that wrap
1280 external applications which provide the actual functionality.
1281 For instance, \fIwget\fP is used to download files and uzbl-browser
1282 includes a script that calls wget with appropriate options in
1283 a prepared environment.
1284 .PP
1285 Modern web browsers are proud to have addons, plugins, and modules, instead.
1286 This is their effort to achieve similar goals.
1287 But instead of using existing, external programs, modern web browsers
1288 include these functions.
1290 .NH 2
1291 Discussion of the design
1292 .XS
1293 \*(SN Discussion of the design
1294 .XE
1295 .LP
1296 This section discusses uzbl in regard of the Unix Philosophy,
1297 as identified by Gancarz.
1299 .PP
1300 .B "Make each program do one thing well" .
1301 Uzbl tries to be a web browser and nothing else.
1302 The common definition of a web browser is, of course, highly influenced by
1303 existing implementations of web browsers, although they are degenerated.
1304 Web browsers should be programs to browse the web, and nothing more.
1305 This is the one thing they should do.
1306 .PP
1307 Web browsers should not, for instance, manage downloads.
1308 This is the job download managers exist for.
1309 Download managers do primary care about being good in downloading files.
1310 Modern web browsers provide download management only as a secondary feature.
1311 How could they do this job better, than programs that exist only for
1312 this very job?
1313 And how could anyone want less than the best download manager available?
1314 .PP
1315 A web browser's job is to let the user browse the web.
1316 This means, navigating through websites by following links.
1317 Rendering the \s-1HTML\s0 sources is a different job, too.
1318 It is covered by the webkit render engine, in uzbl's case.
1319 Audio and video content and files like PostScript, \s-1PDF\s0, and the like,
1320 are also not the job of a web browser.
1321 They should be handled by external applications \(en
1322 ones which's job is to handle such data.
1323 Uzbl strives to do it this way.
1324 .PP
1325 Remember Doug McIlroy:
1326 .I
1327 ``Write programs that do one thing and do it well.
1328 Write programs to work together.''
1329 .R
1330 .PP
1331 The lesser tenet
1332 .B "allow the user to tailor the environment
1333 matches good here.
1334 There was the question, how anyone could want anything less than the
1335 best program for the job.
1336 But as personal preferences matter, it is probably more important to ask:
1337 How could anyone want something else than his preferred program for the job?
1338 .PP
1339 Usually users want one program for a specific job.
1340 Hence, whenever the task is, for instance, downloading,
1341 the same download manager should be used.
1342 More advanced users might want to have this download manager in this
1343 situation and that one in that situation.
1344 They should be able to configure it this way.
1345 With uzbl, one can use any download manager the user wants.
1346 To switch to a different one, only one line in a small handler script
1347 needs to be changed.
1348 Alternatively it would be possible to query the program to use by
1349 reading a global file or an environment variable, in the handler script.
1350 .PP
1351 Uzbl does neither have its own download manager nor depends on a
1352 specific one, hence uzbl's browsing abilities will not be lowered by having
1353 a bad download manager.
1354 Uzbl's download capabilities will be just as good as the ones of the best
1355 download manager available on the system.
1356 Of course, this applies to all of the other supplementary tools, too.
1358 .PP
1359 .B "Use software leverage to your advantage" .
1360 Uzbl is designed to be extended by external tools.
1361 These external tools are usually wrapped by small handler shell scripts.
1362 Shell scripts are the glue in this approach.
1363 They make the various parts fit together.
1364 .PP
1365 The history mechanism of uzbl shall be presented as an example.
1366 Uzbl is configured to spawn a script to append an entry to the history
1367 whenever the event of a fully loaded page occurs.
1368 The script to append the entry to the history is not much more than:
1369 .DS
1370 #!/bin/sh
1371 file=/path/to/uzbl-history
1372 echo `date +'%Y-%m-%d %H:%M:%S'`" $6 $7" >> $file
1373 .DE
1374 \f(CW$6\fP and \f(CW$7\fP expand to the \s-1URL\s0 and the page title.
1375 .PP
1376 For loading an entry, a key is bound to spawn a load-from-history script.
1377 The script reverses the history to have newer entries first,
1378 then displays \fIdmenu\fP to let the user select an item,
1379 and afterwards writes the selected \s-1URL\s0 into uzbl's command input pipe.
1380 With error checking and corner case handling removed,
1381 the script looks like this:
1382 .DS
1383 #!/bin/sh
1384 file=/path/to/uzbl-history
1385 goto=`tac $file | dmenu | cut -d' ' -f 3`
1386 echo "uri $goto" > $4
1387 .DE
1388 \f(CW$4\fP expands to the path of the command input pipe of the current
1389 uzbl instance.
1391 .PP
1392 .B "Avoid captive user interfaces" .
1393 One could say, that uzbl, to a large extend, actually \fIis\fP
1394 a captive user interface.
1395 But the difference to most other web browsers is, that uzbl is only
1396 the captive user interface frontend (and the core of the backend).
1397 Many parts of the backend are independent of uzbl.
1398 Some are distributed with uzbl, for some external programs,
1399 handler scripts are distributed,
1400 but arbitrary additional functionality can be added if desired.
1401 .PP
1402 The frontend is captive \(en that is true.
1403 This is okay for the task of browsing the web, as this task is only relevant
1404 for humans.
1405 Automated programs would \fIcrawl\fP the web.
1406 That means, they read the source directly.
1407 The source includes all the semantics.
1408 The graphical representation is just for humans to transfer the semantics
1409 more intuitively.
1411 .PP
1412 .B "Make every program a filter" .
1413 Graphical web browsers are almost dead ends in the chain of information flow.
1414 Thus it is difficult to see what graphical web browsers should filter.
1415 Graphical web browsers exist almost only to be interactively used by humans.
1416 The only case when one might want to automate the rendering function is
1417 to generate images of rendered webpages.
1419 .PP
1420 .B "Small is beautiful"
1421 is not easy to apply to a web browser, because modern web technology
1422 is very complex, hence the rendering task is very complex.
1423 Modern web browsers have to consist of many thousand lines of code,
1424 unfortunately.
1425 Using the toolchest approach and wrappers can split the browser into
1426 several small parts, tough.
1427 .PP
1428 As of March 2010, uzbl-core consists of about 3\,500 lines of C code.
1429 The distribution includes another 3\,500 lines of Shell and Python code,
1430 which are the handler scripts and plugins like a modal interface.
1431 Further more, uzbl uses functionality of external tools like
1432 \fIwget\fP and \fInetcat\fP.
1433 Up to this point, uzbl looks pretty neat and small.
1434 The ugly part of uzbl is the web content renderer, webkit.
1435 Webkit consists of roughly 400\,000 (!) lines of code.
1436 Unfortunately, small web render engines are not possible anymore
1437 because of the modern web.
1439 .PP
1440 .B "Build a prototype as soon as possible" .
1441 Plaetinck made his code public, right from the beginning.
1442 Discussion and development was, and still is, open to everyone interested.
1443 Development versions of uzbl can be obtained very simply from the code
1444 repository.
1445 Within the first year of uzbl's existence, a new version was released
1446 more often than once a month.
1447 Different forks and branches arose.
1448 They introduced new features, which were tested for suitability
1449 for the main branch.
1450 The experiences of using prototypes influenced further development.
1451 Actually, all development was community driven.
1452 Plaetinck says, three months after uzbl's birth:
1453 ``Right now I hardly code anything myself for Uzbl.
1454 I just merge in other people's code, ponder a lot, and lead the discussions.''
1455 .[
1456 lwn
1457 uzbl
1458 .]
1461 .NH 2
1462 Problems
1463 .XS
1464 \*(SN Problems
1465 .XE
1466 .LP
1467 Similar to \s-1MH\s0, uzbl, too suffers from being different.
1468 It is sad, but people use what they know.
1469 Fortunately, uzbl's user interface can look and feel very much the
1470 same as the one of the well known web browsers,
1471 hiding the internal differences.
1472 But uzbl has to provide this similar look and feel to be accepted
1473 as a ``normal'' browser by ``normal'' users.
1474 .PP
1475 Though, the more important problem is the modern web.
1476 The modern web is simply broken.
1477 It has state in a state-less protocol,
1478 it misuses technologies,
1479 and it is helplessly overloaded.
1480 The result are web content render engines that must consist
1481 of hundreds of thousands lines of code.
1482 They also must combine and integrate many different technologies,
1483 only to make our modern web accessible.
1484 Website to image converter are hardly possible to run without
1485 human interaction because of state in sessions, impossible
1486 deep-linking, and unautomatable technologies.
1487 .PP
1488 The web was misused to provide all kinds of imaginable wishes.
1489 Now web browsers, and eventually the users, suffer from it.
1492 .NH 2
1493 Summary uzbl
1494 .XS
1495 \*(SN Summary uzbl
1496 .XE
1497 .LP
1498 ``Uzbl is a browser that adheres to the Unix Philosophy'',
1499 that is how uzbl is seen by its authors.
1500 Indeed, uzbl follows the Unix Philosophy in many ways.
1501 It consists of independent parts that work together,
1502 while its core is mainly a mediator which glues the parts together.
1503 .PP
1504 Software leverage can excellently be seen in uzbl.
1505 External tools are used, independent tasks are separated
1506 in independent parts and glued together with small handler scripts.
1507 .PP
1508 As uzbl, more or less, consists of a set of tools and a bit
1509 of glue, anyone can put the parts together and expand it
1510 in any desired way.
1511 Uzbl is very flexible and customizable.
1512 These properties make it valuable for advanced users,
1513 but may keep novice users from using it.
1514 .PP
1515 But uzbl's main problem is the modern web, that makes it hard
1516 to design a sane web browser.
1517 Despite this bad situation, uzbl does a fairly good job.
1520 .NH 1
1521 Final thoughts
1522 .XS
1523 .sp .5v
1524 .B
1525 \*(SN Final thoughts
1526 .XE
1528 .NH 2
1529 Quick summary
1530 .XS
1531 \*(SN Quick summary
1532 .XE
1533 .LP
1534 good design
1535 .LP
1536 unix phil
1537 .LP
1538 case studies
1540 .NH 2
1541 Why people should choose
1542 .XS
1543 \*(SN Why people should choose
1544 .XE
1545 .LP
1546 Make the right choice!
1549 .bp
1550 .TL
1551 References
1552 .LP
1553 .XS
1554 .sp .5v
1555 .B
1556 References
1557 .XE
1558 .ev r
1559 .rm ]<
1560 .de ]<
1561 .LP
1562 .de FP
1563 .IP \\\\$1.
1564 \\..
1565 .rm FS FE
1566 ..
1567 .nr PS -1
1568 .nr VS -1
1569 .[
1570 $LIST$
1571 .]
1572 .nr PS +1
1573 .nr VS +1
1574 .ev
1576 .bp
1577 .TL
1578 Table of Contents
1579 .LP
1580 .PX no