docs/unix-phil
view unix-phil.ms @ 12:018d5f9a2993
wrote about upgrowth and survival (ch03)
| author | meillo@marmaro.de |
|---|---|
| date | Sat, 20 Feb 2010 23:54:43 +0100 |
| parents | b32a43a62cae |
| children | c665b9e7bf8a |
line source
1 .\".if n .pl 1000i
2 .de XX
3 .pl 1v
4 ..
5 .em XX
6 .\".nr PI 0
7 .\".if t .nr PD .5v
8 .\".if n .nr PD 1v
9 .nr lu 0
10 .de CW
11 .nr PQ \\n(.f
12 .if t .ft CW
13 .ie \\$1 .if n .ul 999
14 .el .if n .ul 1
15 .if t .if !\\$1 \&\\$1\f\\n(PQ\\$2
16 .if n .if \\n(.$=1 \&\\$1
17 .if n .if \\n(.$>1 \&\\$1\c
18 .if n .if \\n(.$>1 \&\\$2
19 ..
20 .ds [. \ [
21 .ds .] ]
22 .\"----------------------------------------
23 .TL
24 Why the Unix Philosophy still matters
25 .AU
26 markus schnalke <meillo@marmaro.de>
27 .AB
28 .ti \n(.iu
29 This paper discusses the importance of the Unix Philosophy in software design.
30 Today, few software designers are aware of these concepts,
31 and thus most modern software is limited and does not make use of software leverage.
32 Knowing and following the tenets of the Unix Philosophy makes software more valuable.
33 .AE
35 .\".if t .2C
37 .FS
38 .ps -1
39 This paper was prepared for the seminar ``Software Analysis'' at University Ulm.
40 Mentor was professor Schweiggert. 2010-02-05
41 .br
42 You may get this document from my website
43 .CW \s-1http://marmaro.de/docs
44 .FE
46 .NH 1
47 Introduction
48 .LP
49 Building a software is a process from an idea of the purpose of the software
50 to its release.
51 No matter \fIhow\fP the process is run, two things are common:
52 the initial idea and the release.
53 The process in between can be of any shape.
54 The the maintenance work after the release is ignored for the moment.
55 .PP
56 The process of building splits mainly in two parts:
57 the planning of what and how to build, and implementing the plan by writing code.
58 This paper focuses on the planning part \(en the designing of the software.
59 .PP
60 Software design is the plan of how the internals and externals of the software should look like,
61 based on the requirements.
62 This paper discusses the recommendations of the Unix Philosophy about software design.
63 .PP
64 The here discussed ideas can get applied by any development process.
65 The Unix Philosophy does recommend how the software development process should look like,
66 but this shall not be of matter here.
67 Similar, the question of how to write the code is out of focus.
68 .PP
69 The name ``Unix Philosophy'' was already mentioned several times, but it was not explained yet.
70 The Unix Philosophy is the essence of how the Unix operating system and its toolchest was designed.
71 It is no limited set of rules, but what people see to be common to typical Unix software.
72 Several people stated their view on the Unix Philosophy.
73 Best known are:
74 .IP \(bu
75 Doug McIlroy's summary: ``Write programs that do one thing and do it well.''
76 .[
77 %A M. D. McIlroy
78 %A E. N. Pinson
79 %A B. A. Taque
80 %T UNIX Time-Sharing System Forward
81 %J The Bell System Technical Journal
82 %D 1978
83 %V 57
84 %N 6
85 %P 1902
86 .]
87 .IP \(bu
88 Mike Gancarz' book ``The UNIX Philosophy''.
89 .[
90 %A Mike Gancarz
91 %T The UNIX Philosophy
92 %D 1995
93 %I Digital Press
94 .]
95 .IP \(bu
96 Eric S. Raymond's book ``The Art of UNIX Programming''.
97 .[
98 %A Eric S. Raymond
99 %T The Art of UNIX Programming
100 %D 2003
101 %I Addison-Wesley
102 %O .CW \s-1http://www.faqs.org/docs/artu/
103 .]
104 .LP
105 These different views on the Unix Philosophy have much in common.
106 Especially, the main concepts are similar for all of them.
107 But there are also points on which they differ.
108 This only underlines what the Unix Philosophy is:
109 A retrospective view on the main concepts of Unix software;
110 especially those that were successful and unique to Unix.
111 .\" really?
112 .PP
113 Before we will have a look at concrete concepts,
114 we discuss why software design is important
115 and what problems bad design introduces.
118 .NH 1
119 Importance of software design in general
120 .LP
121 Why should we design software at all?
122 It is general knowledge, that even a bad plan is better than no plan.
123 Ignoring software design is programming without a plan.
124 This will lead pretty sure to horrible results.
125 .PP
126 The design of a software is its internal and external shape.
127 The design talked about here has nothing to do with visual appearance.
128 If we see a program as a car, then its color is of no matter.
129 Its design would be the car's size, its shape, the number and position of doors,
130 the ratio of passenger and cargo transport, and so forth.
131 .PP
132 A software's design is about quality properties.
133 Each of the cars may be able to drive from A to B,
134 but it depends on its properties whether it is a good car for passenger transport or not.
135 It also depends on its properties if it is a good choice for a rough mountain area.
136 .PP
137 Requirements to a software are twofold: functional and non-functional.
138 Functional requirements are easier to define and to verify.
139 They are directly the software's functions.
140 Functional requirements are the reason why software gets written.
141 Someone has a problem and needs a tool to solve it.
142 Being able to solve the problem is the main functional requirement.
143 It is the driving force behind all programming effort.
144 .PP
145 On the other hand, there are also non-functional requirements.
146 They are called \fIquality\fP requirements, too.
147 The quality of a software is about properties that are not directly related to
148 the software's basic functions.
149 Quality aspects are about the properties that are overlooked at first sight.
150 .PP
151 Quality is of few matter when the software gets initially built,
152 but it will be of matter in usage and maintenance of the software.
153 A short-sighted might see in developing a software mainly building something up.
154 Reality shows, that building the software the first time is only a small amount
155 of the overall work.
156 Bug fixing, extending, rebuilding of parts \(en short: maintenance work \(en
157 does soon take over the major part of the time spent on a software.
158 Not to forget the usage of the software.
159 These processes are highly influenced by the software's quality.
160 Thus, quality should never be neglected.
161 The problem is that you hardly ``stumble over'' bad quality during the first build,
162 but this is the time when you should care about good quality most.
163 .PP
164 Software design is not about the basic function of a software;
165 this requirement will get satisfied anyway, as it is the main driving force behind the development.
166 Software design is about quality aspects of the software.
167 Good design will lead to good quality, bad design to bad quality.
168 The primary functions of the software will be affected modestly by bad quality,
169 but good quality can provide a lot of additional gain from the software,
170 even at places where one never expected it.
171 .PP
172 The ISO/IEC 9126-1 standard, part 1,
173 .[
174 %I International Organization for Standardization
175 %T ISO Standard 9126: Software Engineering \(en Product Quality, part 1
176 %C Geneve
177 %D 2001
178 .]
179 defines the quality model as consisting out of:
180 .IP \(bu
181 .I Functionality
182 (suitability, accuracy, inter\%operability, security)
183 .IP \(bu
184 .I Reliability
185 (maturity, fault tolerance, recoverability)
186 .IP \(bu
187 .I Usability
188 (understandability, learnability, operability, attractiveness)
189 .IP \(bu
190 .I Efficiency
191 (time behavior, resource utilization)
192 .IP \(bu
193 .I Maintainability
194 (analysability, changeability, stability, testability)
195 .IP \(bu
196 .I Portability
197 (adaptability, installability, co-existence, replaceability)
198 .LP
199 These goals are parts of a software's design.
200 Good design can give these properties to a software,
201 bad designed software will miss them.
202 .PP
203 One further goal of software design is consistency.
204 Consistency eases understanding, working on, and using things.
205 Consistent internals and consistent interfaces to the outside can be provided by good design.
206 .PP
207 We should design software because good design avoids many problems during a software's lifetime.
208 And we should design software because good design can offer much gain,
209 that can be unrelated to the software main intend.
210 Indeed, we should spend much effort into good design to make the software more valuable.
211 The Unix Philosophy shows how to design software well.
212 It offers guidelines to achieve good quality and high gain for the effort spent.
215 .NH 1
216 The Unix Philosophy
217 .LP
218 The origins of the Unix Philosophy were already introduced.
219 This chapter explains the philosophy, oriented on Gancarz,
220 and shows concrete examples of its application.
222 .SH
223 Examples
224 .LP
225 Following are some examples to demonstrate how applied Unix Philosophy feels like.
226 Knowledge of using the Unix shell is assumed.
227 .PP
228 Counting the number of files in the current directory:
229 .DS I 2n
230 .CW
231 .ps -1
232 ls | wc -l
233 .DE
234 The
235 .CW ls
236 command lists all files in the current directory, one per line,
237 and
238 .CW "wc -l
239 counts the number of lines.
240 .PP
241 Counting the number of files that do not contain ``foo'' in their name:
242 .DS I 2n
243 .CW
244 .ps -1
245 ls | grep -v foo | wc -l
246 .DE
247 Here, the list of files is filtered by
248 .CW grep
249 to remove all that contain ``foo''.
250 The rest is the same as in the previous example.
251 .PP
252 Finding the five largest entries in the current directory.
253 .DS I 2n
254 .CW
255 .ps -1
256 du -s * | sort -nr | sed 5q
257 .DE
258 .CW "du -s *
259 returns the recursively summed sizes of all files
260 \(en no matter if they are regular files or directories.
261 .CW "sort -nr
262 sorts the list numerically in reverse order.
263 Finally,
264 .CW "sed 5q
265 quits after it has printed the fifth line.
266 .PP
267 The presented command lines are examples of what Unix people would use
268 to get the desired output.
269 There are also other ways to get the same output.
270 It's a user's decision which way to go.
272 .SH
273 Pipes
274 .LP
275 The examples show that many tasks on a Unix system
276 are accomplished by combining several small programs.
277 The connection between the single programs is denoted by the pipe operator `|'.
278 .PP
279 Pipes, and their extensive and easy use, are one of the great
280 achievements of the Unix system.
281 Pipes between programs have been possible in earlier operating systems,
282 but it has never been a so central part of the concept.
283 When, in the early seventies, Doug McIlroy introduced pipes for the
284 Unix system,
285 ``it was this concept and notation for linking several programs together
286 that transformed Unix from a basic file-sharing system to an entirely new way of computing.''
287 .[
288 %T Unix: An Oral History
289 %O .CW \s-1http://www.princeton.edu/~hos/frs122/unixhist/finalhis.htm
290 .]
291 .PP
292 Being able to specify pipelines in an easy way is,
293 however, not enough by itself.
294 It is only one half.
295 The other is the design of the programs that are used in the pipeline.
296 They have to interfaces that allows them to be used in such a way.
298 .SH
299 Interface architecture
300 .LP
301 Unix is, first of all, simple \(en Everything is a file.
302 Files are sequences of bytes, without any special structure.
303 Programs should be filters, which read a stream of bytes from ``standard input'' (stdin)
304 and write a stream of bytes to ``standard output'' (stdout).
305 .PP
306 If the files \fIare\fP sequences of bytes,
307 and the programs \fIare\fP filters on byte streams,
308 then there is exactly one standardized data interface.
309 Thus it is possible to combine them in any desired way.
310 .PP
311 Even a handful of small programs will yield a large set of combinations,
312 and thus a large set of different functions.
313 This is leverage!
314 If the programs are orthogonal to each other \(en the best case \(en
315 then the set of different functions is greatest.
316 .PP
317 Programs might also have a separate control interface,
318 besides their data interface.
319 The control interface is often called ``user interface'',
320 because it is usually designed to be used by humans.
321 The Unix Philosophy discourages to assume the user to be human.
322 Interactive use of software is slow use of software,
323 because the program waits for user input most of the time.
324 Interactive software requires the user to be in front of the computer
325 all the time.
326 Interactive software occupy the user's attention while they are running.
327 .PP
328 Now we come back to the idea of using several small programs, combined,
329 to have a more specific function.
330 If these single tools would all be interactive,
331 how would the user control them?
332 It is not only a problem to control several programs at once if they run at the same time,
333 it also very inefficient to have to control each of the single programs
334 that are intended to work as one large program.
335 Hence, the Unix Philosophy discourages programs to demand interactive use.
336 The behavior of programs should be defined at invocation.
337 This is done by specifying arguments (``command line switches'') to the program call.
338 Gancarz discusses this topic as ``avoid captive user interfaces''.
339 .[
340 %A Mike Gancarz
341 %T The UNIX Philosophy
342 %I Digital Press
343 %D 1995
344 %P 88 ff.
345 .]
346 .PP
347 Non-interactive use is, during development, also an advantage for testing.
348 Testing of interactive programs is much more complicated,
349 than testing of non-interactive programs.
351 .SH
352 The toolchest approach
353 .LP
354 A toolchest is a set of tools.
355 Instead of having one big tool for all tasks, one has many small tools,
356 each for one task.
357 Difficult tasks are solved by combining several of the small, simple tools.
358 .PP
359 The Unix toolchest \fIis\fP a set of small, (mostly) non-interactive programs
360 that are filters on byte streams.
361 They are, to a large extend, unrelated in their function.
362 Hence, the Unix toolchest provides a large set of functions
363 that can be accessed by combining the programs in the desired way.
364 .PP
365 There are also advantages for developing small toolchest programs.
366 It is easier and less error-prone to write small programs.
367 It is also easier and less error-prone to write a large set of small programs,
368 than to write one large program with all the functionality included.
369 If the small programs are combinable, then they offer even a larger set
370 of functions than the single large program.
371 Hence, one gets two advantages out of writing small, combinable programs.
372 .PP
373 There are two drawbacks of the toolchest approach.
374 First, one simple, standardized, unidirectional interface has to be sufficient.
375 If one feels the need for more ``logic'' than a stream of bytes,
376 then a different approach might be of need.
377 But it is also possible, that he just can not imaging a design where
378 a stream of bytes is sufficient.
379 By becoming more familiar with the ``Unix style of thinking'',
380 developers will more often and easier find simple designs where
381 a stream of bytes is a sufficient interface.
382 .PP
383 The second drawback of a toolchest affects the users.
384 A toolchest is often more difficult to use for novices.
385 It is necessary to become familiar with each of the tools,
386 to be able to use the right one in a given situation.
387 Additionally, one needs to combine the tools in a senseful way on its own.
388 This is like a sharp knife \(en it is a powerful tool in the hand of a master,
389 but of no good value in the hand of an unskilled.
390 .PP
391 However, learning single, small tool of the toolchest is easier than
392 learning a complex tool.
393 The user will have a basic understanding of a yet unknown tool,
394 if the several tools of the toolchest have a common style.
395 He will be able to transfer knowledge over one tool to another.
396 .PP
397 Moreover, the second drawback can be removed easily by adding wrappers
398 around the single tools.
399 Novice users do not need to learn several tools if a professional wraps
400 the single commands into a more high-level script.
401 Note that the wrapper script still calls the small tools;
402 the wrapper script is just like a skin around.
403 No complexity is added this way,
404 but new programs can get created out of existing one with very low effort.
405 .PP
406 A wrapper script for finding the five largest entries in the current directory
407 could look like this:
408 .DS I 2n
409 .CW
410 .ps -1
411 #!/bin/sh
412 du -s * | sort -nr | sed 5q
413 .DE
414 The script itself is just a text file that calls the command line
415 a professional user would type in directly.
416 Making the program flexible on the number of entries it prints,
417 is easily possible:
418 .DS I 2n
419 .CW
420 .ps -1
421 #!/bin/sh
422 num=5
423 [ $# -eq 1 ] && num="$1"
424 du -sh * | sort -nr | sed "${num}q"
425 .DE
426 This script acts like the one before, when called without an argument.
427 But one can also specify a numerical argument to define the number of lines to print.
429 .SH
430 A powerful shell
431 .LP
432 It was already said, that the Unix shell provides the possibility to
433 combine small programs into large ones easily.
434 A powerful shell is a great feature in other ways, too.
435 .PP
436 For instance by including a scripting language.
437 The control statements are build into the shell.
438 The functions, however, are the normal programs, everyone can use on the system.
439 Thus, the programs are known, so learning to program in the shell is easy.
440 Using normal programs as functions in the shell programming language
441 is only possible because they are small and combinable tools in a toolchest style.
442 .PP
443 The Unix shell encourages to write small scripts out of other programs,
444 because it is so easy to do.
445 This is a great step towards automation.
446 It is wonderful if the effort to automate a task equals the effort
447 it takes to do it the second time by hand.
448 If it is so, then the user will be happy to automate everything he does more than once.
449 .PP
450 Small programs that do one job well, standardized interfaces between them,
451 a mechanism to combine parts to larger parts, and an easy way to automate tasks,
452 this will inevitably produce software leverage.
453 Getting multiple times the benefit of an investment is a great offer.
454 .PP
455 The shell also encourages rapid prototyping.
456 Many well known programs started as quickly hacked shell scripts,
457 and turned into ``real'' programs, written in C, later.
458 Building a prototype first is a way to avoid the biggest problems
459 in application development.
460 Fred Brooks writes in ``No Silver Bullet'':
461 .[
462 %A Frederick P. Brooks, Jr.
463 %T No Silver Bullet: Essence and Accidents of Software Engineering
464 %B Information Processing 1986, the Proceedings of the IFIP Tenth World Computing Conference
465 %E H.-J. Kugler
466 %D 1986
467 %P 1069\(en1076
468 %I Elsevier Science B.V.
469 %C Amsterdam, The Netherlands
470 .]
471 .QP
472 The hardest single part of building a software system is deciding precisely what to build.
473 No other part of the conceptual work is so difficult as establishing the detailed
474 technical requirements, [...].
475 No other part of the work so cripples the resulting system if done wrong.
476 No other part is more difficult to rectify later.
477 .PP
478 Writing a prototype is a great method to become familiar with the requirements
479 and to actually run into real problems.
480 Today, prototyping is often seen as a first step in building a software.
481 This is, of course, good.
482 However, the Unix Philosophy has an \fIadditional\fP perspective on prototyping:
483 After having built the prototype, one might notice, that the prototype is already
484 \fIgood enough\fP.
485 Hence, no reimplementation, in a more sophisticated programming language, might be of need,
486 for the moment.
487 Maybe later, it might be neccessary to rewrite the software, but not now.
488 .PP
489 By delaying further work, one keeps the flexibility to react easily on
490 changing requirements.
491 Software parts that are not written will not miss the requirements.
493 .SH
494 Worse is better
495 .LP
496 The Unix Philosophy aims for the 80% solution;
497 others call it the ``Worse is better'' approach.
498 .PP
499 First, practical experience shows, that it is almost never possible to define the
500 requirements completely and correctly the first time.
501 Hence one should not try to; it will fail anyway.
502 Second, practical experience shows, that requirements change during time.
503 Hence it is best to delay requirement-based design decisions as long as possible.
504 Also, the software should be small and flexible as long as possible
505 to react on changing requirements.
506 Shell scripts, for example, are more easily adjusted as C programs.
507 Third, practical experience shows, that maintenance is hard work.
508 Hence, one should keep the amount of software as small as possible;
509 it should just fulfill the \fIcurrent\fP requirements.
510 Software parts that will be written later, do not need maintenance now.
511 .PP
512 Starting with a prototype in a scripting language has several advantages:
513 .IP \(bu
514 As the initial effort is low, one will likely start right away.
515 .IP \(bu
516 As working parts are available soon, the real requirements can get identified soon.
517 .IP \(bu
518 When a software is usable, it gets used, and thus tested.
519 Hence problems will be found at early stages of the development.
520 .IP \(bu
521 The prototype might be enough for the moment,
522 thus further work on the software can be delayed to a time
523 when one knows better about the requirements and problems,
524 than now.
525 .IP \(bu
526 Implementing now only the parts that are actually needed now,
527 requires fewer maintenance work.
528 .IP \(bu
529 If the global situation changes so that the software is not needed anymore,
530 then less effort was spent into the project, than it would have be
531 when a different approach had been used.
533 .SH
534 Upgrowth and survival of software
535 .LP
536 So far it was talked about \fIwriting\fP or \fIbuilding\fP software.
537 Although it is just a verb, it does imply a specific view on the work process
538 it describes.
539 The better verb, however, is to \fIgrow\fP.
540 .PP
541 Creating software in the sense of the Unix Philosophy is an incremental process.
542 It starts with a first prototype, which evolves as requirements change.
543 A quickly hacked shell script might become a large, sophisticated,
544 compiles program this way.
545 During its lifetime, it will get extended, rearranged, rebuilt (from scratch).
546 Growing software matches the view that ``software is never finished. It is only released.''
547 .[
548 FIXME
549 gancarz p.26
550 .]
551 In this view, building a software is an evolutionary process;
552 it begins with the initial prototype and ends when the software is not used anymore.
553 .PP
554 Successful software is used by many for a long time.
555 This implies that the software is needed, useful, and better than alternatives.
556 Darwin talks about: ``The survival of the fittest.''
557 .[
558 FIXME
559 .]
560 Transferred to software: The most successful software, is the fittest,
561 is the one that survives.
562 The fitness is affected mainly by three properties:
563 .IP \(bu
564 portability of code
565 .IP \(bu
566 portability of data
567 .IP \(bu
568 range of usability
569 .IP \(bu
570 reuseability of parts
571 .LP
572 foo
574 .PP
575 text files
576 .PP
577 reuse of independent parts
585 .NH 2
586 Results
587 .LP
588 The unix phil is an answer to the sw design question
589 .LP
590 tool chains empower the uses of sw
594 .NH 1
595 Case study: nmh
597 .NH 2
598 History
599 .LP
600 MH, nmh.
601 They are old.
603 .NH 2
604 Contrasts to similar sw
605 .LP
606 vs. Thunderbird, mutt, mailx, pine
607 .LP
608 flexibility, no redundancy, use the shell
610 .NH 2
611 Gains of the design
612 .LP
614 .NH 2
615 Problems
616 .LP
620 .NH 1
621 Case study: uzbl
623 .NH 2
624 History
625 .LP
626 uzbl is young
628 .NH 2
629 Contrasts to similar sw
630 .LP
631 like with nmh
632 .LP
633 addons, plugins, modules
635 .NH 2
636 Gains of the design
637 .LP
639 .NH 2
640 Problems
641 .LP
642 broken web
646 .NH 1
647 Final thoughts
649 .NH 2
650 Quick summary
651 .LP
652 good design
653 .LP
654 unix phil
655 .LP
656 case studies
658 .NH 2
659 Why people should choose
660 .LP
661 Make the right choice!
663 .nr PI .5i
664 .rm ]<
665 .de ]<
666 .LP
667 .de FP
668 .IP \\\\$1.
669 \\..
670 .rm FS FE
671 ..
672 .SH
673 References
674 .[
675 $LIST$
676 .]
677 .wh -1p