This report is an html draft of HaT.TR.96.01, a paper being submitted for external publication in the IBM Systems Journal. Once published, copyright transfers to that journal. Authorization for copies should be obtained there. This early version is being made available for professional peer communication. Since this file was auto-converted and touched up, some html marks may be bad. Please let me know if your browser objects.
The notions of the changed environment are that people are different, people are non-linear, people make mistakes, and people work in certain ways better than others. Studying how people work makes sense if you care about your project; planning to recover from mistakes make sense if you care about quality; increasing technology does not always increase effectiveness; and, the number of ways of creating software is increasing with the number of people creating it. People still need to communicate, and the distances and amounts are larger.
Technology has improved productivity; feedback; the ability to communicate over distances . It increases the amount that can be done by one person, the appropriateness of what is built, and the geographic spread of the team.
This article celebrates the strengths, foibles, and diversity of people, how that has changed application development - and not changed it.. What is the same is the general nature of people and the way that dominates application development. What is different is that that fact is widely recognized. Most modern projects contain one or more of: a user, a human-computer interaction specialist, a sociologist, psychologist, or ethnographer.
Technology has a secondary influence on all of this, because the people factors dominate. Extreme distribution of developers, due to telecommunications, is one particular place where technology has changed the business of software development.
I should like to work from several points in this article:
There are two gateways into the discussion: the factors that drive application development and the way the projects themselves are laid out. The first part of this article deals with project size. Are projects getting larger or smaller? Why, and what should we expect in the future? The second part deals with the human factors. The third part takes a look at selected characteristics of projects, to see how those factors are showing up.
The music running through the article is that there are a lot of people developing software. They think differently, they work differently. They also have a great deal in common. The human factor in software development is being recognized and technology is supporting the human element.
(Note: A small explanation on the use of the term "human factors" in this article is in order. For now, we are interested in the way people work to create software, the human factors in creating software. Most articles containing the phrase "human factors" are interested in the interaction of humans with a particular piece of software, perhaps the one being developed. To keep these separate, I use the phrase "human-computer interaction" when referring to this latter, and "human factors in software development" for the main topic of the article.)
Yet, there always will be things that cannot be built even by 8 or 10 of the best people in a given number of years. The painful part is that to improve on that small team, one must add exponentially more people for each marginal improvement. This is because there is a reduction in both skill and productivity per person as the staff size increases. The Mythical Man-Month walks through the math in its chapter on large projects, and Surviving Your First Object-Oriented Project contains a model showing both the initial drop and later growth in problem size that can be addressed as staff size grows, due to these factors.
As individuals and small teams have been taking on larger problems, so have large teams taken on larger tasks. Figure 1 shows a picture of project ambitiousness over time for three staff sizes, the individual, the small or medium-sized team, and the large team. It shows all three staff sizes delivering more complex systems over the years.
Reading it in the other direction, it shows that the staff needed to deliver any particular problem size is diminishing. This is great news! There are orders of magnitude more individuals available than there are teams. Shifting projects to smaller teams (ideally, individuals) is probably the only way to produce the orders of magnitude more pieces of software that are being requested (i.e., get out of the software crisis).
Figure 1 could lead us to believe one of two things. If we believe that the problems size has a limit, the implication of the graph is that one day, all projects will be done by individuals. On the other hand, if there is no limit to the size of software we want to build, the three team sizes will never go away. There will always be something to build that requires the large team. This is the scenario I expect.
Our thirst for computation and system complexity is growing, not slowing. To give this growth a perspective, let us compare thirst for computation against the phenomenal, exponential growth of computing capability. Figure 2(a) contains two curves. One shows exponential growth, as predicted and measured for computing capability ("Moore's Law"). The other shows growth according to the factorial function. The real difference between the two curves does not show up well in Figure 2(a). The difference shows up in Figure 2(b), which plots them on a logarithmic scale. Computing capability, or any other exponential growth function, becomes a straight line on the log scale, but the factorial function still curves up out of sight. It is quite possible that our thirst hunger for computation matches a function that grows faster than exponential (there are many such functions) rather than the exponential. The available computing power will not catch up. It is not that computing capability is growing exponentially, but that computing capability is only growing exponentially, and our demand is growing faster. This explains part of why our computers get faster, but our word processing software gets slower.
It could, of course, be the case that although the computations themselves will grow in size, the problem complexity will not. This is unlikely. One of the characteristics of people is that they want things personalized, just their way. Here are two examples from everyday life.
One of the simplest gadgets imaginable is the refrigerator. It keeps things cold. Now it keeps different things at different temperatures (butter, vegetables, food, ice). The doors might open to the left or the right, or from the middle. The freezer can be on top or on bottom. There might be a separate ice or water dispenser. Then there are all the colors and sizes. Just for a refrigerator. Next consider a car. Reading the sticker on the side of the car gives an idea all the variations in demand. A windowing application program presents many more variations than the car.
So, regardless how simple and elegant the inside of the system is,
New software is being expected to track more information and relate it in more ways than before. Weather, identified as a chaotic system, will always require more sophisticated models and presentations to give us the weather reports we want. Software systems are having to include more features and accommodate more variations. So, software projects will continue to use staffs at all sizes. The issues this article covers will be with us for a long time to come.
That people are different is no surprise to us (when we bother to think about it). That difference is cause for celebration as well as frustration. It is fortunate that some people like working on the innards of a system, some like working on the user interface, some like marketing, and some like writing prose. Since groups of people with mixed backgrounds, personalities and thinking styles typically out-perform homogeneous groups of people, it is worth getting people with different characteristics on the same team. This immediately produces friction, since different people also have more trouble communicating with each other. Thus it is that some computer projects are taking on staff with backgrounds in psychology, sociology and family therapy to help keep the project teams running.
Delivering a system hinges on the fact that somewhere, key people, such as programmers, testers and installers know they have to make the system work, despite whatever the managerial, sales, requirements, analysis, and design people said, and whatever the process designer wrote for the process. Many projects fail for having followed the process exactly. Many succeed for not following the process. I have used processes, studied processes, and designed processes. My current, professional suspicion is that no one, at this time, is capable of writing a precise process for software creation that will work as written. Too many unexpected things come up.
Projects succeed because people break out of the prescribed process to make sure the system delivers, or because the process is deliberately vague, allowing people to do whatever is necessary. The best modern processes are of the latter form. This is
It is picked up as a theme in the section below on Processes .
The most significant failure mode of people is that they make mistakes. This is no surprise to us, when we stop to think of it. Yet some project managers and executives seem genuinely surprised when their team announces they plan to work an incremental or iterative process. "What do you mean, you don't know how long it will take?", or, even worse, "What do you mean, you plan to do it wrong the first time?" (Implication: "I can go out and hire someone who will promise to do it right the first time.")
People make mistakes in estimation, in requirements, design, typing, checking, installing, etc. We recognize this, and therefore criticize the "waterfall process", which does not take such mistakes into account (although it is still in use, see (P3) ).
leads to prototyping, rapid application delivery, and the incremental, iterative, and risk-management processes. However, these ideas cause a certain amount of trauma, since they introduce uncertainty into the project plan. The manager must first accept (P1) Trust in people to do what is necessary .
Given a choice of going out on a limb to try something new that might work or doing something that is known to be faulty, but is a generally accepted and conservative way of doing business, people often prefer to fail using the conservative method. "We failed, but at least we used the standard method." This is the reason throw-it-over-the-wall, waterfall development is still used, when much better strategies are available.
History has repeatedly shown that if a logical argument clashes with a cultural habit, the logical argument will probably turn out to be right, and the cultural habit chosen.
This conservative failure is one of the human characteristics that modern software development has not dealt with. How can it, since it requires change! We can therefore expect this to continue.
A peculiarly western trait is Invent Here! Some cultures, notably Indian and Japanese, value prior experience. The American and European cultures value originality. From school days on, working individually and creatively are rewarded. In the software industry, those people are recognized, rewarded and promoted who write the better algorithm, the better language, the better class library. Those who "just" put together existing pieces to get a solution quickly, have been and still are looked down upon. Recent literature refers to them as "component assemblers", as though that were a low-skill task. This is the western "Invent Here!" habit.
Invent Here! is reflected in the tendency to down-play other people's work (the Not-Invented-Here syndrome). It ties to ego and reward, which damages reuse programs (see (P5) ). It causes existing solutions and literature to be ignored, at a very high cost in extra labor.
On the basis of cost alone, we should expect every application development department to have access to on-line, internal, or local technical libraries. What better way to cut development costs and improve quality than to use an existing, documented algorithm, data-structure, or solution? And yet it is the rarest group and the rarest individual that manages this. In other words,
Thirty years ago, there was not much inclination to search a computing knowledge base, and not much of a computing knowledge base to search. Today, we have a huge knowledge base available, but still not much inclination to search it. Because of ( P4) , systems are generally designed by intuition and casual invention, just as before.
Structuring rewards is tricky. People act to improve their reward. Measure lines of code per day, and you get copy/paste coding, with huge program line counts. But how do you reward fewer lines? Between (P4) Expect people to Invent Here! and (P5) People act according to their reward , reuse programs are certain to have continuing difficulties. Reuse programs go against basic human characteristics, no matter how you structure the reward system. Rewarding for putting into the library encourages invention, not reuse. Rewarding for making use of the library encourages bloated rather than trim designs.
Reuse happens when it is not in the best interest of the developer to waste time on the innards of the solution. I have only seen this when the developer is a professional from a different field, who is rewarded on their progress in their primary field. They are highly motivated to get the software put together quickly, so they can get on with their work. They happily accept a sub-optimal designs if it can be gathered quickly. Contrast this situation with a devoted programming professional, whose passion and profession is to make better software.
It is a common apology among great thinkers that they cannot keep much in their heads at one time, as though this were not the most common of phenomena.
Software systems long ago left the stage where any one person could think about the whole system at one time. This was as true in the mid-1960's as it is today. The technique for managing it is the same now as then, make each person responsible for a relatively small part of the system.
Raising the programming language from assembler to a high-level language makes an improvement. However, the improvements in (P13) , expressiveness, are countered by (P0) , complexity at the user demand level, with complexity growing faster. If improved programming languages make a linear improvement, and complexity grows faster than exponentially, it is hardly a wonder that programming productivity gains are not keeping up.
In addition to the standard failure modes of individuals, most programming organizations are dominated by a few powerful personalities. This is lamented by people trying to turn programming into a predictable science or engineering discipline (as though those groups are not dominated by individuals).
Is this a failure mode or a success mode? Designs are often better for a dominant personality. Single-party designs have a coherence to them that committee designs notoriously lack. However, selecting a future strategy typically is done better when the experiences of many people can be brought to bear. A dominant personality will obscure the contributions of the quieter people, and may either block or force a choice.
How does the presence of dominant personalities affect software development? Strongly, and not in a way that is amenable to a technology solution. It is for the team sociologist to recognize and adjust for, when putting together or advising the teams. In other words, there is not much you can do but
Several people have worked on the non-linearities of program development. Peter Naur described the task of programming as "theory building", in which the programmers (or their successors) are tasked with constructing a theory of how the problem domain works, and how the software machine works. A team at MCC monitored designers solving a problem. They found the designers did not work in a top-down, bottom-up or middle-out fashion, but bounced around at all levels while working out the problem and the solution. They charted the designers' activity over minutes, and saw that detailed design was interleaved with requirements understanding. Donald Knuth developed "literate programming" to permit software to be designed and documented in a non-linear way. None of works these has had a large impact on the way application projects are set up, but programmers cite them when trying to describe their work.
Experienced designers report that they scan through the decisions coming up to spot the ones going to have major impact on the design. They focus on these, knowing that the other decisions will make less overall difference, or can safely be deferred. It is the designer's experience which indicates the critical ones. The major decisions do not appear in any obvious order, so it appears to other people that the designer is jumping around. In actuality, the expert designer is creating a linear sequence of decisions to be made, based upon impact.
is difficult to support. Standard CASE tools and scheduling tools do not fit. However, groupware tools, such as Lotus Notes, permit developers to report on their decisions as they make them, and link them. Risk management techniques are being used to let the project run in non-linear and unpredicted ways.
People often get their best productivity by copying an existing snippet of design and changing it to meet their current needs. Why does this work so well?
"The world carries its own structure, so that specificity always implies generality (and in this sense, generality is not to be assimilated to abstractness). That is why stories can be so powerful in conveying ideas, often more so than an articulation of the idea itself".
The copied snippet provides both the structure and a sample use of it. The developer can choose to learn the structure, or not. They can get away with just changing the parts that are different. In some cases, a person will change all of the sample, using just the provided structure and the memory of the example.
This copy-modify technique is applied by people everywhere. Cooks copy a recipe and vary just one part. A project manager takes over the previous project's plan and changes every line to reflect the current project. A requirements document or database schema gets copied and altered. An experienced programmer copies her child's hyperlinked program or document to complete a small project without having to learn the nuances of the new medium. In short,
Copy-modify is currently being applied even to completed applications. There is a growing trade between competitors in non-critical applications delivered with both generated, tuned code and the high-level description of it. These are referred to as "application frameworks". The accepted assumption is that the application framework is not quite correct for the buyer, but will take less effort to fix than it would to build from scratch. An example of non-competitive application frameworks suitable for trading traded is a frequent-flier awards system for an airline company. Application frameworks are quite a different notion than providing a "tailorable" system.
People constantly request examples. It is as though they operate internally and directly from the examples. There is some evidence from cognitive psychology that that may be so and there are research projects associated with that idea (see www The Conceptual Metaphor Home Page ). Whether it is actually true that people work directly from examples, it certainly is clear that most people work better given examples than simply working from abstract theory. So it is not only true in expository writing, but in technical development,
The modern style of eliciting application requirements is example-based. We find that people give better requirements when given specific situations to discuss and walk through than when just asked for their input. Even better than discussing the future system is to enact the use of the new system, making an actual performance of it. The performance is both example-based (P10) and tangible (P11) . Some organizations create fictitious employees having different personalities. The users describe how these made-up employees would get their work done. A lazy employee might find ways to defer work until later, while an ambitious employee might find ways to get the most accomplished soonest.
Example- and instance-based design techniques are well received by designers. Where a few, talented individuals can handle the powerful, abstract techniques, the masses of developers have trouble with them. It is not a matter of teaching. Abstract techniques will always attract fewer practitioners, however powerful it might be.
Better than working from an example is working with something tangible. The tangible item is both an example, and it plays into our senses.
An example of working from tangibles is the "CRC" (class-responsibility-collaborator) card exercise. Index cards represent components of the software system under design. The designers write on each card the component's responsibility to the functioning of the system. They pick up the cards and walk through the operation of the system in particular situations. CRC cards have been transferred onto the computer screen by numerous practitioners, but somehow lose their effect there. There is a particular effect in working through the design that comes from reaching over, pointing to a card, picking it up, and perhaps moving it to the side.
Prototyping user interfaces with paper and cardboard produces similar effects. Such prototyping is a "lo-tech" technique. There is a relatively new tension in the software community between hi-tech and lo-tech, as discussed below. The hi-tech has the drawback of being less tangible with the advantage of showing off the latest technology. Lo-tech works to maximize the cognitive effects in design. This tension shows no sign of abating.
The arrival of new technology has not changed this fact, nor is it likely to. On a series of project debriefings, the top two tools named were communications based: group versioning/configuration tools, and communications tools. With extreme distribution, such as home-offices and international work groups, the communications are happening over longer distances, and need to be supported explicitly.
When we program, we are trying to express our thoughts about a problem and a solution we typically do not understand fully, using a language that does not contain many of our accustomed features of expression, to a system that is unforgiving of mistakes. Even leaving out the last three difficulties, just trying to express our thoughts in any exact way is extremely taxing.
Expressing one's thoughts has always been a limit in communications. This is true in law, the other endeavor of exact expression of thought. It is interesting to contemplate concern that lawyers are not getting exponentially faster at creating contracts or laws. It is also true when talking in one's own, ordinary language, whether about personal or technical matters. It is true whether the expression is through painting, music, writing, or speech. When expressing freely, we use many rhetorical forms, simile, metaphor, comparisons, references, drawings and gestures, that are not picked up by our current programming tools.
The field of creating computer understanding for different forms of expression is computational rhetoric . Computational rhetoric is currently most rudimentary and will take a while to develop even a little further. When it finally does develop, programming will still be limited by our ability to think through the problem and the solution, and to work through all the details of how our described solution deals with all the myriad cases presented it. This is not one of our cognitive strengths, but there is no short path to saying that a piece of software is correct. The question of software correctness ultimately boils down to, "Does it do what we have in our minds, even the things we have not gotten around to thinking about yet?"
In other words, exponential growth in productivity is not an appropriate expectation for software creation any more than it is in other fields of thought expression.
Commercially available packages allow individuals to do what teams had to do before, and allow small teams to do what large teams did. Spreadsheets and dedicated simulation packages allow single people to create business simulations that formerly were team efforts. Object-oriented programming was once justified to handle the complexities of user interface programming. A few years later, UI generators became available to create user interfaces using templates and examples, and an entire justification for OO programming went away.
Large organizations around the world are putting a freeze on new development, mandating that systems must be bought, or at worst, bought to be tailored. This real reuse frees up much resource for other, more relevant tasks. The economics of thousands of software packages being deployed is finally making a dent in (P4) Expect people to Invent Here! .
The computational power and technical insights gained in the last 30 years permit one to be less concerned about how one's specifications are implemented. Each higher-level language counts for an improvement in computational rhetoric, and also in executable specification. One can express one's thoughts more succinctly (computational rhetoric), and then not worry about the particular translation to machine code (executable specification).
Because space and time were prime worries in application development 30 years ago, high-level languages were slow to be accepted. There was worry about the space and speed of the compiled code. Eventually, as machines became bigger and faster and compiler algorithms became better, few people needed to worry about the space and time of the compiled code. The similar pattern today is the concern in going from high-level languages to very-high-level (LISP, Smalltalk, Prolog, APL2), declarative and functional languages. Once again, improvements in machine power and compiler sophistication are making them more acceptable. Smalltalk and LISP, once laboratory oddities for the extreme computer power they drained, are increasingly being used to deploy production applications. This trend will continue, and will benefit application development teams. It is a case of technology better supporting the languages of thought and expression (P13) .
If people make mistakes, then one would do well to provide feedback quickly. One would hope that technology could have improved our rate of feedback. It has.
Feedback for programming was once per day when the turnaround time for program compilation was 1 day using batch runs of punched cards. Desk checking the program was worth all the time it took. Time sharing and interactive workstations took that to a minute for syntactical errors and minutes or small hours for behavioral errors. Presently, incremental compilation gives syntactical feedback in one or a few seconds, and some systems even permit the programmer to change the program while it is executing, which reduces behavioral feedback time from minutes to seconds.
Very high-level languages like APL2 and CLOS are used to get early feedback on proposed system architectures and algorithms even when they cannot be used for application deployment. At the user interface, pictorial feedback from interface generators is in minutes for certain kinds of commonly used interfaces. Behavioral feedback is still usually a matter of days or weeks, because the behavior has to be programmed.
The greatest social impact of technology on application development has come from the ability to support extreme distribution of the development team. Outsourcing the programming to a different country and working from home are now the order of the day. They require very fast communications, which are now widely available. Extreme distribution raises interesting challenges for satisfying the need of people to communicate with each other. Misunderstandings happen easier if two people are not seeing each others' faces and using a vocabulary of expression made common by repeated use in similar contexts.
The ability to communicate rapidly over global distances has shifted large-scale economics. Companies are sending their employees home (or the employees are asking to work from home), which frees up office space. The ability to set up satellite links has allowed India, Singapore, and South Korea to grow significant custom programming shops, spurring local population and economic growth there.
I could hope that the extreme distribution movement would improve our ways of communicating design information. I suspect it will not , based upon the rest of the human characteristics. Instead of working out standard ways to communicate a software design effectively, communication problems will probably be fixed by people talking to each other directly, running programs remotely, and simply outsourcing larger parts of the system.
On a small enough project, the few people may have multiple skills that enable them to talk to the users, gather requirements, mix user interface, infrastructure, and domain programming, and manage the project plan. On a larger project, that is less likely. The competence and time demands are greater. Therefore, modern teams are staffed with specialists, specialists in:
Not every team-building exercise actually builds a team. Conversely, however, many successful project groups have pointed to their team-building days at the start of the project as having helped them work more effectively. It is a case of team building being advantageous but not sufficient. Because of this, some managers take team building lightly. Thinking that it might do some good, and probably no harm, they allocate a few hours for it.
Part of the difference between success and failure is how seriously the organization treats the team-building concept. Consider on one hand a 2-day stay at a hotel, with spouse (or special one). The project staff are given certain activities together to learn to work and communicate together. The entire group is given time and activities to get to know each other socially. Contrast this with 3 hours in the cafeteria doing a few games and activities. It is not surprising that these latter tend to come back viewing team building as a waste of time. When done well, the team building has been done well, the staff typically do not look back on it as a waste of time, but actually do work better, with fewer tensions between groups.
Just as there is an industry of team-workers, there is an industry of facilitators. These are people whose sole job is to stand in front of a group of people, helping them trade information and reach a decision. The facilitators may have no background in the material being discussed, but they know when the discussion is getting off-track, when people are using actions that are not constructive, and they know ways to intervene and get the group moving forward again.
With the increased recognition that fulfillment and satisfaction increase productivity, managers are increasingly pressured to act as psychologists. One manager complained that she felt her job was more psychologist than manager. An increasing number of psychologists are being employed to train managers and to visit groups having team difficulties.
"Real" users are increasingly being requested as part of the project team. It is not uncommon to see a project plan requesting users one day per week over the life of the project. Occasionally, a project will manage to get a user placed directly on the development team. The purpose of having these users is to reduce the errors in requirements. The use of "real" users is a recognition that programmers, analysts and managers are not best suited for stating the requirements.
The most recent shift in team composition is the growing geographical distribution between team members, as mentioned earlier. Some of the staff may work at home, some may work in a different country. Some may work part time. Their special needs are being met with additional communication. This is recognition of the social forces permitting and obliging the increased distribution, as well as recognition of the primary force that communication presents.
Tools are fueled by and also fuel these trends. Electronic brainstorming and team facilitation tools record and transmit the ideas generated by the people. Telecommunications permits distributed communication. Groupware tools permit distributed collaboration. The current approach is to let the programs support the people in their work, rather than to try to replace the people.
People are different, and work differently. This shows up in a fascinating spectrum of technical approaches being proposed for developing programs. The different approaches have neither demonstrated failure and faded away, nor gained domination based on success. Rather, each has found kindred spirits in various programmers, and grown as the population has increased. Thus, for every principle named earlier, there is an example of its reverse fervently being used somewhere.
Offsetting concrete design techniques, such as the CRC cards mentioned earlier, are abstract and mathematical techniques, such as the Dijkstra and Gries' program derivation techniques. Formal notations (Lotos, Z, VDM, etc.) offset informal languages and notations (Smalltalk, Forth, C, etc.). Logic programming counters object-oriented programming. Rigorous design processes (Cleanroom) to counter the more common informal ones.
Formal program derivation goes against "people make mistakes", "rapid feedback", and "people work from examples and tangibles." Yet it has found a large group of proponents over the last 30 years, who keep working toward the goal of creating programs that are provably correct. Dijkstra repeatedly demonstrated the invention of outstanding algorithms using mathematical techniques, Gries has moved some of that work into mainstream computer science education. A few communications protocols have actually been proved safe, and certain programming languages and notations, such as VDM, Z, Unity, and Lotos, not only are not going away, they are growing (Lotos is an international specification standard). The Cleanroom methodology applies the notion of decomposed, provable programs to very large systems.
Pulling in the other direction is a large and growing group, those favoring informal methods, prototyping and rapid experimentation. On "provable" correctness, these methods are arguably inferior. At the same time, they are arguably superior in terms of delivering systems quickly and matching the nature of humans.
The practicing programmer is involved with this debate daily, not only on program derivation technique, but on whether the language supports static type checking or not. From the creation of Pascal and C over 25 years ago, the two camps have been locked in debate, neither able to displace the other. Successive generations of languages has not reduced that debate, nor shown a clear winner. Current banner-carriers of the difference are C++ and Eiffel (for static type checking) and Smalltalk (against static type checking).
What is intriguing is that neither the formalist nor the informalist group has succeeded in driving away the other. It does not look as though either will. So far, the informalists have delivered the most systems, while the formalists are holding the vision for the future.
Matching the formal vs. informal programming tug-of-war is the question of how rigorous the development process should be. Cleanroom techniques demonstrated a series of successes in the late 1980's and early 1990's, with a formal process that prevents programmers from testing their own code. Code is tested by another team, who rates the programming team. Testing is carefully set up to establish a mean-time-between-failure schedule, using carefully thought out probabilities for each driving action.
The opposite, far more common approach, may be described as "melee". It is intended to maximize the freedom of movement of each developer, tying the results together in a minimal but sufficient way (remember (P1) Trust in people to do what is necessary ?). This approach has also been demonstrated on large programs. One approach to handling the variations in actions between people is to build the system each night, with large regression tests, so that developers get a continual sense of "develop - deliver". Risk management and the usual, more sedate forms iterative development fall in between these two extremes.
Is the presence of diversity a sign that the general statements about human factors earlier in the article are false? Not at all. They are true in a broad sense, by population and by degree. What is also true is that people differ, and this difference is showing up in the myriad combinations of ways that software can be built.
The other key cognitive factor in design is time-to-feedback. Seymour Cray made a point about feedback and design techniques in a videotaped talk he gave on his design career. Coming out of university, he was the proud and proficient owner of an extra-large slide rule. Early on his first job, he was given a design assignment, and set about it, diligently calculating the parameters in the design for several days. Walking the halls, he met an experienced designer, who showed him that it was simpler just to apply a few rules of thumb, build a prototype, and test it to see where it was off. From that, a few adjustments could be made to the design, and it could be brought within spec. In short, a little bit of feedback can replace a lot of analytic work.
Programming has followed this track. People sometimes complain that with rapid, interactive programming, they actually make more errors in programming, and spend more time fiddling with their code. Yet it is almost impossible to peel developers away from their screens to desk-check or derive their work. Rapid feedback also has introduced Rapid Application Development as a technique for deploying software. The idea is simply to get the software out so fast that the users can be brought in a few days to comment on the software. Of course, days is a very long time for feedback, which provides the opening for paper and lo-tech user interface design.
Given a choice between a cognitively matched method and a hi-tech method, (a) the cognitive method will be generally more effective, (b) the hi-tech method will generally be selected by management, (c) the cognitive method will generally be applied by busy practitioners in the trenches.
Any tool is welcomed as it reduces a burden on the programmers, and resisted as it attempts to make decisions for them. Looking back at the development of high-level languages, the principal complaint was that the compiler could not make as good decisions about instruction sequencing and register allocation as humans. Eventually, it became clear that the compiler could, and that there more interesting decisions to make, such as those concerning structure and algorithm. This same complaint will repeat itself every time a new level of decision making is given to the automating software. The complaint will last until the software proves its capability, and the humans find more interesting decisions to make.
Risk management is a task allocation strategy in which assignments are made based upon the estimated damage of failure of the assignment. It addresses the non-linearity of development, and requires a greater amount of trust in the individuals. It is up to each person or team to assess where their current greatest risks lie, and to address those risks. The role of project management is to help them assess their relative risks. An assessed risk may be in an algorithm, a product, a staffing plan, requirements, or any other place. It is the hardest to adopt, perhaps since the least concrete has been written on it (see www The Risk Management Catalog ), and since it requires trusting the actions of the most individuals on the team. In personal discussions, project leaders who use risk management techniques, say that it is by far the most effective, since once trained, every individual on the project is maximizing the effect of their work.
Iterative is a scheduling and staging strategy which includes planned rework of pieces of the system. It addresses errors and learnings in the requirements and design of the system itself. Booch gives this sort of learning a value by calling it "gestalt, round-trip design". His term emphasizes the human characteristic of learning by completing. Iterative development is hard to plan, because it is hard to guess in advance how many errors or major learnings will take place. Some projects simply fix the number of iterations to three: the draft design, the final design, the tested design.
Incremental is a scheduling and staging strategy in which pieces of the system are developed at different rates or times, and integrated as they are developed. It addresses errors and new learnings in the overall methodology. After a section of the system is built, the methodology is examined to find where it was wrong or could be improved. Either the team structure, the techniques, or the deliverables might be changed. Incremental is the simplest to learn of the three new processes, since it is straightforward to plan how many pieces to cut the system into. It has become a critical success factor for modern projects.
Non-linearities in the design process are being supported at the process level by scheduling by delivered system milestones (incrementally), instead of by the phase deliverables, requirements, analysis, design, code. Requirements, analysis, design and programming may be done interleaved for different aspects of the increment. The design need not progress in a fashion that simply matches the increase in the number of bubbles on a design chart. Rather, it progresses by settling selected design issues, some of which will be delivering functionality. Decision-tracking tools are appropriate for this style of work. A groupware tool is used to capture and comment on the decisions made on each aspect.
Beyond the process strategies, there is a general resurgence of managing at a personal level. What does resurgence mean? Successful managers have long stated that they get valuable insights from wandering the corridors and listening to the random comments they hear. This has recently been blessed with the term, "management by walking around." The existence of a recognized and advocated technique helps managers apply what might otherwise be criticized as a waste of time.
What has changed is the recognition of the human factor. Social processes, design techniques, and technology have come to support the weaknesses of people, as well as draw upon their strengths. Fourteen recognitions and adjustments in attitude to take into account the human presence on project were highlighted:
(P0) As a system gets closer to finicky users, its complexity grows, possibly faster than exponential.
(P1) Trust in people to do what is necessary.
(P2) Adjust for people making mistakes.
(P3) People prefer to fail conservatively than risk succeeding differently.
(P4) Expect people to Invent Here!
(P5)People act according to their reward.
(P6) People can keep only a small amount in their heads.
(P7) Recognize the presence of dominant personalities.
(P8) Let people work in unpredictable sequences
(P9) Provide something to alter.
(P10) Provide examples.
(P11) Provide something tangible, where possible.
(P12) The communications load can soon dominate the project.
(P13) Software creation is limited by the ability of people to express their thoughts.
(P14) Simply buy it.
Offsetting any basic characteristic of people is that there are many more people building software. This gives rise to many ways of working, many conflicting ways. One should not expect that trend to diminish in the future, but rather to stabilize, as the populations in support of each method grow.
Technology has a secondary influence, because the people factors dominate. Its greatest two effects are in permitting extreme distribution of the project staff and workforce, and in supporting the human factors, such as giving feedback and permitting non-linear work processes. Technology does not always increase effectiveness, as when it conflicts with basic cognitive or human characteristics.
Finally, it is not just the design techniques and the technology that are taking into account the human characteristics, but the management techniques and the overall development process as well. Management by walking around and management by risk reduction support the basic concepts of humans working non-linearly, communicating heavily, making mistakes and then delivering a working system anyway.
Cockburn, A., Surviving Your First Object-Oriented Project, Addison-Wesley, 1996.
Naur, P., "Programming as theory building", in Computing: A Human Activity, ACM Press, 1992, pp.37-49.
Knuth, D., Literate Programming, Stanford University, 1992.
Lave, J. and Wenger, E., Situated Learning: Legitimate Peripheral Participation, Cambridge Univ. Press, 1993.
Johnson-Laird, P. and Byrne, R. Deduction, Lawrence Erlbaum Associates, 1991.
Lakoff, G., "Conceptual Metaphor Home Page", world-wide web page "http://cogsci.berkeley.edu/", 1996.
Cunningham, W., and Beck, K., "A laboratory for teaching object-oriented thinking," ACM SIGLPLAN 24(10):1-7, 1989.
McCarthy, J., Dynamics of Software Development, 1995.
Cockburn, A., "The Risk Management Catalog", world-wide web page "http://members.aol.com/acockburn/risktcata/risktoc.htm", 1996.
Booch, G., Object-Oriented Analysis & Design with Applications, Benjamin-Cummings, 1994.