Is Building and Managing Software Products a Social Activity?

Fred Brooks, a particularly insightful manager working at IBM in the 1970s, noticed that adding more software programmers to a project that is already late makes the project even more late. He coined this principle as "The Mythical Man-Month", a grim piece of advice to all future managers. While this phenomenon has been widely understood (and also widely ignored), there are other strange phenomenon in the world of software development that are equally interesting and counter-intuitive.

The most surprising of these, at least to me, is what is known as "Conway's Law". Contrary to the popular stereotype propagated by Hollywood that portrays software programmers as self-centered, individualistic nerds, programming is a fundamentally social activity. You need to talk to lots of people on your and other teams to understand the architecture, the interfaces and various other details of the system. Melvin Conway, who originally coined the term, observed that because of the social nature of programming, the design of an underlying software system closely reflects the social relationships between its programmers. 

That is to say, if you have two lead architects designing a system, you're likely to end up with a two-module project. This phenomenon was first observed in the 1970s, when compiler design teams with 4 lead engineers ended up with 4-pass compilers. If your team had only 3 lead engineers, they were likely to design a 3-pass compiler. The design of the interfaces between the modules is also likely to be heavily influenced by the relationship between the lead programmers. If your system has 2 architects that are good friends, you're likely to end up with a two well defined modules with clean interfaces and detailed contracts. If the two are not very good friends, they're likely to come up with a design that relies too heavily on standard interfaces (like XML/WebServices) that may ultimately not be flexible enough for the system.

The most grim example of this phenomenon was demonstrated by two groups of programmers working on NASA's Mars Climate Orbiter. The spacecraft, costing some $300 million, crashed on Mars because of badly designed interface between two modules - One group assumed some Integers would be in metric units, while the other assumed they're be in English units.

This phenomenon has a profound effect on software quality, but rarely gets the attention it deserves from managers of the software, despite lots of evidence for this effect from both Harvard researchers and Microsoft. Managers of software teams, especially in small startup environments, need to ensure that the team they're building is compatible with the software being building. If you've hired 3 architects to design your e-commerce website, you're very likely to end up with a 3-module design (frontend, middleware and backend), even if this is not the most optimal way to do things.

Another surprising reality of software design is known as the "second-system" effect. If an Engineer is designing a new system similar to another one that they've previously worked on, they will almost always over-engineer it and make it way too complex. The Engineers will try to solve all the limitations and shortcomings of the first system on their second attempt, trying to cram in everything that they didn't get done the first time. I've seen numerous examples of this in my career, and this effect is literally deadly - It can kill your startup, because of the unnecessary complexity that will get introduced. The unnecessary complexity will keep delaying the product, and the software will soon become too complicated, even before you've had a chance to launch it. Startups are particularly vulnerable, because they tend to hire engineers "with relevant experience" to what they're trying to build.

Managers rarely tend to think of these things when planning, but it is absolutely important that they do. Software is designed to be very precise, but in the end, it suffers from the limitations of its creators, who are, after all, human. Being aware of the various psychological and social effects at play during software design will help you get the software built and launched faster.