Ontogeny recapitulates phylogeny

Well, that’s quite a mouthful, I agree – ontogeny recapitulates phylogeny. What it reflects is rather interesting. The phases in the development of an individual organism mimic to an extent its ancestral form. For example, the human embryo develops gills bars which are then modified for other purposes. But, “to an extent”. Biologist point out that the so called recapitulation occurs at a very coarse level, bereft of details [The Sciences of the Artificial, Third Edition by Herbert A. Simon, MIT Press, 1996].

But this serves as an exciting analogy for something similar that happens while we are learning a subject. The student’s progression seems to resemble the progression of the discipline from childhood to maturity.

Take for example, software engineering. First there was waterfall, then the iterative and incremental methods, and then agile. The streamlined regularity of waterfall is most useful for the tyro software engineer; there are copious milestones and checkpoints so that (s)he has enough guidance to get along. The essence of iterative and incremental development is appreciated with experience, when the power of feedback becomes manifest. Finally, agile in all its facility marks the transition from instruction to interaction. The software engineer now has sense enough not to need micro-management, (s)he can leverage the typical loose-coupling of an agile team. (Indeed, a criticism of agile has been its over-dependence on “premium people” to be successful). So the progression of a software engineer from a neophyte to a seasoned pro is marked by a progressing embrace of the more mature development methodologies.

What is the implication of this trend on how we teach software engineering to students? Do we introduce waterfall, then iterative and incremental development, then agile, in sequence? Or should we indoctrinate students in the most mature methodology of the day and touch upon others as historical footnotes? In conventional sciences the dogma of the day is “taught”, while its ancestors may (or may not) be mentioned – for example ether in physics, or phlogiston in chemistry. But unlike ether and phlogiston, waterfall is not merely of historical import, systems of relevant scope are successfully built using waterfall, very much in the present.

So must software engineers learn and unlearn (or at least learn to supersede) a methodology before moving on to the next? Or is there a more organic advancement?

Blueprint versus Recipe

Simon presents two different but complementary perspectives of complex systems: state descriptions versus process descriptions, or simply, blueprint versus recipe. For a simple system, the congruence of these two views is almost intuitive. A circle is locus of points all of which are at an equal distance from another point (the center). To draw circle, anchor one arm of a compass at a fixed point (the center), and rotate the other arm until it returns to the starting point. The first statement is a blueprint for a circle, the second, a recipe for drawing a circle. A blueprint characterizes the world as sensed, while a recipe reflects upon the way it as acted upon. [The Sciences of the Artificial, Third Edition by Herbert A. Simon, MIT Press, 1996].

For software systems, endowed as they are with Brooksean “invisibility” and “unvisualizability”, neither of the blueprint or recipe views are easy to arrive at. Describing the structure of code may not necessarily be a complete or even accurate description of the software subsuming the code. And although directions for constructing a unit of code can be specified with precision, there is no guarantee the desired behavior will be derived from that code.

Then how should we go about learning how to build software systems? Do we focus on absorbing knowledge about the blueprint or acquiring skills around the process? This confusion is often reflected in software engineering textbooks. Some authors take the descriptive route, trying to narrate software's characteristics – in all its quirks – using natural language. Others assume away all the messiness and indulge in the purity of mathematical models and their transformations. The former allows for easier reflection of the blueprint, the latter for the recipe. Both approaches are useful, but not one in the exclusion of the other.

That elusive balance is the key.