Sunday, October 10, 2010

Ontogeny recapitulates phylogeny

Well, that’s quite a mouthful, I agree – ontogeny recapitulates phylogeny. What it reflects is rather interesting. The phases in the development of an individual organism mimic to an extent its ancestral form. For example, the human embryo develops gills bars which are then modified for other purposes. But, “to an extent”. Biologist point out that the so called recapitulation occurs at a very coarse level, bereft of details [The Sciences of the Artificial, Third Edition by Herbert A. Simon, MIT Press, 1996].

But this serves as an exciting analogy for something similar that happens while we are learning a subject. The student’s progression seems to resemble the progression of the discipline from childhood to maturity.

Take for example, software engineering. First there was waterfall, then the iterative and incremental methods, and then agile. The streamlined regularity of waterfall is most useful for the tyro software engineer; there are copious milestones and checkpoints so that (s)he has enough guidance to get along. The essence of iterative and incremental development is appreciated with experience, when the power of feedback becomes manifest. Finally, agile in all its facility marks the transition from instruction to interaction. The software engineer now has sense enough not to need micro-management, (s)he can leverage the typical loose-coupling of an agile team. (Indeed, a criticism of agile has been its over-dependence on “premium people” to be successful). So the progression of a software engineer from a neophyte to a seasoned pro is marked by a progressing embrace of the more mature development methodologies.

What is the implication of this trend on how we teach software engineering to students? Do we introduce waterfall, then iterative and incremental development, then agile, in sequence? Or should we indoctrinate students in the most mature methodology of the day and touch upon others as historical footnotes? In conventional sciences the dogma of the day is “taught”, while its ancestors may (or may not) be mentioned – for example ether in physics, or phlogiston in chemistry. But unlike ether and phlogiston, waterfall is not merely of historical import, systems of relevant scope are successfully built using waterfall, very much in the present.

So must software engineers learn and unlearn (or at least learn to supersede) a methodology before moving on to the next? Or is there a more organic advancement?

Blueprint versus Recipe

Simon presents two different but complementary perspectives of complex systems: state descriptions versus process descriptions, or simply, blueprint versus recipe. For a simple system, the congruence of these two views is almost intuitive. A circle is locus of points all of which are at an equal distance from another point (the center). To draw circle, anchor one arm of a compass at a fixed point (the center), and rotate the other arm until it returns to the starting point. The first statement is a blueprint for a circle, the second, a recipe for drawing a circle. A blueprint characterizes the world as sensed, while a recipe reflects upon the way it as acted upon. [The Sciences of the Artificial, Third Edition by Herbert A. Simon, MIT Press, 1996].

For software systems, endowed as they are with Brooksean “invisibility” and “unvisualizability”, neither of the blueprint or recipe views are easy to arrive at. Describing the structure of code may not necessarily be a complete or even accurate description of the software subsuming the code. And although directions for constructing a unit of code can be specified with precision, there is no guarantee the desired behavior will be derived from that code.

Then how should we go about learning how to build software systems? Do we focus on absorbing knowledge about the blueprint or acquiring skills around the process? This confusion is often reflected in software engineering textbooks. Some authors take the descriptive route, trying to narrate software's characteristics – in all its quirks – using natural language. Others assume away all the messiness and indulge in the purity of mathematical models and their transformations. The former allows for easier reflection of the blueprint, the latter for the recipe. Both approaches are useful, but not one in the exclusion of the other.

That elusive balance is the key.

Saturday, October 2, 2010

Some Reflections on Software Engineering Education

Education is an explosive topic. Harmonious people become highly intolerant when talking about education, especially other people’s. To play it safe, I have qualified “education" in this section’s title with “software engineering" and qualified “software engineering education" with “some reflections". I do not claim what is reflected here is either a complete or consistent commentary on education in the abstract.

Krutchen has conjectured “that the half-life of software engineering ideas is roughly five years". That is, five years from a given date, 50% of what constitutes the current state of the art “... will have been forgotten or seriously marginalized – not really worth teaching an undergraduate software engineering student" (The biological half-life of software engineering ideas. IEEE Softw., 25(5):10–11, 2008). This seems quite plausible to those associated with software engineering for more than five years. This also raises serious questions on what should be taught to the student of software engineering. If what is cool today – to borrow the latest lingo – will be cold in five years, what hot topics should now be taught?

This is a problem common to any fast evolving field. Hamming’s remark in the general context of science and engineering education is apt for software engineering also: “... teachers should prepare the students for the student’s future, not the teacher’s past" (The Art of Doing Science and Engineering: Learning to Learn. CRC., 1997). That is easier said than done; certainly for those of us not endowed with Hamming’s wisdom and experience. But I believe it can serve as an enduring vision for the way software engineering is taught and learnt. We have full knowledge of our past, and none about the student’s future. Yet, we must strive to impart an understanding of the problems today’s students will be called upon to solve tomorrow. This a significant challenge; how does one go about it?

The progress of any discipline can be seen as a window moving from left to right across a continuum of change. The frame of the window represents the present. On the left we have concepts and techniques that worked before but have since been found to be inadequate. On the right we have exploratory results that look promising but are yet to be tried beyond controlled conditions. In the middle, within the window’s frame, lie today’s state-of-the-art, nourished by insights from the left, and feeding experience to the right. For a field like software engineering the window may move relatively fast, but this continuum is common across fields. A key aspect of preparing students for the future is to sensitize them to the moving nature of the window. In addition, we need to culture an appreciation of what worked before, what works now, and what is likely to work in the future. In the treatment of topics in this book, I have been influenced by this view.

On my good days I can convince myself I am not that old (yet); still a lot has changed from the time I was an undergraduate student. Then, the snazziest gadget we had was a wrist-watch (which only gave us time); blackberry meant black berry; the Web was something like the Aurora Borealis, we knew it existed, but nearly no one had seen. The biggest difference between then and now is indeed the proliferation of the Web. Generally, education can be seen as a combination of information and insight. Without the Web, teachers and books were the only source of the former as well as the latter. But with the Web, information is just clicks away. But insight is still hard to glean. Insight comes from distilling information in a particular context, and depends heavily on experience and intuition of the distiller. The challenge for today’s teachers and textbooks is to shift away from merely purveying information. We have to give students the insights they need to make sense of the information that is now so easy to gather.