Comparing Design Notes over Coffee

My architect friend Ken and I had coffee the other day and caught each other up on the latest trends in our fields. Ken and his wife designed our house. Ken and I like to talk about meaningful design processes (me about software, Ken about buildings and homes) and learn from each other’s experiences.

I explained to Ken the new trend in software called agile development which emphasizes teamwork, close customer involvement, evolutionary design, programmers as designers, and incrementally delivering value to customers. On a typical agile team there isn’t much distinction between designing and programming. Design is viewed as something that is constantly happening while you are coding. In fact one view held by extremists that I find somewhat disturbing is to equate any design up front as being the same as Big Design Up Front which is equated with being wasteful. Since you will change your ideas during implementation anyway, why invest any time inventing pie-in-the-sky solutions? (In case you can’t tell, I take a more moderated view—it all depends on the situation and the context. I believe a little upfront design never hurts, and sometimes more is needed).

More traditional processes tend to divide development into phases: problem requirements are gathered and analyzed, solutions are designed and implemented. Although even that description is too simplistic. The Rational Unified Process talks about overlapping phases and delivering incrementally. So what’s the big difference between agile and more traditional approaches?

One big difference between agile and more traditional approaches is what stands between the developer and the problem. Traditional software development teams have analysts or business architects who gather requirements, write them down, and then work closely with developers/programmers who implement the software. These people hold the torch for what the user needs and wants (and can afford) throughout the process. The good analysts I know are skilled at talking with users and business experts and at translating their requirements into terms that developers understand. They bridge the complex world of the business and the detailed world of the programmer. On agile teams, developers work directly with their “customers”. Middlemen are eliminated. As little documentation as possible is created. There’s direct, open communication. The software design takes shapes in a very organic manner. Things are not planned out to the nth degree beforehand.

Ken noted that a similar trend in building architecture, especially the environmentally friendly architecture that he’s interested in. There’s the traditional design process which Christopher Alexander believes makes for lifeless buildings, and the more organic, buider-as-designer process (which is a haven for old hippies, according to Ken). It is into this world divided into disparate camps, that Ken wants to bring his creativity and vision. He’s worked in a big architectural firm and felt far removed from the customer’s needs. On the other hand, do-it-yourselfers tend to not want help. Where does such a passionate designer like Ken fit in and make a living? I told him he has to find a new market and educate potential customers on the benefits of taking responsibility and engaging in the design of their environmentally friendly homes with a designer like him.

I told him I find myself in a similar situation. But I haven’t had to create a totally new market. More often than not, I work with those who follow fairly traditional development processes. There I add value by talking about responsibility-driven design. I also help people who want to build flexible, well-designed systems get more skilled at articulating what they want and how to build it. I also try to bring a spark of agility to their processes—finding ways to simplify heavy-handed processes, streamline documentation, and communicate more effectively. I’d like to spend more time with agile teams helping them bring their design skills up another notch while programming like crazy. But so far, that opportunity hasn’t come along.

I’m always fascinated by the parallels between software and the architecture world that Ken and I find whenever we have time for a cup of coffee. My next challenge will be to absorb some of Christopher Alexander’s latest writing (but Ken has warned me that his four volumes are not light reading) before I have another conversation with Ken.

Why Objects?

As I’ve been working on a position statement for an OOPSLA panel reflecting on the roots of modern software development practices while looking to the future, I’ve been thinking hard about why I got hooked on object technology. Compared with structured programming and design, objects seemed significantly better at handling complexity. Object programming languages were an earth shattering improvement over the procedural and assembly languages I used when I first encountered structured design techniques. Instead of simply following conventions, object programming language constructs forced me to bundle together meaningful operation and data. Object-oriented methodologies generally incorporate the principles of structured design but OOD seems much more than an incremental improvement over SD. Instead of focusing on a thread of control and managing its complexity via procedural decomposition and structured control constructs, object design enables me to break a composition into thousands of semi-autonomous entities with structured roles and responsibilities. Objects offer me a completely different way to think about computation. This way of thinking empowers me to deal with a level complexity that I could never have dealt with only using structured design techniques. Object technology encourages me to form abstractions—objects—and to design how small neighborhoods of them interact.

Responsibility-driven design offers thinking tools that enable developers to conceive of an implementation in terms of interacting roles and their responsibilities. It provides a vocabulary for describing designs that helps developers communicate complex ideas and make tradeoffs more effectively. Agile practices, by emphasizing working code that satisfies customers, seek to reduce accidental complexity by admonishing you to design simply and grow complexity only when needed. Eric Evans, in Doman-Driven Design: Tackling Complexity in the Heart of Software, offers tactics for identifying, preserving, and sharing a common domain model. Refactoring tools have taken tediousness out of making changes and modern application development environments have made it possible for development teams to “hum” by testing and building incrementally. These all represent progress.

But at the end of the day, they cannot reduce the complexity inherent using diverse tools, platforms, and technologies that make up a typical sprawling IT system. While OOD/OOP did give us an order of magnitude improvement over previous techniques and tools, we still don’t have the order-of-magnitude better approach we need to sort today’s complex environments and minimize the gaps and seams that are inherent when diverse technology comes together in a complex system. In the meantime, thank goodness for the framework builders who give us various ways of linking objects with relational databases and for little languages and tools that take the tedium out of repetitive (error prone) tasks of gluing things together. We live in a complex world where objects will continue to make a lasting, significant contribution. What will be the next breakthrough in software development that will subsume the principles of OOD (and transitively SD) and provide the next order of magnitude improvement? I’m not sure. While I don’t think there are any silver bullets out there, I look forward to discovering and encountering even more effective practices, technologies, and techniques that allow us to address inherent complexity head on.

Don’t Look for Grisgris

Gris-gris
African fetishes, charms or amulets.
Objects believed to have magical or spiritual powers to protect against evil or injury.

I received an email from a colleague pointing out a blistering critique of object technology in an article by Richard Mansfield, OOP Is Much Better in Theory Than in Practice, and a website devoted to defrocking object-oriented myths. The article starts out on this gloomy note and only gets more negative:

“Think object-orient programming (OOP) is the only way to go? You poor, misguided soul. Richard Mansfield contends that OOP is just the latest in a history of ideas that sound good in theory but are clumsy in practice.”

My colleague found this rant rather depressing. He has assumed for years that modern systems are ideally coded using OOAD---but still has struggled to make it happen in his own projects. He asked, what do I think of this kind of criticism?

Here’s my take. Any one who raises questions and concerns is good to listen to. There certainly initially was a lot of hype around objects. In the early days it was oversold. On the other hand, many of Mansfield’s comments remind me of someone who likes take broad swipes at something just for argument’s sake as well as someone who tweaks things for database performance. For him, objects get in they way. He’d much prefer to build software machinery optimized for relational database access. Reminds me of the classic ACM Article by Donald Knuth extolling the virtures of GOTO statements. GOTOs are useful sometimes. Objects, in my experience are even more useful---but not universally so. I know lots of people who have benefited from doing the hard work of creating a design with reusable classes with planned variations. Creating a model of interacting objects is hard intellectual work. Writing piles of code with cut-and-paste reuse is easier. It just is harder to maintain. I also know people who don’t see much benefit of objects because most of the code they write simply shovels data to and from a database with little behavior other than validity checks in between. While they may use objects to create GUIs and get results back from the database, they rarely invent objects of their own. And unfortunately, object programming languages still don’t co-exist very neatly with relational technology.

The value of object programming and design is apparent when you have significant behavior in your application and you create a model of interacting objects that does some significant work. Sure, you don't need objects, just like you don’t need high level programming languages. But objects can sure come in handy. To discount the benefits of object-oriented designs simply because people don’t create good designs using them (another one of those correlation=cause false arguments in this anti-object writing) isn’t a rational argument. That’s like saying diets don’t work because fat people diet! There are also tangible benefits to developers who leverage extensive class libraries to build complex systems. I would hate to have to build all the complex software we do today without them!

I thanked my colleague for pointing me to this website and article. I'm going to give them a more careful reading which will probably result in some blog entries, if nothing else. Untangling any truth in a rant from fallacy is always good practice. But at the end of the day, we shouldn’t be looking for grisgris to be universally applied with blind faith to ward off all software ills. Instead we should be looking for solutions that work well for a given situation. I find objects a pretty good tool for a wide class of problems. In my own modeling and design over the last two years I’ve also looked at creating solutions using declarative approaches and seen how rule engines and objects mix (not very well unless you are very skilled at abstracting and writing generic rules with lots of meta-context). The value of objects hasn’t dimmed; my toolkit has simply expanded.

The Cost of Inertia

Last week I closed out a safety deposit box that I had rented but hadn’t touched for over 20 years. In theory, I paid for the box to hold tax returns and valuables, but never visited it after placing some “starter” something into it when I opened the account (what was it?). The bank branch in our town had closed years ago and the safety deposit box had been relocated to a branch in a nearby town. So I had to drive 6 miles for this little errand. I was damned if I was going to pay $39 for another year’s rental!! For some reason, I decided to take action. I’d rather donate the fee to the Oregon Food Bank, instead of lining the pockets of a bank. To my amusement, the box had 3 coins in it—2 Susan B Anthony dollars and a 500 lire piece. Over the years those coins have cost me $600.

Why did it take me so long to stop paying my annual fee and clean out the box? Plain and simple: inertia. As defined by dictionary.reference.com:

The tendency of a body to resist acceleration; the tendency of a body at rest to remain at rest or of a body in straight line motion to stay in motion in a straight line unless acted on by an outside force.
Resistance or disinclination to motion, action, or change.

In the busyness of life it’s all too easy to let things slide rather than fix ‘em. I didn’t miss that $39 in my pocket, but I didn’t like paying for something I wasn’t using, either. As a small act of, well, determination, I took an hour out of my busy day and “fixed” the problem. Over the past year I’ve shut down my unused dialup service (saving $10/month, and readjusted my banking to eliminate most monthly fees). Now if I could figure out how to get my phone company to coalesce my home and DSL lines (that’s a long story not worth recounting here), I’d save myself another $300 per year. But two painfully long phone calls to my phone company haven’t fixed things yet. I’ve got to muster some determination before I try again. Every so often I get the itch to cut out waste.

I had lunch today with a software manager who described some costs of inertia in her organization. Inertia that makes for tedious retyping of data from one system into another instead of writing software that could handle the majority of data transfers. Inertia that keeps a 30 year old system chugging along, even though it hasn’t aged gracefully. Most software, as it ages, gets more complex and more difficult to maintain.

There’s a cost to inertia. In my case, a few hundred bucks a year. In companies with inefficient processes and creaky software, the cost can be quite high—in dollars as well as the expense of people working at tedious (unnecessary) tasks. Lean software development practices aim to strip away “excess waste” in processes. But applying lean thinking to legacy systems and maintenance projects isn’t nearly as cool as using it on new initiatives. I wish it were. There may be no glory in chipping away at cruft built up because of inertia. But there should be. At the end of the day you will have left the world in a slightly better state. I know I have. My check to the food bank is in the mail.

Timely events in Use Cases

Last week in a use case writing course I was teaching, a student presented me with a use case where he had been puzzling over how to accurately express the passage of time. His use case occurred over a period of several days and went something like this

Day 1:
The user does this
And this
And the system does this
Day 2:
The system does this
Another action..
Day 3:
The system does this
The user does this
And then things are finished by…

Although it accurately reflected the passage of time in the current system, he wasn't satisfied with it. He wanted to describe a new version that could be more responsive. Instead of waiting for nightly data feeds, what if the system could process data more quickly? Expressing this in terms of time passing (day 1, day 2, etc.) seemed too concrete and limiting. After probing a bit, I suggested he consider restating this passage of time as “events” that enabled the use case to move forward. For example, “Day one” could become “Prediction made”, “Day two” could become “Project Completed”, and so forth. This shift in thinking--from describing elapsed time to describing events in a complex process--brought clarity to this use case and will influence the system re-design.

People can get bogged down because they don’t know how to express passage of time or lengthy pauses in action. Some might be tempted to “fix” the above use case by splitting it into three smaller one with appropriate pre and post conditions. That technical solution would only obscure the user’s overarching goal.

My eventful solution, another alternative, was inspired by Michael Jackson’s Problem Frames. Jackson emphasizes that “the point of software development is to build machines that interact with the world and change it.” Jackson is big on describing problems in terms of domains and software “machines” having direct interfaces where shared phenomena flow between them. He cautions, “Don’t think of an interface as a queue or pipe or stream of messages flowing between the domains; instead think of events and states and values as being shared between the connected domains. Each interface is an interface of shared phenomena.” This notion of shared phenomena has been difficult for me to wrap my head around. I don’t typically think about what’s going out there in the world when I'm focusing on writing usage descriptions. But Jackson’s viewpoint is slowly starting to influence my thinking. Sometimes eventful interjections can help explain why the user or the system is doing what they are doing and lift you out of writing uninformative “skin level deep” descriptions.