Software Architecture

Today, even the notion that there should be software architecture is up for debate! Regardless of what verdict one arrives at,  it is a good thing to examine whether something should exist prior to investing time studying it.

[editorial note: this post commences an occasional series on Software and Enterprise Architecture, in which I hope to collect all the principles into one convenient place. We begin at the beginning: should there even BE a discipline of software architecture? ]

There are a number of "nay-sayers." But I think they can all be classified as variants on the notion of "evolutionary design," which contrasts strongly with the notion of engineering, design, and deliberative architecture. The idea is that design is an emergent property that will manifest itself as the work unfolds. The Wikipeida article puts it this way:

Emergent design in agile software development[edit]

Emergent design is a consistent topic in agile software development, as a result of the methodology's focus on delivering small pieces of working code with business value. With emergent design, a development organization starts delivering functionality and lets the design emerge. Development will take a piece of functionality A and implement it using best practices and proper test coverage and then move on to delivering functionality B. Once B is built, or while it is being built, the organization will look at what A and B have in common and refactor out the commonality, allowing the design to emerge. This process continues as the organization continually delivers functionality. At the end of an agile release cycle, development is left with the smallest set of the design needed, as opposed to the design that could have been anticipated in advance. The end result is a smaller code base, which naturally has less room for defects and a lower cost of maintenance.^[1]

As emergent design is heavily dependent upon refactoring, practicing emergent design without a comfortable set of unit tests is considered an irresponsible practice.^{[citation needed]}

I have to laugh at the last line: "practicing emergent design without .. .[snip] ... is considered an irresponsible practice." It reminds me of the Moliere comedy, Le Bougeoise Gentilhomme, where the man finds he has been speaking prose all his life and didn't even know it. Likewise, many a team has been practicing "emergent design" for decades and thought they just hadn't used an architect.

There are a lot of parallels here with methodology pathology. Just adopting an agile or iterative methodology is not a silver bullet.  Process does not automate success! [The Rothering Principle] TheHackerCIO has seen plenty of projects with churning iterations that never seem to get anywhere; just as he has seen Waterfall projects that can't get out of one phase and into another, or that allow phase leakage. Process alone cannot ensure the absence of pathology. The Capability Maturity Model of Carnegie-Mellon seemed like a good thing, and I was a proponent of it, until I worked with outsourcing companies in a particular subcontinent (which will remain unnamed), all of whom were CMM level-5 certified, and none of whom could deliver one useable module of code to our project. I'd love to have seen a post-mortem analysis of how their coding failures made it back into the "feedback-loop" of their CMM program!

One manifestation of this attack is the TDD methodology. Taken rigorously, it directly assails the need for architecture: you are to take your assigned User Story from your Product Backlog, and after coding your first test case -- mind you, not one line of actual code has yet been written!!! -- after coding that test case, you now enter the "red" state, and are to determine ... what?

• The best way to structure your code to allow for future eventualities? No! 
• The best approach to code this method, so that you can reuse existing code? No!
• The most elegant set of methods that will allow for maximal flexibility? No! 
• A higher degree of generality to accommodate use cases that are going to come? No!

The principle adhered to here is YAGNI!

In contrast to attempting to architect, engineer, or design, one is supposed to take the directest, clearest, fastest path to implement the user story, at least a portion of the user story, and nothing but the user story. You are to "just do it," as the Nike slogan goes, and when your test-case passes, you enter the "green" state. At this point, you are to consider refactoring the code you just wrote, to achieve whatever an Architect or designer might have attempted upfront.

This is a direct assault on the notion of architecture, or on the need for it.  There are others.

Next up: Patterns and Pattern Languages

I Remain,

TheHackerCIO

E Versus T at a Riot

CEOs in one panel discussed with CTOs in another at last Friday's LACTO Forum. A kind of "E" versus "T." Surprisingly, both groups agreed.

A major discussion point centered around CEOs influence in the company. Especially, about the unintended consequences of their comments. It's almost as if CEOs need to distinguish between ex-cathedra pronouncements, and ober dicta. For those of you who aren't lawyers, and TheHackerCIO hopes that's everyone: ober dicta are statements said in passing -- they are remarks, or observations, -- and are not legally binding. Both the CEOs and CTOs were troubled by how such a passing remark could affect the direction of the team.

If this new direction is not communicated to the rest of the team, then it can result in different people or groups pulling in differing directions. The CEOs were troubled by this unintended consequence. The CTOs suggested that it was the duty of everyone, and especially senior management to re-validate such directional messages across the organization. That way, by enhancing communication, the CTO can at least know if a questionable direction has been posited: he can investigate in private with the CEO, if necessary, and a formal position can then be established for everyone.

The CEOs further lamented that this can even extend to unintended consequences from their seeming enthusiastic about something!

Personally, I've never experienced this problem, but it's good to file away for the future.

The CTOs had an interesting technique for communicating with the CEO about a proposal. So long as it's not wrong headed altogether, instead of saying "no," just say, "OK, but that will take 4 more engineers and we will deliver 4 months later than scheduled." This is an excellent bit of advice, and reminds me of the negotiations used in Agile programming.

The event was hosted by Riot Games, who are awesome -- both for hosting us, and for their well -organized development. They provided a tour to those interested, and TheHackerCIO is always interested in technology tours! This one is particularly heartening, because the Agile artifacts are all up on the walls everywhere you go. Post-its are in particular places to indicate remaining work and burn-down charts track the progress on every sprint. The tour-guide mentioned that in their post-mortem analyses, every time they missed out on cross-functional teams, they had problems. It's kind of cool at Riot that you can see the whole pipeline of their work as you move down the hallway: from creative inception and content development, to more technical elaboration and finally to testing/ playing. They even have simulated cafes -- because many of their game users have their User Experience in a computer cafe.

All in all, a most profitable session!

All The News That's Fit To Piss Me Off

The Love-of-my-life asked me, "Don't you run out of things to write about? How do you pick what to blog about?" And TheHackerCIO replied to her, "I write about whatever pisses me off!

"And that's an renewable resource."

Inexhaustible.

To adapt the banner tag from the New York Times, TheHackerCIO blogs "All the News That's Fit to Piss Me Off."

One of my business development partners told me, "At client X, it seems that the every project originates from anger! Someone important gets upset and then a project to correct it is started."

It might not be the best way to organize development management, but it definitely ties requirements back to values and priorities.

These two stories illustrate a central point about development. It is the frustration of values that brings about the need for creativity and work. That's why requirements are inexhaustible. So long as not everything imaginable is now achievable, we have work. We have requirements. We have User Stories; and a Product Backlog. And the only way to level these requirements to the time available for programming is by the mechanism of cost. The inexorable law of supply and demand is what reduces the infinite demand for new features to the limited supply of programmer labor. The Agile technique of bidding points -- a form of estimation -- against User Stories is a wonderful mechanism for directly linking the demand for requirements to the supply of programmer labor.

Since programmer's backgrounds are not all created equal and since the "requirements" are not standardized commodity contracts, like Pork-Belly Futures Contracts on the Chicago Mercantile Exchange, we need a way to standardize them and price them. And this Agile technique is the best such pricing mechanism I've seen in any methodology. It directly addresses all the issues, and with a direct tie back to the bidder. The programmer has  "skin in the game." The winner, after all, must deliver the User Story in the time he bid.

But this doesn't reduce the total anger, because every new feature leads to the imagination of  ten more! Or a hundred!

Anger is a beautiful thing.

I Remain, Ever Angrily,

TheHackerCIO