Conceptual Integrity Part 1 ... or Why Committees Can't Do Doodly

Last night at DevTalk LA (that Geeky local book club), having finished our book, we read two articles. We do this in-between books, to allow members a little extra time to get the book in-hand. Next week, we'll start the eagerly anticipated RESTful Web APIs by Richardson & Amundsen.

Add caption

RESTful Web APIs

This book appears to be a complete rewrite of their earlier RESTFul Web Services (2007). It was, by universal acclamation of Dev Talk LA, one of the better books we've recently read, and advance opinion holds the re-write to take things on the next level and clear up all our questions.

The two articles can be (and should be!) read on-line here:

•  Design and Research by Paul Graham
•  Why Why Functional Programming Matters by Reg Braithwaite

We spent most of the time on Paul Graham's essay.

If you haven't read Paul Graham, you need to stop reading here and get on with it. Buy his book. Read his essays. Just do it. The short-version of why is this: Paul Graham is a twice-successful Startup enterpreneur who built his success upon technical excellence. He's a big proponent of the rapid development he found possible through the use of Lisp, of which his is an advocate and presently is designing/promoting Arc, a Lisp derivative of his own. But there are two additional factors that make him a compelling figure.

First, he's done something other than technology. He's also a painter. And the interplay of what he has learned from both shows clearly in his very insightful and reflective essays. Secondly, he understands, in a very no-nonsense way the value and importance of passion. From last nights essay, for instance, came this timeless quote:

To make something good, you have to be thinking, "wow, this is really great," not "what a piece of shit; those fools will love it." 

Which brings us to the point of today's essay.

Conceptual Integrity.

It's important.

Really important.

Paul Graham gets it, too.

As he puts it, (highlights mine)

Notice all this time I've been talking about "the designer." Design usually has to be under the control of a single person to be any good. And yet it seems to be possible for several people to collaborate on a research project. This seems to me one of the most interesting differences between research and design. 

There have been famous instances of collaboration in the arts, but most of them seem to have been cases of molecular bonding rather than nuclear fusion. In an opera it's common for one person to write the libretto and another to write the music. And during the Renaissance, journeymen from northern Europe were often employed to do the landscapes in the backgrounds of Italian paintings. But these aren't true collaborations. They're more like examples of Robert Frost's "good fences make good neighbors." You can stick instances of good design together, but within each individual project, one person has to be in control. 

I'm not saying that good design requires that one person think of everything. There's nothing more valuable than the advice of someone whose judgement you trust. But after the talking is done, the decision about what to do has to rest with one person.
Why is it that research can be done by collaborators and design can't? This is an interesting question. I don't know the answer. Perhaps, if design and research converge, the best research is also good design, and in fact can't be done by collaborators. A lot of the most famous scientists seem to have worked alone. [ed. note: see my "Never Hire The Greatest Scientist The World Has Ever Known"] But I don't know enough to say whether there is a pattern here. It could be simply that many famous scientists worked when collaboration was less common. 

Whatever the story is in the sciences, true collaboration seems to be vanishingly rare in the arts. Design by committee is a synonym for bad design. Why is that so? Is there some way to beat this limitation? 

I'm inclined to think there isn't-- that good design requires a dictator. One reason is that good design has to be all of a piece. Design is not just for humans, but for individual humans. If a design represents an idea that fits in one person's head, then the idea will fit in the user's head too.

I wanted to collect these ideas into one place and last night was a wonderful impetus to do so.  But this posting is already too long, so it looks like we have to kick off a series. And what a wonderful topic for a series! Conceptual Integrity. And, I'm very happy to have one of my heroes, Paul Graham, give as forceful and thoughtful a kickoff as could be imagined.

I Remain,

TheHackerCIO

The 10x Engineer (part 2)

[part 1 may be read here.]

ITWorld linked to TheHackerCIO's blog on "The 10x Engineer", can be read here. Rereading your own posting has a strange feeling. There's always so much more one could have written. But there is only so much time in the day for blogging. Or in the night, as the case may be. In fact, my blog postings are written at night, after a hard-days hacking for clients. But I pop in and re-read them quickly before posting, just to try to keep typos at bay.

The additional point I should have made about 10x Engineers are two-fold. First, there are plenty more anecdotal examples I could have specified. At one very recent client, their team-lead/head-developer  (let's pretend her name is Sally, so we can distinguish this anecdote from the previous instance which featured Bill!) easily produces more than all the rest of her team, which is on the order of 10.  I never got a full head-count, for several were remote, so I don't know exactly. But you get a sense of it from hearing the names and adding them into the known quantities in the home office.

The second point, which is far more important, derives from both this example and the previously cited one. In both cases, with Bill & Salley, they each  not only were at least an order-of-magnitude more productive, but also an order-of-magnitude harder workers! Any time I stopped into the office, no matter how late, Bill was hard at work, headphones plugged in, and plugging away. Sally seemed to live at the office, working often until midnight and at least one day on the weekend.

So there's a major cost to being a 10x engineer. You shoulder a great deal of the burden. You become Atlas.



I Remain,

TheHackerCIO

The 10x Engineer

Shanley attacks the "Myth" of the 10x Engineer in her blog.

 On the other hand, in Quora, there is an interesting discussion positing the existence of 100x and 1000x engineers.

Is this a myth?

One point Shanley makes is that:

There is no conclusive body of scientific research to suggest that the 10x engineer is in fact a real phenomenon, much less one that provides us with a deep and actionable understanding of the factors, conditions and stipulations of their existence. 

Make no mistake. I'm going to disagree with Shanley. But first, I'll partically agree. There is no scientific research to back up this notion, because controlling the variables is extremely difficult, if not impossible. Also, finding an objective measure of productivity is problematic.

That doesn't mean that the phenomenon is unreal. It means that science -- at present -- is unable to study it. There are plenty of things that fall outside of the purview of the Scientific Method of Experimentation, controlled variables, and double-blind-placebo-controlled systematic testing. Political behavior, for example, cannot be subjected to this, despite the misnomer of "Political Science."

However, as to the 10x engineer, I can cite plenty of anecdotal evidence: while recognizing that this is not scientific, nor an attempt to establish the fact scientifically. Using a concrete example to think about has a lot of positives advantages; keeping our ideas tied to reality being one of the most important!

At one client we had a Programmer/Analyst, Bill, we'll call him, who designed and wrote detailed specifications for approximately 60% of the modules to be developed. The remaining 40% were divvied up between the other 8 team members. And the complexity of the 60% was far in excess of anything found in the other 40%. Bill's were basically the crucial accounting and financials portion, while the rest were less important ancillary functions. Nice to have, but unessential. I got one of these areas -- how to monitor the Oil and Natural gas production relative to the lease provisions and actual cost of production.

That's pretty close to an order of magnitude more work being performed by Bill than by the rest of us. This was borne out by the programming staff who were ramped up on each team to code the final product. Bill's team was close to 20 or 25 (it's hard to specify exactly, because team size is not a constant value in most projects; they expand and contract).  The other teams all put together amounted to about the same.

That didn't mean that Bill was better than us. Yes, he was more productive. He got a whole lot more done than any of us. I didn't really envy him the position. He was the one who was always there late into the evening, while I went home to my wife. He was the one who had trouble getting away even for lunch, or a coffee break, while I headed out to Downtown Subscriptions for a premium Expresso.  But he was also recognized as indispensable to the project. At one point, his rate got increased above ours. But only about 10-15 % higher. Not enough to compensate him for the additional effort required, in my opinion. I don't think Bill was 10x better, whatever "better" means. He wasn't 10x more intelligent, either. But he was 10x more productive, there is 0 doubt.

But I didn't begrudge Bill his extra rate premium. I didn't resent him for the overtime hours. And I didn't envy the security he attained through his work. I merely reflected that with another such worker, the team would have been complete! Of course, it hadn't been possible to locate another such worker. And I don't know of any process or method for so doing. They seem to arise spontaneously. I can assure you that if I had interviewed Bill, I would never have guessed him to be a 10x outlier, in advance. I don't think Google would have selected him; I doubt he was very good at brain-busting puzzles. I don't think that having him write a spec at the white-board would have helped anyone spot the potential.

But Bill's accomplishment I celebrated and uphold to this day as spectacular, inspiring, and praise-worthy. Without it, the team would have needed another ten employees and the product would have been much worse, in accordance with the Cartesian Law that the more minds involved, the less the stamp of one clear integrating plan. [Cf. the Mediations] Why should Bill's accomplishment be denigrated, denied, or claimed to be mythical? The achievement was Herculean, heroic and Heroes should be rewarded.

In contrast, Shanley claims of the 10x Engineer, that:

It over-focuses on the role of the individual and individual contribution in success, reinforcing Silicon Valley’s tendency towards hero worship, elitism and destructive individualism while ignoring the context of situation and privilege.

This is bizarre. How can you possibly "over-focus" on individual contribution, when that is the basis for any collective accomplishment? What is wrong with worshiping Heroes for their accomplishments? What in the world has situation and privilege got to do with this? In the case I used above, "Bill," was in reality a minority and a woman, and hardly came from a privileged background. She was brought in at the beginning, just like everyone else, so there was no situation there. And she was told, "If the client doesn't like you, you're out of here, pronto!," as she related to me at one point. So, I'm sorry, I must have missed out from Shanley how I should not uphold the accomplishments of a minority female struggling against prejudice and stereotypes and yet producing an order of magnitude more work than anyone else on the project!

I'm sorry (OK, I'm not, but it sounds good, rhetorically) but ANY individual accomplishment an order of magnitude greater than the other team members is worthy of hero worship, worthy of considering that person an "elite," and far from destructive. In fact, it is pure inspiration for us to strive toward this as a goal. The more productive we are, the better our lives will be, and the better-off everyone will be.

So please, let's not have any more of this gratuitous attack on the "Lone Genius," or the "10x Engineer."  Let's recognize that there are amazing outliers. Let's celebrate them. Let's be encouraged to emulate them.

[part 2 may be read here.]

I Remain,

TheHackerCIO