kief's blog

Duct tape and technical debt

I received an email from my boss with a link to that Spolsky piece on duct tape programmers which had previously got my blood up. The boss obviously doesn't read my blog, but he's been around the block enough that he didn't take Spolsky's piece at face value.

Nevertheless, when Martin Fowler's recent post on technical debt came through my reader, I shot the link back to my boss.

Reckless debt may not be inadvertent. A team may know about good design practices, even be capable of practicing them, but decide to go "quick and dirty" because they think they can't afford the time required to write clean code. I agree with Uncle Bob that this is usually a reckless debt, because people underestimate where the DesignPayoffLine is. The whole point of good design and clean code is to make you go faster - if it didn't people like Uncle Bob, Kent Beck, and Ward Cunningham wouldn't be spending time talking about it.

Duct tape design topic (not Fowler's term, he doesn't refer to Spolsky's post at all) is only a quarter of the quadrant in Fowler's post, the whole post is worth reading.

The inverse to the technical debt metaphor is the investment metaphor, i.e. investing time to build a solid architecture, or refactor, or whatever. This metaphor is widely used, as it's a natural one. Of course, it potentially runs counter to the YAGNI principle. Interestingly, Martin Fowler is not an uncritical advocate of YAGNI, he expresses his ambivalence in his Is Design Dead piece.

I think it would be worth drawing up a quadrant for YAGNI v. BDUF similar to the one for technical debt. I'm not sure what the axes would be. One would probably be the level of knowledge confidence you have.

"Knowledge confidence" refers to how well you understand your requirements and the technologies and patterns you will use to address them, and the confidence you have that these will not change. In other words, are you sure whether you're gonna need it, or could it end up being de-scoped to hit a delivery date? What's the cost of putting in a simpler solution that won't necessarily address every requirement, then refactoring it as you reach that requirement in the backlog? How does it compare to the cost of developing a more complex solution up front that supports requirements you never get to, meaning there are requirements in between that get left out as well because you spent time on the complexity you didn't need?

The Engineer and Hacker Continuum

Joel Spolsky's recent piece on Duct Tape Programmers takes a characteristically extreme position that programmers who get stuff done simply and quickly are wonderful. I say characteristically extreme, because Spolsky has a columnist's approach to writing, i.e. take an extreme position that will get people worked up, although (actually because) it has obvious holes to pick.

I like to say that programmers fall on a Engineer / Hacker Continuum, and that each end has both positive and negative examples. On the engineering end, you have people who like to thoroughly think through what they are doing. The negative version of the engineer is the over-engineering architecture astronaut. They make everything harder than it needs to be. Their first reaction to any change request is that it's probably impossible, and in any case will be difficult and expensive, needing major parts of the system to be refactored, frameworks ripped out, etc.

On the opposite end of the spectrum is the Hacker. Managers love the hacker, because their first reaction to anything is that it'll be easy, and by the time the managers finish discussing whether to do it, the Hacker has already knocked it up. This is Joel's Duct Tape Programmer.

The evil side of the Hacker is that many (not all) of this type of programmer get things done quickly because they don't bother to think everything through, much less test it (Joel's dream programmer thinks unit tests are a luxury). They've coded it up, the code compiles, and they can click through the happy path on their development machine. Ship it! Oh, wait, I forgot to commit my code to the repository. Done, ship it!

Of course this results in bugs, customer complaints, and general disaster, at least if the company has any real customers. It works great for startups, because it gives them something to demo ("we have a working application"), and nobody is using it so the bugs don't matter.

So the positive flipside of the Engineer is that they take care, think things through, and design and build solid applications.

I've worked with Hackers who develop solid, clean, simple code, They're wonderful, they get stuff done quick. I've also worked with Engineers who keep their designs simple and effective. Also wonderful. But more often I've worked with developers who will slide into the negative versions of these if left to their own devices.

A development manager who follows Spolsky's principle of worshipping the Duct Tape Programmer, thinking they're always going to get the positive version of the Hacker work (code that's developed quickly, **and** is solid, fast, and scalable because it's been written simply and cleanly), are eventually going to end up tearing their hair out, not understanding why nobody can get the app to stop crashing, why new features take forever to develop even for their golden Hackers because nobody can untangle the spaghetti (in reality the Hackers end up writing new apps, because it's faster, which creates a new kind of spaghetti).

Is Spring still lightweight?

The Spring Framework emerged as the lightweight alternative to EJB for Java developers. It was simple, sensible, and had low overhead for designing, developing, and running applications. Over time it has grown from the platform of choice of subversive, technically-driven teams who want to get things done effectively, into the platform of choice for Enterprise Java development teams.

This is great, the best technology has won. But now Spring has become a serious company, and acquired by a bigger company which itself is majority owned by a massive company. And on the way, it's become something which smells to me like an Enterprise Solution.

I'm not talking about the corporate structure - I've got no problem with that in itself. And I'm not talking about the core framework itself, or even the family of extra components that have been acquired. I'm trying to get my head around the development suites, the application servers, the other application servers, (and now I see they even have a web server?).

Now, I have great respect for the guys who developed a lightweight Java platform that the pointy haired boss will sign off on using for corporate projects. If they are similarly packaging quality, useful open source software like Eclipse, Tomcat, and OSGi server, and so on, so that it makes life easier for designing, developing, and running well-engineered applications based on the principles of simplicity, then rock on, let me at it.

But I can't make heads or tails of the Spring Source suite of solution offerings. I have a look at the pricing, and I can't see the practical difference between these and offerings from IBM and the rest. What I do see is going down this road makes my cloud strategy awkward. I can't dynamically add and remove server images in response to useage requirements if I have to count the CPU's and pay a thousands of bucks every time I do so (waiting for the purchase to be approved by corporate each time). So it's not very lightweight from that perspective.

And I can't even work out what each Spring Source product does and whether it makes sense for me to even consider spending the cash, I quickly get bogged down by solution-speak every time I try to get a quick understanding. Then I saw a notice about Discovery Days, and hey! a day-long seminar which goes over the various Spring Source solution offerings and explains what the hell they are, sounds perfect. Oh, wait, it costs £400. They want me to pay to go to their sales presentation.

Bleh.

Chrome is not compelling enough for me. Yet.

I gave Google's Chrome web browser another go, and after a bit less than a week I've gone back to FireFox. There are two reasons for this.

First, although Chrome might be faster when using Google's online apps, more stable since each tab is a separate process, etc., from a user experience point of view there is simply nothing compelling enough to make me _need_ to keep using Chrome. It isn't ten times better than the alternatives in any aspect of its use that I really notice on a regular basis. Firefox doesn't crash very often on me.

The second reason is plugins. Firefox has loads of plugins, which in effect means it has loads more functionality than is available in Chrome. In practice, there are probably only two or three plugins that I truly rely on. The first time I tried Chrome, shortly after it's release, I used it at work, and not for very long, because of FireFox's built in functionality for password management, and the password timeout plugin to enhance it. I use a lot of passwords when browsing at work, for the various online tools, test deployments of software, etc. I can't remember them all, but I don't want to just let Chrome remember them, and make them available to anyone who gains accesses to my PC. So I use a master password, and a timeout plugin that makes sure the master password is required fairly frequently.

This time around I used Chrome at home, and it turns out I don't rely on FireFox plugins quite as much in that context. However, last night I was surfing and ended up delving into a thread of links that seemed quite interesting for work, and wanted to save them to Delicious. I'm used to using a FireFox plugin for this, and realized I'm incapable of living without it. That was the end of Chrome for me, for this round anyway.

Chrome will probably add a plugin API, but we'll see whether it helps it become more than a me-too in the browser market. Firefox has won a 22% share of the market by doing compelling things that IE didn't do at first, and didn't do well when they finally added them (tabbed browsing and plugins). Chrome's intended killer features seem to be that it runs complex online apps faster and more stably. Unfortunately for Chrome, FireFox is not currently slow or crashy enough to make this a clear win for most users, but perhaps as the online applications become increasingly complex it will matter.

Lean and Agile blogrolls

Here are some blogs I'm currently following about Lean proceses. Some are focused on software development, some cover lean processes more broadly.

These are blogs focused on Agile software development as well as Lean:

These are pulled dynamiclly from my Google Reader account, so will change whenever I change my subcriptions.

One indicator of the growth of the Cloud model

From Richard Waters at the FT, via Nicholas Carr.

According to Microsoft research chief Rick Rashid, around 20 per cent of all the servers sold around the world each year are now being bought by a small handful of internet companies - he named Microsoft, Google, Yahoo and Amazon. That is an amazing statistic, and certainly not one I’d heard before. And this is before cloud computing has really caught on in a big way.

Of course, this figure doesn't really prove a trend, the quote above is too vague. How big is a "small handful", and is this an increased concentration over previous time periods (and what is the time period we're talking about as "now"?) The implication is yes, and it is credible from what we're told about these companies' rapidly growing data centre expansion.

Cloud Camp London

I went to last night's London CloudCamp, which I enjoyed quite a bit. I'm not going to go into pontifications about cloud computing - I think it's a brilliant concept, not mature yet, and an excellent option for online service startups.

A few random notes from the Camp:

SOFEA, a flawed approach to a presentation architecture

I'm doing a bit of research into presentation technologies for use in J2EE apps, with accessibility as a particularly key requirement, and an eye on SOA, and extensibility (porlets or not?), among other things. I ran across a year-old article on TSS about a white paper and blog by Ganesh Prasad, Rajat Taneja and Vikrant Todankar.

Nice description of what Lean development is all about

I'm becoming a big fan of Lean Software Development, a particular strand of Agile Development methodologies, although we are a mostly-Scrum shop. I find Corey Ladas's description a very tidy explanation of the key philosophies of Lean: