Nevertheless, when Martin Fowler's recent post on technical debt came through my reader, I shot the link back to my boss.
Reckless debt may not be inadvertent. A team may know about good design practices, even be capable of practicing them, but decide to go "quick and dirty" because they think they can't afford the time required to write clean code. I agree with Uncle Bob that this is usually a reckless debt, because people underestimate where the DesignPayoffLine is. The whole point of good design and clean code is to make you go faster - if it didn't people like Uncle Bob, Kent Beck, and Ward Cunningham wouldn't be spending time talking about it.
Duct tape design topic (not Fowler's term, he doesn't refer to Spolsky's post at all) is only a quarter of the quadrant in Fowler's post, the whole post is worth reading.
The inverse to the technical debt metaphor is the investment metaphor, i.e. investing time to build a solid architecture, or refactor, or whatever. This metaphor is widely used, as it's a natural one. Of course, it potentially runs counter to the YAGNI principle. Interestingly, Martin Fowler is not an uncritical advocate of YAGNI, he expresses his ambivalence in his Is Design Dead piece.
I think it would be worth drawing up a quadrant for YAGNI v. BDUF similar to the one for technical debt. I'm not sure what the axes would be. One would probably be the level of knowledge confidence you have.
"Knowledge confidence" refers to how well you understand your requirements and the technologies and patterns you will use to address them, and the confidence you have that these will not change. In other words, are you sure whether you're gonna need it, or could it end up being de-scoped to hit a delivery date? What's the cost of putting in a simpler solution that won't necessarily address every requirement, then refactoring it as you reach that requirement in the backlog? How does it compare to the cost of developing a more complex solution up front that supports requirements you never get to, meaning there are requirements in between that get left out as well because you spent time on the complexity you didn't need?
]]>Nevertheless, when Martin Fowler's recent post on technical debt came through my reader, I shot the link back to my boss.
Reckless debt may not be inadvertent. A team may know about good design practices, even be capable of practicing them, but decide to go "quick and dirty" because they think they can't afford the time required to write clean code. I agree with Uncle Bob that this is usually a reckless debt, because people underestimate where the DesignPayoffLine is. The whole point of good design and clean code is to make you go faster - if it didn't people like Uncle Bob, Kent Beck, and Ward Cunningham wouldn't be spending time talking about it.
Duct tape design topic (not Fowler's term, he doesn't refer to Spolsky's post at all) is only a quarter of the quadrant in Fowler's post, the whole post is worth reading.
The inverse to the technical debt metaphor is the investment metaphor, i.e. investing time to build a solid architecture, or refactor, or whatever. This metaphor is widely used, as it's a natural one. Of course, it potentially runs counter to the YAGNI principle. Interestingly, Martin Fowler is not an uncritical advocate of YAGNI, he expresses his ambivalence in his Is Design Dead piece.
I think it would be worth drawing up a quadrant for YAGNI v. BDUF similar to the one for technical debt. I'm not sure what the axes would be. One would probably be the level of knowledge confidence you have.
"Knowledge confidence" refers to how well you understand your requirements and the technologies and patterns you will use to address them, and the confidence you have that these will not change. In other words, are you sure whether you're gonna need it, or could it end up being de-scoped to hit a delivery date? What's the cost of putting in a simpler solution that won't necessarily address every requirement, then refactoring it as you reach that requirement in the backlog? How does it compare to the cost of developing a more complex solution up front that supports requirements you never get to, meaning there are requirements in between that get left out as well because you spent time on the complexity you didn't need?
]]>I like to say that programmers fall on a Engineer / Hacker Continuum, and that each end has both positive and negative examples. On the engineering end, you have people who like to thoroughly think through what they are doing. The negative version of the engineer is the over-engineering architecture astronaut. They make everything harder than it needs to be. Their first reaction to any change request is that it's probably impossible, and in any case will be difficult and expensive, needing major parts of the system to be refactored, frameworks ripped out, etc.
On the opposite end of the spectrum is the Hacker. Managers love the hacker, because their first reaction to anything is that it'll be easy, and by the time the managers finish discussing whether to do it, the Hacker has already knocked it up. This is Joel's Duct Tape Programmer.
The evil side of the Hacker is that many (not all) of this type of programmer get things done quickly because they don't bother to think everything through, much less test it (Joel's dream programmer thinks unit tests are a luxury). They've coded it up, the code compiles, and they can click through the happy path on their development machine. Ship it! Oh, wait, I forgot to commit my code to the repository. Done, ship it!
Of course this results in bugs, customer complaints, and general disaster, at least if the company has any real customers. It works great for startups, because it gives them something to demo ("we have a working application"), and nobody is using it so the bugs don't matter.
So the positive flipside of the Engineer is that they take care, think things through, and design and build solid applications.
I've worked with Hackers who develop solid, clean, simple code, They're wonderful, they get stuff done quick. I've also worked with Engineers who keep their designs simple and effective. Also wonderful. But more often I've worked with developers who will slide into the negative versions of these if left to their own devices.
A development manager who follows Spolsky's principle of worshipping the Duct Tape Programmer, thinking they're always going to get the positive version of the Hacker work (code that's developed quickly, **and** is solid, fast, and scalable because it's been written simply and cleanly), are eventually going to end up tearing their hair out, not understanding why nobody can get the app to stop crashing, why new features take forever to develop even for their golden Hackers because nobody can untangle the spaghetti (in reality the Hackers end up writing new apps, because it's faster, which creates a new kind of spaghetti).
]]>I like to say that programmers fall on a Engineer / Hacker Continuum, and that each end has both positive and negative examples. On the engineering end, you have people who like to thoroughly think through what they are doing. The negative version of the engineer is the over-engineering architecture astronaut. They make everything harder than it needs to be. Their first reaction to any change request is that it's probably impossible, and in any case will be difficult and expensive, needing major parts of the system to be refactored, frameworks ripped out, etc.
On the opposite end of the spectrum is the Hacker. Managers love the hacker, because their first reaction to anything is that it'll be easy, and by the time the managers finish discussing whether to do it, the Hacker has already knocked it up. This is Joel's Duct Tape Programmer.
The evil side of the Hacker is that many (not all) of this type of programmer get things done quickly because they don't bother to think everything through, much less test it (Joel's dream programmer thinks unit tests are a luxury). They've coded it up, the code compiles, and they can click through the happy path on their development machine. Ship it! Oh, wait, I forgot to commit my code to the repository. Done, ship it!
Of course this results in bugs, customer complaints, and general disaster, at least if the company has any real customers. It works great for startups, because it gives them something to demo ("we have a working application"), and nobody is using it so the bugs don't matter.
So the positive flipside of the Engineer is that they take care, think things through, and design and build solid applications.
I've worked with Hackers who develop solid, clean, simple code, They're wonderful, they get stuff done quick. I've also worked with Engineers who keep their designs simple and effective. Also wonderful. But more often I've worked with developers who will slide into the negative versions of these if left to their own devices.
A development manager who follows Spolsky's principle of worshipping the Duct Tape Programmer, thinking they're always going to get the positive version of the Hacker work (code that's developed quickly, **and** is solid, fast, and scalable because it's been written simply and cleanly), are eventually going to end up tearing their hair out, not understanding why nobody can get the app to stop crashing, why new features take forever to develop even for their golden Hackers because nobody can untangle the spaghetti (in reality the Hackers end up writing new apps, because it's faster, which creates a new kind of spaghetti).
]]>This is great, the best technology has won. But now Spring has become a serious company, and acquired by a bigger company which itself is majority owned by a massive company. And on the way, it's become something which smells to me like an Enterprise Solution.
I'm not talking about the corporate structure - I've got no problem with that in itself. And I'm not talking about the core framework itself, or even the family of extra components that have been acquired. I'm trying to get my head around the development suites, the application servers, the other application servers, (and now I see they even have a web server?).
Now, I have great respect for the guys who developed a lightweight Java platform that the pointy haired boss will sign off on using for corporate projects. If they are similarly packaging quality, useful open source software like Eclipse, Tomcat, and OSGi server, and so on, so that it makes life easier for designing, developing, and running well-engineered applications based on the principles of simplicity, then rock on, let me at it.
But I can't make heads or tails of the Spring Source suite of solution offerings. I have a look at the pricing, and I can't see the practical difference between these and offerings from IBM and the rest. What I do see is going down this road makes my cloud strategy awkward. I can't dynamically add and remove server images in response to useage requirements if I have to count the CPU's and pay a thousands of bucks every time I do so (waiting for the purchase to be approved by corporate each time). So it's not very lightweight from that perspective.
And I can't even work out what each Spring Source product does and whether it makes sense for me to even consider spending the cash, I quickly get bogged down by solution-speak every time I try to get a quick understanding. Then I saw a notice about Discovery Days, and hey! a day-long seminar which goes over the various Spring Source solution offerings and explains what the hell they are, sounds perfect. Oh, wait, it costs £400. They want me to pay to go to their sales presentation.
Bleh.
]]>This is great, the best technology has won. But now Spring has become a serious company, and acquired by a bigger company which itself is majority owned by a massive company. And on the way, it's become something which smells to me like an Enterprise Solution.
I'm not talking about the corporate structure - I've got no problem with that in itself. And I'm not talking about the core framework itself, or even the family of extra components that have been acquired. I'm trying to get my head around the development suites, the application servers, the other application servers, (and now I see they even have a web server?).
Now, I have great respect for the guys who developed a lightweight Java platform that the pointy haired boss will sign off on using for corporate projects. If they are similarly packaging quality, useful open source software like Eclipse, Tomcat, and OSGi server, and so on, so that it makes life easier for designing, developing, and running well-engineered applications based on the principles of simplicity, then rock on, let me at it.
But I can't make heads or tails of the Spring Source suite of solution offerings. I have a look at the pricing, and I can't see the practical difference between these and offerings from IBM and the rest. What I do see is going down this road makes my cloud strategy awkward. I can't dynamically add and remove server images in response to useage requirements if I have to count the CPU's and pay a thousands of bucks every time I do so (waiting for the purchase to be approved by corporate each time). So it's not very lightweight from that perspective.
And I can't even work out what each Spring Source product does and whether it makes sense for me to even consider spending the cash, I quickly get bogged down by solution-speak every time I try to get a quick understanding. Then I saw a notice about Discovery Days, and hey! a day-long seminar which goes over the various Spring Source solution offerings and explains what the hell they are, sounds perfect. Oh, wait, it costs £400. They want me to pay to go to their sales presentation.
Bleh.
]]>First, although Chrome might be faster when using Google's online apps, more stable since each tab is a separate process, etc., from a user experience point of view there is simply nothing compelling enough to make me _need_ to keep using Chrome. It isn't ten times better than the alternatives in any aspect of its use that I really notice on a regular basis. Firefox doesn't crash very often on me.
The second reason is plugins. Firefox has loads of plugins, which in effect means it has loads more functionality than is available in Chrome. In practice, there are probably only two or three plugins that I truly rely on. The first time I tried Chrome, shortly after it's release, I used it at work, and not for very long, because of FireFox's built in functionality for password management, and the password timeout plugin to enhance it. I use a lot of passwords when browsing at work, for the various online tools, test deployments of software, etc. I can't remember them all, but I don't want to just let Chrome remember them, and make them available to anyone who gains accesses to my PC. So I use a master password, and a timeout plugin that makes sure the master password is required fairly frequently.
This time around I used Chrome at home, and it turns out I don't rely on FireFox plugins quite as much in that context. However, last night I was surfing and ended up delving into a thread of links that seemed quite interesting for work, and wanted to save them to Delicious. I'm used to using a FireFox plugin for this, and realized I'm incapable of living without it. That was the end of Chrome for me, for this round anyway.
Chrome will probably add a plugin API, but we'll see whether it helps it become more than a me-too in the browser market. Firefox has won a 22% share of the market by doing compelling things that IE didn't do at first, and didn't do well when they finally added them (tabbed browsing and plugins). Chrome's intended killer features seem to be that it runs complex online apps faster and more stably. Unfortunately for Chrome, FireFox is not currently slow or crashy enough to make this a clear win for most users, but perhaps as the online applications become increasingly complex it will matter.
]]>First, although Chrome might be faster when using Google's online apps, more stable since each tab is a separate process, etc., from a user experience point of view there is simply nothing compelling enough to make me _need_ to keep using Chrome. It isn't ten times better than the alternatives in any aspect of its use that I really notice on a regular basis. Firefox doesn't crash very often on me.
The second reason is plugins. Firefox has loads of plugins, which in effect means it has loads more functionality than is available in Chrome. In practice, there are probably only two or three plugins that I truly rely on. The first time I tried Chrome, shortly after it's release, I used it at work, and not for very long, because of FireFox's built in functionality for password management, and the password timeout plugin to enhance it. I use a lot of passwords when browsing at work, for the various online tools, test deployments of software, etc. I can't remember them all, but I don't want to just let Chrome remember them, and make them available to anyone who gains accesses to my PC. So I use a master password, and a timeout plugin that makes sure the master password is required fairly frequently.
This time around I used Chrome at home, and it turns out I don't rely on FireFox plugins quite as much in that context. However, last night I was surfing and ended up delving into a thread of links that seemed quite interesting for work, and wanted to save them to Delicious. I'm used to using a FireFox plugin for this, and realized I'm incapable of living without it. That was the end of Chrome for me, for this round anyway.
Chrome will probably add a plugin API, but we'll see whether it helps it become more than a me-too in the browser market. Firefox has won a 22% share of the market by doing compelling things that IE didn't do at first, and didn't do well when they finally added them (tabbed browsing and plugins). Chrome's intended killer features seem to be that it runs complex online apps faster and more stably. Unfortunately for Chrome, FireFox is not currently slow or crashy enough to make this a clear win for most users, but perhaps as the online applications become increasingly complex it will matter.
]]>These are blogs focused on Agile software development as well as Lean:
These are pulled dynamiclly from my Google Reader account, so will change whenever I change my subcriptions.
]]>These are blogs focused on Agile software development as well as Lean:
These are pulled dynamiclly from my Google Reader account, so will change whenever I change my subcriptions.
]]>According to Microsoft research chief Rick Rashid, around 20 per cent of all the servers sold around the world each year are now being bought by a small handful of internet companies - he named Microsoft, Google, Yahoo and Amazon. That is an amazing statistic, and certainly not one I’d heard before. And this is before cloud computing has really caught on in a big way.
Of course, this figure doesn't really prove a trend, the quote above is too vague. How big is a "small handful", and is this an increased concentration over previous time periods (and what is the time period we're talking about as "now"?) The implication is yes, and it is credible from what we're told about these companies' rapidly growing data centre expansion.
]]>According to Microsoft research chief Rick Rashid, around 20 per cent of all the servers sold around the world each year are now being bought by a small handful of internet companies - he named Microsoft, Google, Yahoo and Amazon. That is an amazing statistic, and certainly not one I’d heard before. And this is before cloud computing has really caught on in a big way.
Of course, this figure doesn't really prove a trend, the quote above is too vague. How big is a "small handful", and is this an increased concentration over previous time periods (and what is the time period we're talking about as "now"?) The implication is yes, and it is credible from what we're told about these companies' rapidly growing data centre expansion.
<!--break-->
A few random notes from the Camp:
]]>A few random notes from the Camp:
I enjoyed Neil Bartlett's lightning talk about OSGi and the Cloud. I'm gearing up for an OSGi pilot project at the moment, and am hoping to attend Neil's course at Skills Matter next week, but hadn't thought through the usefulness of OSGi in a cloud. Given that getting the full value out of a cloud infrastructure means being able to rapidly and automatically provision an arbitrary number of servers, and then keep them all updated, this is a natural fit. It's a shame he didn't manage to finish in 5 minutes.
My notes are poor, but I think it was a guy from Qwest Software who did an excellent job of making a very key point for people hoping to move their company onto a cloud infrastructure. In brief, you are not going to get the approval from the C?Os unless you can prove that it will be cheaper than your current infrastructure. And you can't do that if you can't tell them how much your current infrastructure currently costs in the same terms as the Cloud, i.e. how much does it cost to add a new service, including not just hardware and vendor costs but the support and other costs that go along with it. He asked how many people in the room can identify that cost with their current infrastructure right now, and I didn't see anyone raise their hand. Hmm.
I was amused when they took a voice vote on the number of people planning to attend each of the breakout sessions. It sounded like there were about the same number of people going to 'Open Cloud' as going to 'Enterprise Cloud', but the latter group sounded much less enthusiastic.
]]>The basis of the article is that for the purposes of the presentation layer, the architecture should consider three elements, application download (which may be loading a web page, or downloading a rich client), presentation flow, and data interchange. They essentially argue that these must all come together in the client, thus reconciling the thick-vs-thin client camps by effectively defining server-side control of presentation as an anti-pattern called Front Controller.
I tend to agree with two of the commenters on the TSS article, Keith Weinberg and Ivan Smirnov, in that SOFEA is an approach which works in some circumstances, not all.
Actually, I would say the conceptual split between application, presentation, and data defined by the SOFEA paper is fine and useful, a slightly different approach to describing MVC. But the authors are quite insistent that the MVC can only live in the browser using AJAX or else an actual thick client. This is an arbitrary definition which, if followed dogmatically, means SOFEA is useless in a number of situations, including my own.
The number one reason is accessibility, which I mentioned above as a requirement. SOFEA requires all users to use a browser which fully supports the flavour of AJAX used by the application, which cuts out a number of devices and browsers (most mobile phones, for instance, and screen readers) and users in locked-down IT environments.
The power of assembling the presentation on the server-side is that it can be delivered to a broad range of devices. I don't quite buy the idea (as the SOFEA authors responded to the above posters) that the presentation sitting anywhere but the browser automatically classifies it as an anti-pattern. The browser is an arbitrary place to draw the line, if the architecture of an application joins the application, presentation, and data following a "good" design pattern, the fact that it does so on computer A rather than computer B is irrelevant.
]]>]]>People with different skills have to work together to deliver product features. Don't build features that nobody needs right now. Don't write more specs than you can code. Don't write more code than you can test. Don't test more code than you can deploy.
I took my new Nokia N95 with me to Turkey last month and played with the GPS and Google Maps quite a bit, and did the occasional browsing and twitter posting as well. Using maps on a phone chews up a lot of data since it constantly downloads images, so I made sure to only use it from WiFi spots at my in-laws' or friends' houses. I wasn't worried about accidentally downloading the images from a local phone network on 3G or GPRS since I never set up a local access point, and anyway I have my phone set to always prompt me to choose the data connection to use.
I was very pleased with my phone, bragging that WiFi and GPS were a surprisingly useful bell and whistle when used in combination abroad.
I got the phone bill yesterday. £50 in roaming data charges.
Apparently, T-Mobile tracks how much data you transfer over wifi, and bills you for it. Never mind that you are transferring data over your own network connection, nothing to do with T-Mobile other than having bought the phone from them.
Now I'm scared to use my phone. What if I agree a business deal over my t-mobile phone, will T-Mobile deduct a commission from my phone bill? I once negotiated a raise over my mobile, do I owe T-Mobile a percentage of my salary every month? Forever?
I found a forum post from someone who made the same discovery. I'm sure it's embedded in the Terms and Conditions.
I've been a T-Mobile customer for 10 years. Unfortunately, it's 16 months until I can freely switch. But I will.
]]>In my team, we need a Lead/Manager for the Support Team, and a Support Engineer. This has proved a tricky position to specify, since these people need to be all-rounders. They need to be able to manage our infrastructure, from OS up, so they need to be strong on Linux, scripting, and "infrastructure" tools and concepts, like monitoring, configuration management, etc. But they also need to be able to support the applications themselves, which requires knowing about Java app servers (Tomcat in particular), databases, and web servers. On top of that, they need to be customer-support focused - they don't deal with the end users directly, but do need to deal with internal customers and key people, particularly technical contacts, at our customers and partners.
On top of that we need a Senior Java Architect and an Architect/Senior Developer type. These are again tricky, I firmly believe in Architects being hands-on, and we are an agile team, so they can't just be UML monkeys. But the core development team is offshore in Poland, and our software is complex, so it will be a challenge to get up to speed with it, and be able to interface between the non-technical people in London and the developers there, take ownership of planning a major revision. Cool roles really, but a fairly new thing for us, so we can't be 100% certain how the role will work in practice.
We're also going to hire a QA manager and test engineer.
Fun times here!
]]>The responsibilities for the support team involve:
Our company combines the agility and rapid pace of a four year old software start-up with the stability and resources of an international, publically listed publishing company.
We develop and provide a software service that serves as a healthcare knowledge source across multiple healthcare organisations. We deliver this as a hosted service, and also as software that partner organisations can host.
The Map of Medicine application suite is a Java-server based web application that runs on Linux or Windows on a Tomcat application server using a MySQL, SQL Server, or Oracle database.
Our hosting infrastructure currently spans a production farm in redundant data centres, and a development/test "lab", and will expand to another data centre internationally. Our production systems are primarily Linux with Apache and Tomcat, with some Windows servers running SQL Server. Our test lab uses virtual servers on VMWare ESX, running Oracle, MySQL, and various other bits of technology.
Ensuring that our applications are highly available, reliable, and responsive is a key driver, so we implement monitoring, security, and centralized configuration management using mainly open source tools such as Nagios, Puppet, Subversion, and OpenLDAP, as well as custom scripts. Load balancing, database clustering, and data-centre failover are other cornerstones of our availability strategy.
We must be able to rapidly configure and deploy our applications for testing, pilot projects, and expansion. Centralized configuration management is essential to this, as is our use of virtualization technologies.
We also provide knowledge management services internally and externally, and provide leadership to the rest of the company on how to effectively use these. These are based around our wiki and bug tracker using Confluence and Jira.
The Support and Infrastructure Team Lead needs to be an experienced system administrator who can lead a team of 4+ in planning and implementing new technologies and expanding and improving use of existing technologies. This is a hands-on technical role. The Team Lead needs to be highly customer-focused, able to discuss daily operational problems and issues with non-business people and customers. They will also need to be able to think longer term and be involved in planning new applications and services and consult with client-organisations to plan their technical implementation of our software.
The responsibilities for the support team involve:
Our company combines the agility and rapid pace of a four year old software start-up with the stability and resources of an international, publically listed publishing company.
We develop and provide a software service that serves as a healthcare knowledge source across multiple healthcare organisations. We deliver this as a hosted service, and also as software that partner organisations can host.
The Map of Medicine application suite is a Java-server based web application that runs on Linux or Windows on a Tomcat application server using a MySQL, SQL Server, or Oracle database.
Our hosting infrastructure currently spans a production farm in redundant data centres, and a development/test "lab", and will expand to another data centre internationally. Our production systems are primarily Linux with Apache and Tomcat, with some Windows servers running SQL Server. Our test lab uses virtual servers on VMWare ESX, running Oracle, MySQL, and various other bits of technology.
Ensuring that our applications are highly available, reliable, and responsive is a key driver, so we implement monitoring, security, and centralized configuration management using mainly open source tools such as Nagios, Puppet, Subversion, and OpenLDAP, as well as custom scripts. Load balancing, database clustering, and data-centre failover are other cornerstones of our availability strategy.
We must be able to rapidly configure and deploy our applications for testing, pilot projects, and expansion. Centralized configuration management is essential to this, as is our use of virtualization technologies.
We also provide knowledge management services internally and externally, and provide leadership to the rest of the company on how to effectively use these. These are based around our wiki and bug tracker using Confluence and Jira.
The Support and Infrastructure Team Lead needs to be an experienced system administrator who can lead a team of 4+ in planning and implementing new technologies and expanding and improving use of existing technologies. This is a hands-on technical role. The Team Lead needs to be highly customer-focused, able to discuss daily operational problems and issues with non-business people and customers. They will also need to be able to think longer term and be involved in planning new applications and services and consult with client-organisations to plan their technical implementation of our software.
So the first pattern I came up with in that post was around keeping people involved throughout the process of planning, rolling out, and running a service, rather than having each of these things done by a different set of people, relying on the mythical "knowledge transfer" process. I'm sure there's a lot more to say on that one, but for the moment I'd like to get another idea out.
This pattern can be called DeployOften. I've found that a good rule in most areas of life is: if you find something difficult to get right, do it more often. This is obvious in some contexts - it's called practice - but in day to day business it goes against the grain to seek out painful tasks and repeat them more than is necessary to get through the job. The benefit is that the more you do something, the better you get at it, and it becomes less painful.
The principle of doing difficult things more often is found throughout agile development methodologies, an obvious example being test-driven development. Where testing is usually a painful and dull process, typically skipped or skimped, agile makes it a central behaviour, in fact the very first thing you should do when coding is to write up automated tests.
A difficult and painful part of service management is deploying new or updated software, part of what ITIL calls the transition phase. One organisation I know of tries to release updates to their software every 3 months, and each time it's a trial. Deploying the software to the server inevitably turns up surprises, and acceptance testing by users drags on with multiple rounds as updated releases are built to fix problems discovered.
This is in spite of automated nightly and iteration builds, which somehow never bring out the same issues that come out even on staging servers, which use snapshots of current live data sets, and more rigidly mimic the live deployment environment.
The DeployOften pattern suggests that the operations team should deploy each iteration build onto a production-like environment rather than waiting for the nearly-complete release. This will raise cries from the ops people, who don't exactly have light workloads already. But by deploying every two weeks they will get it down to a very quick process, and also turn up deployment problems much sooner in the development cycle, which should raise the developers' awareness of the kinds of things they need to keep in mind to make deployment easier.
]]>My focus has been much more on the technical than on the people processes for running and supporting the infrastructures. In my current job, the team I've brought in and the infrastructure we've built has reached a decent level, although there's certainly plenty more to do technically. But looking at what we've done and what I want to do next, I've realized thatrather than moving on and doing the same thing at another company, the more interesting challenge will be to take things to the next level.
The next level for me is going beyond the technical to focus on the people and processes. The technology infrastructure is going to grow in size and sophistication, including spreading to multiple data centres globally, but the technical challenges seem like more of the same to me. The challenges that seem newer and more intriguing to me personally are more along the lines of how the hell we're going to organize and coordinate people doing development, infrastructure, and support in three, four, or more countries.
So I went on a course in ITIL version 3. Yikes. ITIL is basically a blueprint for organizing a huge IT operation with lots of bureaucratic processes, forms, and signoffs that will make it nearly impossible to get anything done, and ensure that responsibilities are divided so that nobody who is doing anything productive sees the big picture.
I don't think it has to be this way. I actually did find the course useful, although not as useful as it could have been given that most of the people on it were more interested in ticking off the certification than getting ideas on how to improve the organisations they work at. There were some pretty interesting people there, some of whom were obviously interested in fixing real problems "back home". If the course had been more of a workshop where we shared war stories and ideas, it would have rocked.
A lot of the concepts in ITIL are useful, I think it's more a matter of using your head when applying them, making sure to adapt the ideas in ways that fit your needs and objectives. It's very easy to see how an organisations, especially large ones, take the ITIL material and use it to build horribly inefficient IT structures. I've worked with companies that use ITIL this way, and the course shed light on how they got this way.
The biggest problem with ITIL is that it's presented with clearly defined "phases" of strategizing, desiging, deploying ("transitioning"), operating, and improving IT services. This is an invitation to a waterfall model, where (as in at least one organization I know of) each phase can even be run by a completely different team of people.
So one group designs the service, hands it off to another than rolls it out (tests and installs it), and then hands off to a completely separate team that supports it. In the organisation I've encountered, the operations team hasn't got the vaguest clue about the service.
Of course the transition process involves "knowledge transfer" where the people who set up the service train the support team, but anybody who's done this stuff in the real world should know better.
Knowledge that is transferred in a handover process is never, ever, ever going to be learned as well as knowledge that comes from actually being involved throughout the whole process. Having some hands-off manager (ahem) overseeing things all the way through doesn't cut it, the people who will actually be diving into runtime problems with an application need to have gotten their hands dirty trying to install the application, and even have pitched into meetings where the details of how the application should be integrated into the infrastructure.
Otherwise, you're going to end up in the situation of my nameless organisation, one which is actually often held up as an exemplar of ITIL. They host an application on their servers, installed by the transition team, and their support team had training on how to log into the server and investigate problems with it. But when users call up with problems, the support people, who probably support dozens of applications, have forgotten all of this. They call up the software vendor - who have no access to the servers.
Can you imagine how incompetent your organisation looks when it's clear that your support people have no idea that the application they support is run by their own company?
But I do think it's possible to take many of the ideas of ITIL and apply them in a more agile manner. A bit of Googling shows I'm not the only one who thinks so, but that there doesn't seem to have been much work done on the idea, at least publicly. It's certainly something that would take a bit of thought and work.
My first thought, clearly, is that an agile IT services process would have to embrace the lean management principle of empowerment by having the "workers" (for lack of a better word) involved throughout the process.
I've also thought that the kanban approach to agile is paricularly suited to a sysadmin team, since it does away with the iterations/release cycle in favor of a queue of tasks that people pull from when they find they've got spare capacity.
Anyway, I'm looking forward to thinking this stuff through and trying out ideas over the next year. Although I'm going to be far less hands-on technically, my focus does need to involve a thorough understanding of the technical aspects of what we're doing, so I don't think I'm going to become a total suit.
]]>