Azedi Technology

Here are some blogs I'm currently following about Lean proceses. Some are focused on software development, some cover lean processes more broadly.

These are blogs focused on Agile software development as well as Lean:

These are pulled dynamiclly from my Google Reader account, so will change whenever I change my subcriptions.

From Richard Waters at the FT, via Nicholas Carr.

According to Microsoft research chief Rick Rashid, around 20 per cent of all the servers sold around the world each year are now being bought by a small handful of internet companies - he named Microsoft, Google, Yahoo and Amazon. That is an amazing statistic, and certainly not one I’d heard before. And this is before cloud computing has really caught on in a big way.

Of course, this figure doesn't really prove a trend, the quote above is too vague. How big is a "small handful", and is this an increased concentration over previous time periods (and what is the time period we're talking about as "now"?) The implication is yes, and it is credible from what we're told about these companies' rapidly growing data centre expansion.

I went to last night's London CloudCamp, which I enjoyed quite a bit. I'm not going to go into pontifications about cloud computing - I think it's a brilliant concept, not mature yet, and an excellent option for online service startups.

A few random notes from the Camp:

I'm doing a bit of research into presentation technologies for use in J2EE apps, with accessibility as a particularly key requirement, and an eye on SOA, and extensibility (porlets or not?), among other things. I ran across a year-old article on TSS about a white paper and blog by Ganesh Prasad, Rajat Taneja and Vikrant Todankar.

I'm becoming a big fan of Lean Software Development, a particular strand of Agile Development methodologies, although we are a mostly-Scrum shop. I find Corey Ladas's description a very tidy explanation of the key philosophies of Lean:

T-Mobile has quite a scam going.

I took my new Nokia N95 with me to Turkey last month and played with the GPS and Google Maps quite a bit, and did the occasional browsing and twitter posting as well. Using maps on a phone chews up a lot of data since it constantly downloads images, so I made sure to only use it from WiFi spots at my in-laws' or friends' houses. I wasn't worried about accidentally downloading the images from a local phone network on 3G or GPRS since I never set up a local access point, and anyway I have my phone set to always prompt me to choose the data connection to use.

I was very pleased with my phone, bragging that WiFi and GPS were a surprisingly useful bell and whistle when used in combination abroad.

I got the phone bill yesterday. £50 in roaming data charges.

Apparently, T-Mobile tracks how much data you transfer over wifi, and bills you for it. Never mind that you are transferring data over your own network connection, nothing to do with T-Mobile other than having bought the phone from them.

Now I'm scared to use my phone. What if I agree a business deal over my t-mobile phone, will T-Mobile deduct a commission from my phone bill? I once negotiated a raise over my mobile, do I owe T-Mobile a percentage of my salary every month? Forever?

I found a forum post from someone who made the same discovery. I'm sure it's embedded in the Terms and Conditions.

I've been a T-Mobile customer for 10 years. Unfortunately, it's 16 months until I can freely switch. But I will.

We're looking to hire again. This time I've got 6 positions to fill in my team, and we've got 4 others.

In my team, we need a Lead/Manager for the Support Team, and a Support Engineer. This has proved a tricky position to specify, since these people need to be all-rounders. They need to be able to manage our infrastructure, from OS up, so they need to be strong on Linux, scripting, and "infrastructure" tools and concepts, like monitoring, configuration management, etc. But they also need to be able to support the applications themselves, which requires knowing about Java app servers (Tomcat in particular), databases, and web servers. On top of that, they need to be customer-support focused - they don't deal with the end users directly, but do need to deal with internal customers and key people, particularly technical contacts, at our customers and partners.

On top of that we need a Senior Java Architect and an Architect/Senior Developer type. These are again tricky, I firmly believe in Architects being hands-on, and we are an agile team, so they can't just be UML monkeys. But the core development team is offshore in Poland, and our software is complex, so it will be a challenge to get up to speed with it, and be able to interface between the non-technical people in London and the developers there, take ownership of planning a major revision. Cool roles really, but a fairly new thing for us, so we can't be 100% certain how the role will work in practice.

We're also going to hire a QA manager and test engineer.

Fun times here!

We need someone to take up the leadership of the London-based support team at the Map of Medicine, which manages and supports our server-based software products. This is a hands-on technical role suitable for someone with senior-level technical skills, ready to move into or improve in a team-lead position. We are also looking for a junior syadmin type, probably someone looking for their second job in the field. The contact email is tech dash jobs at map of medicine dot c o m. No recruiters wanted.

What the team does

The responsibilities for the support team involve:

• Delivering and supporting our applications to our customers (we provide second and third line support, first-line helpdesk support is handled by client organizations), internally and externally,
• Helping new and existing customers with the technical aspects of implementing and integrating our software into their systems and processes
• Running, improving, and expanding our hosting infrastructure, particularly as we grow internationally.

Our company combines the agility and rapid pace of a four year old software start-up with the stability and resources of an international, publically listed publishing company.

Details

We develop and provide a software service that serves as a healthcare knowledge source across multiple healthcare organisations. We deliver this as a hosted service, and also as software that partner organisations can host.

The Map of Medicine application suite is a Java-server based web application that runs on Linux or Windows on a Tomcat application server using a MySQL, SQL Server, or Oracle database.

Technologies we use

Our hosting infrastructure currently spans a production farm in redundant data centres, and a development/test "lab", and will expand to another data centre internationally. Our production systems are primarily Linux with Apache and Tomcat, with some Windows servers running SQL Server. Our test lab uses virtual servers on VMWare ESX, running Oracle, MySQL, and various other bits of technology.

Ensuring that our applications are highly available, reliable, and responsive is a key driver, so we implement monitoring, security, and centralized configuration management using mainly open source tools such as Nagios, Puppet, Subversion, and OpenLDAP, as well as custom scripts. Load balancing, database clustering, and data-centre failover are other cornerstones of our availability strategy.

We must be able to rapidly configure and deploy our applications for testing, pilot projects, and expansion. Centralized configuration management is essential to this, as is our use of virtualization technologies.

We also provide knowledge management services internally and externally, and provide leadership to the rest of the company on how to effectively use these. These are based around our wiki and bug tracker using Confluence and Jira.

The Role

The Support and Infrastructure Team Lead needs to be an experienced system administrator who can lead a team of 4+ in planning and implementing new technologies and expanding and improving use of existing technologies. This is a hands-on technical role. The Team Lead needs to be highly customer-focused, able to discuss daily operational problems and issues with non-business people and customers. They will also need to be able to think longer term and be involved in planning new applications and services and consult with client-organisations to plan their technical implementation of our software.

Required Skills

• Strong knowledge of Linux and Windows server operating systems,
• Strong knowledge of web-application technologies including Apache, load-balancing, virtual web-hosting, HTTP protocol,
• Knowledge of SQL databases, including installation and principles of tuning, and at least basic SQL,
• Experience in managing multi-server infrastructures including monitoring, configuration, user and host directory services (LDAP, DNS), shared storage, backups, etc.,
• Scripting languages (Bourne/Bash shell scripting, Perl),
• Excellent written and verbal communication,
• Experience working in a technical support environment, understanding of issue tracking systems, Service Level Agreements (SLAs), first/second/third line support arrangements and other technical support processes,
• Ability to manage and prioritize a fast-paced, rapidly changing workload keeping customer and business needs at the fore,

Desirable Skills

• Java application servers, particularly Tomcat,
• Server virtualization technology, particularly VMWare ESX,
• Shared storage, particularly iSCSI,
• Experience of healthcare systems or working with or in a clinical environment
• Presentation skills

I'd like to start on a set of rules for running and supporting onlines services in a way that takes advantage of lean production principles. This is carrying on the thoughts stirred up by my recent exposure to ITIL, which is pretty much the IT infrastructure equivalent of the waterfall development principle.

So the first pattern I came up with in that post was around keeping people involved throughout the process of planning, rolling out, and running a service, rather than having each of these things done by a different set of people, relying on the mythical "knowledge transfer" process. I'm sure there's a lot more to say on that one, but for the moment I'd like to get another idea out.

This pattern can be called DeployOften. I've found that a good rule in most areas of life is: if you find something difficult to get right, do it more often. This is obvious in some contexts - it's called practice - but in day to day business it goes against the grain to seek out painful tasks and repeat them more than is necessary to get through the job. The benefit is that the more you do something, the better you get at it, and it becomes less painful.

The principle of doing difficult things more often is found throughout agile development methodologies, an obvious example being test-driven development. Where testing is usually a painful and dull process, typically skipped or skimped, agile makes it a central behaviour, in fact the very first thing you should do when coding is to write up automated tests.

A difficult and painful part of service management is deploying new or updated software, part of what ITIL calls the transition phase. One organisation I know of tries to release updates to their software every 3 months, and each time it's a trial. Deploying the software to the server inevitably turns up surprises, and acceptance testing by users drags on with multiple rounds as updated releases are built to fix problems discovered.

This is in spite of automated nightly and iteration builds, which somehow never bring out the same issues that come out even on staging servers, which use snapshots of current live data sets, and more rigidly mimic the live deployment environment.

The DeployOften pattern suggests that the operations team should deploy each iteration build onto a production-like environment rather than waiting for the nearly-complete release. This will raise cries from the ops people, who don't exactly have light workloads already. But by deploying every two weeks they will get it down to a very quick process, and also turn up deployment problems much sooner in the development cycle, which should raise the developers' awareness of the kinds of things they need to keep in mind to make deployment easier.