I've been disappointed to see otherwise switched-on technical groups, and even high-priced 'managed hosting' companies, fail to take advantage of virtualization, even as they spend (and charge) loads of money to migrate physical servers onto virtualized infrastructures such as VMWare.

Moving from an OS running directly on hardware to an OS running on software that pretends to be hardware opens amazing possibilities, akin to the shift from paper to online data management. You're replacing immutable, physical servers with data, which means you can treat your infrastructure the same way you treat any other data - you can automate it, copy it, transfer it, and basically put it at the mercy of anything you can script.

While it's disappointing to see people use virtualization to replicate the experience of hosting on immutable, physical hardware, it's truly appalling to see hosting vendors offer this as a service, and add a premium price on top. It's only more annoying when they call it 'Cloud'.

I've recently seen a tender response from a big name, international hosting provider which basically offers to provide you with a couple of dedicated hardware boxes with ESX installed on them. Reasonable, if not ideal; obviously it's more powerful to have access to a pool of hardware resources that you share with other customers, so you can flex capacity when you need to, but there are reasons why a customer might not want to go this way.

What blew my mind was the way virtual machines on these boxes would be provided. For each VM, you pay a setup fee, and a monthly management fee. These fees are in pretty much the same ballpark as what you would pay for dedicated hardware for each of these VM's. But this company also charges a very hefty fee for each physical ESX box, so you're actually paying quite a bit more for N virtual machines than you would pay for N physical servers.

On top of this, you lose the flexibility that virtualization allows. Need to spin up a new image to prototype some changes to your application configuration? Request a price, get an invoice, raise a PO, pay a setup fee, and commit to three months of paying for the new image. Need to clone a running image to debug a production bug? Similar story. Expand capacity for a few weeks to support a marketing campaign? Implement an upgrade strategy that involves cloning, upgrading, testing, then swapping the clone into production? Even after you've jumped through the budgeting and purchasing hoops, you'll need to send your 'managed' hosting vendor a change request and wait a few days for their engineers to use the virtualization management console for you to carry out each step.

I'm not a Capability Maturity Model kind of guy, but I could see the benefit of having one for virtualization, to help enterprise CIO types understand what they should be demanding from vendors. The lowest level would involve using virtualization to replicate physical hardware, and the next would introduce flexibility in managing instances, supporting the types of use cases I described above. Higher levels of maturity would be more cloud-like, particularly around self service in provisioning images, flexible capacity management, and dynamic provisioning. I see higher levels of capability moving away from Infrastructure as a Service and towards providing a development and deployment platform that abstracts the details of servers, i.e. Platform as a Service.

A virtualization CMM would be grossly abused by marketers, but something like it might provide a few clues, and stomp out the practice of hosting providers offering virtualization without the benefits.

My first experience with the RedHat Network reminds me of a major limitation of commercial platforms that doesn't get much press. You actually get less than you do with free alternatives like apt-get and yum.

I'm setting up a new hosting infrastructure for a client which, among other things, involves moving from the free Fedora to commercial Redhat Linux. Although I've managed Redhat machines before, this is my first time using the Redhat Network (RHN) for installing and updating software.

In the past I've used apt-get on Debian, and yum on Fedora, and found them a godsend. Set up properly, it takes minimal effort to keep multiple systems up to date and consistent, whereas when I've had to go the "by-hand" route, machines invariably ended up with older versions of software. It's just too hard to keep up with all the various packages installed on various servers, not to mention the headache of chasing down various dependencies and resolving conflicts when you do upgrade or install a new package.

Why would someone pay £500 per month for a server when they could pay less than £100 from a different provider? The short answer is support. But it's a more complicated story than that.

One of my clients had a key server die this week, one of many they have with a cheap hosting provider. After investigating a bit, it was clear that it was not a system error - I was briefly able to examine the system logs using the provider's web based recovery tool, with no evidence of problems, but then the server stopped responding even to the tool.

So I called support. The first-line support guy verified that the machine wasn't responding and couldn't be recovered with the online tool, so he referred it to engineering at the data center.

However, he couldn't tell me when they would investigate the issue. It depends on how busy they are. With a real hosting provider, you get an SLA that includes response times. You also get someone who will communicate with you, to make you feel like they're on the case. I only got the promise that I would get an email when it was fixed, no direct contact, no phone call.

In the end it took three hours for them to fix the server. This isn't an unreasonable amount of time, given the cost of the service, but during those three hours my client decided that the three new servers they had decided to add that very morning should be gotten from another provider instead. They'll probably end up paying five times the price, but they know they'll get better support.

The key point here isn't that the service was poor. They fixed the problem fairly quickly. It isn't even just that they could not promise a response time - with thousands of customers paying less than a hundred pounds a month, they can't afford to make guarantees.

But even a cheap and cheerful support organisation should have a way to give updates on their progress, even if it is just via semi-automated emails. Let me know when your engineers have started investigating the problem. Let me know when they've identified the cause, and when they've fixed it.

The frosting with these guys came when they sent the form-email that the problem was fixed. I replied, asking if they could tell me what went wrong. The reply came the next day: look at your server logs.

That's lame. That's passing the buck. It's not in the server logs, because it's not an OS problem. Hard booting didn't fix the issue, and whatever did fix it didn't involve changing any configuration files or such on the server. So it was a hardware or network issue of some kind.

I guess I'll never know. And I guess I'll never use these guys again.

Here are some of the hosting providers I've worked with.

DreamHost

DreamHost offers shared hosting accounts with an amazing amount of functionality for a very low price. Lots of virtual domains, mysql instances, and tons of extras. They also have dedicated servers, but they seem to be in flux. When I looked at them a while ago they had dedicated Debian servers for under $100 per month, but these are now "coming soon", and the pricing for dedicated RedHat servers (as opposed to VPS servers) aren't promising. So I'm not sure if their dedicated servers are as great a value as their shared hosting.