I know this is getting old, but here's another nice example of why you should be using Free Software for everything that's really important. KVM works very well; I know it misses some of the nice, pretty interfaces but at least it won't stab you in the back at the next upgrade.
We were faced with a very similar dilemma when we set up our virtualized cluster... Do we go with a free open-source solution like KVM, forgoing an easy-to-use GUI, or do we go with a more expensive solution such as VMware?
We eventually decided to build a KVM-based cluster, and while we were already extremely glad we did before the vSphere 5 licensing change, this latest development only serves to confirm the wisdom of our choice. We have enterprise-level support if we need it, the virsh command-line interface is very straightforward and easy to pick up, and we have not shackled our fates to an organization that can yank the rug out from under us whenever they like. Moreover, for Linux-based folks who find themselves pondering their options, consider that VMware requires that you run Windows in your cluster if you want the full advantages of vSphere (e.g., live migration). Because of the added maintenance and security concerns, we were quite loathe to introduce any Windows operating systems into our cluster environment, and the unfortunate state of the VMware world is that it's extremely Windows-centric.
The best part? Not only do we have a fast, rock-solid virtualization solution in place, but the easy-to-use GUI we wanted is also on the horizon. Take one look at the Archipel Project at http://archipelproject.org/, and I think you'll agree its interface puts vSphere to shame. Archipel is not yet ready for critical production environments and (last I checked) currently lacks full support for libvirt-based storage APIs (e.g., for LVM-backed virtual storage pools), but development is progressing steadily. I, for one, am looking forward to getting the best of both worlds (liberated + easy-to-administer software) in our data center when the time is right. If that interests you, head on over to https://github.com/primalmotion/archipel and fork away!
What do you use to manage your KVM cluster? I'm not aware of any package that does that, although I haven't looked that deeply. I have been meaning to try out Eucalyptys/Ubuntu Enterprise Cloud (which sounds like similar functionality to a KVM cloud) but haven't done so yet.
If you have a blog, would you write something up about Archipel? That looks pretty damned nice, and I'd like to hear the perspective of an implementer (alongside reading docs).
I'm all about getting FOSS some visibility in my company.
Absolutely. I'm already half-way done writing an article on Archipel, and I'll be sure it gets posted to HN when it's ready.
As an aside, Antoine (the author of Archipel) just told me that beta 3 will be released next week, after which he'll be focusing on expanding the VM storage options.
I have two iSCSI Targets. I want to pull a LUN from each, and mirror on the KVM Host system, exposing the mirrored disks for my guests.
Compared to using ZFS on OpenSolaris with Xen 3.1, this process is incredibly cumbersome and unreliable.
Plus I just don't get the need for libvirt. It seems an incredibly complex and useless abstraction over a simply documented configuration file for Guests, and a tool to start/stop/add/remove them.
I'm sure a big part of that may just be that I'm not familiar enough with KVM. Still, it's 2011. Tying all your data to a single host with limited redundancy and depending on live-migration seems like a fundamentally flawed approach to me with a lot of needless complexity.
Libvirt is not needed to use KVM but the purpose has been to provide stability (and an abstraction as you mentioned) over the changes QEMU has had in various releases.
now, the version of KVM I'm using is really, really old, but as far as I can tell, KVM, by itself, does not do any locking. It's pretty easy to start a single guest twice when using kvm by itself, which will irreparably corrupt your instance.
So yeah, I'd strongly recommend that you use libvirt (or some other wrapper that handles things like locking the block devices) if you use KVM.
With xen, on the other hand, it handles that level locking for you out of box, so personally I see no reason to use libvirt. The libvirt devs seem pretty focused on KVM anyhow; Xen support, at least in the past, was pretty poor, so personally, I use the native xen tools for xen.
(I'm not saying this is a reason to use xen instead of KVM; I'm just saying that if you do use KVM, you should also use libvirt.)
> I have two iSCSI Targets. I want to pull a LUN from each, and mirror on the KVM Host system, exposing the mirrored disks for my guests.Compared to using ZFS on OpenSolaris with Xen 3.1, this process is incredibly cumbersome and unreliable.
I don't see the relation with KVM. There is nothing special in the way it uses open-iscsi or mdadm to access iSCSI targets and set up mirrors.
Maybe you're unfamiliar with these Linux-specific tools and mistakenly attribute to KVM your difficulties which are really Linux problems.
> Plus I just don't get the need for libvirt.
I don't use libvirt; I wrote a couple of scripts that allow me to do what I need with KVM. libvirt is useful only if you need to manage a whole lot of VMs.
No, I'm aware they're Linux problems. It just seems the mind-share is on KVM these days so I generally consider Linux Host Virtualization == KVM.
You're absolutely correct though. It's not really KVM's fault. It's the Linux iscsi, software RAID, and volume management capabilities that are so lacking.
It just happens that that makes using KVM much less appealing.
I want:
1. Reliable Snapshots and Replication
2. Simple Volume Management
3. Human readable device names
4. Consistency of volumes/devices between reboots
5. An "uncorruptable" FS backing the guests
6. Reliable Virtualization
OpenSolaris is the only platform I know addresses these concerns. Linux doesn't come close despite it feeling like it's virtualization options are more mature.
FreeBSD may be a good fit with VirtualBox, but VBox on OpenSolaris and OpenIndiana was unstable for me at least. I'd like to give FreeBSD a try sometime.
On libvirt, that's about what I expected. I guess I just wanted to complain about it in general. ;-)
"why you should be using Free Software for everything that's really important"
I'm sorry, but having regularly been in a situation where I'm trying to get sign-off on large tech project the first question is "what is the support package like" and the question never asked because caring about that kind of thing is below their pay grade is "but is this software free and open?"
That's because organizations like the one you work have governance models that aren't actually focused on delivering IT services.
If you use open-source software, you don't need to procure anything, right? At one level, that's great, but to people who run the contracts unit or procurement team, that doesn't compute. In their world, you hassle people for a discount and fight over contract terms.
Why do you think that Red Hat sells support contracts in the guise of a software license?
Answer: Because the processes that enable big companies/government to spend money on services are vastly different than software. On a services contract, you need to negotiate statements of work, etc. For a software contract, you just need to buy a software SKU and a maintenance/support SKU.
Small little detail about myself, I helped out in creating a unified driver package for ESXi and wrote a couple custom *nix net drivers for hardware VMWare wouldn't support (cheap commodity hw). So I agree with you that after I had ditched VMware for KVM I was in heaven. Shoutout to those ppl that used my stuff, and try KVM!
