Of virtual machines and gained productivity - and hardware

Of virtual machines and gained productivity - and hardware


I spend most of my time doing Web programming - basically, tinkering and cleaning up some professional websites that require maximum accessibility, and efficient coding while remaining very simple. This needs XHTML+CSS+ECMAscript and some PHP glue; and while I have no problem running a LAMP test server on my main machine, up until now I needed a spare machine just to do testing under Windows.

Not anymore.

(Revised: some typos, missing brackets, and an 'extra' on kqemu configuration)

Virtual Goodness

A Faint Hope

Beforehand, I used a spare machine, some refurbished stuff that was supposed to move to my boss' wife's medical cabinet but remained under my desk for months before I decided to use it to do some IE ugliness testing.

In short, throwing CSS and Jscript hacks around to make my websites work as similarly as possible under Internet Explorer 6 as it did under W3C-compliant browsers while keeping those free of intrusive IE-only code. With the existence of IE7, I would have even needed two of those machines...

Luckily, since I wasn't the only one in this situation, there are already ways to at least test a website using several IEs under a single Windows machine; and the spare was used to run Windows XP (it came with it), with IE7 installed and IE 4,5 and 6 as 'standalone' installs. From 4 to 1 extra machines needed for testing: sweet.

But then, that machine was needed for a temporary worker in-shop: since I couldn't scrap enough budget to buy a new machine, my test machine had to be freed and formatted like all other machines in the shop (that's another part of my job). As such, no more multiple IE test machine; back to square one.

Holy Grail

Having read about virtual machines, I started looking for more data; what softwares were available, easy to use, and the least expensive?

Right at that moment, the Mandriva French user mailing list got a post saying that Kqemu was finally GPL'ed, and not a binary blob with glue as it used to be.

I knew about Qemu, which is Fabrice Bellard's work on a multi-machine, multi-purpose emulator originally based on the Bochs emulator (but now radically different); learning more about the Qemu kernel module (kqemu), I was surprised to learn that it allowed almost native speed emulation of complete x86 machines on x86-based GNU/Linux systems.

So I decided to give it a test drive, first by installing the version bundled with my distribution, then using the latest tarballs.

Frankly, I was very soon amazed, but it required some tinkering on my part.

The Quest for Ultimate Performance

First, I had to create the virtual machine; that requires creating a virtual drive image that will be stocked in a single, huge file; the rest of the system is emulated by Qemu. Basically,

  • my current CPU: an Athlon64 2600+ (with some extra clock speed, however the overclock was prompted by Gimp sluggishness, not Qemu)
  • an Intel 440FX chipset (probably the only way you'll see a 686 AMD on an Intel chipset - heh)
  • a Cirrus Logic 5446 card (can be replaced with a generic VESA 2.0 card for higher resolutions)
  • an NE2000 compatible card
  • a CD-ROM reader (automatic since version 0.9)
  • an ACPI controller

Additionally, I could have added:

  • a USB controller
  • a sound card: PC speaker, Sound Blaster 16 or Ensoniq AudioPCI ES1370
  • multiple core emulation

Reading the documentation extensively, I decided to emulate a single core machine with no USB, no sound, 300 Mb of RAM and 4 Gb of hard disk space. Thus I created a RAW disk image, asked Qemu to boot from my cdrom drive, and started installing Windows XP Professional on it (I had a spare license). For now I didn't care about performance.

Suffice to say that it is better with such 'guest' OS to disable ACPI (possible Blue Screen Of Death caused by 'virtual' virtual IRQs being mistreated).

Compress a Real Machine into a 800 Mb file

The install was a bit longuish (GNU/Linux systems spoiled me, I guess), but I was soon booting into Windows 'proper'; updates were installed (reboot), cleaned up (reboot), useless services shut down (reboot), IE7 installed (reboot), additional updates installed (reboot), and then finally I could install MultipleIE. Having cleaned up the system, defragmented the disk, freed up 'recovery'-wasted space, resized the swap file (reboot), I decided that the image was ready.

Notice the amount of rebooting; having those take place on a virtual machine hosted on a Linux-based system with plenty of RAM made those oh so fast...

I then decided to convert my 4 Gb image to a smaller, compressed, read-only image (the system would basically boot up from the same point in time and all modifications would be saved in /dev/null): the qcow2 formal allowed that. However, I was soon puzzled by a strange thing.

The new image was no smaller than the original, RAW image; I had expected to go down to, at least, the virtual drive's occupied size (by then, around a gigabyte). But then I remembered.

Windows file systems don't clean clusters when files they contain are deleted or moved. So I had to find something to clean free disk space. While Linux does the same thing, you at least have system tools to clean up after yourself if you so desire. No such luck with Windows.

So I installed Eraser 5.8, configured it to run a simple, zero-fill, pass. And then I redid the conversion.

I could finally run multiple Internet Explorer versions from a 800 Mb virtual machine that always boot fresh in a window. But it was a bit slow sometimes.

The Race is ON

Thus I decided to attack the accelerator; but for that, I needed to actually recompile Qemu from source (it requires a gcc 3.x version, gcc 4.x won't work) and then compile Kqemu (requires kernel headers and the same compiler than the one used to compile the kernel). That was soon done.

I was a bit anxious, due to the fact that until now I had run Qemu on a 64-bit machine to emulate a 32-bit machine; the accelerator required both host and guest machines to use the same CPU.

The compilation went without an itch (Qemu required I specified the compiler at .configure time, but the rest was pretty much painless), the kernel module loaded without problem, and I could finally boot my accelerated virtual machine - using qemu-system-x86_64 -kernel-kqemu - and watch.

At first, I saw no difference; Qemu would tell me that it couldn't find /dev/kqemu (while it was obviously there). So I tried to start Qemu as root (not for long!), there was no error message; thus, I tried chmod 666 /dev/kqemu.

Holy Dooly, that thing's FAST!

In order for me not to have to log in as root every time I wanted to start that virtual machine, I did some config files editing:

  • in /etc/modprobe.preload, load kquemu;
  • in /etc/rc.local, i added: echo 1024 > /proc/sys/dev/rtc/max-user-freq and chmod 666 /etc/kqemu for a 'warning-free' Qemu loading

While the user mode emulation reminded me of an early try at running XP on an old laptop with very little RAM, a very slow hard drive and a puny processor, the kernel-accelerated version of the same was blazing fast, booting in a flash and launching applications at pretty much native speed.

You got Power

I was stumped for a while on how to manage the emulation while it was running; the documentation mentioned a monitor, but for the life of me I didn't know how to access it; it allows commits to disk while running in snapshot mode, memory state dump and recovery (meaning you can save a running guest while it runs and restore it to that same point in time later on), stop the emulation (guest is frozen) and resume it (be careful, the guest's clock needs to be reset), unmounting and remounting devices (cds, USB devices) and some other useful things.

Simply press Ctrl+Alt+2 for the monitor console, Ctrl+Alt+1 goes back to the emulated system's screen.

Ain't that sweet?

Conclusion

The accelerated system boots in under a few seconds; CPU use goes down to almost zero when the virtual machine is started and idling. The only thing I have to watch for is that sometimes, the guest system's clock goes forward a bit. But frankly, that thing is FAST: it enjoys close to native CPU speed and benefits from Linux's accelerated disk access on a compressed, read only system; the emulated machine takes up 300 Mb of RAM, but only because a lower value (default would be 128 Mb) means emulated swap thus some emulation slowdown.

It's even more comfortable to run an emulated machine than it was to run two machines side by side. For one thing, that's one less machine to load an antivirus for (it starts clean every time), and one less hard disk+fans noise.

Bibliography

The Qemu homepage

Category: 

Comments

Anonymous visitor's picture
Submitted by Anonymous visitor (not verified) on

Great read. I was wondering how this would compare to virtual box? Did anyone have some experience with it? It is also GPL.

Pat

Anonymous visitor's picture
Submitted by Anonymous visitor (not verified) on

Might be time to exit before Vista 'virtualisation tax' kicks in. See browsershots.org, for example.

Mitch Meyran's picture

...it is just that: a collection of screenshots. When you need to test CSS, DHTML or ECMAscript behaviours, browsershot is useless.
Not to mention it may take a few hours and some retries before you get your browser shot.

Right now I use it to ensure that Safari works (but I'm not too concerned about Safari/Webkit, I test under Konqueror...).
---
A computer is like air conditioning: it becomes useless when you open windows.

Anonymous visitor's picture
Submitted by Anonymous visitor (not verified) on

Let me see. This is cool and all but why do I just dual boot?

#1 Native speeds.

#2 Full features. Full use of resources.

#3 I don't have to have double the memory.

KISS

Hmmm, maybe if we simply speed up booting, running in a window will not be so cool.

If you want to run popular Win apps, WINE does a fair job and as if native (with out Windows it's self.) But then, we don't need Win apps with so much to choose from, in open source.

Get Kubuntu. If you have to work on Windows, use your old computer. While a dual boot is fun for comparison, todays Kubuntu really doesn't even need it(but you may want to start his way). Either I can do it in Kubuntu or I don't care about it.

VM's are for developers (like the author).

I sometimes use WINE (easy install) to run the actual Windows version of Firefox and the windows plugins as if native on Kubuntu; just to play. I also have IE6 with WINE (yes in Kubuntu) but I NEVER need it.

I also use KMyMoney now instead of Quicken so Microsoft (and Intuit) can suck eggs.

FYI

Mitch Meyran's picture

About #1: frankly, I can't find speed differences between emulated and native XP in such a setup: my machine, although not top-notch, has far more oomph than needed to run XP at full speed. Disabling themes has a drastic impact on XP usually, especially on non-accelerated video cards, and it holds true here - so if you go that way, shut down the 'Themes' service and revert to 'legacy' dektop themes.
For one thing, that gets rid of the Fisher-Price look everywhere, even on the login screen.
About #2: the only thing the emulated system lacks is a 3D card. The rest's here. Using the accelerator, system overhead is very low.
About #3: you don't need double the memory. In fact, memory/disk management on recent Linux kernel, being much better than Windows', allow you to setup a virtual machine with close to your system's full RAM capacity, and still do stuff on the side. Spare 128 Mb for Linux (or even only 64 Mb if you run lightweight desktop managers), give the rest to Qemu: enjoy.

I must admit I didn't try other VMs recently; Qemu seduced me due to its very small size, versatility, and relative ease of use when you don't fear the command line. It's proven itself fast and stable until now, and when it ain't broke...
---
A computer is like air conditioning: it becomes useless when you open windows.

Anonymous visitor's picture
Submitted by Anonymous visitor (not verified) on

ok.. here goes..

i have been using virtualization now for probalby 1yr.. presently using VMware's free server.. but being an admin demands (in my opinion) that i learn and keep up with what is out there... aswell as the fact that id gladly use A GPL'd peice od software rather than a non-gpl...

as of late.. i have been giving a few other "free" as in beer Virtualization software a try.. first i tried virtual box... it is almost identical to vmware's server when it comes to the management console... however.. there are limitiations.. of the top of my head.. there was lack of support for a few things in the free version.. the good thing is that just like vmware you do not need a newer processor with the virtualization extensions... performance seemed to be on par with VMware...

then i tired the virtual iron.. the first thing that hits you is that you NEED to have a newer processor with the virtualization extension, which adds to the cost of free.. it is defenetly alittle more complicated to setup but not so hard that it cannot be done.. but it defenetly more complicated... performance was nice.. i cannot say it was like native.. but im sure it was close.. the free versions limits you to a single processor VM and i think a single managed box... what i did not like was the fact that all the configuration is stored on the management console not on the box being managed..

last but not least is xenexpress... this also needs the virtualization extension.. it is alittle more complicated then vmware but alittle easier than virtual iron.. performance is on par with virtual iron.. Oh and one other thing.. virtual iron and xenexpress both do not afficially support any windows version less then XP... on the possitive side.. the configuration stays on the managed box.. so management can be done from any pc as long as you have the software installed.. aswell.. as if ytou so not have any management pc runing the box will still come up and run your vm's.

i have not settles on any of the new ones yet.. still have to do more testing.. but just to give you guy alittle headsup maybe....

NOW.. as to WHY virtualize and not dual boot.. well.. you have everything running at once first comes to mind.. i run a Virtual windows machine ontop of my pclinuxos... if i need to pop into windows real quick to do something.. it is there running no need for a second machine.. no need to shut down linux and get into windows.. and i can have all the windows apps i need running without having to play with wine.. another plus is the fact that if you need to try something and you are affraid it may screwup yur windows install.. just clone your windows VM ,install the software, test it.. once you convinced your good to go.. delete the vm and run it on your production VM.. once you have used virtualization.. everything else is just antiquated..

Anonymous visitor's picture
Submitted by Anonymous visitor (not verified) on

I saw Microsoft has its own virtualisation software out for free. But here's the rub - you can only stick MS OS's on it!

Mitch Meyran's picture

No, I think that Virtual PC 2005 release 2 can actually run Fedora Core. Note that MS didn't create Virtual PC, they bought it 2 years ago, along with the devs.

Why you'd run a stable system as guest with an unstable host though, that stumps me.
---
A computer is like air conditioning: it becomes useless when you open windows.

Author information

Mitch Meyran's picture

Biography

Have you ever fixed a computer with a hammer, glue and a soldering iron? Why not? It's fun!