Avoiding Memory Issues with Virtual Machines

In my last post I discussed a few advantages of using virtual machines for development. This post deals with a huge downside – memory usage.

When you run a virtual machine you’re running two operating systems (or more) on your hardware – the guest and the host. Both require resources just to be active. If your hardware doesn’t have enough memory to run both operating systems, you’re in for a world of hurt as the host will be constantly paging to try and keep up.

There are a couple of ways to mitigate this problem. Obviously you can just go and buy more RAM. More memory for the host means that you can allocate more to the guest. Not a big deal if your hardware is new and you can still get memory for it. The next obvious thing is to run your host as lean as possible. Close your email, your music player, twitter, etc to free up some memory. After this you should try and tidy up what you’re running on the guest. Only run the programs you absolutely need to be running in order to keep paging low.

One more thing you can do to trim your guest memory footprint is to avoid using the piece of the OS that takes up a large chunk of resources: the windowing system. How do you do this? With Linux, runlevel 3 is everything but the GUI, so that’s the first step. A quick search for how to do this with your distribution will bring up all kinds of resources. After you switch your runlevel your system will boot with a command prompt only.

From there you can use x-forwarding to do your work. In short, programs are run on the guest  but the GUI is displayed on the host. Thus, there is no overhead on the remote machine for the window environment and you can use allocate less memory to the VM and get the same performance. (This isn’t limited to VMs; you can use XForwarding on any two computers connected to a network and have the appropriate software.)

There was no configuration needed on my setup. My guest is CentOS 6 and my host is OSX Lion. I booted the VM, then started a terminal on my Mac. In the terminal I typed the following:

$ ssh -X ryan@myvmname

This sets up an x-forwarding session. From there you can run your favourite windowed programs, such as emacs, vim, jEdit, add/remove programs, etc. I won’t go into any more detail here because there isn’t any; if your setup is different there will be resources online to help.

Using this setup we were able to cut the memory allocation of the guest in half with no change in performance. It’s a pretty simple way to get the advantages of all Linux has to offer with the convenience of your favourite desktop.

Developing with Virtual Machines

Using virtual machines for development provides a number of advantages over setting up environments directly on your laptop or desktop. I’m running CentOS 6.3 in VirtualBox, and I’m going to give you three reasons why I prefer developing this way.

Security Blanket

Virtual Box lets you take snapshots of your VM at any time, even when it’s running. I usually use this feature when I’ve gotten some weird configuration working, or I’ve done system updates. This gives me the ability to go back to a known good state if something goes wrong. I will also take a snapshot of my system before I try a potentially risky spike or upgrade. That way if things don’t go as planned, I simply revert to the snapshot I took when everything was working – no reinstalls or reconfiguration are necessary.

System Portability

There’s a feature in Virtual Box called Export Appliance. This creates an industry standard disk image that most virtual managers can import. I like to think of snapshots as an incremental backup and appliances as a full backup. I’ll install an OS, run the updates, install the tools I need, and make any configuration changes (taking snapshots along the way!). Once everything is ready, I can create the appliance and send it to others so they can import it into their virtual manager and have a development-ready OS up and running in under 5 minutes.

One Host, Many Guests

The physical machine you have is your host (your laptop or desktop) and the VMs that you have on your system are the guests. You can have as many different guests as your hard drive space will allow. You might do this if you like CentOS for PHP and Ubuntu for Rails (or vice-versa). You might also do this if you wanted to have a separate VM for each of your clients (which is really easy when all you have to do is import the appliance again). This ensures you don’t mess up one client’s environment while you’re upgrading something for another.

There are a few drawbacks to running VMs for developing, but the ones I’ve come across are pretty insignificant compared to the benefits mentioned above. In my next post, I’ll describe how we got around our main problem with running VMs on a laptop: memory.