Tuesday, August 26, 2008

gem update taking forever: the solution

I've been running into this little problem for a long time: gem update takes slightly longer than forever. I've Googled around a bit, and it seems I'm not the only one suffering.

One of the posts I read about this, mentioned that gem is a piece of **** and tries to load the whole of rubyforge.net into memory, which fails and / or takes forever on low memory machines. I know this to be true, because my old laptop (an HP Omnibook XE3 with 256MB), which I use as a server, has serious trouble updating my Rails gems. When I installed this machine with the server version of Ubuntu Hardy, I did what I usually do: the swap I created was twice as big as the RAM, so it has 512MB of RAM.

For updating gems, this seems to be too little. Yes, that made me cry too.

Adding more swap was not that easy, because I used up the entire disk for other stuff. Omnibooks don't have much diskspace. What to do, what do to?

Enter nbd. NBD allows me to share a swapfile from my desktop, which is then used as swapspace on my server. I can verify this works, because my server is updating it's gems as we speak, with a swapfile used over nbd. Pretty cool.

Is this hard to set up? Not at all.

First, we create a swapfile of about on the computer we want to share swapspace from. Do this with

$ dd if=/dev/zero of=/tmp/swapfile bs=1024k count=1024

The swapfile is now in /tmp. Then we make it a swapfile by running

$ chmod 777 /tmp/swapfile
$ mkswap /tmp/swapfile

Now, just install nbd-server and modprobe nbd on the computer you wish to share a swapfile from. On my computer, running an Intrepid Alpha, nbd-server did not create a stub configuration file during install. If there is no configuration file, nbd-server will not start. Just run

$ sudo dpkg-reconfigure nbd-server

to create on and use some sensible values, like pointing it to the file we just created.

Next, on the client, install the nbd-client package, modprobe nbd and run

$ sudo dpkg-reconfigure nbd-clent

as necessary, using the same sensible values. Tell it to use the nbd as swapspace.

Last, run

$ sudo gem update

succesfully :-)

Of course, if you have oodles of diskspace left, you can also create a local swapfile and use that. I didn't because a) I have no spare diskspace on the server and b) always prefer the over-engineered solution.

1) The way I set up nbd above is insecure. Please keep this in mind. I am only using this for a gem update, so for me, this is not a problem. If you plan to use the nbd permanently, please, use a more secure setup.
2) I'm honestly not sure whether all the steps in the creation of the swapfile are necessary. I copied some of this from a tutorial on ubuntuforums about nbd. Credit to the poster of this thread.


HIteshGanger said...

what can a windows user do for such problem

wzzrd said...

I have absolutely no idea, sorry. Haven't touched a Windows box in months, if not years.

Anonymous said...

Getting these types of updates is the one exact thing that I have been doing all the time. From these types of solutions we are sure to get rid of the problem that I have been facing.