At first it was just (extremely noticeable) performance degradation, but minutes after telling my manager about it and asking for a new disk it was starting to look more serious: directory listing (file manager, fancy filename completion in my editor and good old ls) was failing on my home directory.

I figured I had to get as much data off the drive as possible before I shut the machine off, rsync wouldn’t work as it needs to get directory listings. Fortunately Roger pointed me at ddrescue, which enabled me to carve a partition on an external drive and copy do a direct clone of the failing drive. ddrescue works by following the British mantra of “Keep calm and carry on”, that is if ddrescue finds some data it can’t copy it gives up and moves on copying as much data as possible rather than exiting because corrupt/unrecoverable data was found.

I managed to recover most of my data with ddrescue and while it was going I had to set up another machine to use while I was waiting for the replacement disk. This gave me opportunity to try out a backup solution and various tactics to make my data easier to recover.

The biggest problem, email. Having all my mail filtering and account configuration trapped in Evolution meant that I rely on it more than I am comfortable with. Based on recommendations and usage in the office I decided to try offlineimap and imapfilter.

Offlineimap is a Python program that will download your mail and store it in the maildir format, meaning that you can use any of a range of mail programs to read your mail. Imapfilter is a Lua program that will connect to an Imap server and move mail around on the server. You write simple Lua scripts whereby you instantiate an Imap object then write bits of code to move messages around based on the mail headers:

msgs = imapacc.INBOX:is_unseen() *
imapacc.INBOX:contain_field(“X-mailing-list”, “linux-fsdevel@vger.kernel.org”)
imapacc.INBOX:move_messages(imapacc[‘INBOX/linux-fsdevel’], msgs)

Brilliant! Now my email is stored in a Mail folder in my home directory and imapfilter connects to the mail servers directly and sorts my messages into appropriate mail boxes.

For backup Ross recommended duplicity, another Python program. This program uses librsync and GnuPG to create GPG encrypted incremental backups.

After the first run duplicity will only backup what has changed, and all of your backups are stored in GPG encrypted tar archives. The first run didn’t take too long to backup my 20GB home directory and I expect future backups will be even swifter.

Now that I have my new disk installed and my data in a more recoverable format with a simple backup solution I thought I’d try and address the cause of the disk destruction.

Blunting the teeth (trying to stop Poky so easily destroying disks)

Poky, and OE, are known as destroyers of disks, for understandable reasons. Building an operating system means a lot of disk activity, to increase the lifespan of the disk it would be good to reduce the amount of disk activity.

Now I’m not expecting my new disk to fail as rapidly, it’s an old school platter-based drive, but as I do all of my development on a single disk machine (laptop) the simplest solution; to use a separate disk for builds, is impractical.

I decided to create a separate partition for Poky builds (simple thanks to LVM) and use a filesystem setup which would hopefully reduce the amount of writes.

I chose to use an Ext4 filesystem without a journal. With this set up I get the modern filesystem goodness (extents and delayed allocation should equate to less disk activity) and also fewer writes through not using a journal (as it’s a partition for temprorary build data, so I’ll not be upset if I lose any/all of it). I strongly considered Xfs and Btrfs, I decided against Xfs after reading about it’s slow directory creation and deletion (Poky does a lot of this) but will likely try Btrfs soon, though I’m more inclined to play around with that on a set up that can benefit from its snapshot support.

As far as I can tell you can’t disable journalling on an existing ext4 filesystem, you need to create the filesystem without a journal. Fortunately I was working with a new disk where I’d left plenty of unused space. I created a new logical volume to house the partition, then created an ext4 partition in that logical volume using the default filesystem features (from /etc/mke2fs.conf) only, with the journal option disabled.

mkfs.ext4 /dev/mapper/vg_scimitar-lv_srv -L srv -O ^has_journal,extents,huge_file,flex_bg,uninit_bg,dir_nlink,extra_isize

Et voila, hopefully that will increase the longevity of the disk.

While losing several days of coding time to this was initially a little frustrating it was fun and productive to play around with storage, backup and mail solutions to make my working environment a little more pleasant and my data more secure.

Reading

A Sony PRS 600 E-Book reader was gifted to me this Christmas, and what a fantastic piece of kit it is.

I’ve been enjoying several public domain texts from Feedbooks as well as a bunch of geeky PDF’s.

The PDF handling is not as bad as I expected, at standard zoom it’s just about readable in good light although much more pleasant to read at M(edium) or L(arge) text size, despite the slightly weird line wrapping you end up with.
Unfortunately tables and diagrams are completely destroyed when the text size is increased so I tend to set it back to S(mall|tandard) text size for those bits and sometimes use the Zoom function to make text more legible in low light.

The touch function is fun and sometimes even handy for turning pages and looking up words as well as highlighting and annotating the more academic reading materials.

I’ve been using Calibre to manage the device. While the UI is not to my taste (photo-realistic icons, busy interface, etc) the functionality is excellent and extremely stable. I should really donate some money to the project.

Listening

The iPod Shuffle just wasn’t cutting it as a portable audio player once I moved to London and had lengthy commutes to fill my ears on. I tried using my Android phone but the short battery life made this a chore, so I decided to pick up a new audio player.

I wanted something relatively cheap which had to have a screen (yes Apple, a screen!), a reasonable amount of storage (more than 2GB) and allow me to put music on it by simply copying files. For extra points it would play Ogg Vorbis and Flac audio.

The Sandisk Sansa Clip fits all of my requirements. With 8GB storage, support for all the formats I care about (and more) and a built in FM tuner it’s a steal at less than fifty notes. I’ve only had it a couple of days but battery life seems reasonable and it charges in a few hours over USB.

Watching

Long have I been meaning to build a PVR machine, and this Acer (Veriton N260G) computer was too cheap to pass up on.

Pair it with a Hauppage WinTV-NOVA-T DVB-T tuner and an install of MythTV and I have a low power PVR running with little cost or effort.

I’ve a few niggles to iron out on the software side but this seems like a more than capable machine at less than two tonne. I’ve not tried watching live TV while recording or trying to shove high def through it but both of these are on my agenda. For now it’s happily (and very quietly) humming away recording things for me.

MythTV is a bit of a pig though, if I was building a media PC without PVR functionality I’d be running Boxee or XBMC and would have had it running in less than an hour from no-OS to full system. Alas that was not the case with the MythTV set up…

Because of this I’ve decided that I’m going to keep a set of notes against ConnMan git for budding UI developers.

These notes will sit on this here website and be updated each time there’s a change in the ConnMan interfaces.
Once the interfaces stabilise for ConnMan 1.0 I’ll write the information up nicely and post on the ConnMan website or some such.

See the first draft here: Writing connectivity UIs for ConnMan.

Hung out with a bunch of cool people, attended some great talks, absorbed the general atmosphere and got inspired to do some hacking.

Guadec hacks:

• Started working on a ConnMan plugin to implement the shared connectivity namespace I proposed in my talk. This needs some more work and may involve me getting more familiar with ConnMan internals so it’s not ready yet.
• GUPnP context manager for ConnMan. GUpNP has NetworkManager and Unix back-ends for creating context manager objects so I sat down and hammered a ConnMan implementation too.I didn’t have enough network to pull all of the dependencies to build it so I’m going to do that in the office this week and have Ross look over and hopefully merge it before next week end.
• Of course, I did some carrick and gconnman hacking and even more meta-hacking (discussing features and bugs with others and helping people get familiar with the code).

While I was away Thomas made the Moblin Gtk theme rock even harder and I merged some of Jussi’s awesome visual enhancements to Carrick such that when I updated my netbook I got this piece of prettiness:

The software I’ve been working on for the last few months is starting to look really quite nice and it’s even getting to a stage where it’s mostly usable. Great stuff!

Guadec has inspired me to work on a few more things which I hope to be blogging about soon. My current #1 home-time-hacking priority is gconnman; I want to add GObject introspection annotations and think I’ve worked out a way to make it slightly more efficient.

More on both these things soon as we all decided at Guadec that we need to blog more about the neat stuff we’re working on with Moblin. It truly is the nuts.

Of course there where various questions about NetworkManager; “Why aren’t you using it”, “Why don’t you switch to it now 0.7 is released”, etc. Questions I was entirely unable to answer, various hand waving, etc.

On the whole the talk was productive though, I proposed a shared D-Bus API for common connectivity queries and after the talk Darren Kenny of Sun Microsystems told me he was interested in implementing this for the OpenSolaris connectivity daemon NWAM. Win!
I’ll be working on a ConnMan plugin and getting in touch with Darren soon.

I’ve boldly thrown up a talks page where you can get at my slides.

GCDS is really great so far. Saw a good talk by Will Thomson on “Profiling and Optimising D-Bus APIs”. His D-Bus profiling tool, Bustle, looks great and I’m going to share it with the ConnMan team.

Back to the talks.