Building a Diskless MythTV Frontend with Mythbuntu 10.04 - Lucid Lynx - Part 1

I just upgraded to Mythbuntu 10.04 which was released just a few days back on April 29th.  Since I was already running Mythbuntu 9.10 on my MythTV backend server I was able to upgrade it to Lucid using the well documented and fairly simple steps on the ubuntu.com site:

$ sudo apt-get install update-manager-core

$ sudo do-release-upgrade

After upgrading the backend server I decided to finally figure out why my diskless frontend server wasn't working.  After a day of investigating it turned out to be a faulty stick of memory.  With that replaced, I needed to configure a diskless frontend on the MythTV backend server so I could use PXE to boot into a frontend without having to worry about anything else.  Since Mythbuntu hasn't had a graphical control panel to create a diskless frontend since Karmic, this blog post documents everything I had to do to get my diskless frontend up and running.  Some of the setup already existed but I'm documenting it here for future use.

Requirements for PXE Booting

PXE is a way for computers to boot up using resources found on the network.  On some computers you have to press F12 or some other key to get them to PXE boot but on all computers that have network cards that support PXE you should be able to set the first boot option in the BIOS to something like "Network" to allow your computer to PXE boot.

During a PXE boot a network card will perform the following tasks without any intervention from any OS stored anywhere on the computer:

1. automatically obtain an IP address,
2. automatically obtain a boot kernel and initial ramdisk, and
3. boot up the computer using that kernel and initial ramdisk 

After booting up the kernel using the initial ramdisk in step 3, the OS can either use a built-in hard drive or the computer's RAM as a root disk or it can NFS mount a volume from a remote server and use it as a root disk.  The NFS scenario is most common for computers that PXE boot and is the one I use.

For a computer to be able to automatically obtain an IP address you need a DHCP server in your network.  However, this DHCP server has to be configured so it can inform the PXE booting computer where the kernel and initial ramdisk files are located.  The kernel and initial ramdisk files are hosted on a TFTP server.  Finally the NFS mounted root disk needs an NFS server which will export the directory of files that will become the root directory of a diskless MythTV frontend.

In my home network I have a separate DHCP server hosted on a Linux machine that used to be my main Linux server.  Now it just serves DHCP until I can move that functionality into my Cisco router.  The other two pieces of the puzzle i.e. the TFTP server and the NFS server are handled by the Mythbuntu backend server and are closely tied to each other.  However, there is no reason why they can't be split up onto their own servers.

The next post will focus on the steps needed to build a diskless image using the Linux Terminal Server Project's utilities.  The post after that will focus on configuring the individual servers needed to put it all together.  These three posts should serve as a more or less complete guide to setting up a diskless Mythbuntu frontend.

Switching your Rails Database from SQLite3 to PostgreSQL or MySQL

I deployed a minimal new Rails application I wrote over a period of a couple of hours and started testing it with valid production data.  It's basically a very simple CRUD application for a very specific audience.  Very basic stress testing (using Apache's ab) showed me that it didn't work well when requests were coming in with concurrency > 2. I realized I'd started the project using SQLite3 as the database and that's probably where the bottleneck is since it can handle requests with a concurrency of 1 just fine.  The application is deployed using mod_passenger (my first experience with it) and is configured to never tear down application instances due to idle timeouts.  With 6 application instances listening, a concurrency of 6 should have been a cinch.

In any case, now that I need to switch it to using a real database like PostgreSQL or MySQL, there were no obvious solutions that would allow me to keep the data I already had in my production database.  Everything out there talks about going from development to test to production each of which environments gets their own schema but nothing else.  Migrations allow you to keep your production data in place but that's not the same as dumping it and loading it.  Using database-specific dump/load utilities might result in SQL that needs to be tweaked before it can be loaded into another database type.

In comes the Yaml Db plugin by Orion Henry and Adam Wiggins.  It does exactly what one would expect.  Similar to the way schema.db is database-agnostic, the Yaml Db has a database-agnostic dump format and a similar load format.  One of the specific use cases mentioned on that site is "One common use would be to switch your data from one database backend to another."

Excellent!  Btw, thank you George for the tip!  You know who you are :)

Spaces in /etc/fstab

I needed to mount a Samba share from my windows gaming / media center PC to my MythTV backend so I could navigate to all my videos from a single location instead of having to worry about which server they were on.  To that end I shared out the Videos folder from my account on the Windows PC.  Since my user on the Windows PC is "Shahbaz Javeed" that presented a problem when trying to auto-mount it using fstab on my MythTV host - spaces aren't allowed in any field of /etc/fstab.  Any spaces are considered field delimiters.  The solution is to escape all spaces with \040.  My /etc/fstab entry now looks like so:

//frey/Users/Shahbaz\040Javeed/Videos   /var/lib/mythtv/videos/frey cifs     guest,ro       0       0

This works swimmingly!

CheckPoint VPN-1 SecureClient on Snow Leopard

It turns out that the CheckPoint VPN-1 SecureClient for Leopard (OSX 10.5) doesn't work on Snow Leopard (OSX 10.6) due to differences between the two versions in the kernel and the way kextload works.Harald has a blog entry detailing how to fix the Leopard package so it installs on Snow Leopard and then fix the installed files so they properly run as well.  He alludes to a method to fix the package itself so you can install it on multiple Macs without manually making those changes.

I took it upon myself to modify the package and am providing it here.  Feel free to use it at your own risk.  You can verify the authenticity of the file by running the following command to get its checksum:

cksum SecureClient-VPN-1.zip


You should get the following output:


3505974925 22321216 SecureClient-VPN-1.zip

Boot Camp x64 is Unsupported on this Computer Model

Apple says that only certain models of Mac are x64 compatible when using Bootcamp 3.0 that comes with Snow Leopard and possibly Leopard.  Imagine my surprise when my 17" MacBook Pro wasn't one of them.  I'd hoped their flagship portable would be on the list.  I got the somewhat curt error message "Boot Camp x64 is Unsupported on this Computer Model" and the Bootcamp installer refused to continue.  The solution turned out to be simple.  I located the bootcamp64.msi file on the Snow Leopard disc and ran it in Vista compatibility mode.  Everything installed just fine.  So there, Apple!

OSX and /etc/resolv.conf

I recently went back to a Mac laptop and encountered an interesting issue.  I needed to make some changes to /etc/resolv.conf to reflect a modified search path and since /etc/resolv.conf is a symlink to /private/etc/resolv.conf I edited the latter file.  All was well until I connected to a different network.  Now my /private/etc/resolv.conf file, which clearly states that it's an auto-generated file, wasn't updated resulting in the "host" command (among other things) breaking.  After posting on Apple's forums I ended up answering my own question.

/private/etc/resolv.conf is itself a symlink to /var/run/resolv.conf which is the file that is auto-generated.  I ended up discovering that after looking at the /private/etc/resolv.conf file in my oldest Time Machine backup.  That'll learn me.

Accessing the KDE Wallet from the Cmdline

I needed to write a script that would contact my Exchange server at work via IMAP and list all the messages in the Calendar folder. The idea was to see if it was possible to perform a one-way sync from the Exchange server to a specific calendar in Kontact. I was going to embed my IMAP password in the script - security hole, I know - but we have a password policy that requires the password to change every 30 days. Since I didn't want to edit the script every month I decided to see if it was possible to use the password for kmail that's already stored in my KDE wallet.

I didn't have any luck finding a perl interface to the KDE wallet. However, thanks to the good folks at #kde on freenode, I found that the KDE wallet - and lots of other applications as well - expose their interfaces over D-Bus. This was the first time I'd dealt with D-Bus so it took some getting used to but I figured out how to read my kmail password from my KDE wallet. KDE comes with the handy qdbus program that allows command-line testing of the D-Bus interface.

What follows are step-by-step instructions on how to use qdbus to open your KDE wallet and read your kmail password. I'll incorporate this into my one-way sync experiment using the Net::DBus perl module but I wanted to put this out there in case someone else was looking at that.

Introduction to qdbus
A quick intro to qdbus before we get started so you can explore other options instead of just kwalletd:

The following command shows all applications exposing a DBus interface:

$ qdbus

:1.50
org.gtk.vfs.Daemon
:1.51
:1.52
:1.54
org.kde.kwalletd
:1.56
org.kde.printer-applet-3206
:1.57
net.update-notifier-kde-3203
:1.58

The numbers and strings refer to applications, however since most applications expose a recognizable string it's common to use just the strings and ignore the numbers.

The following command lists all the DBus paths exposed by the kwalletd application:

$ qdbus org.kde.kwalletd
/
/MainApplication
/modules
/modules/kwalletd

The /MainApplication path is mainly used when you want to interact with the application itself and you'll find many applications that expose a /MainApplication path. I haven't explored this much but it looks like it should be interesting.

With that introduction to qdbus you should have enough to explore further on your own.

Getting a Password from a KDE Wallet
The following steps will open your default KDE wallet and get your kmail password. Each step will have an explanation, the command issued and the output of that command.

We will use the /modules/kwalletd path in the DBus interface for org.kde.kwalletd for all our password-getting needs. You can get a list of all the methods and signals exposed in the /modules/kwalletd path by using the following command:

$ qdbus org.kde.kwalletd /modules/kwalletd

method bool org.kde.KWallet.isOpen(QString wallet)
method bool org.kde.KWallet.isOpen(int handle)
method bool org.kde.KWallet.keyDoesNotExist(QString wallet, QString folder, QString key)
method QString org.kde.KWallet.localWallet()
method QString org.kde.KWallet.networkWallet()
method int org.kde.KWallet.open(QString wallet, qlonglong wId, QString appid)
method int org.kde.KWallet.openAsync(QString wallet, qlonglong wId, QString appid, bool handleSession)
method int org.kde.KWallet.openPath(QString path, qlonglong wId, QString appid)

Formatting's a bit messed up going forward. Not sure why :-(

However, let's get to work obtaining the password. First, we will open the default KDE wallet - called kdewallet. We will call the org.kde.KWallet.open method which expects a wallet name string, what appears to be a wallet id (similar to a file handle it seems) and finally an application id string. We will use "kdewallet" as the wallet name since that's the name of the default wallet in KDE. We don't know the value of the wallet id so we'll just specify 0. The application id is interesting because KDE wallet prompts the currently logged in user with the application id of any applications that call the org.kde.KWallet.open method which we're abbreviating to just open since that uniquely identifies it in the method list for the /modules/kwalletd path. Specifying a meaningful id here goes a long way to helping the user click on "Allow", "Allow Once" or "Allow Never". With all that in mind, let's use the following command:

$ qdbus org.kde.kwalletd /modules/kwalletd org.kde.KWallet.open kdewallet 0 "KOrganizer-Exchange 1-way Sync"

470467109

This results in popping up a dialog box like so:


For the purpose of these experiments, I chose "Allow Once". Once I've allowed it, the qdbus call returns a wallet id - similar to a file handle - that we'll use in all our other method calls. I did see that if I waited too long the dialog box remains visible but qdbus times out. However, the next time you make the org.kde.KWallet.open call it returns a valid wallet id without prompting which means the permission grant is persistent. I'll have to deal with the timeouts in my perl code somehow. The next step is to see what's stored in my wallet. This isn't strictly necessary if you already know what you want but serves to walk through my own discovery process. Notice I'm passing in the newly given wallet id as well as the full application id as I sent earlier.

$ qdbus org.kde.kwalletd /modules/kwalletd folderList 470467109 "KOrganizer-Exchange 1-way Sync"

AdobeAIR
Amarok
Form Data
Network Management Passwords
bilbo
kblogger
kmail
mailtransports

Let's list the contents of the kmail folder:

$ qdbus org.kde.kwalletd /modules/kwalletd entryList 470467109 kmail "KOrganizer-Exchange 1-way Sync"

account-242017858
account-990222852

I know the account-242017858 account is the one I need the password from because the other account is older. So let's see how to retrieve that password:

$ qdbus org.kde.kwalletd /modules/kwalletd readPasswordList 470467109 kmail account-242017858 "KOrganizer-Exchange 1-way Sync"

account-242017858: [the password here]

There you go folks! That's all it takes. Please let me know if you found this helpful.