We have a criminal jury system which is superior to any in the world; and its efficiency is only marred by the difficulty of finding twelve men every day who don't know anything and can't read.
— Mark Twain
Marc's realm

Creating and using disk images mini-howto

Introduction

I had the need of booting Linux from a compact flash card in an embedded device. And for producing small quantities in-house it was necessary to have disk images from these flash cards that could just be dd'ed onto the the flash cards. Since it took me some time to figure out how to do that I've written this guide in hope others may find it useful.

Our first method to get these disk images was to boot the embedded device via Knoppix, mount the compact flash, copy the necessary files via scp onto the compact flash, chroot onto the flash, run LiLo and reboot. Afterwards we'd dd the complete flash content into a file. Not very entertaining. And especially hard to automate when you have a new release very often.

So we wanted to automate the image creating process as much as possible. Using an USB CF reader/writer we thought this shouldn't be too hard, but it turned out that when we copied the files onto the flash and chroot'ed into it lilo refused to run (can't remember why, sorry).

So we got the idea of producing bootable mini-images, where we would mount the partition using the loopback device, copy the files in, unmount the image and dd that complete image onto the compact flash (complete with MBR, partition table, everything).

Problem is, again lilo is making problems: you can't just update the kernel by copying a new one over the old one. You have to run lilo again. And grub was out since neither of us managed to get it to work (while grub seems to be very good, the configuration is an unnecessarily hairy nightmare Almost the same technique as described in this mini-howto can be use with grub if you dig it. And saves you the FAT16 partition as well.). Alternatives: booting DOS, using LOADLIN or SYSLINUX. Obviously, SYSLINUX is the cleaner solution.

This simple technique described can also be used with any other medium, like USB sticks for example.

Creating a disk image

And this is how you do it:

  1. Insert CF into reader/writer. We assume that the CF is now accessible as /dev/sda.

  2. Since our raw CF's had lots of garbage on it we zero out the complete CF (helps compressing the image later on We used 64MB CF's when I wrote the first version of this documents. When zero'd, partitioned and formated these compressed down to just 4200 bytes with bzip2... nice ratio :-) An even nicer ratio is that of a 2GB hard disk image we've done: it compresses from 2GB down to just 18613 bytes.).

    dd if=/dev/zero of=/dev/sda
  3. Create partitions: we need at least one boot partition (FAT12 or FAT16, but not FAT32) and a root partition (we used Ext3).

    fdisk /dev/sda
  4. Format the partitions.

    mkfs.msdos /dev/sda1
    mkfs.ext3 /dev/sda2
  5. Install SYSLINUX on boot partition.

    syslinux -s /dev/sda1
  6. Install master boot record (found in SYSLINUX source directory).

    dd if=mbr.bin of=/dev/sda
  7. Mount the boot partition.

    mount /dev/sda1 /mnt
  8. Copy the kernel image onto boot partition.

    cp bzImage /mnt/kernel.bzi
  9. Create SYSLINUX configuration file.

    cat >/mnt/syslinux.cfg <<"EOF"
    DEFAULT kernel

    LABEL kernel
    KERNEL kernel.bzi
    APPEND root=/dev/hdc2
    EOF
  10. Umount the boot partition.

    umount /mnt
  11. Save the final image.

    dd if=/dev/sda of=image.bootable

You can then mount the root directory and copy all your files into it, and even update the kernel by just copying a new bzImage onto the boot partition. No need to run any program like LiLo afterwards.

If you just want to copy the partitioned space then you may want to read on about mounting the disk image and then come back here: you need to calculate the size, which is (<end block number of the last partition> + 1) * 512. Then give dd the additional option count=<size>.

Mounting the disk image

There are two ways to mount the partition.

The clean way

First, we need to determine the offset of the partition. This is quite easy: just type fdisk -ul <device>. The option -ul means list the partitions on the device and assume a unit size of 512 byte. This looks something like this:

tetsuo:~ # fdisk -ul /dev/sda

Disk /dev/sda: 256 MB, 256376832 bytes
8 heads, 62 sectors/track, 1009 cylinders, total 500736 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *          62       19839        9889    4  FAT16 <32M
/dev/sda2           19840      231135      105648   83  Linux
/dev/sda3          231136      442431      105648   83  Linux
/dev/sda4          442432      471199       14384   83  Linux

Now all we need to do is a little math to get the offset: we need to multiply the start block by 512. E.g. if we wanted to mount the first partition we'd have an offset of 62 * 512 = 31744. The second partition has an offset of 19840 * 512 = 10158080. Now that we have the offset we can mount the partition:

mount -o loop,offset=10158080 image.bootable /mnt

This would mount the second partition on /mnt. Linux recognizes it as ext3 if it is formatted as ext3 and the kernel supports ext3, so no need for a -t ext3 option to mount.

The dirty way

There is also a hard way to find the formatted partitions if you can't calculate the offsets for some reason:

for ((i=0 ; $i < 10000 ; i=$i + 1)) ; do
    mount -o loop,offset=$(($i * 512)) image.bootable /mnt && break
done

If there is a partition within the first 10000 blocks, it gets mounted eventually :-) Just type "mount" to get the offset...

Final comments

After we've unmounted the disk image we can now just dd the disk image to a new compact flash:

dd if=image.bootable of=/dev/sda

Easy as that.

There are several ways to force Linux to re-read the partition table after we've written a disk image with partition table to an empty compact flash. Propably the best way is to run:

partprobe

This program is part of GNU parted. If it's not installed then you might succeed with the following command:

/sbin/sfdisk -R /dev/sda

In the rare case that you have neither, there's still a hack: unload the USB module and load it again:

modprobe -r usb-uhci && modprobe usb-uhci

Credits

This document was written and is ©opyrighted 2003,2006,2010 by Marc Haisenko. Thanks to the SYSLINUX author H. Peter Anvin for finding an unnecessary step in the creation process. This moved to the chapter "Final comments". Manicalic told me about partprobe and sfdisk to re-read the partition table. If you have further comments/additions/corrections please mail them to me. You may copy and distribute this document as long as you include this credit section and my name. You may modify it and add your name to this section as well.