Don't Let BootDisk Creator Eat Your Thumb Drive

Have you used USB thumb drives instead of CDs for booting your computers into automation? If you have, you'll probably have been most disappointed that suddenly the rest of your multi-gig drive was useless for anything else. Or perhaps you've just gone demented after trying for hours to partition up your flash drive into several useful partitions? If so then this article might help. Today I'll show why working with flash drives happens to be so tricky under Windows (and of course provide a couple of tips for getting round these Microsoft imposed limitations).

I'll also show you how to blow away the artificial 1GB and 2GB limits imposed by Bootdisk Creator when formatting your Linux and DOS automation environments onto USB drives by taking control of the formatting process. The will help you realise the full capacity of your USB drives, allowing you to store your images, documents and that plethora of utils you've picked up over the years. Those that use WinPE can sit back all happy and snug -these issues are unlikely to affect you!

Introduction

Over the last couple of years, USB Flash Drives have been plaguing the workplace and have for many become a data security nightmare. But that aside, they are really nifty devices and now being commonly used by Altiris Administrators for booting computers into automation. This is because of a combination of factors,

• USB Flash devices are now very low in cost
• They have excellent read performance (but dodgy as hell write performance....)
• Flash Drives are (in my experience) more robust and reliable than Compact Disks
• USB ports are more reliable than CD Drives
• No specialist software is required to write to them
• BIOS manufacturers now support as standard USB removable devices as a boot devices.

From the Altiris Admin's perspective, Flash Drives are not the complete answer -older desktops and laptops may not have the necessary BIOS extensions to boot from USB drives. So whilst these devices are great (and now my preferred light-touch imaging tool) I still always keep those automation CDs handy. But like those floppies, I'm all too aware that CDs will too become a relic of the past.

When I first started working with Flash drives I was really, really confused. Windows didn't seem to like dealing with them at a low level. It doesn't like deleting partitions on them, and if you're lucky enough to be able to get Windows to create a partition on one, that will be the only partition you will ever create on it. So, I tried bypassing the whole shebang by partitioning the drive with Linux into a couple of fat or NTFS partitions, but no matter how hard I tried when I booted back into Windows only the first partition was ever seen.

This became problematic to the extreme when I then wanted to use these flash drives as automation boot devices. My 4GB flash drive when formatted for Linux with Bootdisk creator turned by 4GB drive into a 1GB fat partition. The rest of the 3GB now from a Windows point of view unusable -wasted. As I wanted to put images on the drive, as well as my usual handy utils I decided so see what was going on, and how to fix it. After some googling, I found I was not alone.

Microsoft and Removable Media

Here is a little known amazing fact about Windows and removable media. Windows by-design will only let you see one partition. That's right -Microsoft deemed a long time ago that removable media should only contain a single partition which is why if you try to partition a removable device you are only ever offered the option of creating a single partition spanning the whole device.

Further, if you try to cheat by partitioning up your device in Linux, Windows will only present the first viewable partition as a drive letter. And you will not be able to convince it otherwise without some hackery. This hackery focuses on altering the Removable Media Bit, which the device sends Windows on request so the OS knows whether to treat the device as removable or not. This can be done within the device itself, or on-the-fly within Windows.

Hackery 1: Flipping the removable bit

When you insert a flash drive into your trusty Windows box, the device is interrogated with a SCSI data enquiry. The response contains the Removable Media Bit which is 0 for a fixed disk, and 1 for a removable one. If you are really lucky, your flash drives will either support the flipping of this removable bit which allows you to change their personality type from a removable drive to a fixed drive. Thus, by flipping the removable bit, Windows will be able to treat your flash drive as a fully functional drive and allow you to partition it as you see fit.

I've only been able to find one utility on the internet which does this, the Lexar application BootIt, which can be downloaded from pendriveapps.com. The idea is you simply execute this utility, and it will talk to the firmware and get it to toggle that Removable Media Bit.

If this works for you, (or you can sanction purchasing USB sticks that do support it) then you are damn lucky and you can go forth and partition your drive to your heart's content.

There are some important considerations though when using BootIt,

1. As previously mentioned, not all Flash Drives support the flipping of the Removable Media Bit (none of my drives support this)
2. By changing this bit, you might be circumventing your company's security policies regarding data security removable media. This can get you into a lot of trouble, so if your company has such policies (or you visit company's that do) please don't do this.
3. Autorun.inf will no longer work. Microsoft's autorun triggers only work for removable media, so making your drive fixed will not allow you to leverage the autorun capability. In my mind, this is actually good considering the amount of malware which spreads this way...

Hackery 2: Using the Windows Fixed-Disk Driver

Although Windows will by default try and use its removable disk drivers for accessing Flash drives, you can of course force Windows on a computer-by-computer basis to use a fixed drive driver for your flash drives.

All you need to do is,

1. Download the compact flash driver (available on lancelhoff.com)
2. Modify the driver inf file to reflect your flash drives vendor and device ID
3. Use Device Manager to force Windows to use this driver.

Although this is messy, it does have some uses if you are creating USB keys for distribution to IT Staff to facilitate imaging when you take into account the Windows feature which limits the partitions you can view. Updating the device driver allows you to modify your computer so that you can create additional partitions and write images to them, whilst preventing any tampering from others whom you distribute the flash drives too. This means that you'll be safe in the knowledge that, by-and-large, your IT staff will not even know there are gigs of images hidden on there. Their windows computers will by default never be able to browse anything but that first viewable partition on the drive -the Altiris Automation partition.

So, let's see how this works.

1. Download the Compact Flash Filter driver

   This driver intercepts and filters IOCTL_STORAGE_QUERY_PROPERTY, the I/O Control which Windows uses to determine the properties of storage devices. If this driver finds that the Flash Device hits its Removable Media Bit set, it resets it to zero on the fly so that Windows XP is fooled into handling the CompactFlash devices as a fixed disk.

2. Discover the Device Instance ID information for your Flash Drive

   Open up Device Manager, and expand the "Disk Drives" device class branch. Locate your USB Flash Drive, and in the device properties highlight the Device Instance ID value in the details tab and hit CTRL-C

   

3. Update the driver inf file

   Right-click and edit the inf file cfadisk.inf which you downloaded in the zip from step 1. Replace the text device_instance_id_goes_here with the device instance ID of your flash disk (which you should tucked away in your buffer, so hit CTRL+V). Once you made the change, save the inf file.

   (Note: You can of course add multiple device lines here to add local disk support for all of your flash drives...)

4. Update the driver in Device Manager

   In device manager, open up your "Disk Drivers" device branch again and in Properties, navigate to the driver tab and force the driver to update from Compact Flash Driver you just updated. You'll be warned it isn't digitally signed, but proceed anyway.

   Within minutes you'll be able to delete partitions, create partitions, choose your own partition sizes and filesystem types. The feeling of absolute power you feel at this point is dizzying, and I highly recommend having a lie down before social contact with your colleagues becomes a frightening possibility.

   This solution is rather neat, but is a bit of a bugger though in most scenarios, so for what follows let's assume your Windows computers in the main will still only see the first viewable partition on your Flash drives.

Altiris and Removable Media

Having learned that from the Microsoft angle things are a little frustrating, what do we really want then from out removable media regarding their use as automation boot devices? Well, I've come slowly to the conclusion that overall, its best to try and format these USB devices as one big drive. Dump images in a folder called images, and your personal files and apps in other suitable named folders. This way you can get the most from each device without compromising security or requiring driver hacks on each computer you visit.

So, let's take a look at the Altiris side of the removable media house. It turns out that using the native Altiris toolset to convert your flash drives into automation boot disks has just one problem -it's a bit wasteful. In general you will not be allowed to take advantage of all your flash drive space, and indeed your multi-Gig drive can be effectively neutered down to a 1GB device.

Bootdisk Creator will destroy what ever is already on the drive and repartition to its pre-programmed defaults.

For example, Deployment Server 6.9 SP2 will format a 4GB USB Stick as follows when prepping your device for the various automation environments,

• WinPE. Drive reformatted as 4GB FAT32.

  

• Linux. Drive reformatted as 1GB FAT16

  

• DOS. Drive reformatted to 2GB FAT16

  

So, with the exception of WinPE, the picture is looking a bit disappointing -your multi-gig flash drive has been neutered to either a 1GB or 2GB Drive. Sure you can boot to Linux and add other FAT or NTFS partitions, but what's the point? Windows will never let you see them. The way around this is to install automation onto these disks yourself. A scenario which in fact BootDisk creator has catered for very nicely.

Creating DOS Automation on a Flash Drive

As we've seen, DOS automation as created by BootDisk Creator on a flash drive is limited to 2GB. So, let's now make our 4GB Flash drive usable with DOS at its full capacity.

On Deployment Server do the following,

1. In Bootdisk Creator, right-click your DOS configuration and select the option to "create bootdisks".
2. In the Boot Disk Media Type wizard screen, select the option to copy the bootfiles to a folder. Note the location in your temporary folder -copy it to your buffer with CTRL-C

   

   And click Next

3. When the file copy is complete, transfer these files to your desktop computer.

On your desktop, do the following

1. Download and Install HP's USB Disk Storage Format Tool
2. Insert your USB Stick, and wait for it to be recognised.
3. Run the utility (Start Menu -> All Programs -> Hewlett Packard Company)
   --> Configure the filesystem as FAT32
   --> Change the Volume label to Altiris (not essential, but setting the label gives you some confidence all is well)
   --> In Format options, select "Create a DOS startup disk"
   --> Configure the location of the DOS files you've just copied from the Deployment Server

   

   Click Start

4. Once the drive has been formatted and the system files written open up your flash drive. You should find it populated with COMMAND.DOM, IO.SYS and MSDOS.SYS as show below,

   

5. Now copy the rest of your files over to the USB stick (Note: Don't overwrite the existing files). Your USB drive should now look as follows,

   

6. Just in case you feel that the above drive isn't pleasant to look at, and would rather the drive looked blank you can of course hide the files using attrib. Open up a command prompt, and change you drive to the USB Flash Drive. In my case, my flash drive is on E:\
   

   Attrib /S /D +h E:\*.*
   

   Anyone who now views the flash drive with the default explorer settings to not show hidden files and folders should see a blank disk. At this point, I usually create my placeholder directories docs, images & utils.

So, if you're here you've now got your big bootable DOS automation drive. Create some folders like docs, apps and images and that's it! Don't forget to try and USB boot it...

Creating Linux automation on a Flash Drive

Creating a Linux automation partition is nearly as straightforward as the DOS procedure executed in the last step. The principles are the same,

1. Format the drive as one big partition
2. Write the master boot record and boot file
3. Copy the rest of the automation files

The drive format can once again be done with HP's USB Storage Format Tool, but this time we can't use the msdos system files. With Linux, we have to use a Peter Anvin tool called SYSLinux to write the master boot record, activate the partition and copy the boot file.

So, what is SYSLinux? In short it's a highly specialised bootloader designed to boot Linux systems from FAT filesystems. This seems odd at first glance -why on earth would you want to boot into Linux off a FAT filesystem, when Linux has its own filesystems for this purpose? I suspect the answer here lies in Peter Anvin's other creation ISOLinux -his sister tool designed for the creation of Live Linux CDs. The Altiris developers simply decided to use the same framework for both CDs and Flash Drives.

For robustness, we're going to change the ordering slightly of this processes 1-3 above. What we're going to do is format the drive, copy over the Linux files and then write our master boot record and system boot file.

To begin then, on Deployment Server do the following,

1. Create the Linux Automation Files

   In Bootdisk Creator, right-click your Linux configuration and select the option to "create bootdisks". In the Boot Disk Media Type wizard screen, select the option to copy the bootfiles to a folder. Note the location in your temporary folder -copy it to your buffer with CTRL-C. Click Next

   

2. When the file copy is complete, transfer these files to your desktop computer.

Now, on your Windows Desktop,

1. Download and install HP's USB Disk Storage Format Tool
2. Insert your USB Stick, and wait for your drive to be recognised
3. Run the utility (Start Menu -> All Programs -> Hewlett Packard Company)
   --> Configure the filesystem as FAT32
   --> Change the Volume label to Altiris

   And click Start (note this time we are not making a DOS startup disk)

   

4. Now copy the rest of the Linux automation files (created earlier on the server) over to the flash drive.

   

5. At this point, you've got all the files in place, but the drive isn't bootable. The masterboot record needs to be written, and the bootloader ldlinux.sys updated. Download syslinux-3.82.zip and extract to C:\syslinux on your desktop
6. In a command prompt, type the following:

   C:\syslinux\win32\syslinux -m -a E:

   where E: is in my case the drive letter for my removable drive. Change this to suite your setup. The Syslinux option -m writes tells syslinux to write the master boot record and the option -a tells syslinux to make the target partition bootable.

7. The drive should now be bootable, but once again you might find it a bit cluttered to look at. Let's resolve again with a quick attrib command. Open up a command prompt and type,
   

   Attrib +h E:\*.*
   

   At this point, you'll now have a seemingly blank drive which you'll be able once again to add images, applications and documents up to the full capacity of the flash drive.

Summary

In today's article, I have shown how you can get the most from your flash drives by avoiding the restrictive BootDisk creator built-in formatting. Once you do this a few times, this process is actually faster than formatting your sticks directly with the Bootdisk Creator. Importantly, this method can be realised completely within Windows -no requirement to use time consuming tricks like GParted under Linux to expand partitions and the like.

The fundamental reason why all this is necessary is the Windows shortcoming of neutering its disk management capabilities when removable drives are on the menu. Its a shocking omission really. The two hacking tips concerning the direct altering and filtering of the Removable Media Bit do go someway to resolving this, but its not ideal for environments where such practices constitute a severe security risk.

I should re-iterate here that the Bootdisk Creator formatting limitations appear only to be an issue for DOS and Linux. Bootdisk Creator appears to behave very nicely with WinPE -but then I've only tested with 4GB drives. Either way, formatting your flash drives with WinPE under BootDisk Creator will give you ample space for an image or two and those critical utils without going through a manual process similar to those detailed above.

Kind Regards,
Ian./