Archiving Large Files to a FAT Drive

Book DriveManaging personal data storage is one of those ongoing challenges, especially as digital data sources increase in size and frequency. One of my archive solutions is a Western Digital Book Drive (so named because of it's size). It's 500 gigabytes and has both Firewire and USB2 connections, perfect for use on both my MS Windows XP system and Macintosh PowerBook G4. It's been painless, up till now.

To use it on both systems I need to use the FAT file system, which is the file system it comes formatted with out-of-the-box. This was fine until I started video editing. Now I have files exceeding the maximum file size FAT can handle. This is doubly bad since I do my video editing on my Powerbook with it's 80 gigabyte main disk drive and CDROM. Not only do I need all the space I can get but most video projects exceed 640 megabytes, so archiving to CD is tedious in the extreme.

OS X Unix to the rescue. I've solved the problem of archiving to my Book Drive by using the tar and split commands. Open a terminal console and go to the directory you keep your projects in. For example, Movies. For the sake of this example, we'll say the files are in a directory called Project1. To archive:

~/Movies dale$ tar cvzf Project1archive.tgz Project1
~/Movies dale$ split -b 1000m Project1archive.tgz Project1archive.tgz-
~/Movies dale$ ls
Project1archive.tgz-aa Project1archive.tgz-ab Project1archive.tgz-ac Project1archive.tgz Project1

In this example, we archived all the contents of the Project1 directory into a single tarball, then with the split command chunked the tarball into 1 gigabyte files. Split automatically appends aa, ab, ac, etc to whatever prefix you give it. In our example, the prefix was "Project1archive.tgz-".

I can now copy the files to my FAT hard drive, since none of the files exceed the maximum FAT file size. The Project1archive.tgz file is no longer required and can be deleted.

To reverse the process, copy the files back and:

~/Movies dale$ cat Project1archive.tgz-* > Project1archive.tgz
~/Movies dale$ tar xvzf Project1archive.tgz Project1

Do a number of dry runs to insure it works for you before deleting your original project files for real!

If you have a great way of archiving large files please leave a comment and share!

Comments

I didn't realize that there was a limit to the size of the file! Bummer!

An easy PC solution for splitting files is Dekabyte's Filesplitter. Totally free and you can set the size of each part of the file and program will split the file into equal pieces of the size you specified with the left over in a third file. Also great for splitting files for emailing or burning since it comes built-in with pre-set sizes for floppy, CDR and DVDR. Pretty spiffy for a free program!

http://www.dekabyte.com/filesplitter/