Fun with ZFS and Microsoft VHD (Virtual Hard Disk) over the internet

I have this inexpensive on-line storage provider which allows CIFS mounts over the internet to an "unlimited" storage pool. Unlimited in this case is a 2.6TB file system that is usually 95% full and I email them if I'm unable to upload to it. I wouldn't say it is the answer to your storage issues at home or at the office, but for $19 a year, it is cheap entertainment for me.

I started playing off by trying to figure out the different ways I could use it. First and simple, mount it up and copy files. A little boring, but functional none the less. They also offer RSYNC, so naturally, I set up some scripts with my web hosting provider to backup various family websites I have there. The boring stuff stops here.

The next bright idea I had centered around my home NAS, an OpenSolaris/ZFS server that provides a few TBs to the home network with rolling snapshots and replication to my old file server I shoved in the top of a rack in the office lab. The process is pretty simple, and with my method provides you an encrypted remote file system to boot.

mount -F smbfs -o user=(youruserid) //dav.onlinestoragesolution.net/(youruserid) /mntoss (enter your password)

The first step is to create a file on the remote mount. I used this command to create a "sparse" file.

dd if=/dev/zero of=/mntoss/raw/rawimage001.szd bs=1 count=0 seek=50G

Now we have a "50" GB file for your zfs pool.

lofiadm -c aes-256-cbc -a /mntoss/raw/rawimage001.szd (enter your passphrase... don't forget it or you won't see your files again)

zpool create yourpool /dev/lofi/1 (only do this once though, for subsequent zpool imports use zpool import -d /dev/lofi yourpool after you have done your lofiadm)

if you want to get the most out of your storage and network, zfs set compression=gzip-9 yourpool

If I recalled this from memory correctly you should have a new zpool mounted up on your box and you can start copying files and rest at night knowing they are encrypted.

After playing around with this I decided to try the same thing in Windows 7 using VHDs. I created the VHD as a dynamically expanding disk trying to mimic the same sparse layout we did above. Next I created a partition, NTFS file system (50GB) and told it to do a quick format. Then waited......and waited.....and after about 20 minutes I said...well, I can't post that but you get the idea.

I canceled the format which took awhile as well. Then thought back in time to one of the Veritas VVR tricks I've used when setting up new replication pairs; create the volume small (like 10 megs), create the pair relationship and let it replicate. Then after that finishes (rather quickly) grow the volumes to their final large size saving countless hours and stressed out network links from copying zeros between sites.

This worked for my remotely mounted VHD file as well. I re-created the partition but this time made it 10MB. After it quick formatted (which took a minute or so) I navigated back into Disk Management and extended the volume to 50GB which also only took about a minute.

Happy remote storing, if you come up with an interesting way to use it, do share!

Fun with ZFS and Microsoft VHD (Virtual Hard Disk) over the internet