Teaching VMs to Share and Share Alike

I've always marveled at how a child with no interest in a toy must have it the moment another child picks it up.

My vSphere guest VMs refused to share a raw device mapped LUN that we use for in-band management of HDS arrays. My current version (4.0.0 build 261574) refuses to even allow me to present the same raw LUN to multiple VMs, something I know prior versions allowed me to do so long as only one of them used it at a time. While this works, it's a pain to manage in a dynamic lab environment where anyone might need to use one.

Enter SCSI Bus Sharing, allowing multiple VMs to um, share a SCSI bus. This feature is more traditionally used for clustering VMs, but it turns out to suit my needs quite well.

There are two modes for SBS: virtual sharing and physical sharing. Virtual sharing means VMs on the same host can share the SCSI bus; physical sharing means VMs across servers can share the bus.

Unfortunately, neither mode will allow VMotion of a running VM (or suspended; see below), hence I probably wouldn't recommend this for general production use.

But for lab use it makes a lot of sense. Here's how I recommend doing it.

1. Remove the command device RDM from any guest OS configuration(s).
2. In the VI Client Datastores view, browse to a shared datastore and create a folder (ie, "cci" or "cmd-devs")
3. In the VI Client Hosts and Clusters view, browse to a host's Storage Adapter configuration and find the raw LUN to which you want to map. Right-click the LUN and select "Copy identifier to clipboard"
   
4. Open an SSH session to an ESX host and cd into /vmfs/volumes/'datastore'/'folder'
5. Create a raw device map: vmkfstools -createrdm /vmfs/devices/disks/ :
   vmkfstools --createrdmpassthru /vmfs/devices/disks/naa.60060e800564a400000064a4000000ff uspvm_cmddev_00ff.vmdk
   
6. Back in the VI Client, go to VMs and Templates and select your first VM and "Edit Settings..."
7. "Add..." a new hard disk. Select "Use existing virtual disk" and when prompted browse to the datastore and folder where the command device vmdk is located.
8. Map the RDM to an unused SCSI node (X:Y where X is the SCSI controller and Y is the SCSI bus ID of the new hard disk). Most commonly 1:0 should be available, but you need something other than SCSI controller 0.
9. Mark the new disk as Independent and select Persistent mode.
10. Click OK to close the Virtual Machines Properties dialog.
11. Re-open the VM Properties dialog box and select the new SCSI controller. On the right-hand side select the Physical bus sharing mode.
12. Close the dialog box and restart your VM.

Repeat as necessary. Remember you will be unable to VMotion these guests while they are running. VI Client will report the guest has a SCSI controller engaged in bus sharing. (Interestingly, I was able to VMotion a suspended guest, but the guest automagically migrated back to the original host when I powered it back on.)

Teaching VMs to Share and Share Alike