Difference between physical and virtual mode RDM is only the way VM interacts to RDM, in Physical mode the commands from VM is direct to RDM no interference at hypervisor level, so snapshot is not possible.
In Virtual mode, we can take snapshots of VMs. There are some more limitations physical vs virtual but snapshot is something important to know.
https://kb.vmware.com/s/article/2009226 > Difference between Physical compatibility RDMs and Virtual compatibility RDMs
Below are some scenarios:
Scenario 1: (Physical & Virtual mode vMotion)
The vMotion of VM attached with Physical or Virtual mode RDM will work.
Scenario 2: (Physical & Virtual mode Storage vMotion)
The storage vMotion for Physical or Virtual will work as is, if we do not alter any options during the storage vMotion.
Scenario 3: (Storage vMotion with Physical Mode RDM – Changing disk format in migration wizard)
We cannot change the disk format, for example to thin provision if the physical mode RDM attached VM is powered on.
Scenario 4: (Storage vMotion with Virtual Mode RDM – Changing disk format in migration wizard)
If we do storage vMotion and change the disk format for example to thin provision, then the virtual mode RDM will get converted into VMDK without alerting the users about the conversion.
Scenario 5: (Migrating VMs with physical mode RDM for Microsoft clustering/SQL clustering)
Storage vMotion will not work because of SCSI Controller & physical bus sharing mode.
Procedure to do:
1. Shutdown VM2 and VM1 in this order.
2. Remove RDM from VM2.
3. Storage vMotion the VM1 to another storage.
4. Add the RDM back to VM2 and power the VM on.
Scenario 6: (Storage vMotion of VM with virtual mode RDM to VSAN datastore)
This will by default convert the virtual mode RDM to virtual disk, because VSAN by default is thin provisioned. So even if we don’t choose any disk format change during storage vMotion, the virtual mode RDM will be converted.
If you don’t want this to happen, then you need to remove the RDM and storage vMotion the VM to VSAN Datastore first and then attach the virtual mode RDM back.
Scenario 7: (Cloning VM with virtual mode RDM)
This will convert the RDM into VMDK.
Scenario 8: (Making template of VM with Virtual mode RDM)
This will convert the VM to template, the VMs deployed from this VM will have VMDK not RDM.
Other Pointers worth noting:
* While deleting RDM from VM edit setting, there is an option to “Delete files from datastore” this one will not cause any impact to data if it is RDM, because it will only delete the pointer files. However, while doing this for a traditional VMDK, we will lose all the data if this option is selected.
* If we are planning to convert a physical mode RDM to VMDK but without downtime to VM, then we can remove the Physical mode RDM from VM (If the RDM is not being used by application and it is idle) and add it back as Virtual mode RDM and do the conversion while the VM is online.
* Any VMs that has RDM / NFS directly to the guest like windows iSCSI initiator within windows VM, this will not be captured in virtual layer anywhere. So need be careful during migration etc.
* Post conversion of RDMs to VMDK, at storage end make the RDM LUN offline and leave it for a while before it can be permanently deleted.
* Host boot times or storage rescans can take a long time with physical mode RDM, VMware recommends implementing perennial reservation for all ESXi hosts hosting VM nodes with physical RDMs.
For Microsoft Clustering or SQL clustering setup for lab, here is quick steps:
1. Take two Windows VMs.
2. Edit settings of both the VMs and add new SCSI Controller.
3. Add Physical Mode RDM to 1st VM under the newly added SCSI Controller.
4. Add the same physical mode RDM added to 1st VM under the newly added SCSI Controller for the 2nd VM, this time you need to pick add existing disk and add the .vmdk (descriptor file) from 1st VM. This file is pointing file to actual RDM.
Some important links: (Thanks to all for helping me to put all the above together for a project)
https://www.youtube.com/watch?v=X9YomsRznFY – All the above scenarios has been done in lab, thanks to TAM Lab 078 – Migrating & Converting RDMs to VMDKs
https://kb.vmware.com/s/article/1005241 – Migrating virtual machines with Raw Device Mappings (RDMs) (1005241)
https://www.youtube.com/watch?v=u6vfIdoD6Go – Migrating from RDM to shared VMDK
https://core.vmware.com/resource/vsphere-7-rdm-shared-vmdk-migration – https://core.vmware.com/resource/vsphere-7-rdm-shared-vmdk-migration
https://www.youtube.com/watch?v=oLjLsjYTiFU – Migrating virtual machines with Raw Device Mappings
https://kb.vmware.com/s/article/2009226 – Difference between Physical compatibility RDMs and Virtual compatibility RDMs
https://blogs.vmware.com/virtualblocks/2020/07/31/using-perennially-reserved-flag-wsfc-rdms/ – Perennial Reservation
https://kb.vmware.com/s/article/1016106 – ESXi host takes a long time to start during rescan of RDM LUNs
https://kb.vmware.com/s/article/2001823 – Identifying Virtual Machines with RDMs – PowerCLI
https://kb.vmware.com/s/article/1004814 – Identifying Virtual Machines with RDMs – vSphere Client
https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.vm_admin.doc/GUID-4236E44E-E11F-4EDD-8CC0-12BA664BB811.html – Add an RDM Disk to a Virtual Machine
https://kb.vmware.com/s/article/1017530 – Raw Device Mapping for local storage
https://kb.vmware.com/s/article/2147661 – Microsoft Windows Server Failover Clustering (WSFC) with shared disks on VMware vSphere 6.x: Guidelines for supported configurations
https://docs.vmware.com/en/VMware-vSphere/6.7/vsphere-esxi-vcenter-server-67-setup-mscs.pdf – Setup for Failover Clustering and Microsoft Cluster Service
https://blogs.vmware.com/apps/2019/05/wsfc-on-vsphere.html – Hosting Windows Server Failover Cluster (WSFC) with shared disks on VMware vSphere: Doing it right!