I just ran into a MAJOR issue with the SimpliVity Backups and vCloud Director. As you know vCloud Director consolidates datastores into one large logical datastore cluster and abstracts this storage resource just like it does with the Virtual Data Center. During the migration from a legacy StoreVirtual Storage Area Network (SAN) to SimpliVity Hyperconverged storage, vCloud Director will randomly move the VMX configuration file and VMDK disk files to different datastores for a given Virtual Machine (VM). When this happens the SimpliVity backups will fail. SimpliVity can ONLY backup the VMDKs files if there are located in the same datastore as the VMX VM configuration file. You may receive the error: "SimpliVity Partial VM Backup Detected." In the case of a scattered VM, this means that the VMDK was NOT backed up if it resides on a different datastore than the VMX file. I found this support link https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-sv14832en_us&docLocale=en_US, however I did not receive the error message: "Backup failed due to scattered VMDK files", I only received the "SimpliVity Partial VM Backup Detected." I opened a case with HPE support and they acknowledged this issue. I suggested they at least change the error message to something more meaningful like "SimpliVity Backup Failed" or better yet support a SimpliVity Backup with a scattered VM. Fortunately, the fix is relatively simple you just have to perform a Storage vMotion and move all of the VMDK files to the same datastore and folder where the VMX resides. Here's the steps I performed to fix this issue:
- Review each VM.
- Right click, All HPE SimpliVity Actions, Search Backups.
- Review the SimpliVity Backup Size. If the SimpliVity size is smaller than the expected backup size, you probably have an incomplete backup, because of a scattered VM.
- If the VM is scattered note the Datastore where the SimpliVity backup resides.
- If the VM is scattered, perform a storage vMotion of the VM to the datastore where the SimpliVity backup is stored. This will move all of the VMDKs and VMX files to the same folder on the datastore.
- Clear the SimpliVity Partial VM Backup warning message and make sure it doesn't come back for the VM.
- Make sure to check all of the VMs even the ones that do not have the warning. I caught a few VMs that didn't have the warning of a partial SimpliVity Backup that were still scattered.
Bottom line - do NOT store the vmx and vmdks for a VM in different datastores or the SimpliVity backup will fail!
When we discussed the backup strategy with SimpliVity it was their opinion that a SimpliVity Backup was adequate to protect the VMs, especially if the SimpliVity backups were replicated to a remote cluster. We strongly recommend to all of our clients to store their backups off line and off site at some point for additional protection. If your ESXi infrastructure is compromised a hacker can delete all of your on-line backups, steal your data, and permanently delete all of your company's data. For this reason, we still use Veeam to backup SimpliVity VMs both to disk and tape. Veeam can properly backup a scattered VM. This additional backup really saved us in this case, because the SimpliVity Backups were not properly working for scattered VMs. You can never be too careful with backups! It's better to have multiple copies of the same VM versus not having one at all. To quote my friend Mark Minasi, backup failures can be an "RGE – Resume Generating Event." Avoid RGEs and make sure you get good backups!