I have two Hyper-V HA clusters setup with one replicating to the other. Each cluster has 2 2012r2 Hyper-V hosts and is pointed to a SMBv3 file server for storage.
Saturday morning the primary(Active) cluster started randomly moving VM's from one server to the other. Went in and manually moved everything back like I wanted it and seemed ok.
Sunday morning I have NO VM's left. Every VHDX on EVERY VM is completely trashed... Booting into OS recovery on the VM and going to command prompt shows that it does see the partitions on drives but sees the file systems as RAW. Trying to open or mount the VHD's on the host or on the storage server both result in a File or Directory is corrupted and unreadable.
Thankfully the replica cluster did it's job and I booted everything back up.
What on earth could have happened on the first cluster to kill ALL the VM's? I have never seen the like, 12 VM's all at once have their HDDs trashed, I want to figure out what went wrong so I can make sure it doesnt happen again.
Anyone have any thoughts?