Environment:
Two Windows 2012 Hyper-V nodes in a failover cluster
Single 9TB CSV on an Equalogic 4100x SAN with iSCSI
All VMs and VHDs live in this CSV
VMM 2012, DPM 2012
Last week my primary file server experienced catastrophic file corruption on a 2tb data volume VHD. The issue was first discovered because DPM was reporting that it failed a backup due to file corruption, and by the time we decided to shut down many folders/files were already lost. That server has been rebuilt and data restored from backup. Root cause was never determined but it was attributed to a 6 month old checkpoint that i deleted the night before in VMM.
Fast forward a few days and now my engineering server is reporting corruption in it's data volume (1tb VHDX). No snapshots or checkpoints have been created or ever existed on this VM except those taken by DPM 3 times a day. My Dell folks have run diagnostics on the SAN and Microsoft has looked at my Hyper-V and DPM systems looking for clues, nothing so far. I did find an issue in DPM which might cause corruption in a CSVhttp://support.microsoft.com/kb/2802159/en-us but MS did not think it was related.
At this point, my boss is starting to feel that moving to Hyper-V/System Center was a mistake and I dont blame her. 3 days of major system interruptions and many long nights for me, all after moving to a system which promised better availability. How does one do root cause analysis on a corrupted disk to isolate possible causes? How can I protect against corruption when the corruption causes the backup to fail? Why does Microsoft label my case Priority A "critical" and then not call me back for several hours? Can these kinds of corruptions inside the VHD stem from Hyper-V? Virus?