Scenario
I have a 3 Windows 2012 R2 Hyper-V hosts which is participating in a cluster, all of the virtual machines are hosted on CSV's. We also use Dynamic Memory for most of the servers
We do regular reboots every week for our virtual machines, but we've found that sometimes, a virtual guest will suddenly stop requesting additional memory. At the time of writing, I have identified 10 machines - on different hosts, and even different clusters which are exhibiting this behaviour. They're also running OS's ranging from Windows 7 to Windows 2008 R2 and Windows 2012.
I have selected one to demonstrate below.
Here, you can see the virtual machine is configured to use Dynamic Memory
and here, you can see that it's not actually demanding more than the start memory, despite the fact that I've overloaded the servers virtual memory by using MSPAINT.
Workaround
Rebooting the virtual machine does not resolve the issue, but live migrating the virtual machine from the host it is on to another host suddenly allows it to demand more memory. One can then live-migrate it back to the original host and the problem
is no longer there.
Troubleshooting
We have performed the following troubleshooting steps:
- Verified that the Integration Services are up-to-date (currently running 6.3.9600.16384)
- Verified that the virtual server has up-to-date Windows Updates
- Checked the Hyper-V-Worker\Admin event log on the current host - no errors pertaining to this VM.
- Verified that the Hyper-V host has sufficient RAM (currently only using 542 out of 768 GB)
- Checked the System event log on the virtual machine - nothing out of the ordinary to report.
- Verified that all services which start with the word "Hyper-V" on the guest machine is running.
Using the following command, I can easily identify which machines are affected (this command applies to SCVMM only):
Get-SCVirtualMachine | where {$_.DynamicMemoryEnabled -eq $true -and $_.DynamicMemoryDemandMB -eq $_.Memory} | ft Name,HostName,OperatingSystem,Memory,DynamicMemoryDemandMB,DynamicMemoryMinimumMB,DynamicMemoryMaximumMBI do not know any diagnostic program which can give me more information regarding the memory status of a server - specifically one which can tell me more information about Memory demand, and why it's not working.
Considering that the problem is resolved when I live migrate the affected machine, it would suggest that the issue is with the host, but as live migrating it back no longer exhibits the issue - that seems to negate that theory. Additionally, the other 103 virtual machines which run on this host seem to not show this issue.
It's almost as if the machine is being ran with static memory, and Hyper-V doesn't realise it's dynamic, but the moment it's live migrated, it suddenly realises this and enables dynamic memory.
Any thoughts?