VirtualBox: vboxmanage controlvm hangs

T

tingo

I run VirtualBox for virtual machines, and have done that for some years now (I have started testing bhyve, but that isn't relevant here). A few days agon, one of my virtual machines wasn't responding, and in the course of trying to figure out why, I discovered that the machine had crashed, the VBox.log file for the machine says:
Code: 
59:20:49.257805 ACPI: Reset initiated by ACPI
59:20:49.257844 Changing the VM state from 'RUNNING' to 'RESETTING'
59:20:49.258172 PIT: mode=3 count=0x10000 (65536) - 18.20 Hz (ch=0)
59:20:49.259791 PDMR3Reset: after     1 ms, 1 loops: 1 async tasks - ahci/0
I was already messing around with 'vboxmanage controlvm' so I tried to shutdown the machine with
Code: 
tingo@kg-v7$ vboxmanage controlvm FreeBSD-trap poweroff
0%...10%...20%...
and there it hangs. ps on the host shows
Code: 
root@kg-v7# ps awx -t pts/6                                                       PID TT  STAT     TIME COMMAND
50492  6  Is    0:00.03 -sh (sh)
50627  6  S+   19:23.18 /usr/local/lib/virtualbox/VBoxManage controlvm FreeBSD-trap poweroff
Currently, there is one other vm running on this host, and it is working and unaffected by this problem. vboxmanage list runningvms output
Code: 
tingo@kg-v7$ vboxmanage list runningvms
"FreeBSD-v5" {5312412c-353b-41bf-9535-497e1c45de44}
VirtualBox processes on the host
Code: 
root@kg-v7# ps awx | grep VBox
42080  -  S        11:11.58 /usr/local/lib/virtualbox/VBoxXPCOMIPCD
42082  -  S        30:58.24 /usr/local/lib/virtualbox/VBoxSVC --auto-shutdown
42083  -  S       298:59.87 /usr/local/lib/virtualbox/VBoxHeadless --comment FreeBSD-v5 --startvm 5312412c-353b-41bf-9535-497e1c45d
42239  -  S        43:33.71 /usr/local/lib/virtualbox/VBoxHeadless --comment FreeBSD-trap --startvm 36178dfc-fe58-40f8-ba9e-787a4b6
59735  1  S+        0:00.00 grep VBox
50627  6  S+       19:24.35 /usr/local/lib/virtualbox/VBoxManage controlvm FreeBSD-trap poweroff
Has anyone else seen this behavior? Should I do more to figure out why it hangs (before I just kill it)?

The host runs
Code: 
root@kg-v7# uname -a
FreeBSD kg-v7.kg4.no 11.0-STABLE FreeBSD 11.0-STABLE #0 r307729: Fri Oct 21 22:34:13 CEST 2016     root@kg-v7.kg4.no:/usr/obj/usr/src/sys/GENERIC  amd64
and the VirtualBox port version is
Code: 
root@kg-v7# pv virt*
[Reading data from pkg(8) ... - 549 packages found - done]
virtualbox-ose-5.1.6_1      <  needs updating (port has 5.1.10)
virtualbox-ose-kmod-5.1.6   <  needs updating (port has 5.1.10)
Not necessarily relevant, but the guest vm that hangs runs FreeBSD too
Code: 
root@trap# freebsd-version -ku
10.2-RELEASE-p18
10.2-RELEASE-p22
root@trap# uname -a
FreeBSD trap.kg4.no 10.2-RELEASE-p18 FreeBSD 10.2-RELEASE-p18 #0: Sat May 28 08:53:43 UTC 2016      root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64
 
tingo said:
Should I do more to figure out why it hangs (before I just kill it)?
Click to expand...
Seems to me this is a virtualbox problem, maybe ask them?
From my experience, once the VM is in a transition state (Running->something else) there is nothing you can do but kill the process.

tingo said:
59:20:49.259791 PDMR3Reset: after 1 ms, 1 loops: 1 async tasks - ahci/0
Click to expand...
This seems to hint at some unfinished io (async task ahci channel 0)
The process however is in sleep (S) mode. This suggests that VBoxHeadless is just thinking there is unfinished business - again virtualbox problem.
 
Everything is always easier if it is SEP (somebody else's problem). :-)
The reason I asked is because I'm using the FreeBSD port, running on a FreeBSD host. You know, just in case somebody else had seen this under FreeBSD and was interested in more data.
I'll wait a bit more and then I'll just kill it.
 
Yeah, I'd just kill it. Then work to update your VirtualHost installation. The bug may already have been fixed (5.1.6 -> 5.1.10).
 
I've just setup the latest VirtualBox (5.1.12) using pkg on a fresh FreeBSD 10.3 install and see the same (or similar) hang when trying to shutdown or save state of a VM. I've tried a FreeBSD guest, an Ubuntu guest and a Centos guest and see the same issue. The VBox.log shows the the VM has finished running and has sent a request to power off to the VBoxHeadless process but the headless process never exists.

Code: 
01:55:05.739398 PDMR3PowerOff: Driver 'VD'/1 on LUN#1 of device 'lsilogicscsi'/0 took 1 040 032 432 ns to power off
01:55:05.795245 PDMR3PowerOff: 1 935 014 100 ns run time
01:55:05.795286 Changing the VM state from 'POWERING_OFF' to 'OFF'
01:55:05.923560 Console: Machine state changed to 'Stopping'
01:55:05.952638 Console::powerDown(): A request to power off the VM has been issued (mMachineState=Stopping, InUninit=0)
01:55:05.952654 Display::handleDisplayResize: uScreenId=0 pvVRAM=000000080fc00000 w=640 h=480 bpp=32 cbLine=0xA00 flags=0x1

This happens with all guests so looks like an issue with the FreeBSD version of VBoxHeadless to me. My FreeBSD has no GUI installed so everything runs headless and is running root on ZFS so not sure if that is causing an issue.

Have you had any luck solving this issue? If not, I'll try the VirtualBox forums.

Thanks
Russell
 
No, I haven't solved the issue. I finished upgrading up my "production" (I use it only for myself) VirtualBox host to FreeBSD 10.3-stable, installed VirtualBox 5.1.14 and moved my guests back onto the machine. I haven't seen a guest crash or hang since (knock on wood).
Details:
Code: 
root@kg-vm# pv virt*
[Reading data from pkg(8) ... - 59 packages found - done]
virtualbox-ose-kmod-5.1.14  =  up-to-date with port
virtualbox-ose-nox11-5.1.14_2  =  up-to-date with port
root@kg-vm# uname -a
FreeBSD kg-vm.kg4.no 10.3-STABLE FreeBSD 10.3-STABLE #0 r311695: Sun Jan  8 20:58:02 CET 2017     root@kg-vm.kg4.no:/usr/obj/usr/src/sys/GENERIC  amd64
 
I guess I need to wait until I can access virtualbox-5.1.14 then. I'm running 10.3-RELEASE and using pkg to install and it looks like 5.1.12 is the latest version (I did a fresh test install of FreeBSD and VirtualBox last night). Hopefully 5.1.14 will be available soon and I can try that.

Code: 
$ pkg version -vR | grep virt
virtualbox-ose-kmod-5.1.12         =   up-to-date with remote
virtualbox-ose-nox11-5.1.12_1      =   up-to-date with remote

Code: 
$ uname -a
FreeBSD kylie-fbsd 10.3-RELEASE-p11 FreeBSD 10.3-RELEASE-p11 #0: Mon Oct 24 18:49:24 UTC 2016     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64
$ freebsd-version -uk
10.3-RELEASE-p11
10.3-RELEASE-p16

Thanks

Russell
 
Yes, the VBoxHeadless process doesn't terminate when the guest powers off. The log I posted shows the guest saying it has shutdown and requested power off (and using VNC I can see that it has fully shutdown) but the VBoxHeadless process doesn't exit as it should. If I use VBoxManage controlvm to shut the machine down, with a VNC window open, I see exactly the same issue and the VBoxManage controlvm process hangs also, waiting for VBoxHeadless to terminate.

I can confirm that if I don't have a VNC window open to the guest when the guest powers off, then the VBoxHeadless process terminates as expected. I can initiate shutdown from the VM through VNC provided I then close the VM window before it powers off.

This is as described in the ticket I posted. I thought it was related to your issue because if I use VBoxManage controlvm to shut down the guest, that would hang also.

Thanks

Russel
 
Ok, so when you say "graphical desktop" in the ticket you mean "VNC"? Sorry, I didn't connect the dots there.
In my case, there wasn't a graphical desktop (or VNC) involved, but vboxmanage controlvm still hung...
 
When I first posted here, I didn't realise the VNC connection to the VM was the cause. I had only installed guests without graphical desktops also but used VNC to bring up their console during installation and booting. I subsequently found that ticket mentioning the graphical desktops but ultimately the issue occurs just because of the VNC connection, not the graphical desktop in the guest.

But yes, it does appear my issue is separate from yours so sorry for posting to your thread.
 
i have this exact same problem... will stop just fine if no vnc active connection; 10.3 w/ virtualbox 5.1.14
last message when it stucks to stopping state is:
Code: 
00:00:43.103726 Console::powerDown(): A request to power off the VM has been issued (mMachineState=Stopping, InUninit=0)
00:00:43.103756 Display::handleDisplayResize: uScreenId=0 pvVRAM=0000000000000000 w=720 h=400 bpp=32 cbLine=0x0 flags=0x1
 
You must log in or register to reply here.
Back
Top