bhyve hw.vm.* settings for a dual Xeon L5640

huckfinn

huckfinn

Hi, I'm running FreeBSD 11.1 on an Fujitsu Siemens RX300 S6, with two Xeon L5640. I've to use for software legacy reaons an Ubuntu 16.04 LTS to develop a large set of PhaseOne IIQ images (SDK is released for Linux 16.04). I use bhyve and it worked very well out of the box . Thank you FreeBSD developers for that fine peace of software and the documentation. Now I want to provide a maximum of system resources to the guest OS, to optimize the calculation time .

The tool dmesg gives me the following informations:
Code: 
CPU: Intel(R) Xeon(R) CPU           L5640  @ 2.27GHz (2266.79-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x206c2  Family=0x6  Model=0x2c  Stepping=2
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x29ee3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,DCA,SSE4.1,SSE4.2,POPCNT,AESNI>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
  TSC: P-state invariant, performance statistics
real memory  = 51539607552 (49152 MB)
avail memory = 50015694848 (47698 MB)

...

FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs
FreeBSD/SMP: 2 package(s) x 6 core(s) x 2 hardware threads

I guess I've to set the parameter but have no idea how to set the number of sockets:
Code: 
...
hw.vmm.topology.threads_per_core = 2
hw.vmm.topology.cores_per_package = 6
in /etc/rc.conf.

From the forum discussion Thread bhyve-cpu-vcpu-cores-nad-threads.60169 I've learned, that the number of CPU's which can be declared with the parameter bhyve -c NUM is calculated by:
Code: 
NUM = sockets * hw.vmm.topology.threads_per_core * hw.vmm.topology.cores_per_package

Questions:

1. Is common or are there side effects to give the VM the all CPU resources?
2. Is there a paramter to tell the VM environment to use all sockets?
3. Without setting any hw.topology.* parameter, I can address 16 CPU's. How does the bhyve resource management works here and is there a tutorial, how to optimize the a configuration?

Bash: 
#!/bin/sh

VM_NAME=ubuntu-16.04
VM_CPUS=16
VM_MEM=16G
VM_GRUB=$VM_ROOT/$VM_NAME.map
VM_IMG=$VM_ROOT/img/$VM_NAME.img

grub-bhyve -m $VM_GRUB -r hd0,msdos1 -M $VM_MEM $VM_NAME &&\
bhyve -A -H -P\
                  -s 0:0,hostbridge\
                  -s 1:0,lpc\
                  -s 2:0,virtio-net,tap0\
                  -s 3:0,virtio-blk,$VM_IMG\
                  -s 4:0,ahci-cd,$VM_ISO\
                  -l com1,stdio\
                  -c $VM_CPUS -m $VM_MEM\
                  $VM_NAME
#EOF

Thank you huck.
 
I've learned, that the number of CPU's which can be declared with the parameter bhyve -c NUM is calculated by:
Click to expand...

The maximum number of vcpu in bhyve is 16, regardless of the sysctl settings. They only control how the processors appear in the guest. Whether there's an actual benefit to specifying multiple cores per package (due to shared caches) probably needs the devs to answer. One of the main reasons for these sysctl's is that some Windows versions will happily run with 1 or 2 multi-core packages, but not with 3+ packages.

With a 16 vcpu guest, you just have 16 "bhyve vcpu" threads competing for cpu time, just like any other process -

Code: 
  PID USERNAME   PRI NICE   SIZE    RES STATE   C   TIME    WCPU COMMAND
26185 root        79    0   281M  6348K RUN     1   0:02  36.20% bhyve{vcpu 15}
26185 root        80    0   281M  6348K RUN     1   0:03  33.90% bhyve{vcpu 9}
26185 root        79    0   281M  6348K RUN     3   0:03  27.33% bhyve{vcpu 11}
26185 root        80    0   281M  6348K CPU3    3   0:04  24.56% bhyve{vcpu 4}
26185 root        78    0   281M  6348K RUN     3   0:04  24.44% bhyve{vcpu 0}
26185 root        79    0   281M  6348K RUN     2   0:03  24.21% bhyve{vcpu 5}
26185 root        79    0   281M  6348K RUN     0   0:03  23.59% bhyve{vcpu 7}
26185 root        79    0   281M  6348K RUN     0   0:03  23.59% bhyve{vcpu 3}
...
 
Nope, maxcpu is still currently hardcoded to 16

/sys/amd64/include/vmm.h
Code: 
#define VM_MAXCPU       16                      /* maximum virtual cpus */

There's some talk of possibly converting this to a sysctl. Also a possible change to allow cpu/cores/threads to be specified per guest via the bhyve command line. Hopefully something will get in before FreeBSD 12, maybe even an 11.x release, but nothing seems to have been committed so far.
 
You must log in or register to reply here.
Back
Top