Solved tap interface doesn't seems to receive anything

nerozero

nerozero

Hello,

I have an issue with tap interface participating the bridge. The ]tap interface doesn't seems to receive anything, but it does send staff.
I have recently update the BSD
# uname -mrs -> FreeBSD 12.1-RELEASE-p7 amd64

Here is what I done:
Code: 
# bge0 - lan interface
# sysctl net.link.tap.up_on_open
net.link.tap.up_on_open: 1

# ifconfig bridge0 create
# ifconfig tap11 create
# ifconfig bridge0 addm tap11 bge0

# ifconfig bge0 up # just in case...
# ifconfig tap11 up

# ifconfig tap11
tap11: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=80000<LINKSTATE>
    ether 00:bd:02:1a:f7:0b
    groups: tap
    media: Ethernet autoselect
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
    Opened by PID 3777

# ifconfig bridge0
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 02:76:87:07:be:00
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: bge1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 2 priority 128 path cost 2000000
    member: tap11 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 5 priority 128 path cost 2000000
    member: bge0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 1 priority 128 path cost 55
    groups: bridge
    nd6 options=9<PERFORMNUD,IFDISABLED>


Now if I run dhclient inside virtual machine (which is using tap11) here is what I got in tcpdump:

Code: 
tcpdump -vv -i <interface>
---------------------------------------------------------------------------
BRIDGE

14:23:00.740429 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from 00:a0:98:c4:01:57 (oui Unknown), length 300, xid 0x146abb0f, Flags [none] (0x0000)
      Client-Ethernet-Address 00:a0:98:c4:01:57 (oui Unknown)
      Vendor-rfc1048 Extensions
        Magic Cookie 0x63825363
        DHCP-Message Option 53, length 1: Discover
        Parameter-Request Option 55, length 7: 
          Subnet-Mask, BR, Time-Zone, Default-Gateway
          Domain-Name, Domain-Name-Server, Hostname
        Client-ID Option 61, length 19: hardware-type 255, 98:c4:01:57:00:01:00:01:26:9c:6b:51:00:a0:98:c4:01:57
14:23:00.741089 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    10.51.0.1.bootps > 10.51.0.54.bootpc: [udp sum ok] BOOTP/DHCP, Reply, length 300, xid 0x146abb0f, Flags [none] (0x0000)
      Your-IP 10.51.0.54
      Client-Ethernet-Address 00:a0:98:c4:01:57 (oui Unknown)
      Vendor-rfc1048 Extensions
        Magic Cookie 0x63825363
        DHCP-Message Option 53, length 1: Offer
        Server-ID Option 54, length 4: 10.51.0.1
        Lease-Time Option 51, length 4: 86400
        Subnet-Mask Option 1, length 4: 255.255.255.0
        BR Option 28, length 4: 10.51.0.255
        Default-Gateway Option 3, length 4: 10.51.0.1
        Domain-Name Option 15, length 4: "itcs"
        Domain-Name-Server Option 6, length 4: 10.51.0.1


--------------------------------------------------------------------------
TAP

14:22:41.641348 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from 00:a0:98:c4:01:57 (oui Unknown), length 300, xid 0x68a8bd2c, secs 58, Flags [none] (0x0000)
          Client-Ethernet-Address 00:a0:98:c4:01:57 (oui Unknown)
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message Option 53, length 1: Discover
            Parameter-Request Option 55, length 7: 
              Subnet-Mask, BR, Time-Zone, Default-Gateway
              Domain-Name, Domain-Name-Server, Hostname
            Client-ID Option 61, length 19: hardware-type 255, 98:c4:01:57:00:01:00:01:26:9c:6b:51:00:a0:98:c4:01:57
14:23:00.740422 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from 00:a0:98:c4:01:57 (oui Unknown), length 300, xid 0x146abb0f, Flags [none] (0x0000)
          Client-Ethernet-Address 00:a0:98:c4:01:57 (oui Unknown)
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message Option 53, length 1: Discover
            Parameter-Request Option 55, length 7: 
              Subnet-Mask, BR, Time-Zone, Default-Gateway
              Domain-Name, Domain-Name-Server, Hostname
            Client-ID Option 61, length 19: hardware-type 255, 98:c4:01:57:00:01:00:01:26:9c:6b:51:00:a0:98:c4:01:57
14:23:09.172556 IP (tos 0x10, ttl 128, id 0, offset 0, flags [none], proto UDP (17), length 328)
    0.0.0.0.bootpc > 255.255.255.255.bootps: [udp sum ok] BOOTP/DHCP, Request from 00:a0:98:c4:01:57 (oui Unknown), length 300, xid 0x146abb0f, secs 8, Flags [none] (0x0000)
          Client-Ethernet-Address 00:a0:98:c4:01:57 (oui Unknown)
          Vendor-rfc1048 Extensions
            Magic Cookie 0x63825363
            DHCP-Message Option 53, length 1: Discover
            Parameter-Request Option 55, length 7: 
              Subnet-Mask, BR, Time-Zone, Default-Gateway
              Domain-Name, Domain-Name-Server, Hostname
            Client-ID Option 61, length 19: hardware-type 255, 98:c4:01:57:00:01:00:01:26:9c:6b:51:00:a0:98:c4:01:57

Please pay attention to tap mac address and the address which is visible from the vm guest:
mac on guest 00:a0:98:c4:01:57
mac on tap 00:bd:02:1a:f7:0b

also the received packet counter on the guest machine shows 0 all the time....

here is the output of the netstat: (credits to <RhodiumToad> on freebsd irc)
- the amount of Oerrs on bridge, slowly increasing...
- the amount of Opkts on tap11 - stay the same while Ipkts increasing ...


Code: 
# netstat -in
Name    Mtu Network       Address              Ipkts Ierrs Idrop    Opkts Oerrs  Coll
bge0   1500 <Link#1>      4c:d9:8f:b6:0b:9d 232753649     2     0 119002207     0     0
bge0      - 10.51.0.0/24  10.51.0.97        232729261     -     - 118991519     -     -
bge1   1500 <Link#2>      4c:d9:8f:b6:0b:9e     9068     0     0    27695     0     0
lo0   16384 <Link#3>      lo0                    780     0     0      780     0     0
lo0       - ::1/128       ::1                      0     -     -        0     -     -
lo0       - fe80::%lo0/64 fe80::1%lo0              0     -     -        0     -     -
lo0       - 127.0.0.0/8   127.0.0.1              780     -     -      780     -     -
bridg  1500 <Link#4>      02:76:87:07:be:00    39509     0     0 119030052 13351     0
tap11  1500 <Link#5>      00:bd:02:1a:f7:0b     2035     0     0      150     0     0

I think i'am not the only one who experience the same issue: https://forums.freebsd.org/threads/iohyve-networking-issue.76037/#post-468970
also please note, no firewalls are enabled....


Thanks in advance...
 
Last edited by a moderator:
You seem to have messed up your bridge. You have bge0 and bge1 attached to the bridge. igb0 has an IP address, igb1 has not. Why are they both attached to the bridge? I would suggest removing bge0 from the bridge. Use bge1 as your 'uplink' and tie the tap* interface to that bridge. Reserve igb0 as a dedicated interface to manage the host itself.
 
SirDice, Thanks for reply!
SirDice said:
You seem to have messed up your bridge
Click to expand...
No, i'm not.
bge1 was added to the bridge for testing reason. I had a notebook connected to the bge1 and it does obtain IP address and works perfectly. no issues what so ever...
even more, if I assign IP address to a tap device say:
ifconfig tap11 inet 10.51.0.222/24 it become pingable from the entire network. it is possible i'm wrong but IMHO this is the bhyve issue....

here some more info. If I set
sysctl set net.link.tap.debug=1
i can see the tap messages in the /var/log/messages log:
Code: 
Jul 13 02:56:16 srvhost kernel: tap11 is closed. minor = 0xb
Jul 13 02:56:16 srvhost kernel: tap11: link state changed to DOWN
Jul 13 02:56:16 srvhost kernel: tap11 is open. minor = 0xb
Jul 13 02:56:16 srvhost kernel: tap11: link state changed to UP
Jul 13 02:56:17 srvhost kernel: tap11 starting
Jul 13 02:56:38 srvhost syslogd: last message repeated 5 times
Jul 13 02:56:52 srvhost kernel: tap11 writing, minor = 0xb
Jul 13 02:56:52 srvhost kernel: tap11 starting
Jul 13 02:56:52 srvhost kernel: tap11 writing, minor = 0xb
Jul 13 02:56:52 srvhost syslogd: last message repeated 2 times
Jul 13 02:56:53 srvhost kernel: tap11 starting
Jul 13 02:56:53 srvhost kernel: tap11 writing, minor = 0xb
Jul 13 02:56:55 srvhost syslogd: last message repeated 5 times
Jul 13 02:56:55 srvhost kernel: tap11 starting
Jul 13 02:56:57 srvhost kernel: tap11 writing, minor = 0xb
Jul 13 02:56:59 srvhost kernel: tap11 starting
Jul 13 02:57:00 srvhost kernel: tap11 writing, minor = 0xb
Jul 13 02:57:00 srvhost kernel: tap11 starting
Jul 13 02:57:00 srvhost syslogd: last message repeated 1 times
Jul 13 02:57:01 srvhost kernel: tap11 writing, minor = 0xb
Jul 13 02:57:03 srvhost kernel: tap11 starting
Jul 13 02:57:06 srvhost syslogd: last message repeated 3 times
Jul 13 02:57:08 srvhost kernel: tap11 writing, minor = 0xb
Jul 13 02:57:08 srvhost kernel: tap11 starting
Jul 13 02:57:22 srvhost syslogd: last message repeated 3 times
(yes, it was 3am... :( my brain was almost offline)

Couple of minutes ago I restarted the bsd and immediately after starting I ran the bhyve, it does take the IP from DHCP and ping was working for couple of seconds ... then it stops ....

:(
 
nerozero said:
bge1 was added to the bridge for testing reason.
Click to expand...
Don't add igb0 interface to the bridge. Leave it as a dedicated management interface. Use a dedicated interface (igb1) to bridge your VMs with. Don't add both igb0 and igb1 to the bridge as you did here:
Code: 
# ifconfig bridge0
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 02:76:87:07:be:00
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: bge1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 2 priority 128 path cost 2000000
    member: tap11 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 5 priority 128 path cost 2000000
    member: bge0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 1 priority 128 path cost 55
    groups: bridge
    nd6 options=9<PERFORMNUD,IFDISABLED>

nerozero said:
even more, if I assign IP address to a tap device say:
Click to expand...
Don't add an IP address to the tap interfaces.

nerozero said:
it is possible i'm wrong but IMHO this is the bhyve issue....
Click to expand...
No, you just fail to understand how bridge(4) works.

This is my configuration, it has been running perfectly fine for quite some time now.
Code: 
# vm list
NAME            DATASTORE  LOADER     CPU  MEMORY  VNC           AUTOSTART  STATE
case            default    bhyveload  4    4096M   -             Yes [3]    Running (2532)
jenkins         default    bhyveload  4    4096M   -             Yes [5]    Running (3059)
kdc             default    none       2    2048M   0.0.0.0:5901  Yes [2]    Running (2196)
debian          stor10k    grub       2    4096M   -             No         Stopped
gitlab          stor10k    bhyveload  4    6144M   -             Yes [10]   Running (4837)
gitlab-runner   stor10k    bhyveload  4    4096M   -             Yes [11]   Running (4859)
kibana          stor10k    bhyveload  4    6144M   -             Yes [1]    Running (1999)
lady3jane       stor10k    bhyveload  4    4096M   0.0.0.0:5900  Yes [12]   Running (4774)
plex            stor10k    bhyveload  4    4096M   -             Yes [6]    Running (2582)
riviera         stor10k    bhyveload  2    4096M   -             Yes [9]    Running (4559)
sdgame01        stor10k    bhyveload  4    4096M   -             No         Stopped
tessierashpool  stor10k    bhyveload  4    32768M  -             Yes [4]    Running (76800)
wintermute      stor10k    bhyveload  4    4096M   -             Yes [8]    Running (4358)
Code: 
# vm switch list
NAME     TYPE      IFACE       ADDRESS  PRIVATE  MTU   VLAN  PORTS
servers  standard  vm-servers  -        no       9000  11    lagg0
public   standard  vm-public   -        no       9000  10    lagg0
Code: 
# ifconfig lagg0.10
lagg0.10: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9000
        description: vm-vlan-public-lagg0.10
        ether 00:25:90:f1:58:39
        inet6 fe80::225:90ff:fef1:5839%lagg0.10 prefixlen 64 scopeid 0xa
        groups: vlan vm-vlan viid-bdfd6@
        vlan: 10 vlanpcp: 0 parent interface: lagg0
        media: Ethernet autoselect
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
# ifconfig lagg0.11
lagg0.11: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 9000
        description: vm-vlan-servers-lagg0.11
        ether 00:25:90:f1:58:39
        inet6 fe80::225:90ff:fef1:5839%lagg0.11 prefixlen 64 scopeid 0x8
        groups: vlan vm-vlan viid-8bf4d@
        vlan: 11 vlanpcp: 0 parent interface: lagg0
        media: Ethernet autoselect
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
Code: 
# ifconfig vm-public
vm-public: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
        ether fe:d9:f0:91:05:f5
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: tap11 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 22 priority 128 path cost 2000000
        member: tap10 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 21 priority 128 path cost 2000000
        member: tap9 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 20 priority 128 path cost 2000000
        member: tap8 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 19 priority 128 path cost 2000000
        member: tap6 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 17 priority 128 path cost 2000000
        member: tap5 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 16 priority 128 path cost 2000000
        member: tap4 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 15 priority 128 path cost 2000000
        member: tap3 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 14 priority 128 path cost 2000000
        member: lagg0.10 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 10 priority 128 path cost 2000000
        groups: bridge vm-switch viid-4c918@
        nd6 options=1<PERFORMNUD>
Code: 
# ifconfig vm-servers
vm-servers: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
        ether 6e:8c:4e:5e:3d:7f
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: tap7 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 18 priority 128 path cost 2000000
        member: tap2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 13 priority 128 path cost 2000000
        member: tap1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 12 priority 128 path cost 2000000
        member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 11 priority 128 path cost 2000000
        member: lagg0.11 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 8 priority 128 path cost 2000000
        groups: bridge vm-switch viid-d5539@
        nd6 options=1<PERFORMNUD>

To clarify, there are NO IP addresses on the tap*, bridge(4), lagg0.10, lagg0.11, igb1 and igb2 interfaces. The ONLY interface that has an IP address is igb0, which I use as a dedicated management interface for the host. That interface is NOT connected to any of the bridges or LAGG interfaces.
 
SirDice, here is my current entire rc.conf file:
Code: 
hostname="srvhost"
   
# // network->interfaces //
#  
gateway_enable="YES"
ifconfig_bge0="inet 10.51.0.97 netmask 255.255.255.0"
defaultrouter="10.51.0.1"
   

# // network->services //

sshd_enable="YES"
   
# // virtualization //
kld_list="nmdm vmm"
   
cloned_interfaces="bridge0 tap11 up"
ifconfig_bridge0="addm bge0 addm tap11"

zfs_enable="YES"
dumpdev="NO"
sendmail_enable="NONE"

the bhyve run script:
Code: 
/usr/sbin/bhyve -A -H -P -c 4 -m 4096M \
    -s 0:0,hostbridge \
    -s 2:0,virtio-net,tap11 \
    -s 4:0,ahci-cd,/common/distro/recovery/systemrescuecd-6.0.7.iso \
    -s 29,fbuf,tcp=0.0.0.0:5900 \
    -s 30,xhci,tablet \
    -s 31:0,lpc -l com1,stdio \
    -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd sysrescuecd

the idea is to pass the network to the guest machine....
if you see something stupid and/or wrong - please, please, please tell me exactly where.... I have tested all possible ways..... I'm out of options....
no firewall, almost no software, nothing...
 
Let me repeat what I said:
SirDice said:
Don't add igb0 interface to the bridge. Leave it as a dedicated management interface. Use a dedicated interface (igb1) to bridge your VMs with.
Click to expand...

Remove gateway_enable, you don't need it. The host doesn't route anything.

Change this:
Code: 
ifconfig_bridge0="addm bge0 addm tap11"
To:
Code: 
ifconfig_bridge0="addm bge1 addm tap11"
Attach a cable to bge1 to your switch.
 
SirDice,
this doesn't seems to help...
here is what I did according to your suggestion:
Code: 
bge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=c019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
    ether 4c:d9:8f:b6:0b:9d
    inet 10.51.0.97 netmask 0xffffff00 broadcast 10.51.0.255
    media: Ethernet autoselect (1000baseT <full-duplex,master>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bge1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=c0099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE>
    ether 4c:d9:8f:b6:0b:9e
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
    inet 127.0.0.1 netmask 0xff000000
    groups: lo
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 02:76:87:07:be:00
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: bge1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 2 priority 128 path cost 2000000
    member: tap11 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 5 priority 128 path cost 2000000
    groups: bridge
    nd6 options=1<PERFORMNUD>
tap11: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=80000<LINKSTATE>
    ether 00:bd:a9:d0:07:0b
    groups: tap
    media: Ethernet autoselect
    status: active
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
    Opened by PID 3335

The issue is still there ... amount of received packets on the guest machine - 0, the tcpdump shows similar situation... dump on bridge side works OK, dump on the tap side shows only outgoing packets ....

the weird staff that the very same configuration does worked 2 month ago while i tested the machine ... not i will take a brake and will reinstall machine using earlier bsd image .without updates... hope this will help... or i don't know.. lost hope
 
This is may error or create a cloned interface named up:
Code: 
cloned_interfaces="bridge0 tap11 up"
Doesn't appear to have an impact though, but remove that up in any case, it doesn't belong there.
 
I have manually created bridge and tap, I up-ed both ... the result is the same...
Anyway, SirDice, Thank you for quick replies! I will post my progress .... but first I will take a brake... need to sleep a bit...
 
Can you give the VW switch a network/IP address? The VM guest OS will possess an IP address within the network range. You can then edit your pf.conf "set skip interface bridge0". You may want to issue the set skip interface again specifically for the VM switches - e.g. set skip interface vm-public. And do not forget to NAT the traffic for the IP block/interface.
 
So .... The issue with network was redirect stdin to /dev/null ....

how to replicate:
Code: 
#!/bin/sh

/usr/sbin/bhyve -A -H -P -c 4 -m 4096M \
    -s 0:0,hostbridge \
    -s 2:0,virtio-net,tap11 \
    -s 4:0,ahci-cd,/common/distro/recovery/systemrescuecd-6.0.7.iso \
    -s 29,fbuf,tcp=0.0.0.0:5900 \
    -s 30,xhci,tablet \
    -s 31:0,lpc -l com1,stdio \
    -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd sysrescuecd < /dev/null &

pay attention that /dev/null is fed as stdin ...

now if you run vm - it will perfectly run, but - will not receive any packets threw the tap interface ....

but if you run change the script to:

Code: 
#!/bin/sh

touch /root/bhyve-stdin
/usr/sbin/bhyve -A -H -P -c 4 -m 4096M \
    -s 0:0,hostbridge \
    -s 2:0,virtio-net,tap11 \
    -s 4:0,ahci-cd,/common/distro/recovery/systemrescuecd-6.0.7.iso \
    -s 29,fbuf,tcp=0.0.0.0:5900 \
    -s 30,xhci,tablet \
    -s 31:0,lpc -l com1,stdio \
    -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd sysrescuecd < /root/bhyve-stdin &

it will run perfectly

could someone please explain to me the roots of "Why I'm dumb"?

PS.
SirDice, bridge does work well by sitting on the same interface as the host machine ethernet.
here is the my ifconfig which does work:

Code: 
ifconfig
bge0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=c0099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE>
    ether 4c:d9:8f:b6:0b:9d
    inet 10.51.0.97 netmask 0xffffff00 broadcast 10.51.0.255
    media: Ethernet autoselect (1000baseT <full-duplex>)
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
bge1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=c019b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,VLAN_HWTSO,LINKSTATE>
    ether 4c:d9:8f:b6:0b:9e
    media: Ethernet autoselect
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
    options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
    inet6 ::1 prefixlen 128
    inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
    inet 127.0.0.1 netmask 0xff000000
    groups: lo
    nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    ether 02:76:87:07:be:00
    id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
    maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
    root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
    member: tap11 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 5 priority 128 path cost 2000000
    member: bge0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
            ifmaxaddr 0 port 1 priority 128 path cost 55
    groups: bridge
    nd6 options=9<PERFORMNUD,IFDISABLED>
tap11: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=80000<LINKSTATE>
    ether 00:bd:77:24:f7:0b
    groups: tap
    media: Ethernet autoselect
    status: active
    nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
    Opened by PID 88830


Thanks :)
 
benoitc, If I recall correctly, the serial port was directed into stdio, which blocked communication.
now days I'm using vm-bhyve port to manage virtual machines
 
You must log in or register to reply here.
Back
Top