PF Allow all traffic to bhyve guests through PF

[tekollt]

Hey,

I have a host running different services, and among them also use bhyve to run some virtual machines.

There is a default block rule that prevents traffic to flow, i thought that the following would override that for my guests but that is not the case. I assume i am missing or overlooking something.

pass in quick on vm-public all
pass out quick on vm-public all

For example, to allow external DNS lookup for my VM i had to define

dns_port = "{ 53 }"
pass in quick proto udp to port $dns_port keep state

Obviously i dont have much experience with PF, but i based on my knowledge i would say that its not feasible to have to define all outbound traffic as this. What
I would really like, is that all outbound traffic to be uninterupted, while only allowing port 443 inbound to my guest VM without exposing 443 on the host.

This is a bridged setup

set skip on lo0
scrub in all

# allow dynamic NAT configuration (e.g. luemuctl)
nat-anchor "bhf-nat/*"

# block all incoming and allow all outgoing traffic
block return in log all
pass out quick all

# allow icmp6
pass in quick proto icmp6 all

# allow icmp4 (ping)
pass in quick inet proto icmp all icmp-type { echoreq, unreach }

# allow ssh
pass in quick proto tcp to port ssh

# Allow bhyve VM
pass in quick on vm-public all
pass out quick on vm-public all

dns_port = "{ 53 }"
dhcp_udp = "{ 67, 68 }"
ecb_tcp = "{ 443 }"



pass in quick on em0 proto tcp from any port $ecb_tcp dns_port to any port $ecb_tcp
pass in quick on em0 proto udp from any port $dhcp_udp dns_port to any port $dhcp_udp

pass in quick proto udp to port $dns_port keep state
pass in quick proto tcp to port $dns_port keep state

Any help to get me started would be much apriciated

 

[SirDice]

I assume i am missing or overlooking something.

Yes, you're overlooking the fact that packets travel from one interface to another. You're only allowing traffic on one of them.

Suppose you have two interfaces, for simplicity's sake; em0 and em1. If you need to allow traffic through the host you need to allow the traffic to come in on em0 and go out em1. And for traffic the other way around, allow incoming on em1 and outgoing on em0. Rules must be set on both interfaces or else the block in all would apply.

pass in quick on vm-public all 
pass out quick on vm-public all

This only allows traffic to flow in and out of the vm-public interface (guessing by the name that's a sysutils/vm-bhyve bridge(4)). That bridge(4) interface has an 'uplink' interface. You will need to set up rules to allow traffic to pass that interface too. Traffic out of your VMs will go in on vm-public and out that uplink interface.

 

[tekollt]

Yes, you're overlooking the fact that packets travel from one interface to another. You're only allowing traffic on one of them.

Suppose you have two interfaces, for simplicity's sake; em0 and em1. If you need to allow traffic through the host you need to allow the traffic to come in on em0 and go out em1. And for traffic the other way around, allow incoming on em1 and outgoing on em0. Rules must be set on both interfaces or else the block in all would apply.


This only allows traffic to flow in and out of the vm-public interface (guessing by the name that's a sysutils/vm-bhyve bridge(4)). That bridge(4) interface has an 'uplink' interface. You will need to set up rules to allow traffic to pass that interface too. Traffic out of your VMs will go in on vm-public and out that uplink interface.

Got it, that mde more sense. I managed to solve it with the followigng

dhcp = "{ 67, 68 }"
ext_if = "em0"
vm_if = "{ tap0 }"
pass quick on $vm_if all

#Allow bhyve VM`s to use DHCPD ACK
pass in quick on $ext_if proto udp from any port $dhcp to any port $dhcp