Solved Weird Netgraph Behavior With Jail VNET

spmzt

spmzt

Hi,

I'm currently dealing with an intriguing situation that involves the Netgraph and its interaction within a FreeBSD Jail configured with VNET.
Background:
I have loaded the ng_ether module on the host machine. Additionally, I have set up multiple FreeBSD Jails with VNET and they are connected over epair connections directly together.

I want to use ng_tee over my epair interface between two jails. For example I want to run the command below:
ngctl mkpeer epair0a: tee lower left


Here's where things get interesting:
  1. When I attempt to execute the following command directly on the jail:
    ngctl: send msg: Operation not permitted
  2. It makes no sense to run the command under the host because my interface is not exist on the host but I trying it anyway:
    ngctl: send msg: No such file or directory
  3. Strangely, when I run the exact same command within the FreeBSD Jail, it behaves differently and It works!.
To make it more straightforward follow this output:
root@jail:/ # ngctl mkpeer epair0a: tee lower left
ngctl: send msg: Operation not permitted
root@jail:/ # exit
root@host:~ # ngctl mkpeer epair0a: tee lower left
ngctl: send msg: No such file or directory
root@host:~ # jexec jail tcsh
root@jail:/ # ngctl mkpeer epair0a: tee lower left
root@jail:/ #

I'm running FreeBSD 13.2 (kernel and user-land).

Any idea? Should I submit a bug about it?
 
I wonder if bothering with epairs is not the problem. I don't use your approach, because when I use Netgraph, I use ng_bridge and ng_eiface (for jails, VMs uses ng_socket). I didn't try to use ng_tee as you did (I wonder what are you actually trying to achieve, but that's offtopic), but I think since ng_eiface doesn't disappear from the host when assigned to VNET jail, it might actually work nicely with ng_tee.

Edit: to give you a better idea of what I'm doing, I attached a graph from my simple lab.
 

Attachments

  • graph.png
    graph.png
    27.8 KB · Views: 177
algi said:
I wonder if bothering with epairs is not the problem. I don't use your approach, because when I use Netgraph, I use ng_bridge and ng_eiface (for jails, VMs uses ng_socket). I didn't try to use ng_tee as you did (I wonder what are you actually trying to achieve, but that's offtopic), but I think since ng_eiface doesn't disappear from the host when assigned to VNET jail, it might actually work nicely with ng_tee.

Edit: to give you a better idea of what I'm doing, I attached a graph from my simple lab.
Click to expand...
FYI: I do not want the epair to appear in my host. In fact, the thing I want is to connect my jails directly together with epair.
The ng_tee is just an example. Any ngctl command I attempt to run does not work, except when I execute it on the host machine first (even with errors). This peculiar behavior isn't directly linked to ng_tee; it's more of a general quirk.
 
I think the problem might be that epair actually disappears from the host (because of VNET). That's why I proposed to try ng_eiface instead, but perhaps I'm wrong and it wouldn't make any difference for you and the problem is elsewhere.

Edit: When I try "mkpeer ng0_apache: tee lower left" on eiface, I got "ngctl: send msg: Protocol family not supported". Not sure what it means though...
 
But something tells me that you actually don't care about that and you simply want to use your epair no matter what. In that case I wonder if the command actually works from the jail, what do you see in graph / list of nodes in jail and what do you see outside in the host?
 
That would work after running it in jail, plus it's stated in the beginning that the module is preloaded. I still suspect VNET, because for example creating ng_tee outside of jail (on main NIC) works perfectly fine. On the other hand, creating things on eiface that are part of VNET works fine too for me, so it might be a problem with epair.
 
yuripv79 said:
It probably wants some kernel modules to be (pre)loaded?
Click to expand...
I try this before. But I'm not sure.
root@jail:/ # ngctl mkpeer epair0a: tee lower left
ngctl: can't create node: Operation not permitted
root@jail:/ # exit
root@host:~ # kldload ng_tee ng_eiface ng_ether ng_iface
kldload: can't load ng_tee: module already loaded or in kernel
kldload: can't load ng_eiface: module already loaded or in kernel
kldload: can't load ng_ether: module already loaded or in kernel
kldload: can't load ng_iface: module already loaded or in kernel
 
You must log in or register to reply here.
Back
Top