IPoIB between FreeBSD and Linux using Mellanox CX-5

vijay

vijay

Hi all. I am trying to get IPoIB working between FreeBSD (synced to 14.x) and a Linux host. I have enabled the following:
Code: 
options        IPOIB_CM
options        SDP

device        mlx5       # Mellanox CX5/CX6 NIC
device        mlxfw     # Mellanox Monolithic firmware update
device        mlx5en   # Mellanox CX5/CX6 NIC
device        mlx5ib    # Mellanox CX5/CX6 RoCE
device        ipoib      # IP over IB devices
Note that I am trying to run IPoIB, not RoCE.

I can see the device and I can configure it:
Code: 
ib0: flags=8043<UP,BROADCAST,RUNNING,MULTICAST> metric 0 mtu 4092
        uuid: f1f2a676-3b97-11ee-b180-d039ea24a082
        options=80018<VLAN_MTU,VLAN_HWTAGGING,LINKSTATE>
        lladdr 80.0.0.9f.fe.80.0.0.0.0.0.0.b8.59.9f.3.0.d5.6.4e
        inet6 fe80::ba59:9f03:d5:64e%e5a prefixlen 64 scopeid 0x13
        inet 20.20.0.1 netmask 0xffffff00 broadcast 20.20.0.255
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
The port is cabled to the Linux host, but it doesn't look like the link is up based on the above. Can someone help with what I might be missing?
 
Good afternoon.
I assume that on your host with freebsd, the IPoIB interface works in "connected" mode, and on a host with linux interface works in "datagram" mode.
 
Sorry to necro an old thread, but if you aren't running a subnet manager, you will be unable to establish an Infiniband link between hosts. OpenSM should be enabled by default in OpenBSD, but may be gated behind the WITH_OFED_EXTRAS build config flag.
 
You must log in or register to reply here.
Back
Top