Very high interrupt rate for IGC interfaces 2.5G

G

gregober

We are facing a situation where we have igc interfaces with a very high interrupt rate :

Code: 
root: vmstat -i
interrupt                          total       rate
irq4: uart0                         2102          0
irq16: sdhci_pci0                    183          0
cpu0:timer                   10380344172       1021
cpu1:timer                    3163850602        311
cpu2:timer                    2660093077        262
cpu3:timer                    2677358716        263
irq32: nvme0:admin                   537          0
irq33: nvme0:io0                72492126          7
irq34: nvme0:io1                69873086          7
irq35: nvme0:io2                71092624          7
irq36: nvme0:io3                71275619          7
irq37: igc6:rxq0              3056459012        301
irq38: igc6:rxq1             12898754541       1269
irq39: igc6:rxq2              2983594883        293
irq40: igc6:rxq3              3060092764        301
irq41: igc6:aq                         2          0
irq42: igc7:rxq0              4020386059        395
irq43: igc7:rxq1              3802327276        374
irq44: igc7:rxq2              3954831382        389
irq45: igc7:rxq3              4051122920        398
irq46: igc7:aq                         2          0
irq52: igc5:rxq0              1303432406        128
irq53: igc5:rxq1              1345085379        132
irq54: igc5:rxq2              1230924716        121
irq55: igc5:rxq3              1315308648        129
irq56: igc5:aq                         2          0
irq67: igc1:rxq0               993923149         98
irq68: igc1:rxq1                   93765          0
irq69: igc1:rxq2                   88455          0
irq70: igc1:rxq3                  137400          0
irq71: igc1:aq                        20          0
irq72: igc2:rxq0              2502291867        246
irq73: igc2:rxq1              2258132013        222
irq74: igc2:rxq2              2355173884        232
irq75: igc2:rxq3              2201025044        216
irq76: igc2:aq                         2          0
irq94: ix0:rxq0                   508341          0
irq95: ix0:rxq1                   169446          0
irq96: ix0:rxq2                 10331772          1
irq98: ix0:aq                          3          0
irq99: ix1:rxq0               3523337450        347
irq100: ix1:rxq1              4581285686        451
irq101: ix1:rxq2              4713113139        464
irq102: ix1:rxq3              4552805156        448
irq103: ix1:aq                         9          0
Total                        89881119437       8841

We will activate "bindthreads" >> net.isr.bindthreads: 1

We also have noticed that the driver igc does not seem to implement rss (can someone confirm this ?).

What would you suggest in order to circumvent this problem ?
 
Erichans said:
With icg(4) what NIC exactly are you using? Can you post the output of pciconf -lv | grep -B 4 ethernet ?
Click to expand...
Code: 
igc6@pci0:5:0:0:    class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x15f3 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I225-V'
    class      = network
    subclass   = ethernet
igc7@pci0:6:0:0:    class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x15f3 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I225-V'
    class      = network
    subclass   = ethernet
igc4@pci0:7:0:0:    class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x15f3 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I225-V'
    class      = network
    subclass   = ethernet
igc5@pci0:8:0:0:    class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x15f3 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I225-V'
    class      = network
    subclass   = ethernet
igc0@pci0:11:0:0:    class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x15f3 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I225-V'
    class      = network
    subclass   = ethernet
igc3@pci0:12:0:0:    class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x15f3 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I225-V'
    class      = network
    subclass   = ethernet
igc1@pci0:13:0:0:    class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x15f3 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I225-V'
    class      = network
    subclass   = ethernet
igc2@pci0:14:0:0:    class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086 device=0x15f3 subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I225-V'
    class      = network
    subclass   = ethernet


This is based on a Ethernet Controller I225-V.

I am not sure if the level of interupt that we have is "normal" or not really.
 
I don't own anything with an i225 or i226 nic. The high number of interrupts seems odd but, i'm far from an expert/advanced user in this area.

Having said this, I can confirm that, based on the Product brief Intel i225/i226, the i225 does not support RSS (only the i226 does, but currently it is unclear if igc(4) provides this functionality; a quick code scan suggests probably not).

According to the code, the driver is also for the i225's successor: the (improved) i226. igc(4) on FreeBSD seems a project under development/to be fully developed. The man page could also use an update and some additional clarity (PR 280810); Wake on Lan (WoL) also seems not supported: PR 282140.

Forum threads:
My hope is that igc(4) will get more (dev) attention and resources allocated; perhaps through the FreeBSD Foundation. The i225/i226 seem to be two prevalent 2.5 GbE nics. Getting this driver (part of base) fully developed will benefit probably a lot of FreeBSD users in the new 2.5 GbE speed range.

For your interrupt issue, if no additional reactions addressing this arise, I would suggest you take it to a mailing list; alternatively submit a PR. This at least would clearly identify this issue and hopefully also perhaps provide some insight in the nature of these interrupts.
 
You must log in or register to reply here.
Back
Top