sha256: ///var/db/locate.database: Input/output error / cp: ///usr/lib/debug/usr/bin/llvm-objdump.debug: Input/output error

I've removed one bank of memory to see what happened. I have 16 Gb,now I'm running with 8 GB. And I've recreated the ZFS disks previously damaged. Now I'm on Linux and I'm copying the files that I had previously moved elsewhere,there. For the moment I don't see the I/O errors when I copy those files there. But Firefox still has something wrong :

Code:
[81899.514318] traps: Isolated Web Co[120250] general protection fault ip:7c06b111c3f0 sp:7ffd664878d0 error:0 in libxul.so[7c06ac546000+6526000]

According with this post :


There seems to be a bug. Unfortunately no one gave more informations. Take in consideration that I'm having problems with Firefox in Linux and in FreeBSD,so here may come the real X/Y problem,where X = Firefox and Y = it was the ZFS damaged structures. Two problems that are at the same time dependent and independent between / from themself ?
 
What tests did you run?

Some of the tests that you have suggested some months ago. I don't remember well,but I remember the memory stress test. You seem to be very convinced that I have a problem with the memory...but to refute your theory I've removed one bank. Now I want to see what happens and the next step is to re attach the bank removed and remove the other bank. Isn't this experiment more informative than do the test ?

I'm monitoring the ZFS disks,waiting to see if or when they will result again damaged. For the moment they are sane :

Code:
# zpool status -v
  pool: zroot-133
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
        The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
config:

        NAME        STATE     READ WRITE CKSUM
        zroot-133   ONLINE       0     0     0
          sdj       ONLINE       0     0     0

errors: No known data errors

  pool: zroot2
 state: ONLINE
config:

        NAME        STATE     READ WRITE CKSUM
        zroot2      ONLINE       0     0     0
          sdk       ONLINE       0     0     0

errors: No known data errors
 
Now I want to see what happens and the next step is to re attach the bank removed and remove the other bank. Isn't this experiment more informative than do the test ?

That's of course a good thing to do.

But there are more sources of memory corruption than just a bad stick of RAM. You could also have both RAM modules broken.
 
If both are broken,I should see the corruption even using one only module. What are the other sources of memory corruption that you are talking about ?

On the end If I will not be able to understand the causes,I may bring the PC to the repair shop.
 
If both are broken,I should see the corruption even using one only module. What are the other sources of memory corruption that you are talking about ?

CPU namely, mainboard also. Flipping bits in memory can be done pretty much by anything.

On the end If I will not be able to understand the causes,I may bring the PC to the repair shop.

That is likely to be useless. Repair shops usually can barely run memtest86.
 
Code:
# memtester 8000 5
memtester version 4.6.0 (64-bit)
Copyright (C) 2001-2020 Charles Cazabon.
Licensed under the GNU General Public License version 2 (only).

pagesize is 4096
pagesizemask is 0xfffffffffffff000
want 8000MB (8388608000 bytes)
got  8000MB (8388608000 bytes), trying mlock ...locked.
Loop 1/5:
  Stuck Address       : ok     
  Random Value        : ok
  Compare XOR         : ok
  Compare SUB         : ok
  Compare MUL         : ok
  Compare DIV         : ok
  Compare OR          : ok
  Compare AND         : ok
  Sequential Increment: ok
  Solid Bits          : ok     
  Block Sequential    : ok     
  Checkerboard        : ok     
  Bit Spread          : ok     
  Bit Flip            : ok     
  Walking Ones        : ok     
  Walking Zeroes      : ok     
  8-bit Writes        : ok
  16-bit Writes       : ok

Loop 2/5:
  Stuck Address       : ok     
  Random Value        : ok
  Compare XOR         : ok
  Compare SUB         : ok
  Compare MUL         : ok
  Compare DIV         : ok
  Compare OR          : ok
  Compare AND         : ok
  Sequential Increment: ok
  Solid Bits          : ok     
  Block Sequential    : ok     
  Checkerboard        : ok     
  Bit Spread          : ok     
  Bit Flip            : ok     
  Walking Ones        : ok     
  Walking Zeroes      : ok     
  8-bit Writes        : ok
  16-bit Writes       : ok

Loop 3/5:
  Stuck Address       : ok     
  Random Value        : ok
  Compare XOR         : ok
  Compare SUB         : ok
  Compare MUL         : ok
  Compare DIV         : ok
  Compare OR          : ok
  Compare AND         : ok
  Sequential Increment: ok
  Solid Bits          : ok     
  Block Sequential    : ok     
  Checkerboard        : ok     
  Bit Spread          : ok     
  Bit Flip            : ok     
  Walking Ones        : ok     
  Walking Zeroes      : ok     
  8-bit Writes        : ok
  16-bit Writes       : ok

Loop 4/5:
  Stuck Address       : ok     
  Random Value        : ok
  Compare XOR         : ok
  Compare SUB         : ok
  Compare MUL         : ok
  Compare DIV         : ok
  Compare OR          : ok
  Compare AND         : ok
  Sequential Increment: ok
  Solid Bits          : ok     
  Block Sequential    : ok     
  Checkerboard        : ok     
  Bit Spread          : ok     
  Bit Flip            : ok     
  Walking Ones        : ok     
  Walking Zeroes      : ok     
  8-bit Writes        : ok
  16-bit Writes       : ok

Loop 5/5:
  Stuck Address       : ok     
  Random Value        : ok
  Compare XOR         : ok
  Compare SUB         : ok
  Compare MUL         : ok
  Compare DIV         : ok
  Compare OR          : ok
  Compare AND         : ok
  Sequential Increment: ok
  Solid Bits          : ok     
  Block Sequential    : ok     
  Checkerboard        : ok     
  Bit Spread          : ok     
  Bit Flip            : ok     
  Walking Ones        : ok     
  Walking Zeroes      : ok     
  8-bit Writes        : ok
  16-bit Writes       : ok

Done.

Source :

 
Code:
[38585.474837] INFO: task txg_sync:4252 blocked for more than 122 seconds.
[38585.474842]       Tainted: P           O       6.8.0-51-generic #52-Ubuntu
[38585.474843] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[38585.474844] task:txg_sync        state:D stack:0     pid:4252  tgid:4252  ppid:2      flags:0x00004000
[38585.474847] Call Trace:
[38585.474848]  <TASK>
[38585.474851]  __schedule+0x27c/0x6b0
[38585.474855]  schedule+0x33/0x110
[38585.474856]  schedule_timeout+0x95/0x170
[38585.474857]  ? __pfx_process_timeout+0x10/0x10
[38585.474860]  io_schedule_timeout+0x51/0x80
[38585.474862]  __cv_timedwait_common+0x139/0x180 [spl]
[38585.474872]  ? __pfx_autoremove_wake_function+0x10/0x10
[38585.474874]  __cv_timedwait_io+0x19/0x30 [spl]
[38585.474881]  zio_wait+0x119/0x230 [zfs]
[38585.475073]  dsl_pool_sync+0xb7/0x410 [zfs]
[38585.475211]  spa_sync_iterate_to_convergence+0xde/0x220 [zfs]
[38585.475351]  spa_sync+0x321/0x620 [zfs]
[38585.475490]  txg_sync_thread+0x1e7/0x250 [zfs]
[38585.475627]  ? __pfx_txg_sync_thread+0x10/0x10 [zfs]
[38585.475772]  ? __pfx_thread_generic_wrapper+0x10/0x10 [spl]
[38585.475783]  thread_generic_wrapper+0x5c/0x70 [spl]
[38585.475790]  kthread+0xef/0x120
[38585.475793]  ? __pfx_kthread+0x10/0x10
[38585.475795]  ret_from_fork+0x44/0x70
[38585.475797]  ? __pfx_kthread+0x10/0x10
[38585.475798]  ret_from_fork_asm+0x1b/0x30
[38585.475800]  </TASK>
[71397.860222] usb 1-13-port2: disabled by hub (EMI?), re-enabling...
[71397.860549] usb 1-13.2: USB disconnect, device number 10
[71398.177645] usb 1-13.2: new low-speed USB device number 12 using xhci_hcd
[71398.260021] usb 1-13.2: New USB device found, idVendor=145f, idProduct=02c9, bcdDevice= 1.01
[71398.260025] usb 1-13.2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[71398.260026] usb 1-13.2: Product: Trust Keyboard
[71398.260027] usb 1-13.2: Manufacturer: SIGMACHIP
[71398.268004] input: SIGMACHIP Trust Keyboard as /devices/pci0000:00/0000:00:14.0/usb1/1-13/1-13.2/1-13.2:1.0/0003:145F:02C9.0008/inpu
t/input12
[71398.350812] hid-generic 0003:145F:02C9.0008: input,hidraw4: USB HID v1.10 Keyboard [SIGMACHIP Trust Keyboard] on usb-0000:00:14.0-13
.2/input0
[71398.352893] input: SIGMACHIP Trust Keyboard Consumer Control as /devices/pci0000:00/0000:00:14.0/usb1/1-13/1-13.2/1-13.2:1.1/0003:14
5F:02C9.0009/input/input13
[71398.404847] input: SIGMACHIP Trust Keyboard System Control as /devices/pci0000:00/0000:00:14.0/usb1/1-13/1-13.2/1-13.2:1.1/0003:145F
:02C9.0009/input/input14
[71398.405017] hid-generic 0003:145F:02C9.0009: input,hidraw5: USB HID v1.10 Device [SIGMACHIP Trust Keyboard] on usb-0000:00:14.0-13.2
/input1
[81899.514318] traps: Isolated Web Co[120250] general protection fault ip:7c06b111c3f0 sp:7ffd664878d0 error:0 in libxul.so[7c06ac54600
0+6526000]
[84160.319820] traps: WebExtensions[111382] general protection fault ip:7c06acc81bde sp:7ffd66485490 error:0 in libxul.so[7c06ac546000+
6526000]
[85765.658576] traps: Isolated Web Co[125898] general protection fault ip:7c06acca2b67 sp:7ffd66488680 error:0 in libxul.so[7c06ac54600
0+6526000]
[92611.396414] traps: WebExtensions[132595] general protection fault ip:75965068d472 sp:7ffdb5f0b500 error:0 in libxul.so[75964ff46000+
6526000]
[93925.494038] Purging GPU memory, 63345 pages freed, 0 pages still pinned, 10553 pages left available.
[125564.987920] usb 1-10: USB disconnect, device number 5
[125568.455794] usb usb2-port1: over-current condition
[125568.477779] usb usb1-port14: over-current condition
[125568.605779] usb usb1-port1: over-current condition
[125568.663786] usb usb2-port2: over-current condition
[125568.733814] usb usb1-port2: over-current condition
[125568.861782] usb 1-6: USB disconnect, device number 3
[125569.119773] usb 1-6: new high-speed USB device number 13 using xhci_hcd
[125569.246630] usb 1-6: New USB device found, idVendor=1058, idProduct=0704, bcdDevice= 1.75
[125569.246633] usb 1-6: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[125569.246635] usb 1-6: Product: External HDD
[125569.246636] usb 1-6: Manufacturer: Western Digital
[125569.246637] usb 1-6: SerialNumber: 57442D575845323039544433303334
[125569.248397] usb-storage 1-6:1.0: USB Mass Storage device detected
[125569.248643] usb-storage 1-6:1.0: Quirks match for vid 1058 pid 0704: 8000
[125569.248664] scsi host4: usb-storage 1-6:1.0
[125570.296203] scsi 4:0:0:0: Direct-Access     WD       3200BMV External 1.75 PQ: 0 ANSI: 4
[125570.296478] sd 4:0:0:0: Attached scsi generic sg6 type 0
[125570.296919] sd 4:0:0:0: [sdg] 625142448 512-byte logical blocks: (320 GB/298 GiB)
[125570.297151] sd 4:0:0:0: [sdg] Write Protect is off
[125570.297153] sd 4:0:0:0: [sdg] Mode Sense: 23 00 00 00
[125570.297320] sd 4:0:0:0: [sdg] No Caching mode page found
[125570.297321] sd 4:0:0:0: [sdg] Assuming drive cache: write through
[125570.335341]  sdg: sdg1 sdg2 sdg3
[125570.335615] sd 4:0:0:0: [sdg] Attached SCSI disk
[125581.270812] usb 1-9: new full-speed USB device number 14 using xhci_hcd
[125581.384809] usb 1-9: device descriptor read/64, error -71
[125586.963784] usb 1-9: new high-speed USB device number 16 using xhci_hcd
[125587.093255] usb 1-9: New USB device found, idVendor=05e3, idProduct=0751, bcdDevice=14.04
[125587.093258] usb 1-9: New USB device strings: Mfr=3, Product=4, SerialNumber=0
[125587.093259] usb 1-9: Product: USB Storage
[125587.093261] usb 1-9: Manufacturer: USB Storage
[125587.095640] usb-storage 1-9:1.0: USB Mass Storage device detected
[125587.095882] scsi host5: usb-storage 1-9:1.0
[125588.154313] scsi 5:0:0:0: Direct-Access     Generic  STORAGE DEVICE   1404 PQ: 0 ANSI: 6
[125588.154542] sd 5:0:0:0: Attached scsi generic sg8 type 0
[125588.390948] sd 5:0:0:0: [sdi] 62333952 512-byte logical blocks: (31.9 GB/29.7 GiB)
[125588.392221] sd 5:0:0:0: [sdi] Write Protect is off
[125588.392224] sd 5:0:0:0: [sdi] Mode Sense: 21 00 00 00
[125588.393485] sd 5:0:0:0: [sdi] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[125588.399266]  sdi: sdi1 sdi2
[125588.399444] sd 5:0:0:0: [sdi] Attached SCSI removable disk
[126000.253481] sdi: detected capacity change from 62333952 to 0
[127498.722961] usb 1-10: new high-speed USB device number 17 using xhci_hcd
[127498.849164] usb 1-10: config index 0 descriptor too short (expected 55, got 32)
[127498.849409] usb 1-10: New USB device found, idVendor=0a5c, idProduct=2764, bcdDevice= 0.00
[127498.849412] usb 1-10: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[127498.849414] usb 1-10: Product: BCM2710 Boot
[127498.849415] usb 1-10: Manufacturer: Broadcom
[127530.957186] usb 1-10: USB disconnect, device number 17
[128395.794666] sd 5:0:0:0: [sdi] 125042688 512-byte logical blocks: (64.0 GB/59.6 GiB)
[128395.797043] sdi: detected capacity change from 0 to 125042688
[128395.798823]  sdi: sdi1 sdi2
                 sdi2: <bsd: sdi5 >
[128395.903453] sd 5:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[128395.903459] sd 5:0:0:0: [sdi] tag#0 Sense Key : Aborted Command [current]
[128395.903462] sd 5:0:0:0: [sdi] tag#0 Add. Sense: Data phase CRC error detected
[128395.903464] sd 5:0:0:0: [sdi] tag#0 CDB: Read(10) 28 00 00 00 00 38 00 00 08 00
[128395.903466] I/O error, dev sdi, sector 56 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[128395.985559] sd 5:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[128395.985564] sd 5:0:0:0: [sdi] tag#0 Sense Key : Aborted Command [current]
[128395.985568] sd 5:0:0:0: [sdi] tag#0 Add. Sense: Data phase CRC error detected
[128395.985570] sd 5:0:0:0: [sdi] tag#0 CDB: Read(10) 28 00 00 01 a4 00 00 00 08 00
[128395.985572] I/O error, dev sdi, sector 107520 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0

[128395.987470] sd 5:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[128395.987473] sd 5:0:0:0: [sdi] tag#0 Sense Key : Aborted Command [current]
[128395.987476] sd 5:0:0:0: [sdi] tag#0 Add. Sense: Data phase CRC error detected
[128395.987478] sd 5:0:0:0: [sdi] tag#0 CDB: Read(10) 28 00 00 00 08 00 00 00 08 00
[128395.987480] I/O error, dev sdi, sector 2048 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[128396.086975] sd 5:0:0:0: [sdi] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=0s
[128396.086980] sd 5:0:0:0: [sdi] tag#0 Sense Key : Aborted Command [current]
[128396.086983] sd 5:0:0:0: [sdi] tag#0 Add. Sense: Data phase CRC error detected
[128396.086986] sd 5:0:0:0: [sdi] tag#0 CDB: Read(10) 28 00 00 01 a4 00 00 00 08 00
[128396.086987] I/O error, dev sdi, sector 107520 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[128396.086992] Buffer I/O error on dev sdi2, logical block 512, async page read

Do you think or not that those errors may be caused by the UPS ?

This is a clear error message :

Code:
[125568.733814] usb usb1-port2: over-current condition
 
Do you like this interpretation of the phenomenon ?

This is a USB port error/warning ( not very unusual, a lot of USB connections try to draw more power than the system would like. A UPS usually does not need to draw power). Is the UPS connected via USB? If it is, is the error eliminated by unplugging the USB connection?

Odds are your ZFS drive is drawing too much power. There are 'helper USB cable that connect to (2) USB ports to supply the necessary power.

https://www.amazon.com/ALINNA-Female-Extension-Splitter-Adapter/dp/B098L4HTVG

https://www.amazon.com/CY-Type-C-Power-USB3-0-USB2-0/dp/B0C1WP8MMS?th=1

ZFS is software. What is the hardware?
 
Yes. Ok,I will try to attach the USB disks to the USB hub. I didn't do it because I haven't had these problems for years and I didn't think the problem was that.
 
I've attached the USB disks to my USB hub and now the kernel log is more clean,except for one error that's still there and that certify that I'm affected by the classic X ; Y problem :

Code:
[ 8344.524198] WebExtensions[4280]: segfault at 55fb19c22f3c ip 000075fb1fb1d2f3 sp 00007ffedb7b0a90 error 4 in libxul.so[75fb1af46000+6526000] likely on CPU 12 (core 4, socket 0)

[ 8344.524206] Code: 00 e9 b1 c8 ff ff 48 8b 74 24 18 48 8b 06 49 8b 0b 48 c7 04 c1 03 00 00 00 48 8b 06 49 8b 0b 48 89 6c c1 08 48 83 06 02 eb c5 <41> 8b 55 f4 41 be 01 00 00 00 85 c9 74 b7 85 d2 74 9a 48 8b 74 24

[20836.485594] WebExtensions[9956]: segfault at 55faca86fcd0 ip 000075fb1fb1d54b sp 00007ffedb7b72d0 error 4 in libxul.so[75fb1af46000+6526000] likely on CPU 3 (core 3, socket 0)

[20836.485603] Code: e9 0a fe ff ff 66 2e 0f 1f 84 00 00 00 00 00 4d 39 fc 0f 83 47 c6 ff ff 49 8b 02 48 ff c8 49 89 02 48 85 c0 0f 8e 73 02 00 00 <4b> 8b 6c e5 00 49 ff c4 48 b8 ff ff ff ff ff ff fa ff 48 39 c5 76

[21732.688534] WebExtensions[17063]: segfault at 6988f100000 ip 000075fb1fb04fc5 sp 00007ffedb7b2740 error 4 in libxul.so[75fb1af46000+6526000] likely on CPU 4 (core 4, socket 0)

[21732.688547] Code: 10 bd 8c 00 00 00 45 31 e4 4a 8b 5c e6 08 48 83 fb 02 74 43 48 89 d8 48 83 e0 07 0f 85 d7 00 00 00 48 89 d8 48 25 00 00 f0 ff <48> 83 38 00 75 27 89 d9 c1 e9 03 89 da 41 b8 01 00 00 00 49 d3 e0

So I go for a bug in Firefox. What do you think ?
 
Yes,now I keep them attached to the USB hub. But this time I'm talking about an OS installed to a Sata SSD disk,no USB disks are involved. I keep Linux and FreeBSD installed in different SATA SSD disks and in both the installations Firefox gives the same problem. But the problem is not present when I boot and run Linux (or FreeBSD) virtualized with qemu-kvm or bhyve. In this scenario,Firefox works like a charme.
 
I would blow away your firefox profiles and try with a fresh one.

And then reinstall the firefox packages from scratch. You might have corrupted them while your disks were not working right.
 
Back
Top