AS4004T: Volume inaccessible

Backup and data protection discussion at its finest.

Moderator: Lillian.W@AST

Post Reply
bararu
Posts: 1
youtube meble na wymiar Warszawa
Joined: Tue Sep 29, 2020 7:44 am

AS4004T: Volume inaccessible

Post by bararu »

Hi Folks,

I have a problem

I am a brand-new owner of AS4004T (ADM: 3.5.0.R5D3, BIOS: 1.10) stuffed with four identical and new IronWolf 6TB (ST6000VN0033) drives.
It appears that my relatively young NAS has just dropped an entire array with very sparse logs, and I am utterly confused:
Image

Backstory:
After going through the motions of creating new RAID5 BTRFS array with one spare drive and filling the system with ~2TBs of data the system was powered-off for about a month.
Cue this weekend when the system had the SPARE drive removed, operated for about 20 min and powered off. When SPARE drive was reinstalled, the FileManager had File/folder path does not exist message and upon further inspection the entire Volume was listed as Inaccessible with three drives still appearing to be Healthy.
Image

There are similar threads on this forum (viewtopic.php?f=45&t=9701, viewtopic.php?f=63&t=9168&p=30353) that led me to SSH into the system and check mdstat and dmesg with results below. None, however, list the specifics of the issue at-hand. I really doubt that the array had suffered a simultaneous failure of both drives while the spare was out.

Any thoughts on what is going on or whether this is a result of SPARE drive removal?

Thx.
B

Code: Select all

admin@nasmork:/ $ cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4] 
md126 : active raid1 sda3[0] sdc3[6] sdd3[5] sdb3[4]
      2095104 blocks super 1.2 [4/4] [UUUU]
      
md0 : active raid1 sda2[0] sdc2[6] sdb2[4]
      2095104 blocks super 1.2 [4/3] [UU_U]
      
unused devices: <none>
and

Code: Select all

admin@nasmork:/ $ dmesg
[    6.990829] armada_thermal f06f808c.thermal: Overheat threshold between 93..107
[    7.008264] armada_thermal f2400078.thermal: Overheat threshold between 92..107
[    7.015802] md: linear personality registered for level -1
[    7.021322] md: raid0 personality registered for level 0
[    7.026662] md: raid1 personality registered for level 1
[    7.032002] md: raid10 personality registered for level 10
[    7.037591] md: raid6 personality registered for level 6
[    7.042935] md: raid5 personality registered for level 5
[    7.048273] md: raid4 personality registered for level 4
[    7.053705] device-mapper: uevent: version 1.0.3
[    7.058476] device-mapper: ioctl: 4.34.0-ioctl (2015-10-28) initialised: dm-devel@redhat.com
[    7.067064] device-mapper: cache cleaner: version 1.0.0 loaded
[    7.073300] sdhci: Secure Digital Host Controller Interface driver
[    7.079516] sdhci: Copyright(c) Pierre Ossman
[    7.083934] sdhci-pltfm: SDHCI platform and OF driver helper
[    7.089729] xenon-sdhci f2780000.sdhci: Failed to setup input clk: -517
[    7.096466] ledtrig-cpu: registered to indicate activity on CPUs
[    7.102536] hidraw: raw HID events driver (C) Jiri Kosina
[    7.109306] usbcore: registered new interface driver usbhid
[    7.114911] usbhid: USB HID core driver
[    7.118942] input: Unspecified device as /devices/virtual/input/input0
[    7.125621] Virtual Mouse Driver Initialized.
[    7.130713] coresight-tmc f0809000.etf0: TMC initialized
[    7.136196] coresight-tmc f080a000.etf1: TMC initialized
[    7.141662] coresight-tmc f080b000.etf2: TMC initialized
[    7.147135] coresight-tmc f080c000.etf3: TMC initialized
[    7.152621] coresight-tmc f0811000.etf5: TMC initialized
[    7.158895] coresight-tmc f0812000.etr: TMC initialized
[    7.164303] coresight-funnel f0810000.funnel: FUNNEL initialized
[    7.170654] coresight-etm4x f0c40000.etm0: ETM 4.0 initialized
[    7.176700] coresight-etm4x f0d40000.etm1: ETM 4.0 initialized
[    7.182652] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[    7.189020] IPVS: Registered protocols (TCP, UDP)
[    7.193817] IPVS: Connection hash table configured (size=4096, memory=64Kbytes)
[    7.201211] IPVS: Creating netns size=1840 id=0
[    7.205815] IPVS: ipvs loaded.
[    7.208945] arp_tables: (C) 2002 David S. Miller
[    7.213878] NET: Registered protocol family 10
[    7.218720] NET: Registered protocol family 17
[    7.223196] 8021q: 802.1Q VLAN Support v1.8
[    7.227434] 9pnet: Installing 9P2000 support
[    7.231735] Key type dns_resolver registered
[    7.236359] registered taskstats version 1
[    7.241494] Btrfs loaded, debug=on
[    7.252408] mac_addr 10:bf:48:8d:9a:2e
[    7.265677] mvpp2 f2000000.ppv22 eth0: Using device tree mac address 10:bf:48:8d:9a:2e
[    7.338359] mac_addr 10:bf:48:8d:9a:2f
[    7.351307] mvpp2 f2000000.ppv22 eth1: Using device tree mac address 10:bf:48:8d:9a:2f
[    7.422416] mac_addr 10:bf:48:8d:9a:30
[    7.435151] mvpp2 f2000000.ppv22 eth2: Using device tree mac address 10:bf:48:8d:9a:30
[    7.443571] PCI host bridge /cpn-110-master/pcie@0x620000 ranges:
[    7.449711]    IO 0xf9010000..0xf901ffff -> 0xf9010000
[    7.454879]   MEM 0xf7000000..0xf7efffff -> 0xf7000000
[    7.460196] armada8k-pcie f2620000.pcie: PCI host bridge to bus 0000:00
[    7.466847] pci_bus 0000:00: root bus resource [bus 00-ff]
[    7.472362] pci_bus 0000:00: root bus resource [io  0x0000-0xffff] (bus address [0xf9010000-0xf901ffff])
[    7.481886] pci_bus 0000:00: root bus resource [mem 0xf7000000-0xf7efffff]
[    7.488813] pci 0000:00:00.0: [11ab:0110] type 01 class 0x060400
[    7.488838] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[    7.488882] pci 0000:00:00.0: supports D1 D2
[    7.488886] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    7.488996] iommu: Adding device 0000:00:00.0 to group 2
[    7.494516] pci 0000:01:00.0: [1b4b:9170] type 00 class 0x010601
[    7.494731] pci 0000:01:00.0: reg 0x10: [io  0xfffffff8-0xffffffff]
[    7.494798] pci 0000:01:00.0: reg 0x14: [io  0xfffffffc-0xffffffff]
[    7.494866] pci 0000:01:00.0: reg 0x18: [io  0xfffffff8-0xffffffff]
[    7.494933] pci 0000:01:00.0: reg 0x1c: [io  0xfffffffc-0xffffffff]
[    7.495000] pci 0000:01:00.0: reg 0x20: [io  0xfffffff0-0xffffffff]
[    7.495067] pci 0000:01:00.0: reg 0x24: [mem 0xf7000000-0xf70001ff]
[    7.495134] pci 0000:01:00.0: reg 0x30: [mem 0xf7010000-0xf701ffff pref]
[    7.495371] pci 0000:01:00.0: PME# supported from D3hot
[    7.495602] iommu: Adding device 0000:01:00.0 to group 3
[    7.508666] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[    7.508684] pci 0000:00:00.0: BAR 0: assigned [mem 0xf7000000-0xf70fffff]
[    7.515509] pci 0000:00:00.0: BAR 14: assigned [mem 0xf7100000-0xf71fffff]
[    7.522419] pci 0000:00:00.0: BAR 15: assigned [mem 0xf7200000-0xf72fffff pref]
[    7.529763] pci 0000:00:00.0: BAR 13: assigned [io  0x1000-0x1fff]
[    7.535978] pci 0000:01:00.0: BAR 6: assigned [mem 0xf7200000-0xf720ffff pref]
[    7.543234] pci 0000:01:00.0: BAR 5: assigned [mem 0xf7100000-0xf71001ff]
[    7.550074] pci 0000:01:00.0: BAR 4: assigned [io  0x1000-0x100f]
[    7.556216] pci 0000:01:00.0: BAR 0: assigned [io  0x1010-0x1017]
[    7.562359] pci 0000:01:00.0: BAR 2: assigned [io  0x1018-0x101f]
[    7.568501] pci 0000:01:00.0: BAR 1: assigned [io  0x1020-0x1023]
[    7.574643] pci 0000:01:00.0: BAR 3: assigned [io  0x1024-0x1027]
[    7.580785] pci 0000:00:00.0: PCI bridge to [bus 01]
[    7.585775] pci 0000:00:00.0:   bridge window [io  0x1000-0x1fff]
[    7.591899] pci 0000:00:00.0:   bridge window [mem 0xf7100000-0xf71fffff]
[    7.598720] pci 0000:00:00.0:   bridge window [mem 0xf7200000-0xf72fffff pref]
[    7.606170] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt
[    7.613168] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
[    7.619729] pcie_pme 0000:00:00.0:pcie01: service driver pcie_pme loaded
[    7.619784] aer 0000:00:00.0:pcie02: service driver aer loaded
[    7.619914] ahci 0000:01:00.0: version 3.0
[    7.620169] ahci 0000:01:00.0: AHCI 0001.0000 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
[    7.628305] ahci 0000:01:00.0: flags: 64bit ncq sntf led only pmp fbs pio slum part sxs 
[    7.637016] scsi host0: ahci
[    7.640072] scsi host1: ahci
[    7.643059] ata1: SATA max UDMA/133 abar m512@0xf7100000 port 0xf7100100 irq 145
[    7.650491] ata2: SATA max UDMA/133 abar m512@0xf7100000 port 0xf7100180 irq 145
[    7.658065] PCI host bridge /cpn-110-master/pcie@0x640000 ranges:
[    7.664197]    IO 0xf9020000..0xf902ffff -> 0xf9020000
[    7.669366]   MEM 0xf8000000..0xf8efffff -> 0xf8000000
[    7.674627] armada8k-pcie f2640000.pcie: PCI host bridge to bus 0001:00
[    7.681277] pci_bus 0001:00: root bus resource [bus 00-ff]
[    7.686791] pci_bus 0001:00: root bus resource [io  0x10000-0x1ffff] (bus address [0xf9020000-0xf902ffff])
[    7.696492] pci_bus 0001:00: root bus resource [mem 0xf8000000-0xf8efffff]
[    7.703413] pci 0001:00:00.0: [11ab:0110] type 01 class 0x060400
[    7.703436] pci 0001:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff]
[    7.703478] pci 0001:00:00.0: supports D1 D2
[    7.703482] pci 0001:00:00.0: PME# supported from D0 D1 D3hot
[    7.703586] iommu: Adding device 0001:00:00.0 to group 4
[    7.709101] pci 0001:01:00.0: [1b4b:9170] type 00 class 0x010601
[    7.709316] pci 0001:01:00.0: reg 0x10: [io  0xfffffff8-0xffffffff]
[    7.709383] pci 0001:01:00.0: reg 0x14: [io  0xfffffffc-0xffffffff]
[    7.709450] pci 0001:01:00.0: reg 0x18: [io  0xfffffff8-0xffffffff]
[    7.709517] pci 0001:01:00.0: reg 0x1c: [io  0xfffffffc-0xffffffff]
[    7.709584] pci 0001:01:00.0: reg 0x20: [io  0xfffffff0-0xffffffff]
[    7.709651] pci 0001:01:00.0: reg 0x24: [mem 0xf8000000-0xf80001ff]
[    7.709719] pci 0001:01:00.0: reg 0x30: [mem 0xf8010000-0xf801ffff pref]
[    7.709955] pci 0001:01:00.0: PME# supported from D3hot
[    7.710183] iommu: Adding device 0001:01:00.0 to group 5
[    7.720667] pci_bus 0001:01: busn_res: [bus 01-ff] end is updated to 01
[    7.720683] pci 0001:00:00.0: BAR 0: assigned [mem 0xf8000000-0xf80fffff]
[    7.727506] pci 0001:00:00.0: BAR 14: assigned [mem 0xf8100000-0xf81fffff]
[    7.734417] pci 0001:00:00.0: BAR 15: assigned [mem 0xf8200000-0xf82fffff pref]
[    7.741761] pci 0001:00:00.0: BAR 13: assigned [io  0x10000-0x10fff]
[    7.748148] pci 0001:01:00.0: BAR 6: assigned [mem 0xf8200000-0xf820ffff pref]
[    7.755404] pci 0001:01:00.0: BAR 5: assigned [mem 0xf8100000-0xf81001ff]
[    7.762244] pci 0001:01:00.0: BAR 4: assigned [io  0x10000-0x1000f]
[    7.768559] pci 0001:01:00.0: BAR 0: assigned [io  0x10010-0x10017]
[    7.774877] pci 0001:01:00.0: BAR 2: assigned [io  0x10018-0x1001f]
[    7.781193] pci 0001:01:00.0: BAR 1: assigned [io  0x10020-0x10023]
[    7.787507] pci 0001:01:00.0: BAR 3: assigned [io  0x10024-0x10027]
[    7.793823] pci 0001:00:00.0: PCI bridge to [bus 01]
[    7.798813] pci 0001:00:00.0:   bridge window [io  0x10000-0x10fff]
[    7.805123] pci 0001:00:00.0:   bridge window [mem 0xf8100000-0xf81fffff]
[    7.811944] pci 0001:00:00.0:   bridge window [mem 0xf8200000-0xf82fffff pref]
[    7.819386] pcieport 0001:00:00.0: Signaling PME through PCIe PME interrupt
[    7.826383] pci 0001:01:00.0: Signaling PME through PCIe PME interrupt
[    7.832944] pcie_pme 0001:00:00.0:pcie01: service driver pcie_pme loaded
[    7.832993] aer 0001:00:00.0:pcie02: service driver aer loaded
[    7.833362] ahci 0001:01:00.0: AHCI 0001.0000 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
[    7.841495] ahci 0001:01:00.0: flags: 64bit ncq sntf led only pmp fbs pio slum part sxs 
[    7.850156] scsi host2: ahci
[    7.853210] scsi host3: ahci
[    7.856189] ata3: SATA max UDMA/133 abar m512@0xf8100000 port 0xf8100100 irq 147
[    7.863625] ata4: SATA max UDMA/133 abar m512@0xf8100000 port 0xf8100180 irq 147
[    7.891516] f2702000.serial: ttyS1 at MMIO 0xf2702000 (irq = 28, base_baud = 15625000) is a 16550A
[    7.900728] omap_rng f2760000.trng: Random Number Generator ver. 203b34c
[    7.908786] xhci-hcd f2500000.usb3: xHCI Host Controller
[    7.914229] xhci-hcd f2500000.usb3: new USB bus registered, assigned bus number 1
[    7.921914] xhci-hcd f2500000.usb3: hcc params 0x0a000990 hci version 0x100 quirks 0x00010090
[    7.930498] xhci-hcd f2500000.usb3: irq 29, io mem 0xf2500000
[    7.936343] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[    7.943174] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    7.950431] usb usb1: Product: xHCI Host Controller
[    7.955332] usb usb1: Manufacturer: Linux 4.4.52 xhci-hcd
[    7.960757] usb usb1: SerialNumber: f2500000.usb3
[    7.965697] hub 1-0:1.0: USB hub found
[    7.969477] hub 1-0:1.0: 1 port detected
[    7.973608] xhci-hcd f2500000.usb3: xHCI Host Controller
[    7.979009] xhci-hcd f2500000.usb3: new USB bus registered, assigned bus number 2
[    7.986555] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[    7.994717] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003
[    8.001539] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    8.008795] usb usb2: Product: xHCI Host Controller
[    8.013696] usb usb2: Manufacturer: Linux 4.4.52 xhci-hcd
[    8.019120] usb usb2: SerialNumber: f2500000.usb3
[    8.024014] hub 2-0:1.0: USB hub found
[    8.027792] hub 2-0:1.0: 1 port detected
[    8.033022] xhci-hcd f2510000.usb3: xHCI Host Controller
[    8.038427] xhci-hcd f2510000.usb3: new USB bus registered, assigned bus number 3
[    8.046095] xhci-hcd f2510000.usb3: hcc params 0x0a000990 hci version 0x100 quirks 0x00010090
[    8.054681] xhci-hcd f2510000.usb3: irq 30, io mem 0xf2510000
[    8.060510] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002
[    8.067331] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    8.074635] usb usb3: Product: xHCI Host Controller
[    8.079537] usb usb3: Manufacturer: Linux 4.4.52 xhci-hcd
[    8.084973] usb usb3: SerialNumber: f2510000.usb3
[    8.089886] hub 3-0:1.0: USB hub found
[    8.093666] hub 3-0:1.0: 1 port detected
[    8.108475] xhci-hcd f2510000.usb3: xHCI Host Controller
[    8.113873] xhci-hcd f2510000.usb3: new USB bus registered, assigned bus number 4
[    8.121432] usb usb4: We don't know the algorithms for LPM for this host, disabling LPM.
[    8.129632] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003
[    8.136454] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    8.143712] usb usb4: Product: xHCI Host Controller
[    8.144641] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[    8.144668] ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[    8.161056] usb usb4: Manufacturer: Linux 4.4.52 xhci-hcd
[    8.166489] ata1.00: ATA-10: ST6000VN0033-2EE110, SC60, max UDMA/133
[    8.166493] ata1.00: 11721045168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
[    8.166505] ata2.00: ATA-10: ST6000VN0033-2EE110, SC60, max UDMA/133
[    8.166507] ata2.00: 11721045168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
[    8.193807] ata1.00: configured for UDMA/133
[    8.193808] usb usb4: SerialNumber: f2510000.usb3
[    8.196743] hub 4-0:1.0: USB hub found
[    8.196752] hub 4-0:1.0: 1 port detected
[    8.197300] xenon-sdhci f2780000.sdhci: No vmmc regulator found
[    8.197302] xenon-sdhci f2780000.sdhci: No vqmmc regulator found
[    8.222588] ata2.00: configured for UDMA/133
[    8.227013] scsi 0:0:0:0: Direct-Access     ATA      ST6000VN0033-2EE SC60 PQ: 0 ANSI: 5
[    8.235167] mmc0: SDHCI controller on f2780000.sdhci [f2780000.sdhci] using ADMA 64-bit
[    8.235506] sd 0:0:0:0: Attached scsi generic sg0 type 0
[    8.237691] sd 0:0:0:0: [sda] 11721045168 512-byte logical blocks: (6.00 TB/5.46 TiB)
[    8.237694] sd 0:0:0:0: [sda] 4096-byte physical blocks
[    8.237749] sd 0:0:0:0: [sda] Write Protect is off
[    8.237752] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    8.237777] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    8.250078] scsi 1:0:0:0: Direct-Access     ATA      ST6000VN0033-2EE SC60 PQ: 0 ANSI: 5
[    8.252419] sd 1:0:0:0: Attached scsi generic sg1 type 0
[    8.252501] sd 1:0:0:0: [sdb] 11721045168 512-byte logical blocks: (6.00 TB/5.46 TiB)
[    8.252504] sd 1:0:0:0: [sdb] 4096-byte physical blocks
[    8.252564] sd 1:0:0:0: [sdb] Write Protect is off
[    8.252567] sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
[    8.252593] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    8.296718] xhci-hcd f2510000.usb3: Wait for xhci command complete left 1250 ms
[    8.297782]  sda: sda1 sda2 sda3 sda4
[    8.302265] mmc0: MAN_BKOPS_EN bit is not set
[    8.310525]  sdb: sdb1 sdb2 sdb3 sdb4
[    8.310862] mmc0: new HS200 MMC card at address 0001
[    8.316681] mmcblk0: mmc0:0001 8GTF4R 7.28 GiB 
[    8.320749] mmcblk0boot0: mmc0:0001 8GTF4R partition 1 4.00 MiB
[    8.324817] mmcblk0boot1: mmc0:0001 8GTF4R partition 2 4.00 MiB
[    8.328888] mmcblk0rpmb: mmc0:0001 8GTF4R partition 3 512 KiB
[    8.330660] GPT:Primary header thinks Alt. header is not at the end of the disk.
[    8.330662] GPT:1007615 != 15269887
[    8.330663] GPT:Alternate GPT header not at the end of the disk.
[    8.330664] GPT:1007615 != 15269887
[    8.330665] GPT: Use GNU Parted to correct GPT errors.
[    8.330672]  mmcblk0: p1 p2 p3
[    8.372642] ata4: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[    8.372671] ata3: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[    8.381673] ata3.00: ATA-10: ST6000VN0033-2EE110, SC60, max UDMA/133
[    8.381676] ata3.00: 11721045168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
[    8.384154] ata3.00: configured for UDMA/133
[    8.388308] scsi 2:0:0:0: Direct-Access     ATA      ST6000VN0033-2EE SC60 PQ: 0 ANSI: 5
[    8.388693] sd 2:0:0:0: [sdc] 11721045168 512-byte logical blocks: (6.00 TB/5.46 TiB)
[    8.388695] sd 2:0:0:0: [sdc] 4096-byte physical blocks
[    8.388758] sd 2:0:0:0: [sdc] Write Protect is off
[    8.388761] sd 2:0:0:0: [sdc] Mode Sense: 00 3a 00 00
[    8.388789] sd 2:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    8.389119] sd 2:0:0:0: Attached scsi generic sg2 type 0
[    8.389418] ata4.00: ATA-10: ST6000VN0033-2EE110, SC60, max UDMA/133
[    8.389421] ata4.00: 11721045168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
[    8.391856] ata4.00: configured for UDMA/133
[    8.395987] scsi 3:0:0:0: Direct-Access     ATA      ST6000VN0033-2EE SC60 PQ: 0 ANSI: 5
[    8.396326] sd 3:0:0:0: [sdd] 11721045168 512-byte logical blocks: (6.00 TB/5.46 TiB)
[    8.396329] sd 3:0:0:0: [sdd] 4096-byte physical blocks
[    8.396390] sd 3:0:0:0: [sdd] Write Protect is off
[    8.396392] sd 3:0:0:0: [sdd] Mode Sense: 00 3a 00 00
[    8.396419] sd 3:0:0:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    8.396670] sd 3:0:0:0: Attached scsi generic sg3 type 0
[    8.400675] xhci-hcd f2510000.usb3: Wait for xhci command complete left 1250 ms
[    8.420631] usb 3-1: new high-speed USB device number 2 using xhci-hcd
[    8.442406]  sdd: sdd1 sdd2 sdd3 sdd4
[    8.443231]  sdc: sdc1 sdc2 sdc3 sdc4
[    8.516747] usb 4-1: new SuperSpeed USB device number 2 using xhci-hcd
[    8.542339] usb 4-1: New USB device found, idVendor=0bda, idProduct=0411
[    8.542342] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    8.542344] usb 4-1: Product: 4-Port USB 3.0 Hub
[    8.542346] usb 4-1: Manufacturer: Generic
[    8.544895] hub 4-1:1.0: USB hub found
[    8.545618] hub 4-1:1.0: 4 ports detected
[    8.578752] armada38x-rtc f2284000.rtc: setting system clock to 2020-09-28 23:26:34 UTC (1601335594)
[    8.614555] usb 3-1: New USB device found, idVendor=0bda, idProduct=5411
[    8.621294] usb 3-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    8.628464] usb 3-1: Product: 4-Port USB 2.0 Hub
[    8.633109] usb 3-1: Manufacturer: Generic
[    8.637634] hub 3-1:1.0: USB hub found
[    8.642591] hub 3-1:1.0: 4 ports detected
[    8.712995] sd 0:0:0:0: [sda] Attached SCSI removable disk
[    8.718516] asustor add disk dev sda 
[    8.736935] sd 1:0:0:0: [sdb] Attached SCSI removable disk
[    8.742455] asustor add disk dev sdb 
[    8.852928] sd 3:0:0:0: [sdd] Attached SCSI removable disk
[    8.858452] asustor add disk dev sdd 
[    8.858782] sd 2:0:0:0: [sdc] Attached SCSI removable disk
[    8.864299] asustor add disk dev sdc 
[    8.864459] Freeing unused kernel memory: 352K (ffffffc000e34000 - ffffffc000e8c000)
[    8.872270] Freeing alternatives memory: 76K (ffffffc000e8c000 - ffffffc000e9f000)
[    9.800492] mmcblk: mmc_blk_ioctl_as_set_lock before set lock=0 flag 100
[    9.807337] mmcblk: mmc_blk_ioctl_as_set_lock after set lock=0 flag 100
[    9.813980] EXT4-fs (mmcblk0p3): mounted filesystem with ordered data mode. Opts: (null)
[    9.916271] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[    9.922142] 8021q: adding VLAN 0 to HW filter on device eth0
[    9.955570] IPv6: ADDRCONF(NETDEV_UP): eth2: link is not ready
[    9.961441] 8021q: adding VLAN 0 to HW filter on device eth2
[    9.995110] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
[   10.000979] 8021q: adding VLAN 0 to HW filter on device eth1
[   10.060957] random: dd: uninitialized urandom read (512 bytes read, 64 bits of entropy available)
[   10.121436] mmcblk: mmc_blk_ioctl_as_set_lock before set lock=0 flag 100
[   10.129304] mmcblk: mmc_blk_ioctl_as_set_lock after set lock=0 flag 100
[   10.449245] random: nonblocking pool is initialized
[   10.708774] md: md0 stopped.
[   10.717015] md: bind<sdb2>
[   10.721092] md: bind<sdc2>
[   10.745691] md: bind<sda2>
[   10.750168] md/raid1:md0: active with 3 out of 4 mirrors
[   10.755592] asustor add disk dev md0 
[   10.755658] md0: detected capacity change from 0 to 2145386496
[   10.817568] EXT4-fs (md0): mounted filesystem with ordered data mode. Opts: (null)
[   10.871439] EXT4-fs (md0): re-mounted. Opts: (null)
[   11.223550] EXT4-fs (loop0): mounted filesystem without journal. Opts: (null)
[   11.231454] mmcblk: mmc_blk_ioctl_as_set_lock before set lock=0 flag 100
[   11.238198] mmcblk: mmc_blk_ioctl_as_set_lock after set lock=0 flag 100
[   11.248465] EXT4-fs (mmcblk0p3): mounted filesystem with ordered data mode. Opts: (null)
[   13.421285] mvpp2 f2000000.ppv22 eth2: link up
[   13.426173] IPv6: ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready
[   15.179312] mmcblk: mmc_blk_ioctl_as_set_lock before set lock=0 flag 101
[   15.186082] mmcblk: mmc_blk_ioctl_as_set_lock after set lock=1 flag 101
[   18.276218] bridge: automatic filtering via arp/ip/ip6tables has been deprecated. Update your scripts to load br_netfilter if you need this.
[   18.867452] md: md126 stopped.
[   18.916012] md: bind<sdb3>
[   18.923731] md: bind<sdd3>
[   18.932058] md: bind<sdc3>
[   18.935326] md: bind<sda3>
[   18.942203] md/raid1:md126: active with 4 out of 4 mirrors
[   18.947727] asustor add disk dev md126 
[   18.947794] md126: detected capacity change from 0 to 2145386496
[   18.986971] Adding 2095100k swap on /dev/md126.  Priority:-1 extents:1 across:2095100k 
[   19.015005] md: md1 stopped.
[   19.022607] md: bind<sdb4>
[   19.027138] md: bind<sdc4>
[   19.033661] md: bind<sda4>
[   19.048737] md/raid:md1: device sda4 operational as raid disk 0
[   19.056779] md/raid:md1: device sdc4 operational as raid disk 2
[   19.062787] md/raid:md1: device sdb4 operational as raid disk 1
[   19.084602] md/raid:md1: allocated 105152kB
[   19.088888] md/raid:md1: raid level 5 active with 3 out of 3 devices, algorithm 2
[   19.096463] RAID conf printout:
[   19.096467]  --- level:5 rd:3 wd:3
[   19.096472]  disk 0, o:1, dev:sda4
[   19.096476]  disk 1, o:1, dev:sdb4
[   19.096479]  disk 2, o:1, dev:sdc4
[   19.096510] asustor add disk dev md1 
[   19.096596] md1: detected capacity change from 0 to 11992953782272
[   19.153013] asustor remove disk dev md1 
[   19.163375] md1: detected capacity change from 11992953782272 to 0
[   19.169630] md: md1 stopped.
[   19.172538] md: unbind<sda4>
[   19.208642] md: export_rdev(sda4)
[   19.211993] md: unbind<sdc4>
[   19.244637] md: export_rdev(sdc4)
[   19.247997] md: unbind<sdb4>
[   19.280634] md: export_rdev(sdb4)
[   19.404750] mmcblk: mmc_blk_ioctl_as_set_lock before set lock=1 flag 101
[   19.412913] mmcblk: mmc_blk_ioctl_as_set_lock after set lock=1 flag 101
[   25.483097] ip_tables: (C) 2000-2006 Netfilter Core Team
[   28.151823] sys_stat: module is from the staging directory, the quality is unknown, you have been warned.
[   28.339165] fnotify: module license 'Proprietary' taints kernel.
[   28.452939] Disabling lock debugging due to kernel taint
[   29.840816] iscsi-scst: ***WARNING***: CONFIG_TCP_ZERO_COPY_TRANSFER_COMPLETION_NOTIFICATION not enabled in your kernel. ISCSI-SCST will be working with not the best performance. Refer README file for details.
[   30.025340] Loading iSCSI transport class v2.0-870.
[   30.205175] iscsi: registered transport (tcp)
[   31.812694] usbip_core: USB/IP Core v1.0.0
[   31.833772] usbcore: registered new interface driver usbip-host
[   31.841213] usbip_host: USB/IP Host Driver v1.0.0
[   33.106696] cryptodev: driver 1.9 loaded.
[   33.335612] Bluetooth: Core ver 2.21
[   33.341407] NET: Registered protocol family 31
[   33.347823] Bluetooth: HCI device and connection manager initialized
[   33.358057] Bluetooth: HCI socket layer initialized
[   33.366806] Bluetooth: L2CAP socket layer initialized
[   33.373640] Bluetooth: SCO socket layer initialized
[   33.383610] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
[   33.391037] Bluetooth: BNEP filters: protocol multicast
[   33.399636] Bluetooth: BNEP socket layer initialized
[   33.410297] Bluetooth: RFCOMM TTY layer initialized
[   33.415282] Bluetooth: RFCOMM socket layer initialized
[   33.425245] Bluetooth: RFCOMM ver 1.11
[   33.435732] Bluetooth: HIDP (Human Interface Emulation) ver 1.2
[   33.446132] Bluetooth: HIDP socket layer initialized
[   33.569301] Bluetooth: HCI UART driver ver 2.3
[   33.583070] Bluetooth: HCI UART protocol H4 registered
[   33.598391] Bluetooth: HCI UART protocol BCSP registered
[   33.620347] Bluetooth: HCI UART protocol LL registered
[   33.642666] Bluetooth: HCI UART protocol ATH3K registered
[   33.648279] Bluetooth: HCI UART protocol Three-wire (H5) registered
[   33.656271] Bluetooth: HCI UART protocol Intel registered
[   33.664487] Bluetooth: HCI UART protocol BCM registered
[   33.674256] Bluetooth: HCI UART protocol QCA registered
[   33.714167] usbcore: registered new interface driver btusb
[   43.808355] usbcore: registered new interface driver snd-usb-audio
Top
User avatar
orion
Posts: 3485
Joined: Wed May 29, 2013 11:09 am

Re: AS4004T: Volume inaccessible

Post by orion »

I guess it's something wrong inside software. I don't see any abnormal hardware fault messages. The problem is "[ 19.153013] asustor remove disk dev md1". I think you should report it to asustor directly. https://support.asustor.com/
Top
User avatar
Nazar78
Posts: 2068
Joined: Wed Jul 17, 2019 10:21 pm
Location: Singapore
Contact:
Contact Nazar78

Re: AS4004T: Volume inaccessible

Post by Nazar78 »

Unfortunately I just got the same issue. There's a power outage last night. I've set the NAS to go into safe mode after 15mins and it did before soon after the UPS gets depleted and everything turned off.

When the power restored and the NAS auto start up, ADM shows my vol1 is inaccessible, unknown size and unknown raid level. The disks are listed however inactive state. I guess there's some corruption because some of my settings reverted back to default, like the asustor ID, ntp, network ALB etc. How safe is the safe mode comes into question?

1. Noticed my raid 10 wasn't assembled which I did manually and it works, created the /volume1 and mount it successfully.

2. Noticed my /volume0/usr/etc/volume.conf is corrupted, I rebuilt from scratch and it works, ADM now showing the correct raid 10 and vol1.

The issue now is that whenever the NAS reboots, it doesn't automatically assemble my raid 10. Some how I think the /usr/sbin/raidmand is responsible for this, it's running but I can't find where is the mdadm configuration. So far I can't find anything about this online nor the forum. I'll try to dig in deeper later today and if someone has a clue please help to share your insight because I don't want to go through all the trouble initializing the NAS then restore from backup which I guess would come from Asustor support if I were to contact them.

And also ADM should use the standard way of assembling via mdadm because there's nothing wrong with the raid only ADM settings messed up somewhere then just drop everything telling the user your volume is inaccessible. I booted Ubuntu on the NAS from a separate partition immediately I can see my raid 10 without doing anything.
Last edited by Nazar78 on Thu Oct 01, 2020 2:34 pm, edited 1 time in total.
AS5304T - 16GB DDR4 - ADM-OS modded on 2GB RAM
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps

When posting, consider checking the box "Notify me when a reply is posted" to get faster response
Top
User avatar
Nazar78
Posts: 2068
Joined: Wed Jul 17, 2019 10:21 pm
Location: Singapore
Contact:
Contact Nazar78

Re: AS4004T: Volume inaccessible

Post by Nazar78 »

Ok I managed to fix it, persistent across reboots. I think there's some other corruption like my samba shares not able to edit in ADM but at least volume1 is fixed.

ADM looks for this file to assemble the software raid: /usr/etc/volume.conf. It needs to have the correct info else it'll just bailout. Previously when I reconstruct this file due to corruption, I roughly took and join the UUID_SUB 8th characters from blkid from all the hard disk's 4th partitions:

Code: Select all

 blkid|grep '4: '
But I was wrong real fact that I should use the main UUID instead, replace the "-" with ":". This is also revealed in this command (provided you've already assembled it manually, else just use the blkid command to get the UUID):

Code: Select all

mdadm --query --detail /dev/md1
So for anyone or OP who faced this issue, edit your /usr/etc/volume.conf or /volume0/usr/etc/volume.conf, they are the same file just the path symlinked. After editing just reboot. The file contents looks like below, I've placed "#" as comments. Hope it helps someone...

Code: Select all

[volume1]
Level = 2 # Raid level i.e. 0,1,5,6. For raid 10 set it as 2.
Raid = 4 # Number of active disks?
Total = 4 # Total number of disks?
Option = 0
Ftype = ext4 # File system, btrfs or ext4.
UUID = 09f8f719:23e222de:f8d3s308:3b333f58 # Get the UUID from above comment.
Index = 0,1,2,3 # Disks index starts from zero.
Cachemode = 0
CLevel = 0
CState = -1
CDirty = 0
CUUID =
Cnumber = 0
CIndex =
Cseqcut = No
CsizeMB = 0
AS5304T - 16GB DDR4 - ADM-OS modded on 2GB RAM
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps

When posting, consider checking the box "Notify me when a reply is posted" to get faster response
Top
Post Reply

Return to “Backup and Data Protection”