NAS stuck at merging 0% for 12 hours

Moderator: Lillian.W@AST

Post Reply
kluchgts
Posts: 7
youtube meble na wymiar Warszawa
Joined: Thu Nov 12, 2020 8:34 am

NAS stuck at merging 0% for 12 hours

Post by kluchgts »

Should I be worried about anything? Usually the % counter goes up with anything it does, but this time it's been stuck at 0 with nothing in the log either. I was adding more drives to my raid 10.

edit 1: oh and I can still access my NAS during this time. idk if that's normal or not so I decided to mention it. I haven't tried writing anything though.
User avatar
Nazar78
Posts: 2002
Joined: Wed Jul 17, 2019 10:21 pm
Location: Singapore
Contact:

Re: NAS stuck at merging 0% for 12 hours

Post by Nazar78 »

Try checking the status in SSH:

Code: Select all

cat /proc/mdstat
Or:

Code: Select all

mdadm -D /dev/md*
AS5304T - 16GB DDR4 - ADM-OS modded on 2GB RAM
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps

When posting, consider checking the box "Notify me when a reply is posted" to get faster response
kluchgts
Posts: 7
Joined: Thu Nov 12, 2020 8:34 am

Re: NAS stuck at merging 0% for 12 hours

Post by kluchgts »

My linux is at very beginner levels, what am I looking for here? This is the output of cat /proc/mdstat

Code: Select all

Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md16 : active raid10 sde4[7] sdf4[5] sdc4[3] sdd4[1]
      46857187328 blocks super 1.2 64K chunks 2 near-copies [8/4] [_U_U_U_U]

md1 : active raid10 sdh4[3] sdb4[2] sda4[1] sdi4[0]
      23428593664 blocks super 1.2 64K chunks 2 near-copies [4/4] [UUUU]

md126 : active raid1 sde3[11](S) sdf3[10](S) sdc3[9](S) sdd3[8](S) sdh3[5] sdi3[4] sdb3[7] sda3[6]
      2095104 blocks super 1.2 [4/4] [UUUU]

md0 : active raid1 sdi2[10] sdh2[4] sdb2[7] sda2[6]
      2095104 blocks super 1.2 [10/4] [UUUU______]

unused devices: <none>
User avatar
Nazar78
Posts: 2002
Joined: Wed Jul 17, 2019 10:21 pm
Location: Singapore
Contact:

Re: NAS stuck at merging 0% for 12 hours

Post by Nazar78 »

What's the result of the other command?
AS5304T - 16GB DDR4 - ADM-OS modded on 2GB RAM
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps

When posting, consider checking the box "Notify me when a reply is posted" to get faster response
kluchgts
Posts: 7
Joined: Thu Nov 12, 2020 8:34 am

Re: NAS stuck at merging 0% for 12 hours

Post by kluchgts »

mdadm -D /dev/md* output

Code: Select all

/dev/md0:
        Version : 1.2
  Creation Time : Fri Feb  7 21:01:30 2020
     Raid Level : raid1
     Array Size : 2095104 (2046.34 MiB 2145.39 MB)
  Used Dev Size : 2095104 (2046.34 MiB 2145.39 MB)
   Raid Devices : 10
  Total Devices : 4
    Persistence : Superblock is persistent

    Update Time : Sun Nov 15 00:04:22 2020
          State : clean, degraded
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0

           Name : AS5304T-697B:0
           UUID : 296a2b2e:c4b780f4:9f6cee2e:048b6799
         Events : 26436

    Number   Major   Minor   RaidDevice State
       4       8      114        0      active sync   /dev/sdh2
       6       8        2        1      active sync   /dev/sda2
       7       8       18        2      active sync   /dev/sdb2
      10       8      130        3      active sync   /dev/sdi2
       8       0        0        8      removed
      10       0        0       10      removed
      12       0        0       12      removed
      14       0        0       14      removed
      16       0        0       16      removed
      18       0        0       18      removed
/dev/md1:
        Version : 1.2
  Creation Time : Tue Nov 10 21:04:45 2020
     Raid Level : raid10
     Array Size : 23428593664 (22343.25 GiB 23990.88 GB)
  Used Dev Size : 11714296832 (11171.62 GiB 11995.44 GB)
   Raid Devices : 4
  Total Devices : 4
    Persistence : Superblock is persistent

    Update Time : Fri Nov 13 17:40:19 2020
          State : clean
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0

         Layout : near=2
     Chunk Size : 64K

           Name : VALKYRIE-DONO:1  (local to host VALKYRIE-DONO)
           UUID : f94d591b:59d060cb:82529ca5:b782db3e
         Events : 80800

    Number   Major   Minor   RaidDevice State
       0       8      132        0      active sync set-A   /dev/sdi4
       1       8        4        1      active sync set-B   /dev/sda4
       2       8       20        2      active sync set-A   /dev/sdb4
       3       8      116        3      active sync set-B   /dev/sdh4
/dev/md126:
        Version : 1.2
  Creation Time : Fri Feb  7 21:01:37 2020
     Raid Level : raid1
     Array Size : 2095104 (2046.34 MiB 2145.39 MB)
  Used Dev Size : 2095104 (2046.34 MiB 2145.39 MB)
   Raid Devices : 4
  Total Devices : 8
    Persistence : Superblock is persistent

    Update Time : Tue Nov 10 20:50:09 2020
          State : clean
 Active Devices : 4
Working Devices : 8
 Failed Devices : 0
  Spare Devices : 4

           Name : VALKYRIE-DONO:126  (local to host VALKYRIE-DONO)
           UUID : 09bb3e04:9cce1ae6:72d3fe91:94c2c7b2
         Events : 143

    Number   Major   Minor   RaidDevice State
       5       8      115        0      active sync   /dev/sdh3
       6       8        3        1      active sync   /dev/sda3
       7       8       19        2      active sync   /dev/sdb3
       4       8      131        3      active sync   /dev/sdi3

       8       8       51        -      spare   /dev/sdd3
       9       8       35        -      spare   /dev/sdc3
      10       8       83        -      spare   /dev/sdf3
      11       8       67        -      spare   /dev/sde3
/dev/md16:
        Version : 1.2
  Creation Time : Fri Nov 13 17:40:19 2020
     Raid Level : raid10
     Array Size : 46857187328 (44686.50 GiB 47981.76 GB)
  Used Dev Size : 11714296832 (11171.62 GiB 11995.44 GB)
   Raid Devices : 8
  Total Devices : 4
    Persistence : Superblock is persistent

    Update Time : Sun Nov 15 00:04:44 2020
          State : clean, degraded
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0

         Layout : near=2
     Chunk Size : 64K

           Name : VALKYRIE-DONO:16  (local to host VALKYRIE-DONO)
           UUID : b4c24499:ddcbdb30:d0ebfc10:ec44c7fd
         Events : 70088

    Number   Major   Minor   RaidDevice State
       0       0        0        0      removed
       1       8       52        1      active sync set-B   /dev/sdd4
       4       0        0        4      removed
       3       8       36        3      active sync set-B   /dev/sdc4
       8       0        0        8      removed
       5       8       84        5      active sync set-B   /dev/sdf4
      12       0        0       12      removed
       7       8       68        7      active sync set-B   /dev/sde4
User avatar
Nazar78
Posts: 2002
Joined: Wed Jul 17, 2019 10:21 pm
Location: Singapore
Contact:

Re: NAS stuck at merging 0% for 12 hours

Post by Nazar78 »

There's no activities and /dev/md16 which I believe you're trying to add more disks states the disks are missing. If you're confident, you can attempt to add back the disks from command line then grow the array. Before you do this try reboot first see if the process resume by checking the volume manager and mdstat.

Code: Select all

mdadm /dev/md16 --add /dev/sd[X1]4 /dev/sd[X2]4 /dev/sd[X3]4 /dev/sd[X4]4
X1-4 are the four disks of its 4th partition you're adding e.g. /dev/sdj4 which you'll need to figure it out based on your disks layout. I'm replying using mobile phone it's hard for me to figure your layout and don't want to give the wrong info but it's definitely not the disks that's already in the array.

Code: Select all

mdadm --grow /dev/md16 --raid-devices=8
Then watch the mdstat for its status, CTRL+c to return.

Code: Select all

watch -n1 cat /proc/mdstat
Else you should contact support https://support.asustor.com/.
AS5304T - 16GB DDR4 - ADM-OS modded on 2GB RAM
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps

When posting, consider checking the box "Notify me when a reply is posted" to get faster response
kluchgts
Posts: 7
Joined: Thu Nov 12, 2020 8:34 am

Re: NAS stuck at merging 0% for 12 hours

Post by kluchgts »

when I restart the NAS the status of the volume goes back to good and the drives I was trying to add are listed as spare drives.
User avatar
Nazar78
Posts: 2002
Joined: Wed Jul 17, 2019 10:21 pm
Location: Singapore
Contact:

Re: NAS stuck at merging 0% for 12 hours

Post by Nazar78 »

Ok you can now grow the array to 8 disks with the 4 hot spares. During this process you should see reshape status in the mdstat.

The merging 0% stuck for 12 hours shouldn't happen in the first place. Probably best you also open a ticket with support so they could fix the bug for future ADM release.
AS5304T - 16GB DDR4 - ADM-OS modded on 2GB RAM
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps

When posting, consider checking the box "Notify me when a reply is posted" to get faster response
kluchgts
Posts: 7
Joined: Thu Nov 12, 2020 8:34 am

Re: NAS stuck at merging 0% for 12 hours

Post by kluchgts »

ok so I started the process but I went wrong somewhere and I don't know how much it'll affect me.
There was no md16 so I took a look and thought md126 was correct, but that's one of the mirrors. I ran

Code: Select all

mdadm --grow /dev/md126 --raid-devices=8
and

Code: Select all

watch -n1 cat /proc/mdstat
right after it. md126 went into recovery but finished pretty quickly. I ran the 1st command on md1 which is the raid 10 with the spares assigned to it and it's now reshaping it.
So it looks like I got the right thing going eventually? I just don't know if running the command on md126, one of the mirrors would hurt or not. current output of cat /proc/mdstat

Code: Select all

Personalities : [linear] [raid0] [raid1] [raid10] [raid6] [raid5] [raid4]
md1 : active raid10 sde4[7] sdf4[6] sdc4[5] sdd4[4] sdi4[0] sdh4[3] sdb4[2] sda4[1]
      23428593664 blocks super 1.2 64K chunks 2 near-copies [8/8] [UUUUUUUU]
      [>....................]  reshape =  0.1% (43435456/23428593664) finish=1913.1min speed=203718K/sec

md126 : active raid1 sde3[11] sdf3[10] sdc3[9] sdd3[8] sdh3[5] sdg3[12](F) sdi3[4] sdb3[7] sda3[6]
      2095104 blocks super 1.2 [8/8] [UUUUUUUU]

md0 : active raid1 sde2[14] sdf2[13] sdc2[12] sdd2[11] sdh2[4] sdi2[10] sdb2[7] sda2[6]
      2095104 blocks super 1.2 [10/8] [UUUUUUUU__]
User avatar
Nazar78
Posts: 2002
Joined: Wed Jul 17, 2019 10:21 pm
Location: Singapore
Contact:

Re: NAS stuck at merging 0% for 12 hours

Post by Nazar78 »

Looks good but I'm not sure how many raid 10s you had initially because from your previous post there's md1 and md16.

However if actual fact you indeed have only one raid 10 md1, then everything is good now just wait the reshape to complete.

As for the md126 that's your 2GB swap partition mirror array as long as it's recovered then it's good.
AS5304T - 16GB DDR4 - ADM-OS modded on 2GB RAM
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps

When posting, consider checking the box "Notify me when a reply is posted" to get faster response
Post Reply

Return to “[Official] For AS65xx Series”