How to make your disks sleep!

[dstel]

In your NAS, there is a hidden Volume0 that is in RAID1 on all your disks. Volume0 is unlikely to sleep. If you have some apps on your NAS, most of them use a Volume1. Volume1 is unlikely to sleep.

Well, let's limit the damage to insure that Volume0 and Volume1 uses the same disks so the others can sleep.

Warning, this must be done at each boot. This may be automated in a future version of ADM, we are at 3.5.2RAG2version.

1) Find any volume that is not yours.
ls -l /dev/md*

You'll find md0 but also md126 in my case. So the operation must be done on each of those volumes. From now, I'll give you my example. It's up to you to infer for your own setup.

2) Get the disks that never sleeps from volume1.

Code: Select all

# mdadm --detail /dev/md1
/dev/md1:
        Version : 1.2
  Creation Time : Sat Jul 18 09:07:12 2015
     Raid Level : raid5
     Array Size : 15618877440 (14895.32 GiB 15993.73 GB)
  Used Dev Size : 7809438720 (7447.66 GiB 7996.87 GB)
   Raid Devices : 3
  Total Devices : 3
    Persistence : Superblock is persistent

    Update Time : Sat Nov  7 01:55:28 2020
          State : clean 
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 64K

           Name : AS6508T-D0C0:1  (local to host AS6508T-D0C0)
           UUID : c7b86c6c:0679c9a4:0562baab:4c420d30
         Events : 17390

    Number   Major   Minor   RaidDevice State
       3       8       36        0      active sync   /dev/sdc4
       2       8       52        1      active sync   /dev/sdd4
       4       8       84        2      active sync   /dev/sdf4

So sdc, sdd and sdf will never sleep in my setup. Let's put the volume0 on those. The others will be spare disks.

3) Look at your md0 configuration

Code: Select all

# mdadm --detail /dev/md0
/dev/md0:
        Version : 1.2
  Creation Time : Sat Jul 18 09:06:58 2015
     Raid Level : raid1
     Array Size : 2095104 (2046.34 MiB 2145.39 MB)
  Used Dev Size : 2095104 (2046.34 MiB 2145.39 MB)
   Raid Devices : 8
  Total Devices : 7
    Persistence : Superblock is persistent

    Update Time : Thu Nov  5 08:41:19 2020
          State : clean, degraded 
 Active Devices : 7
Working Devices : 7
 Failed Devices : 0
  Spare Devices : 0

           Name : AS6508T-D0C0:0  (local to host AS6508T-D0C0)
           UUID : f68bd32e:6c63499d:34fbd66a:b7abf589
         Events : 69721

    Number   Major   Minor   RaidDevice State
       8       8       34        0      active sync   /dev/sdc2
      11       8       66        1      active sync   /dev/sde2
       9       8       50        2      active sync   /dev/sdd2
      10       8        2        3      active sync   /dev/sda2
      12       8       18        4      active sync   /dev/sdb2
      13       8      114        5      active sync   /dev/sdh2
      14       8       98        6      active sync   /dev/sdg2
      14       0        0       14      removed

4) Remove and add disks so volume0 uses the same disks than the volume1

Code: Select all

mdadm /dev/md0 --fail /dev/sde2 /dev/sda2 /dev/sdb2 /dev/sdh2 /dev/sdg2
mdadm /dev/md0 --remove /dev/sde2 /dev/sda2 /dev/sdb2 /dev/sdh2 /dev/sdg2
mdadm /dev/md0 --add /dev/sdf2

5) Re-add the other disks as spare (optional but recommended)
For this you need to reduce the size of the array if not, any spare will be immediately added as a true disk since the array would be considered as incomplete

Code: Select all

mdadm --grow /dev/md0 --raid-devices=3

Now, add all the other disks as spare

Code: Select all

mdadm /dev/md0 --add-spare /dev/sde2 /dev/sda2 /dev/sdb2 /dev/sdh2 /dev/sdg2

6) Look at the result

Code: Select all

# mdadm --detail /dev/md/dev/md0:
        Version : 1.2
  Creation Time : Sat Jul 18 09:06:58 2015
     Raid Level : raid1
     Array Size : 2095104 (2046.34 MiB 2145.39 MB)
  Used Dev Size : 2095104 (2046.34 MiB 2145.39 MB)
   Raid Devices : 3
  Total Devices : 8
    Persistence : Superblock is persistent

    Update Time : Thu Nov  5 08:55:51 2020
          State : clean 
 Active Devices : 3
Working Devices : 8
 Failed Devices : 0
  Spare Devices : 5

           Name : AS6508T-D0C0:0  (local to host AS6508T-D0C0)
           UUID : f68bd32e:6c63499d:34fbd66a:b7abf589
         Events : 69778

    Number   Major   Minor   RaidDevice State
       8       8       34        0      active sync   /dev/sdc2
      10       8       82        1      active sync   /dev/sdf2
       9       8       50        2      active sync   /dev/sdd2

       3       8       66        -      spare   /dev/sde2
       4       8        2        -      spare   /dev/sda2
       5       8       18        -      spare   /dev/sdb2
       6       8      114        -      spare   /dev/sdh2
       7       8       98        -      spare   /dev/sdg2

Repeat for all other md you don't know (don't touch to the one containing nvme* since those are the SSD drives). In my case, I had a md126.

Now the "uncontrolled" activities of volume0 and volume1 will only prevent the same three disks (in my case) to go to sleep. The others will go to sleep if they can.

[Nazar78]

Interesting and great suggestion. However I'm curious what happens if you skipped adding the spare drives? Will it continue to work?

Just to share while we're at this topic, I did a different approach though by moving all my volume1 active apps (Asustor's, docker, chroot, Ubuntu Focal Desktop via LXC) to an external ssd and moved the volume0 logs to the ram. The logs are flushed back to the disk upon shutdown or reboot, I followed WD NAS concept I had previously (hinting Asustor folks). This is done automatically at boot so most of the time my volume1 and external volume2 hdds are sleeping only the ssd is active (runs cooler and silent). I could tell by the LEDs they really goes to sleep without checking the system despite having lots of apps running actively.

Not sure though if one starting from scratch is able to natively put all the apps to volume1 as ssd then the rest of the data to a separate volume2 array onwards because I only have 4 slots.

[dstel]

Interesting and great suggestion. However I'm curious what happens if you skipped adding the spare drives? Will it continue to work?

Yes but there is a danger. If you decide to "stop volume X" and remove the drives, your also remove all the volume0. While with spare drives, as soon as you remove a drive, another drive starts and resync. The volume0 is small so the resync is done in less than one minute.

I edited my original post and added the mention "optional but recommended".

Just to share while we're at this topic, I did a different approach though by moving all my volume1 active apps (Asustor's, docker, chroot, Ubuntu Focal Desktop via LXC) to an external ssd and moved the volume0 logs to the ram. The logs are flushed back to the disk upon shutdown or reboot, I followed WD NAS concept I had previously (hinting Asustor folks). This is done automatically at boot so most of the time my volume1 and external volume2 hdds are sleeping only the ssd is active (runs cooler and silent). I could tell by the LEDs they really goes to sleep without checking the system despite having lots of apps running actively.

Not sure though if one starting from scratch is able to natively put all the apps to volume1 as ssd then the rest of the data to a separate volume2 array onwards because I only have 4 slots.

So I didn't do that because I like the volume0 to be on multiple internal disks but I had an application doing a lot of little accesses which were very annoying on the HDDs because of the noise. I moved it to "Volume2" which is a SSD in bay8, it's better!

Note: if you have a free NVME SSD, instead of using it as a cache, you can use it as a volume. However it will not appear in some tools like the monitoring which allow to see only disks from 1 to 8 if they are present.

[Nazar78]

So I didn't do that because I like the volume0 to be on multiple internal disks but I had an application doing a lot of little accesses which were very annoying on the HDDs because of the noise. I moved it to "Volume2" which is a SSD in bay8, it's better!

Yeah placing them in ssd is way better. I have multiple sites facing the web so there's no way of letting those disks going to sleep everytime there's traffic on top of bots and crawlers.

A thing about the system volume0. As safe as it seems having multiple disk copies, they are just mirrors of each other and that doesn't stop corruption occuring on FS level. Recently I encountered a system config corruption. There was a power outage, the NAS was set to go into safe mode 15mins after the UPS took over. The outage took quite sometime and when the UPS got depleted, the NAS shut off while in safe mode. When the power comes back, I partially lost some of the NAS config. I was lucky to have a daily backup of the settings to restore. So I've learned that the system volume0 mirrors are not really that useful unless disks failures. If the system can't boot up due to FS corruption, then the other mirrors are also useless. We can easily dump and restore the system mirror partition images from a newly initialized disk but the data settings are gone without backups.