Migrating RAID volumes

Got a question about our NAS utilities? The answer lies within.

Moderator: Lillian.W@AST

helgrimm
Posts: 36
youtube meble na wymiar Warszawa
Joined: Sun May 08, 2022 5:20 pm
Location: Copenhagen • Denmark

Re: Migrating RAID volumes

Post by helgrimm »

Nazar78 wrote: Edited: From my honest opinion, it's better to speedup the rsync/build process in exchange for less usability because if another disk fail during this process, unless the array is designed to tolerate 2 disks failures, the array will fail and there goes the data. I had RAID10 new disk failure which happened twice over two different new disks in a period of few months.
There goes my argument. These disks are brand spanking new. I'm not hearing what you're saying :)
rebuilding.jpg
rebuilding.jpg (22.49 KiB) Viewed 3426 times
EDIT: Process is through and status changed as expected ->
final.jpg
final.jpg (28.14 KiB) Viewed 3398 times
Last edited by helgrimm on Mon May 16, 2022 6:06 pm, edited 1 time in total.
AS5304T + AS6004U + 4 x SEAGATE EXOS 16TB + 2 x SEAGATE IRONWOLF 8TB + 2 x WD 4TB
User avatar
Nazar78
Posts: 2003
Joined: Wed Jul 17, 2019 10:21 pm
Location: Singapore
Contact:

Re: Migrating RAID volumes

Post by Nazar78 »

helgrimm wrote:Thanks for that information. I do actually have a PC running Ubuntu on an Asus Mini PC that I use as a media center. But say the situation was real - how would I connect the disks? The mini PC only has one SATA connection for the system SSD (2½ inch). Or if I booted my laptop into Linux from USB - again, how to get the disks connected.

On the AS5304T things are a bit different in case you can still SSH into it.
You should be able to enable or install sshd on the Ubuntu Mini PC probably it's already there for you to ssh in. Or just use the GUI over HDMI.

I'll use my 5 bay USB enclosure for recoveries or my available PC sata ports as I don't have the AS6004U but it should be detected as USB enclosure when you connect it to the Asus Mini PC or your laptop. If that's the case, install mdadm if it's not already available, assemble manually using mdadm then mount it to access its contents. On native Linux DE such as Ubuntu Desktop, after installing mdadm it should automatically scan, assemble then popup your mounted device in the file manager.
helgrimm wrote:I've found a product 2 thirds the price of the AS6004U that has 4 slots and offers JBOD, RAID0. RAID1 in 2 pairs and with the lastest USB3. I'm going to go with that one. I'll rest more assured when I know that my data can be moved and accessed independently on almost any external hardware and software. You can have a look if you like - https://www.fantec.de/fileadmin/user_up ... ual_EN.pdf
I've looked at this similar option before but was unsure if this is software or hardware RAID, the latter I assumed. Thing is about hardware RAID is if the controller gets defunct, you will need to replace with the exact controller. Learn it the hard way when my older PC cheap RAID PCI got busted and recovery was a nightmare. Even if it's a software RAID but with an unreleased source, you'll have a hard time dealing with failures trying to poke around the software internals. So in short, normal multi bay USB enclosure is my best bet. At least I can tweak the software RAID using common Linux tools.
helgrimm wrote:I'm going off topic a bit :)

My NAS before the Asustor was a WD MyCloud (I had 2x4TB). Actually one of those 2 volumes I've been talking about copying data from in this thread consists of the two 4TB disks I've yanked from the WD MyCloud enclosures. I have the same NAS as you and my router? You guessed right, it's an Asus RT-AX86U. Now I've read on your homepage that your previous NAS was a WD MyCloud as well. Do you also have a sofa from IKEA? :) :)
Yeah a 4TB first generation, yours is the mirror version. That's my first NAS but I have been playing with RAID and Linux long before, only that I wasn't sure that time if I should splurge on a multi bay NAS.

And that router is a beast! I was running python server (OctoPrint) on it connected wirelessly via USBIP (compiled the kernel modules) to my 3D printer, had since moved the server to the 5304T for better x86_64 support compared to the router's aarch64/arm64. Some of the router's task currently is to host OpenVPN, capture my IP cam via ffmpeg, rotate the captures one month archives, force 2.4GHz clients to connect to 5GHz if they're near by i.e. have good RSSI, monitor the temperature of my cabinet housing all my disks (I didn't let the NAS do this as it will keep the disks awaked when probed by the NAS) and deal with a set of backups to external USB via rsyncd. Few of many things you can achieve with these small devices, just have to explore the possibilities.

We all love Ikea don't we? ¯\_(ツ)_/¯
Last edited by Nazar78 on Mon May 16, 2022 1:31 am, edited 1 time in total.
AS5304T - 16GB DDR4 - ADM-OS modded on 2GB RAM
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps

When posting, consider checking the box "Notify me when a reply is posted" to get faster response
User avatar
Nazar78
Posts: 2003
Joined: Wed Jul 17, 2019 10:21 pm
Location: Singapore
Contact:

Re: Migrating RAID volumes

Post by Nazar78 »

helgrimm wrote:
Nazar78 wrote: Edited: From my honest opinion, it's better to speedup the rsync/build process in exchange for less usability because if another disk fail during this process, unless the array is designed to tolerate 2 disks failures, the array will fail and there goes the data. I had RAID10 new disk failure which happened twice over two different new disks in a period of few months.
There goes my argument. These disks are brand spanking new. I'm not hearing what you're saying :)
Yes! Tell me about it, newly sealed disks, I almost panicked, quickly did the warranty exchange the next day as I don't have spare huge TB of disks laying around, then quickly have it rebuild. The ideal way is actually to have spares assigned to the array so the spare takes over when a single disk failed but it's only a 4 bay.
AS5304T - 16GB DDR4 - ADM-OS modded on 2GB RAM
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps

When posting, consider checking the box "Notify me when a reply is posted" to get faster response
helgrimm
Posts: 36
Joined: Sun May 08, 2022 5:20 pm
Location: Copenhagen • Denmark

Re: Migrating RAID volumes

Post by helgrimm »

Nazar78 wrote:I don't have the AS6004U but it should be detected as USB enclosure when you connect it to the Asus Mini PC or your laptop.
Not on the Windows PC - I got curious and tested that yesterday. Connecting the AS6004U to the Windows Laptop by USB does nothing.
Nazar78 wrote:Yeah a 4TB first generation, yours is the mirror version. That's my first NAS but I have been playing with RAID and Linux long before, only that I wasn't sure that time if I should splurge on a multi bay NAS.
Mine was also the first generation I think. It was 2 separate enclosures. I've had them for many years.
Nazar78 wrote:And that router is a beast! I was running python server [...]
I'm very happy with it for sure :)
AS5304T + AS6004U + 4 x SEAGATE EXOS 16TB + 2 x SEAGATE IRONWOLF 8TB + 2 x WD 4TB
User avatar
Nazar78
Posts: 2003
Joined: Wed Jul 17, 2019 10:21 pm
Location: Singapore
Contact:

Re: Migrating RAID volumes

Post by Nazar78 »

Not on the Windows PC - I got curious and tested that yesterday. Connecting the AS6004U to the Windows Laptop by USB does nothing.
Not even detected as an unknown device in the device manager? If yes we could probably hunt for the driver or beg Asustor to release its source which we can easily compile. What about the Asus Mini PC? What does lsusb (SSH from the NAS) says about the connected AS6004U? I'm sorry about this but unless someone familiar with the AS6004U who could figure this out, only Asustor Support will have the answer.
Last edited by Nazar78 on Mon May 16, 2022 11:08 am, edited 1 time in total.
AS5304T - 16GB DDR4 - ADM-OS modded on 2GB RAM
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps

When posting, consider checking the box "Notify me when a reply is posted" to get faster response
User avatar
Nazar78
Posts: 2003
Joined: Wed Jul 17, 2019 10:21 pm
Location: Singapore
Contact:

Re: Migrating RAID volumes

Post by Nazar78 »

Code: Select all

Mine was also the first generation I think. It was 2 separate enclosures. I've had them for many years.
That's still a cool fun device, although not as polished as a Rasp Pi. Do keep the internal board for future mini projects you might have i.e. home automation.
AS5304T - 16GB DDR4 - ADM-OS modded on 2GB RAM
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps

When posting, consider checking the box "Notify me when a reply is posted" to get faster response
helgrimm
Posts: 36
Joined: Sun May 08, 2022 5:20 pm
Location: Copenhagen • Denmark

Re: Migrating RAID volumes

Post by helgrimm »

Nazar78 wrote:Not even detected as an unknown device in the device manager? If yes we could probably hunt for the driver or beg Asustor to release its source which we can easily compile.
Certainly nothing registers in Disk Management. And not as an unknown in Device Manager either. It's a little hard for me to see if there's any entry changed in there as there's a lot of entries and the list refreshes when you plug/unplug. But there are no unknown devices. I tried to look in the logs, but that just confuses me as it takes more experience than I have to find the way to the right list.
Nazar78 wrote:What about the Asus Mini PC?
There's no mounting or messaging going on when connecting the AS6004U.

EDIT: Added this photo of the output ->

I also ran a fdisk -l in terminal that didn't show any of the disks in the AS6004U.
ubuntu_resized.jpg
ubuntu_resized.jpg (98.43 KiB) Viewed 3401 times
Nazar78 wrote:What does lsusb (SSH from the NAS) says about the connected AS6004U?
  • Bus 002 Device 002: ID 0480:a202 Toshiba America Inc:Canvio Basics HDD
    Bus 002 Device 005: ID 174c:55aa ASMedia Technology Inc.:Name: ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge, ASM1153E SATA 6Gb/s bridge
    Bus 001 Device 001: ID 1d6b:0002 Linux Foundation:2.0 root hub
    Bus 002 Device 007: ID 174c:55aa ASMedia Technology Inc.:Name: ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge, ASM1153E SATA 6Gb/s bridge
    Bus 002 Device 003: ID 174c:3074 ASMedia Technology Inc.:ASM1074 SuperSpeed hub
    Bus 002 Device 004: ID 174c:55aa ASMedia Technology Inc.:Name: ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge, ASM1153E SATA 6Gb/s bridge
    Bus 002 Device 001: ID 1d6b:0003 Linux Foundation:3.0 root hub
    Bus 002 Device 006: ID 174c:55aa ASMedia Technology Inc.:Name: ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge, ASM1153E SATA 6Gb/s bridge
    Bus 001 Device 002: ID 174c:2074 ASMedia Technology Inc.:ASM1074 High-Speed hub
Nazar78 wrote:I'm sorry about this but unless someone familiar with the AS6004U who could figure this out, only Asustor Support will have the answer.
No problem. I'm going for the external enclosure I linked, but it could be fun to see whether this box can be played with somehow - to find out whether it can be used as an actual external enclosure without depending on an Asustor NAS.
Last edited by helgrimm on Mon May 16, 2022 6:02 pm, edited 1 time in total.
AS5304T + AS6004U + 4 x SEAGATE EXOS 16TB + 2 x SEAGATE IRONWOLF 8TB + 2 x WD 4TB
helgrimm
Posts: 36
Joined: Sun May 08, 2022 5:20 pm
Location: Copenhagen • Denmark

Re: Migrating RAID volumes

Post by helgrimm »

Nazar78 wrote:Do keep the internal board for future mini projects you might have i.e. home automation.
I have 2 and I stored them carefully thinking I might put a disk back in one or both, using it as a NAS again. How would one go about using them in other ways?
AS5304T + AS6004U + 4 x SEAGATE EXOS 16TB + 2 x SEAGATE IRONWOLF 8TB + 2 x WD 4TB
User avatar
Nazar78
Posts: 2003
Joined: Wed Jul 17, 2019 10:21 pm
Location: Singapore
Contact:

Re: Migrating RAID volumes

Post by Nazar78 »

For the AS6004U perhaps you could compare the lsusb outputs between when it's connected and not as it's hard to tell which is the device unless you lookup all the VEN_DEV_IDs.

For the spare WD boards you have, you can turn them into another low powered server and/or NAS without the WD's functions. Just use a small SSD with the WD's 4GB partitions but use the debian non-64k page builds so you don't have to cross compile packages and just install them from the standard repo. It'll be just like a native debian server. Connect a USB enclosure or via a hub with external power if you wish to expand its storage further or even with RAID.
AS5304T - 16GB DDR4 - ADM-OS modded on 2GB RAM
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps

When posting, consider checking the box "Notify me when a reply is posted" to get faster response
helgrimm
Posts: 36
Joined: Sun May 08, 2022 5:20 pm
Location: Copenhagen • Denmark

Re: Migrating RAID volumes

Post by helgrimm »

Nazar78 wrote:For the AS6004U perhaps you could compare the lsusb outputs between when it's connected and not as it's hard to tell which is the device unless you lookup all the VEN_DEV_IDs.
Good idea. I'll get around to it :)
Nazar78 wrote: Just use a small SSD with the WD's 4GB partitions but use the debian non-64k page builds so you don't have to cross compile packages and just install them from the standard repo. It'll be just like a native debian server. Connect a USB enclosure or via a hub with external power if you wish to expand its storage further or even with RAID.
I have no idea what you just said, but I appreciate your effort to teach us mere mortals :) :)
AS5304T + AS6004U + 4 x SEAGATE EXOS 16TB + 2 x SEAGATE IRONWOLF 8TB + 2 x WD 4TB
Post Reply

Return to “NAS Utilities”