Quickest way to copy selected folders to connected USB

Backup and data protection discussion at its finest.

Moderator: Lillian.W@AST

Post Reply
dlight69
Posts: 8
youtube meble na wymiar Warszawa
Joined: Sat Jul 31, 2021 7:42 pm

Quickest way to copy selected folders to connected USB

Post by dlight69 »

HI, I want to copy certain folders to a USB drive connected via the rear USB port. By connecting the USB directly to the NAS, rather than using Windows or FTP, I thought I would achieve very high speeds, but using the built-in file explorer app to copy from NAS and paste to USB folder, I get a miserable 4.24 MB/s transfer rate. Copying to my USB 16TB e-sata disks is going to take weeks at this rate. I looked at backup options but these don't seem to offer copying selected folders. I'm not a UNIX expert but I could probably manage to enter the appropriate commands directly if this would speed things up. Can anyone help please?
User avatar
Nazar78
Posts: 2002
Joined: Wed Jul 17, 2019 10:21 pm
Location: Singapore
Contact:

Re: Quickest way to copy selected folders to connected USB

Post by Nazar78 »

"USB 16TB e-sata disks" <- which NAS and specifically which enclosure is this? Take a look at the "External Devices" in ADM, does it state USB 3.x? Probably detected as USB 2, hence the miserable transfer rate. The NAS CPU could also influence the transfer speed. On the other hand, IIRC not many USB enclosure supports > 10TB. I had some old eSata/USB enclosure that detects a 4TB as only 2TB. Yes it does allow me to write data onto the HDD but eventually lead to data corruption.

I've not tested many newer enclosures on my Asustor NAS but I'm using a 5 bay USB3.0 (5Gbps), ORICO brand with 10TB HDD support. It can achieve max transfer at about 350MB/s on the SSD depending on the load (which was tweaked to run the ADM OS and apps) and about 250MB/s to a 4x2TB configured as RAID5 or 80MB/s as single HDD.
Untitled.png
Untitled.png (37.67 KiB) Viewed 2164 times
AS5304T - 16GB DDR4 - ADM-OS modded on 2GB RAM
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps

When posting, consider checking the box "Notify me when a reply is posted" to get faster response
dlight69
Posts: 8
Joined: Sat Jul 31, 2021 7:42 pm

Re: Quickest way to copy selected folders to connected USB

Post by dlight69 »

Hi Nazar78,
Many thanks for your response. My NAS is ASUSTOR AS6510T-D5CB with 10x 16TB e-sata disks. The USB enclosure is Yottamaster 5bay USB3.0 also with 5 x 16TB e-sata disks.
All 5 USB disks are mounted as USB3.0 so the slow transfer is not down to USB2. Do you know how I can log in as UNIX user and maybe execute CP commands which might run faster? Or else I might try FTP or RSYNC?
Any help greatly appreciated.
User avatar
Nazar78
Posts: 2002
Joined: Wed Jul 17, 2019 10:21 pm
Location: Singapore
Contact:

Re: Quickest way to copy selected folders to connected USB

Post by Nazar78 »

Your NAS and USB enclosure should be able to achieve good transfer speeds. Were you trying to copy large amount of smaller files at once? Larger files transfer faster.

To login to the shell, you can enable SSH from the Services -> Terminal -> SSH. Also check the "Enable SFTP service" which you can use some SFTP client such as WinSCP. Don't enable the EZ-Connect option unless you're aware of the security risk.

Simple copy command would be:

Code: Select all

cp /share/my-source-folder /share/USB31-1/my-target-folder
Or with rysnc:

Code: Select all

rsync -avzh /share/my-source-folder /share/USB31-1/my-target-folder
Change the USB31-1 to the ones connected to your NAS, you can find them out easily or with:

Code: Select all

ls -lah /share
You can also use the dd command from my screenshot to perform simple write benchmark. if=source of=target.

For Windows, you can try this NAS performance tester http://www.808.dk/?code-csharp-nas-performance. It's not 100% accurate due to the underlying influence of network speed and SMB but I often use it to quickly test the R/W of my disks via mounted SMB. Getting about 296MB/s read/write on both my connected SSD and RAID5 in the USB enclosure. There's also https://crystalmark.info/en/software/crystaldiskmark/ which you can test using selected folder, network folder of course.
AS5304T - 16GB DDR4 - ADM-OS modded on 2GB RAM
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps

When posting, consider checking the box "Notify me when a reply is posted" to get faster response
dlight69
Posts: 8
Joined: Sat Jul 31, 2021 7:42 pm

Re: Quickest way to copy selected folders to connected USB

Post by dlight69 »

Hi again,
Thanks so much for your help. I'm now using rsync via SSH and copies are moving much faster and I feel much more in control! I'll also check out the NAS performance testers as I also have NAS drives from Synology and QNAP so it will be interesting to compare performance. Thanks again - you're a star!
brujo
Posts: 38
Joined: Tue Nov 15, 2022 9:42 pm

Re: Quickest way to copy selected folders to connected USB

Post by brujo »

Hi
Could you provide more information on how you accomplish it.
I have set port 873 to my QNAP on my Asus router and trying to sync my QNAP to my AS6704T and keep receiving "unable to connect to rsync server" (reff 510).
Thank You
Mario
User avatar
Nazar78
Posts: 2002
Joined: Wed Jul 17, 2019 10:21 pm
Location: Singapore
Contact:

Re: Quickest way to copy selected folders to connected USB

Post by Nazar78 »

brujo wrote:Hi
Could you provide more information on how you accomplish it.
I have set port 873 to my QNAP on my Asus router and trying to sync my QNAP to my AS6704T and keep receiving "unable to connect to rsync server" (reff 510).
Thank You
Mario
Your QNAP needs to have the rsyncd setup properly and enabled, usually in the /etc/rsyncd.conf, setup the path/users something like below then restart the rsyncd daemon:

Code: Select all

pid file = /var/run/rsyncd.pid
log file = /var/log/rsyncd.log
use chroot = yes
uid = admin
gid = root
read only = yes

[WDMyPassport-Backup]
path = /mnt/WDMyPassport/Backup
comment = WD My Passport
read only = false
hosts allow = 192.168.1.0/24 fe80::/64 2400:d803:####:####::/64
auth users = admin
secrets file = /etc/rsyncd.secrets
exclude = /etc/init.d
Then you would do the transfer from your Asustor to the QNAP with (or using the Backup & Restore Remote Sync, note the -z compress switch might not be available on some systems):

Code: Select all

rsync -avzh /share/my-source-folder rsync://admin@192.168.1.1/WDMyPassport-Backup/my-target-folder
You can also use rsync over SSH but it will be slower if you're just doing local transfers which mostly you don't need encryptions.
AS5304T - 16GB DDR4 - ADM-OS modded on 2GB RAM
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps

When posting, consider checking the box "Notify me when a reply is posted" to get faster response
Post Reply

Return to “Backup and Data Protection”