It is currently Sat Dec 03, 2022 7:24 am
All times are UTC + 8 hours

Quickest way to copy selected folders to connected USB

Backup and data protection discussion at its finest.

Quickest way to copy selected folders to connected USB

Postby dlight69 » Mon Sep 19, 2022 9:25 pm

HI, I want to copy certain folders to a USB drive connected via the rear USB port. By connecting the USB directly to the NAS, rather than using Windows or FTP, I thought I would achieve very high speeds, but using the built-in file explorer app to copy from NAS and paste to USB folder, I get a miserable 4.24 MB/s transfer rate. Copying to my USB 16TB e-sata disks is going to take weeks at this rate. I looked at backup options but these don't seem to offer copying selected folders. I'm not a UNIX expert but I could probably manage to enter the appropriate commands directly if this would speed things up. Can anyone help please?
dlight69
 
Posts: 8
Joined: Sat Jul 31, 2021 7:42 pm

Re: Quickest way to copy selected folders to connected USB

Postby Nazar78 » Tue Sep 20, 2022 9:47 am

"USB 16TB e-sata disks" <- which NAS and specifically which enclosure is this? Take a look at the "External Devices" in ADM, does it state USB 3.x? Probably detected as USB 2, hence the miserable transfer rate. The NAS CPU could also influence the transfer speed. On the other hand, IIRC not many USB enclosure supports > 10TB. I had some old eSata/USB enclosure that detects a 4TB as only 2TB. Yes it does allow me to write data onto the HDD but eventually lead to data corruption.

I've not tested many newer enclosures on my Asustor NAS but I'm using a 5 bay USB3.0 (5Gbps), ORICO brand with 10TB HDD support. It can achieve max transfer at about 350MB/s on the SSD depending on the load (which was tweaked to run the ADM OS and apps) and about 250MB/s to a 4x2TB configured as RAID5 or 80MB/s as single HDD.

Untitled.png
Untitled.png (37.67 KiB) Viewed 534 times
AS5304T - 16GB DDR4 - [Internal: 4x10TB Toshiba RAID10, External 5 Bay USB3: 4x2TB Seagate RAID5 + 480GB Intel SSD for ADM-OS/Apps]
When posting, consider checking the box "Notify me when a reply is posted" to get faster response.
User avatar
Nazar78
 
Posts: 938
Joined: Wed Jul 17, 2019 10:21 pm
Location: Singapore

Re: Quickest way to copy selected folders to connected USB

Postby dlight69 » Thu Sep 22, 2022 8:53 pm

Hi Nazar78,
Many thanks for your response. My NAS is ASUSTOR AS6510T-D5CB with 10x 16TB e-sata disks. The USB enclosure is Yottamaster 5bay USB3.0 also with 5 x 16TB e-sata disks.
All 5 USB disks are mounted as USB3.0 so the slow transfer is not down to USB2. Do you know how I can log in as UNIX user and maybe execute CP commands which might run faster? Or else I might try FTP or RSYNC?
Any help greatly appreciated.
dlight69
 
Posts: 8
Joined: Sat Jul 31, 2021 7:42 pm

Re: Quickest way to copy selected folders to connected USB

Postby Nazar78 » Fri Sep 23, 2022 12:59 pm

Your NAS and USB enclosure should be able to achieve good transfer speeds. Were you trying to copy large amount of smaller files at once? Larger files transfer faster.

To login to the shell, you can enable SSH from the Services -> Terminal -> SSH. Also check the "Enable SFTP service" which you can use some SFTP client such as WinSCP. Don't enable the EZ-Connect option unless you're aware of the security risk.

Simple copy command would be:
Code: Select all
cp /share/my-source-folder /share/USB31-1/my-target-folder

Or with rysnc:
Code: Select all
rsync -avzh /share/my-source-folder /share/USB31-1/my-target-folder

Change the USB31-1 to the ones connected to your NAS, you can find them out easily or with:
Code: Select all
ls -lah /share

You can also use the dd command from my screenshot to perform simple write benchmark. if=source of=target.

For Windows, you can try this NAS performance tester http://www.808.dk/?code-csharp-nas-performance. It's not 100% accurate due to the underlying influence of network speed and SMB but I often use it to quickly test the R/W of my disks via mounted SMB. Getting about 296MB/s read/write on both my connected SSD and RAID5 in the USB enclosure. There's also https://crystalmark.info/en/software/crystaldiskmark/ which you can test using selected folder, network folder of course.
AS5304T - 16GB DDR4 - [Internal: 4x10TB Toshiba RAID10, External 5 Bay USB3: 4x2TB Seagate RAID5 + 480GB Intel SSD for ADM-OS/Apps]
When posting, consider checking the box "Notify me when a reply is posted" to get faster response.
User avatar
Nazar78
 
Posts: 938
Joined: Wed Jul 17, 2019 10:21 pm
Location: Singapore

Re: Quickest way to copy selected folders to connected USB

Postby dlight69 » Sat Sep 24, 2022 4:59 pm

Hi again,
Thanks so much for your help. I'm now using rsync via SSH and copies are moving much faster and I feel much more in control! I'll also check out the NAS performance testers as I also have NAS drives from Synology and QNAP so it will be interesting to compare performance. Thanks again - you're a star!
dlight69
 
Posts: 8
Joined: Sat Jul 31, 2021 7:42 pm

Re: Quickest way to copy selected folders to connected USB

Postby brujo » Fri Nov 25, 2022 2:08 am

Hi
Could you provide more information on how you accomplish it.
I have set port 873 to my QNAP on my Asus router and trying to sync my QNAP to my AS6704T and keep receiving "unable to connect to rsync server" (reff 510).
Thank You
Mario
brujo
 
Posts: 19
Joined: Tue Nov 15, 2022 9:42 pm

Re: Quickest way to copy selected folders to connected USB

Postby Nazar78 » Fri Nov 25, 2022 2:27 am

brujo wrote:Hi
Could you provide more information on how you accomplish it.
I have set port 873 to my QNAP on my Asus router and trying to sync my QNAP to my AS6704T and keep receiving "unable to connect to rsync server" (reff 510).
Thank You
Mario

Your QNAP needs to have the rsyncd setup properly and enabled, usually in the /etc/rsyncd.conf, setup the path/users something like below then restart the rsyncd daemon:
Code: Select all
pid file = /var/run/rsyncd.pid
log file = /var/log/rsyncd.log
use chroot = yes
uid = admin
gid = root
read only = yes

[WDMyPassport-Backup]
path = /mnt/WDMyPassport/Backup
comment = WD My Passport
read only = false
hosts allow = 192.168.1.0/24 fe80::/64 2400:d803:####:####::/64
auth users = admin
secrets file = /etc/rsyncd.secrets
exclude = /etc/init.d

Then you would do the transfer from your Asustor to the QNAP with (or using the Backup & Restore Remote Sync, note the -z compress switch might not be available on some systems):
Code: Select all
rsync -avzh /share/my-source-folder rsync://admin@192.168.1.1/WDMyPassport-Backup/my-target-folder

You can also use rsync over SSH but it will be slower if you're just doing local transfers which mostly you don't need encryptions.
AS5304T - 16GB DDR4 - [Internal: 4x10TB Toshiba RAID10, External 5 Bay USB3: 4x2TB Seagate RAID5 + 480GB Intel SSD for ADM-OS/Apps]
When posting, consider checking the box "Notify me when a reply is posted" to get faster response.
User avatar
Nazar78
 
Posts: 938
Joined: Wed Jul 17, 2019 10:21 pm
Location: Singapore

Return to Backup and Data Protection

  • You cannot post new topics in this forum
    You cannot reply to topics in this forum
    You cannot edit your posts in this forum
    You cannot delete your posts in this forum
    You cannot post attachments in this forum
  • Who is online

    Users browsing this forum: No registered users and 1 guest