Quickest way to copy selected folders to connected USB
-
- Posts: 8
- youtube meble na wymiar Warszawa
- Joined: Sat Jul 31, 2021 7:42 pm
Quickest way to copy selected folders to connected USB
HI, I want to copy certain folders to a USB drive connected via the rear USB port. By connecting the USB directly to the NAS, rather than using Windows or FTP, I thought I would achieve very high speeds, but using the built-in file explorer app to copy from NAS and paste to USB folder, I get a miserable 4.24 MB/s transfer rate. Copying to my USB 16TB e-sata disks is going to take weeks at this rate. I looked at backup options but these don't seem to offer copying selected folders. I'm not a UNIX expert but I could probably manage to enter the appropriate commands directly if this would speed things up. Can anyone help please?
- Nazar78
- Posts: 2085
- Joined: Wed Jul 17, 2019 10:21 pm
- Location: Singapore
- Contact:
Re: Quickest way to copy selected folders to connected USB
"USB 16TB e-sata disks" <- which NAS and specifically which enclosure is this? Take a look at the "External Devices" in ADM, does it state USB 3.x? Probably detected as USB 2, hence the miserable transfer rate. The NAS CPU could also influence the transfer speed. On the other hand, IIRC not many USB enclosure supports > 10TB. I had some old eSata/USB enclosure that detects a 4TB as only 2TB. Yes it does allow me to write data onto the HDD but eventually lead to data corruption.
I've not tested many newer enclosures on my Asustor NAS but I'm using a 5 bay USB3.0 (5Gbps), ORICO brand with 10TB HDD support. It can achieve max transfer at about 350MB/s on the SSD depending on the load (which was tweaked to run the ADM OS and apps) and about 250MB/s to a 4x2TB configured as RAID5 or 80MB/s as single HDD.
I've not tested many newer enclosures on my Asustor NAS but I'm using a 5 bay USB3.0 (5Gbps), ORICO brand with 10TB HDD support. It can achieve max transfer at about 350MB/s on the SSD depending on the load (which was tweaked to run the ADM OS and apps) and about 250MB/s to a 4x2TB configured as RAID5 or 80MB/s as single HDD.
AS5304T - 16GB DDR4 - ADM-OS modded on 2GB RAM
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps
When posting, consider checking the box "Notify me when a reply is posted" to get faster response
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps
When posting, consider checking the box "Notify me when a reply is posted" to get faster response
-
- Posts: 8
- Joined: Sat Jul 31, 2021 7:42 pm
Re: Quickest way to copy selected folders to connected USB
Hi Nazar78,
Many thanks for your response. My NAS is ASUSTOR AS6510T-D5CB with 10x 16TB e-sata disks. The USB enclosure is Yottamaster 5bay USB3.0 also with 5 x 16TB e-sata disks.
All 5 USB disks are mounted as USB3.0 so the slow transfer is not down to USB2. Do you know how I can log in as UNIX user and maybe execute CP commands which might run faster? Or else I might try FTP or RSYNC?
Any help greatly appreciated.
Many thanks for your response. My NAS is ASUSTOR AS6510T-D5CB with 10x 16TB e-sata disks. The USB enclosure is Yottamaster 5bay USB3.0 also with 5 x 16TB e-sata disks.
All 5 USB disks are mounted as USB3.0 so the slow transfer is not down to USB2. Do you know how I can log in as UNIX user and maybe execute CP commands which might run faster? Or else I might try FTP or RSYNC?
Any help greatly appreciated.
- Nazar78
- Posts: 2085
- Joined: Wed Jul 17, 2019 10:21 pm
- Location: Singapore
- Contact:
Re: Quickest way to copy selected folders to connected USB
Your NAS and USB enclosure should be able to achieve good transfer speeds. Were you trying to copy large amount of smaller files at once? Larger files transfer faster.
To login to the shell, you can enable SSH from the Services -> Terminal -> SSH. Also check the "Enable SFTP service" which you can use some SFTP client such as WinSCP. Don't enable the EZ-Connect option unless you're aware of the security risk.
Simple copy command would be:
Or with rysnc:
Change the USB31-1 to the ones connected to your NAS, you can find them out easily or with:
You can also use the dd command from my screenshot to perform simple write benchmark. if=source of=target.
For Windows, you can try this NAS performance tester http://www.808.dk/?code-csharp-nas-performance. It's not 100% accurate due to the underlying influence of network speed and SMB but I often use it to quickly test the R/W of my disks via mounted SMB. Getting about 296MB/s read/write on both my connected SSD and RAID5 in the USB enclosure. There's also https://crystalmark.info/en/software/crystaldiskmark/ which you can test using selected folder, network folder of course.
To login to the shell, you can enable SSH from the Services -> Terminal -> SSH. Also check the "Enable SFTP service" which you can use some SFTP client such as WinSCP. Don't enable the EZ-Connect option unless you're aware of the security risk.
Simple copy command would be:
Code: Select all
cp /share/my-source-folder /share/USB31-1/my-target-folder
Code: Select all
rsync -avzh /share/my-source-folder /share/USB31-1/my-target-folder
Code: Select all
ls -lah /share
For Windows, you can try this NAS performance tester http://www.808.dk/?code-csharp-nas-performance. It's not 100% accurate due to the underlying influence of network speed and SMB but I often use it to quickly test the R/W of my disks via mounted SMB. Getting about 296MB/s read/write on both my connected SSD and RAID5 in the USB enclosure. There's also https://crystalmark.info/en/software/crystaldiskmark/ which you can test using selected folder, network folder of course.
AS5304T - 16GB DDR4 - ADM-OS modded on 2GB RAM
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps
When posting, consider checking the box "Notify me when a reply is posted" to get faster response
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps
When posting, consider checking the box "Notify me when a reply is posted" to get faster response
-
- Posts: 8
- Joined: Sat Jul 31, 2021 7:42 pm
Re: Quickest way to copy selected folders to connected USB
Hi again,
Thanks so much for your help. I'm now using rsync via SSH and copies are moving much faster and I feel much more in control! I'll also check out the NAS performance testers as I also have NAS drives from Synology and QNAP so it will be interesting to compare performance. Thanks again - you're a star!
Thanks so much for your help. I'm now using rsync via SSH and copies are moving much faster and I feel much more in control! I'll also check out the NAS performance testers as I also have NAS drives from Synology and QNAP so it will be interesting to compare performance. Thanks again - you're a star!
-
- Posts: 38
- Joined: Tue Nov 15, 2022 9:42 pm
Re: Quickest way to copy selected folders to connected USB
Hi
Could you provide more information on how you accomplish it.
I have set port 873 to my QNAP on my Asus router and trying to sync my QNAP to my AS6704T and keep receiving "unable to connect to rsync server" (reff 510).
Thank You
Mario
Could you provide more information on how you accomplish it.
I have set port 873 to my QNAP on my Asus router and trying to sync my QNAP to my AS6704T and keep receiving "unable to connect to rsync server" (reff 510).
Thank You
Mario
- Nazar78
- Posts: 2085
- Joined: Wed Jul 17, 2019 10:21 pm
- Location: Singapore
- Contact:
Re: Quickest way to copy selected folders to connected USB
Your QNAP needs to have the rsyncd setup properly and enabled, usually in the /etc/rsyncd.conf, setup the path/users something like below then restart the rsyncd daemon:brujo wrote:Hi
Could you provide more information on how you accomplish it.
I have set port 873 to my QNAP on my Asus router and trying to sync my QNAP to my AS6704T and keep receiving "unable to connect to rsync server" (reff 510).
Thank You
Mario
Code: Select all
pid file = /var/run/rsyncd.pid
log file = /var/log/rsyncd.log
use chroot = yes
uid = admin
gid = root
read only = yes
[WDMyPassport-Backup]
path = /mnt/WDMyPassport/Backup
comment = WD My Passport
read only = false
hosts allow = 192.168.1.0/24 fe80::/64 2400:d803:####:####::/64
auth users = admin
secrets file = /etc/rsyncd.secrets
exclude = /etc/init.d
Code: Select all
rsync -avzh /share/my-source-folder rsync://admin@192.168.1.1/WDMyPassport-Backup/my-target-folder
AS5304T - 16GB DDR4 - ADM-OS modded on 2GB RAM
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps
When posting, consider checking the box "Notify me when a reply is posted" to get faster response
Internal:
- 4x10TB Toshiba RAID10 Ext4-Journal=Off
External 5 Bay USB3:
- 4x2TB Seagate modded RAID0 Btrfs-Compression
- 480GB Intel SSD for modded dm-cache (initramfs auto update patch) and Apps
When posting, consider checking the box "Notify me when a reply is posted" to get faster response