ADM backup utility discussion plus clearing broken file system from root

[ChurchOfSlack]

I am using a 6T USB drive for doing ASUSTOR backups. I have about 3.5T on the NAS so this is a comfortable backup drive. It came formatted for NTFS which is convenient enough that I left it. The data I am backing up is about 1.6T flac files, about 1T of pictures (jpg from phone and raw Nikon files). The rest is miscellaneous files - backups of laptops and stuff.

I am using the backup utility that comes with ADM and it isn't very reliable. It has two main problems. On one hand, it is very easy to make one big backup job for everything I need to protect but that one job takes so long it often hangs and doesn't finish. The fix is to break the job into smaller jobs that will finish in about 4 hours and schedule them for different days of the week. Then they will be less likely to time out.

The other problem is that when a job hangs it will often screw up the NTFS file system hanging the whole system. But the worst problem is that all future backups will also hang on the broken file system until the volume is fixed.

At first, since this is a backup, I would reformat the USB drive and restart the backup. That didn't work as well with multiple jobs and over 1T of files (much less the 3.5T I have now). So I figured out how to fsck (clean and repair) the volume online. That's faster and easier than reformatting with web tools and restarting all the backups.

Here are the steps for running fsck on anything but your root volume with specific command syntax for NTFS

1. Find the device
'df -k': find the device name from the list and make a note of it. Mine is /dev/sdh2. You must figure out yours.
2. Unmount the disk. You can use the web app to do safely or use 'umount <device>' from command line as root.
3. Repair the file system (most USB disks will be NTFS but if not, figure out the file system type and substitute)
as root: 'fsck -ft ntfs <device>'

==> don't do this on a mounted volume or on a Mac (HFS+) volume. If the volume is not NTFS, you should figure
out the file system type. To fix things, you have to use -f to force the repair so be careful when giving
the command. Very few guide rails here.

[ChurchOfSlack]

Hey! I wasn't finished!

the -t flag is where you specify the file system type. The example above is for ntfs. I've had good luck with it.

4. remount the USB drive. You can figure out how to mount it using the 'mount' command as root but I find it neater, cleaner and safer to unplug the unmounted USB drive and replug it. That will cause it to automount correctly.

Further note: 'fsck' is a unix utility that will clear out all the 'badness' (technical term) in your file system by unlinking messed up (another technical term) file references. So your data blocks might be there but you can't find them anymore using 'ls' or other file system tools. In practical terms, the disk is now working properly but the affected files are gone.

For a backup, this isn't a problem because you know which job did not complete correctly and the missing backup files can be recreated by re-running the last backup job. If you have data files on your USB drive and run into this problem. The fix will probably cause lost files. I would recommend keeping data on RAID and backing up to single spindle USB drives for cost savings.

Hope this helps those who run into USB file system problems.