Strange cluster allocation, both internal ext4 and external ntfs drives

This is where you can find the latest ADM official release. Feel free to discuss any questions regarding it here.
Post Reply
Elusivity
Posts: 38
youtube meble na wymiar Warszawa
Joined: Thu Jul 25, 2013 8:28 pm

Strange cluster allocation, both internal ext4 and external ntfs drives

Post by Elusivity »

My main query / issue is regarding strange (large) sector sizes, I have a number of external ntfs format and raid-0 ext4 format on the nas itself, with small sub 4kb files it will utilise the expected 4kb sector allocation, however once a file is over 4kb it automatically jumps to 1mb of sectors allocated for the file.

Obviously this is quite wasteful in a folder full of small files or even a folder with both large and small (over 4kb) files in it.

Example, I have a folder with video files and small metadata (sub 1mb) files in it, to a total of 1294 files, 400 of those files are over 1mb, the rest are below, and total space of files are 148gb, however disk used is 230gb.... now I am aware that this can happen when something is formatted with a very large cluster size, however I've confirmed that both the ext4 and ntfs formatted drives are utilising 4kb sector sizes, therefore a 5kb file should only be using 8kb max, and any file should be using space up to the next divisible 4kb chunk, however it is utilising the next 1mb chunk.

I'd love to know why this is and whether it can be fixed? I assume that this is a quirk of the linux OS that ADM runs on as files written to one of the connected NTFS drives under OSX with Paragon NTFS driver, or windows 7, 8.1 and 10 all write to the drives utilising the correct 4kb sector sizes rather than this >4kb scale up to 1mb allocation.

I'm dunning a ADM 2.4.2 and after I backup my raid1 volume on the NAS I'll upgrade to 2.5 however I doubt that this will fix the issue as it's systematic over 13 different drives (11 jbob, 1 raid-0 setup) attached to my AS602T, and I'd need to rewrite all of the offending files in order to correct it anyway.

It's almost like the opposite of thin provisioning, instead of allocating a large file virtually and minimally on the storage pool and allowing it to grow as required it's allocating a large amount of physical disc even though the files will never grow to that size....

I've also posted this is to asustor tech support but maybe someone else has looked into this previously? I've searched and found one instance where another as60x user had something similar but rather than allocating 1mb it was allocating 256mb, which would be far worse, bu the thread ends without much further discussion on this specfic issue. Note that all NTFS drives have been formatted in windows both as GTP and MBR as appropriate (over \ under 2tb), and in osx, ubuntu and windows these drives allocate files based on typical 4kb cluster sizes, it's just when files are written to the drives when connected to the nas that this quirk pops up.
rs277
Posts: 3
Joined: Sat Jan 16, 2016 8:57 am

Re: Strange cluster allocation, both internal ext4 and external ntfs drives

Post by rs277 »

Is the same here on a AS5104T running ADM 2.5.1.RB62 und BIOS 1.21.
Checking the folder properties over the network gives me extremely large size on disk values.
As soon as the file size is larger than 4096 byte the size on the disk jumps to 1 MB.
The next steps after that are 2, 4, 8, 16, 32 and so on. Always rounded up to twice the step size it just exceeded.

BUT, File Explorer and Storage Manager show the values according to the actual file size value, which seems to be correct, because otherwise my NAS drive would be full already.

I found the question in other forums and for different types of NAS. Nobody could really figure out where it comes from. But it seems to be more a reporting issue from the Linux side. Because one of the guys did record the LAN traffic via Wireshark and the packet content also showed the largely exaggerated size.
Post Reply

Return to “[Official] For AS-60X Series”