Format a drive - what allocation unit size

[saltwater]

I'm setting up another Blue Iris machine. I've searched and searched and I cannot locate the information regarding what the allocation unit size should be when formatting a harddrive. The drive to be formatted is an 18TB WD Purple drive. 1024 kilobytes comes to mind but I'd like confirmation please. The default format setting is 8192 bytes. I have the option of going as high as 2048 kilobytes. If it makes any difference I'm setting things up on a Windows 11 machine.

 

[Alaska Country]

Storage configuration

I have a new security system with a 4 terabyte HHD and a 256 gigabyte M.2 SSD. What would be the best storage configuration for this setup? Can I keep the OS and BI together on the M.2 and use the HDD for both new and stored footage or CAN and should I keep new footage on the M.2? Any help...

ipcamtalk.com

 

[saltwater]

Thank you.

 

[spammenotinoz]

4096 bytes will be the default for NTFS and suitable for most people, including CCTV.
When you use H\W raid and large databases it gets a bit more complication, but won't go into that now.

The general rule of large sizes for Media files, doesn't really apply to CCTV.
The key thinking is the larger the less waste, and that's great for your static movie collection, but with CCTV there will be a lot of seeks\read\writes, and when you set it high say 16k, if your disk needs 4k, it actually needs to read 16k that's 4x the work required. If you actually need 8k of data, that data can be spread over different sectors, you get the idea of how this can kill a physical disk quickly.

There are always exceptions, where people can tune to their requirements, but that is for technical people with too much time....

 

[Flintstone61]

You can use the defaults. it works fine.

 

[IAmATeaf]

I set mine to 64k, the thinking being, right or wrong, as that is the size that would be used for SQL server database storage so if it’s good enough …

 

[IAmATeaf]

You can use the defaults. it works fine.

Whatever size you use will work fine but the reason/thinking behind changing from the default is efficiency.

 

[wittaj]

Basically, the allocation unit size is the block size on your hard drive when it formats NTFS.

If you have lots of small files, then it's a good idea to keep the allocation size small so your hard drive space won't be wasted.

So for example if you only have 1k files but have an allocation of 1024, then only 1K of the 1024 is being used and 1023K is not being used

If you have lots of large files like we see for video files, keeping it higher will increase the system performance by having fewer blocks to seek.

As an example, I did a file properties on a random folder on my computer. In this file folder, the actual data is 7.93GB, but it is using 8GB on the drive due to the allocation size. In other words, I have 0.07GB lost that I cannot access to due the allocation sizes. Ideally you want these as close as possible so that you don't have wasted space that you cannot access.



So if your allocation size is too large for the types of files you typically save, then you have a lot of wasted storage space that you cannot use.

 

[Flintstone61]

The noise inside of my 7200 RPM WD Purp got louder when I chose different file allocation unit sizes. on Defaults it's quieter.

 

[spammenotinoz]

Basically, the allocation unit size is the block size on your hard drive when it formats NTFS.

If you have lots of small files, then it's a good idea to keep the allocation size small so your hard drive space won't be wasted.

So for example if you only have 1k files but have an allocation of 1024, then only 1K of the 1024 is being used and 1023K is not being used

If you have lots of large files like we see for video files, keeping it higher will increase the system performance by having fewer blocks to seek.

As an example, I did a file properties on a random folder on my computer. In this file folder, the actual data is 7.93GB, but it is using 8GB on the drive due to the allocation size. In other words, I have 0.07GB lost that I cannot access to due the allocation sizes. Ideally you want these as close as possible so that you don't have wasted space that you cannot access.

View attachment 157958

So if your allocation size is too large for the types of files you typically save, then you have a lot of wasted storage space that you cannot use.

Partially correct, larger allocation space the the less space you waste but it will decrease performance in this use case. For databases you use larger sizes to “read-ahead” but we are mainly writing and the drive has plenty of cache. Any larger than default and you are just over working the drive.

 

[SouthernYankee]

If you are using a complete disk for large video file storage (BVR) continuous recording, I recommend formatting the disk, with a windows cluster size of 1024K (1 Megabyte). This is a increase from the 4K default. This will reduce the physical number of disk writes, decrease the disk fragmentation, speed up access. A cluster is contiguously stored on the drive in windows NTFS.

Remember the drive is writing multiple files one per camera assigned to the drive. Video disks get highly fragmented .
The video file for continuous recording should be set to 3GB or 1Hour. Files over 4GB are very difficult to convert or transfer.

 

[CJ133]

If you are using a complete disk for large video file storage (BVR) continuous recording, I recommend formatting the disk, with a windows cluster size of 1024K (1 Megabyte). This is a increase from the 4K default. This will reduce the physical number of disk writes, decrease the disk fragmentation, speed up access. A cluster is contiguously stored on the drive in windows NTFS.

Remember the drive is writing multiple files one per camera assigned to the drive. Video disks get highly fragmented .
The video file for continuous recording should be set to 3GB or 1Hour. Files over 4GB are very difficult to convert or transfer.

Responding to an older thread, but I have an additional question that I didn't see here.
In an older post you mentioned formatting drives to 90% of their capacity. What is the reason for this and should it still be done on a drive like the OP's 18TB or is it generally ok to let windows format the maximum size?

 

[wittaj]

Responding to an older thread, but I have an additional question that I didn't see here.
In an older post you mentioned formatting drives to 90% of their capacity. What is the reason for this and should it still be done on a drive like the OP's 18TB?

You don't format to 90% of the capacity (I don't even think that is possible LOL), you only set up to use 90% of the capacity of the drive.

The reason for this as when the drive gets full, Windows still needs some space to do its thing, so if you allocated the ENTIRE drive capacity to video, it will stumble trying to write and delete at the same time. You want it to have the ability to have space to temp write to while it is in the process of deleting/writing over.

 

[CJ133]

You don't format to 90% of the capacity (I don't even think that is possible LOL), you only set up to use 90% of the capacity of the drive.

The reason for this as when the drive gets full, Windows still needs some space to do its thing, so if you allocated the ENTIRE drive capacity to video, it will stumble trying to write and delete at the same time. You want it to have the ability to have space to temp write to while it is in the process of deleting/writing over.

Off the top of my head, you can create a partition any size you want. So I took that as create a partition that's only 90% of the drive and leave 10% unused.

But I think I'm understanding now. Use up the entire drive for the partition but only let BI use 90% of it? Does Windows need 2TB of drive in the case of the OPs 18TB?