Write to and read from free disk space using Windows API

Question

Is it possible to write to free clusters on disk or read data from them using Windows APIs? I found Defrag API: https://learn.microsoft.com/en-gb/windows/desktop/FileIO/defragmenting-files

FSCTL_GET_VOLUME_BITMAP can be used to obtain allocation state of each cluster, FSCTL_MOVE_FILE can be used to move clusters around. But I couldn't find a way of reading data from free clusters or writing data to them.

Update: one of the workarounds which comes to mind is creating a small new file, writing some data to it, then relocating it to desired position and deleting the file (the data will remain in freed cluster). But that still doesn't solve reading problem.

What I'm trying to do is some sort of transparent cache, so user could still use his NTFS partition as usual and still see these clusters as free space, but I could store some data in them. Data safety is not of concern, it can be overwritten by user actions and will just be regenerated / redownloaded later when clusters become free again.

Kinsella

simply open volume or disk device. but with this you can read or write sectors. not free clusters. this can do only filesystem driver, which mount volume – Elwood

This sounds like an XY Problem. You indicate you want to access empty disk clusters, but there's no apparent value in doing this. Unless you are implementing a disk defrag tool... What are you really trying to do? – Pangolin

@Pangolin -Unless you are implementing a disk defrag tool - in defrag tool this not need. we use FSCTL_MOVE_FILE IOCTL here. really what space is empty know only filesystem, when partition is mounted. and without lock on volume level - this information can change at any time. so only FS really can do this. any another code not need do this at all – Elwood

@rbmm I'm thinking of storing data in 'free' space which doesn't interfere with normal PC usage, sort of a cache, and can be transparently overwritten by user and then redownloaded later when clusters become free again. So yes, I need to do that on filesystem level and not on raw sectors, because I think it's not possible to implement writing reliably while bypassing Windows FS implementation. So my question is if some kind of API exists for this at FS level? I only found these defrag IOCTL-s but nothing for reading/writing to free space clusters. Or is accessing volume directly the only way? – Kinsella

@Kinsella what you want is absolute impossible and senseless. what save some data on disk - create file. – Elwood

Answer 1

There is no easy solution in this way.

First of all, you should create own partition of the drive. It prevents from an accidental access to your data from OS or any process. Then call CreateFileA() with name of the partition. You will get raw access to the data. Please bear in mind that the function will fail for any partition accessed by OS.

You can perform the same trick with a physical drive too.

The docs

Stopover

What I'm trying to do is using free space to store data transparently for the user. I.e. that he still could use his NTFS filesystem as usual and see free space still as free space. If he would overwrite some of the data stored in free space clusters - that's fine, as it can be easily regenerated/redownloaded later. – Kinsella

No, that is not possible. File system has an exclusive access to own partition. FS can relocate data as it wants without any notification. You cannot run two file systems in one single partition. – Stopover

@Kinsella what is the use case you wish to address? hiding of data? but then downloading data again and again seems not very desirable. I would suggest to create a large container file and do whatever you want inside this file (maybe by using some file system structure). So at least the parent volumes data/system integrity is preserved. – Tainataint

Answer 2

One way could be to open the volume directly via using CreateFile with the volumes UNC path as filename arguement (e.g.: \\.\C:). You now can directly read and write to the volume.

So you maybe can achieve your desired goal with:

• get the cluster size in bytes with GetDiskFreeSpace
• get the map of free clusters with DeviceIoControl and FSCTL_GET_VOLUME_BITMAP
• open the volume with CreateFile with its UNC path \\.\F:
  (take a careful look into the documentation, especially the Remarks sections part about opening drives and volumes)
• seek to the the offset of a free cluster (clusterindex * clusterByteSize) by using SetFilePointer
• write/read your data with WriteFile/ReadFile on the handle, retreived by above CreateFile
  (Also note that read/write access has to be sector aligned, otherwise the ReadFile/WriteFile calls fail)

Please note:

this is only meant as a starting point for your own research. This is not a bullet proof cooking receipt.
Backup your data before messing with the file system!!!

Also keep in mind that the free cluster bitmap will be outdated as soon as you get it (especially if using the system volume). So I would strongly advise against use of such techniques in production or customer environments.

Tainataint