How to create a sparse file on NTFS?

Question

I'm testing a sparse file. But my test code doesn't work well.

HANDLE h = CreateFileW(L"D:\\sparse.test",
        GENERIC_READ|GENERIC_WRITE,
        FILE_SHARE_READ|FILE_SHARE_WRITE|FILE_SHARE_DELETE,
        0,
        CREATE_ALWAYS,
        FILE_ATTRIBUTE_ARCHIVE|FILE_ATTRIBUTE_SPARSE_FILE,
        0);

DWORD d = GetFileAttributes(L"D:\\sparse.test");
// The function returns 32(FILE_ATTRIBUTE_ARCHIVE).
// Where is FILE_ATTRIBUTE_SPARSE_FILE flag?
// How do I make a sparse file.

DWORD written;
WriteFile(h, "aaa", 3, &written, 0);
SetFilePointer(h, 2*1024*1024*1023, 0, FILE_BEGIN);
SetEndOfFile(h);
WriteFile(h, "bbb", 3, &written, 0);

Detestation

Answer 1

You have to create a normal file, then use DeviceIoControl with FSCTL_SET_SPARSE to make it a sparse file.

Subtemperate

@Benjamin: I don't know of another way to do this job. – Subtemperate

It should be mentioned that all data written to the file is always physically allocated, even if it's only 0-bytes. So depending on the use case one might need to additionally use FSCTL_SET_ZERO_DATA: When you perform a write operation (with a function or operation other than FSCTL_SET_ZERO_DATA) whose data consists of nothing but zeros, zeros will be written to the disk for the entire length of the write. To zero out a range of the file and maintain sparseness, use FSCTL_SET_ZERO_DATA. learn.microsoft.com/de-de/windows/desktop/FileIO/… – Demi

Microsoft's own doc say otherwise at other places: When a write operation is attempted where a large amount of the data in the buffer is zeros, the zeros are not written to the file. learn.microsoft.com/en-us/windows/desktop/fileio/sparse-files Maybe the difference is the wording, ALL 0 vs. only some and the behaviour is different in both cases? Wouldn't make much sense to me, though. – Demi

Answer 2

#include <windows.h>
#include <string>
#include <iostream>

HANDLE CreateSparseFile(LPCTSTR lpSparseFileName)
{
    // Use CreateFile as you would normally - Create file with whatever flags 
    //and File Share attributes that works for you
    DWORD dwTemp;

    HANDLE hSparseFile = CreateFile(lpSparseFileName,
        GENERIC_READ | GENERIC_WRITE,
        FILE_SHARE_READ | FILE_SHARE_WRITE,
        NULL,
        CREATE_ALWAYS,
        FILE_ATTRIBUTE_NORMAL,
        NULL);

    if (hSparseFile == INVALID_HANDLE_VALUE)
        return hSparseFile;

    DeviceIoControl(hSparseFile,
        FSCTL_SET_SPARSE,
        NULL,
        0,
        NULL,
        0,
        &dwTemp,
        NULL);
    return hSparseFile;
}

DWORD SetSparseRange(HANDLE hSparseFile, LONGLONG start, LONGLONG size)
{
    // Specify the starting and the ending address (not the size) of the 
    // sparse zero block
    FILE_ZERO_DATA_INFORMATION fzdi;
    fzdi.FileOffset.QuadPart = start;
    fzdi.BeyondFinalZero.QuadPart = start + size;
    // Mark the range as sparse zero block
    DWORD dwTemp;
    SetLastError(0);
    BOOL bStatus = DeviceIoControl(hSparseFile,
        FSCTL_SET_ZERO_DATA,
        &fzdi,
        sizeof(fzdi),
        NULL,
        0,
        &dwTemp,
        NULL);
    if (bStatus) return 0; //Sucess
    else {
        DWORD e = GetLastError();
        return(e); //return the error value
    }
}
int _tmain(int argc, _TCHAR* argv[])
{
    if (argc < 3) {
        std::cerr << "USAGE: SparseFile filename size" << std::endl;
        return 1;
    }

    try {
        ULONGLONG size = std::stoull(argv[2]);
        HANDLE h = CreateSparseFile(argv[1]);
        if (h == INVALID_HANDLE_VALUE) {
            std::cerr << "Unable to create file" << std::endl;
            return 1;
        }
        if (SetSparseRange(h, 0, size) != 0) {
            std::cerr << "Unable to set sparse range" << std::endl;
            return 1;
        }
        LARGE_INTEGER seek;
        seek.QuadPart = size;
        if (!SetFilePointerEx(h, seek, 0, 0)) {
            std::cerr << "Unable to seek to desired offset" << std::endl;
            return 1;
        }
        SetEndOfFile(h);
        CloseHandle(h);
    } catch (const std::exception &ex) {
        std::cerr << ex.what() << std::endl;
    }

    return 0;
}

Sandhi

Does this add any new things to the answer from 2010? – Kine

@ThomasWeller it's a complete working example using the same idea. – Sandhi

@ThomasWeller It calls FSCTL_SET_ZERO_DATA into attention and in my opinion is the better answer therefore. That is needed in case of writing larger blocks of 0-bytes, which would be allocated physically otherwise and should therefore be kept in mind. learn.microsoft.com/de-de/windows/desktop/FileIO/… – Demi