Walking the NTFS Change Journal on Windows 10

Question

I'm trying to read the NTFS Change Journal but I've noticed that all the example code I can find fails on Windows 10, even though it works on Windows 7.

For example, Microsofts own example Walking a Buffer of Change Journal Records works on Windows 7 but when I run the same code on Windows 10 I get an error 87 (The parameter is incorrect) when I call DeviceIoControl with FSCTL_READ_USN_JOURNAL (Note that the earlier call to DeviceIoControl with FSCTL_QUERY_USN_JOURNAL completes successfully and return valid data.).

I've even taken the EXE compiled and working on Windows 7 and copied it to the Windows 10 machine and it still fails, so I believe that Windows 10 may be more strict on parameter validation or something like that?

I am running the code as Administrator, so that's not the issue.

I can't find any other references to this problem, but I get the same issue if I take other peoples example code and attempt to run it on Windows 10.

Edit:

The code itself:

#include <Windows.h>
#include <WinIoCtl.h>
#include <stdio.h>

#define BUF_LEN 4096

void main()
{
   HANDLE hVol;
   CHAR Buffer[BUF_LEN];

   USN_JOURNAL_DATA JournalData;
   READ_USN_JOURNAL_DATA ReadData = {0, 0xFFFFFFFF, FALSE, 0, 0};
   PUSN_RECORD UsnRecord;  

   DWORD dwBytes;
   DWORD dwRetBytes;
   int I;

   hVol = CreateFile( TEXT("\\\\.\\c:"), 
               GENERIC_READ | GENERIC_WRITE, 
               FILE_SHARE_READ | FILE_SHARE_WRITE,
               NULL,
               OPEN_EXISTING,
               0,
               NULL);

   if( hVol == INVALID_HANDLE_VALUE )
   {
      printf("CreateFile failed (%d)\n", GetLastError());
      return;
   }

   if( !DeviceIoControl( hVol, 
          FSCTL_QUERY_USN_JOURNAL, 
          NULL,
          0,
          &JournalData,
          sizeof(JournalData),
          &dwBytes,
          NULL) )
   {
      printf( "Query journal failed (%d)\n", GetLastError());
      return;
   }

   ReadData.UsnJournalID = JournalData.UsnJournalID;

   printf( "Journal ID: %I64x\n", JournalData.UsnJournalID );
   printf( "FirstUsn: %I64x\n\n", JournalData.FirstUsn );

   for(I=0; I<=10; I++)
   {
      memset( Buffer, 0, BUF_LEN );

      if( !DeviceIoControl( hVol, 
            FSCTL_READ_USN_JOURNAL, 
            &ReadData,
            sizeof(ReadData),
            &Buffer,
            BUF_LEN,
            &dwBytes,
            NULL) )
      {
         printf( "Read journal failed (%d)\n", GetLastError());
         return;
      }

      dwRetBytes = dwBytes - sizeof(USN);

      // Find the first record
      UsnRecord = (PUSN_RECORD)(((PUCHAR)Buffer) + sizeof(USN));  

      printf( "****************************************\n");

      // This loop could go on for a long time, given the current buffer size.
      while( dwRetBytes > 0 )
      {
         printf( "USN: %I64x\n", UsnRecord->Usn );
         printf("File name: %.*S\n", 
                  UsnRecord->FileNameLength/2, 
                  UsnRecord->FileName );
         printf( "Reason: %x\n", UsnRecord->Reason );
         printf( "\n" );

         dwRetBytes -= UsnRecord->RecordLength;

         // Find the next record
         UsnRecord = (PUSN_RECORD)(((PCHAR)UsnRecord) + 
                  UsnRecord->RecordLength); 
      }
      // Update starting USN for next call
      ReadData.StartUsn = *(USN *)&Buffer; 
   }

   CloseHandle(hVol);
}

Bessette

Please provide a minimal reproducible example. – Tambac

@IInspectable, the supplied link to msdn.microsoft.com/en-us/library/windows/desktop/… includes a minimal, complete and verifiable example. I'm not sure that I'm allowed to copy paste the code here, but there isn't a much simpler example I could possible write. – Bessette

If you cannot post a minimal reproducible example in the question itself, then that makes the question off-topic. – Tambac

i think your error in using USN_JOURNAL_DATA and READ_USN_JOURNAL_DATA - really this is macros.you need direct use READ_USN_JOURNAL_DATA_V0 or if you use READ_USN_JOURNAL_DATA_V1 you need correct set MinMajorVersion and MaxMajorVersion. guess you use V1 version of structures but init MinMajorVersion and MaxMajorVersion to 0 which is error – Pilkington

Thanks RdMm, yes I came to the same conclusion at the same time. +1 – Bessette

Answer 1

I've managed to work out what the problem is.

The example Microsoft code creates a local variable defined as READ_USN_JOURNAL_DATA, which is defined as:

#if (NTDDI_VERSION >= NTDDI_WIN8)
typedef READ_USN_JOURNAL_DATA_V1 READ_USN_JOURNAL_DATA, *PREAD_USN_JOURNAL_DATA;
#else
typedef READ_USN_JOURNAL_DATA_V0 READ_USN_JOURNAL_DATA, *PREAD_USN_JOURNAL_DATA;
#endif

On my systems (Both the Win10 and Win7 systems) this evaluates to READ_USN_JOURNAL_DATA_V1.

Looking at the difference betweem READ_USN_JOURNAL_DATA_V0 and READ_USN_JOURNAL_DATA_V1 we can see that V0 is defined as:

typedef struct {
    USN StartUsn;
    DWORD ReasonMask;
    DWORD ReturnOnlyOnClose;
    DWORDLONG Timeout;
    DWORDLONG BytesToWaitFor;
    DWORDLONG UsnJournalID;
} READ_USN_JOURNAL_DATA_V0, *PREAD_USN_JOURNAL_DATA_V0;

and the V1 version is defined as:

typedef struct {
    USN StartUsn;
    DWORD ReasonMask;
    DWORD ReturnOnlyOnClose;
    DWORDLONG Timeout;
    DWORDLONG BytesToWaitFor;
    DWORDLONG UsnJournalID;
    WORD   MinMajorVersion;
    WORD   MaxMajorVersion;
} READ_USN_JOURNAL_DATA_V1, *PREAD_USN_JOURNAL_DATA_V1;

Note the new Min and Max Major version members.

So, the Microsoft code is defining a local variable called ReadData that is actually a V1 structure, yet it appears to fill in the data assuming it's a V0 structure. i.e. it doesn't set the Min and Max elements.

It appears that Win7 is fine with this but Win10 rejects it and returns error 87 (The parameter is incorrect).

Sure enough if I explicitly define the ReadData variable to be a READ_USN_JOURNAL_DATA_V0 then the code works on Win7 and Win10, whereas if I explicitly define it as a READ_USN_JOURNAL_DATA_V1 then it continues to work on Win7 but not on Win10.

The strange thing is that the API documentation for READ_USN_JOURNAL_DATA_V1 structure states that it's only supported from Windows 8 on, so it's odd that it works on Windows 7 at all. I guess it's just interpreting it as a READ_USN_JOURNAL_DATA_V0 structure given that V1 version is an extension of the V0 structure. If so then it must be ignoring the size parameter that is passed into DeviceIOControl.

Anyway, all working now. I hope someone finds this a useful reference in the future.

Bessette

Just because you pass in a larger buffer does not mean DeviceIoControl() will populate the full buffer. That is why it has an output parameter for the number of bytes actually written. Pass in a V1 struct and Win7 will likely return dwBytes=sizeof(V0) as output, because it doesn't know about V1. Win10 does, and will validate the version fields if the input buffer is >= sizeof(V1), hence the "invalid parameter" error if you don't fill in the version numbers – Ganef

Answer 2

I came across this exact same issue with the sample code as the OP. At the start of the sample code, you will see a partial initialization of the structure at the point of declaration. A little later in the code, right before the offending call, there is a line that assigns the UsnJournalID into the read data structure.

For Windows 10, though, the other two members of the V1 structure are not initialized. I initialized them right after the UsnJournalID's initialization with:

#if (NTDDI_VERSION >= NTDDI_WIN8)
ReadData.MinMajorVersion = JournalData.MinSupportedMajorVersion;
ReadData.MaxMajorVersion = JournalData.MaxSupportedMajorVersion;
#endif

When I ran my code after doing this, it worked correctly without the error code. I remember reading in the Volume Management API discussion that the Min and Max versions needed to be set. I forgot exactly where, because I've been reading and testing this stuff for a couple days.

Anyway, I hope that clarifies the issue for anyone coming after me.

Buonaparte

Answer 3

Replace the READ_USN_JOURNAL_DATA structure with READ_USN_JOURNAL_DATA_V0 data structure and initialize it.

This worked for me

READ_USN_JOURNAL_DATA_V0 ReadData;
ZeroMemory(&ReadData, sizeof(ReadData));
ReadData.ReasonMask = 0xFFFFFFFF;

Wot