How do we access MFT through C#

Question

I need to access Windows MFT(Master File Table) using C# in my .net application.
I have googled about this and couldn't find any good results. I have been searching for the information from the past 2 days but have been unable to find any information on the same.

I am not looking for exact code to do the same, I am just looking for some information which can get me started.

The only thing I have been able to figure out is that I have to use P/Invoke.
I want to know the functions I would be using to have access to MFT.
If you are able to provide some code sample then that would be great.

Harder

Check codeproject.com/Articles/9293/Undelete-a-file-in-NTFS and just pinvoke the native functions you need. – Deneendenegation

@Deneendenegation Thanks for the link, I had already checked it but couldn't find the relevant information in it. – Harder

Check this link - MFT. Hope you get some idea... – Gynecocracy

The link posted by @KrishnrajRana in the previous comment has expired. Here is a link to it from the Internet Archive. – Bethanybethe

Answer 1

First, you have to have and assert sufficient privileges to access the MFT - this is a pain all by itself. Then, you have to get a handle to a file/folder on the volume - for the calls in the last step...which is to call a Windows API (called DeviceIOControl) in a loop and read the entries from the returned API call - and this is it's own special headache.

Conceptually - this looks like:

static void Main( string[ ] args )
{
  if ( Privileges.HasBackupAndRestorePrivileges )
  {
    using ( var volume = GetVolumeHandle( "C:\\" ) )
    {
      ReadMft( volume );
    }
  }
}

If you take each of these in turn, asserting sufficient privileges is the most obscure part. There's a Windows API to change the privileges of the running token - and you use that to add the necessary privileges. Here's an excerpt from a class that I use to assert those privileges. You could assert a bunch more privileges - but this should be sufficient for reading the MFT.

Your application will need to run under an account that can actually obtain the requisite privileges - an admin account is good. Also, a backup operator will work.

public static class Privileges
{
  private static int asserted = 0;
  private static bool hasBackupPrivileges = false;

  public static bool HasBackupAndRestorePrivileges
  {
    get { return AssertPriveleges( ); }
  }

  /// <remarks>
  /// First time this method is called, it attempts to set backup privileges for the current process.
  /// Subsequently, it returns the results of that first call.
  /// </remarks>
  private static bool AssertPriveleges( )
  {
    bool success = false;
    var wasAsserted = Interlocked.CompareExchange( ref asserted, 1, 0 );
    if ( wasAsserted == 0 )  // first time here?  come on in!
    {
      success =
        AssertPrivelege( NativeMethods.SE_BACKUP_NAME ) &&
        AssertPrivelege( NativeMethods.SE_RESTORE_NAME );

      hasBackupPrivileges = success;

    }
    return hasBackupPrivileges;
  }


  private static bool AssertPrivelege( string privelege )
  {
    IntPtr token;
    var tokenPrivileges = new NativeMethods.TOKEN_PRIVILEGES( );
    tokenPrivileges.Privileges = new NativeMethods.LUID_AND_ATTRIBUTES[ 1 ];

    var success =
      NativeMethods.OpenProcessToken( NativeMethods.GetCurrentProcess( ), NativeMethods.TOKEN_ADJUST_PRIVILEGES, out token )
      &&
      NativeMethods.LookupPrivilegeValue( null, privelege, out tokenPrivileges.Privileges[ 0 ].Luid );

    try
    {
      if ( success )
      {
        tokenPrivileges.PrivilegeCount = 1;
        tokenPrivileges.Privileges[ 0 ].Attributes = NativeMethods.SE_PRIVILEGE_ENABLED;
        success =
          NativeMethods.AdjustTokenPrivileges( token, false, ref tokenPrivileges, Marshal.SizeOf( tokenPrivileges ), IntPtr.Zero, IntPtr.Zero )
          &&
          ( Marshal.GetLastWin32Error( ) == 0 );
      }

      if ( !success )
      {
        Console.WriteLine( "Could not assert privilege: " + privelege );
      }
    }
    finally
    {
      NativeMethods.CloseHandle( token );
    }

    return success;
  }
}

Once you're past that hurdle, the rest is - well...still a festival of obscurity. You have to get a handle to a file or folder - with backup semantics. You can more-than-likely just open a FileStream on any old file on the volume you're after and the FileStream will have a handle you can use for subsequent calls. This isn't precisely what my application did - but my app had to do things this answer doesn't have to do.

  internal static SafeFileHandle GetVolumeHandle( string pathToVolume, NativeMethods.EFileAccess access = NativeMethods.EFileAccess.AccessSystemSecurity | NativeMethods.EFileAccess.GenericRead | NativeMethods.EFileAccess.ReadControl )
  {
    var attributes = ( uint ) NativeMethods.EFileAttributes.BackupSemantics;
    var handle = NativeMethods.CreateFile( pathToVolume, access, 7U, IntPtr.Zero, ( uint ) NativeMethods.ECreationDisposition.OpenExisting, attributes, IntPtr.Zero );
    if ( handle.IsInvalid )
    {
      throw new IOException( "Bad path" );
    }

    return handle;
  }

For ReadMft - There is a rather complex windows API function - DeviceIOControl - that takes buffers with an epic variety of inputs and returns buffers containing a mind-bending variety of outputs. It's a kind of catch-all API for querying information about various devices - and the volume containing the MFT is a device.

To read the MFT, you call DeviceIOControl with a device IO control code of FSCTL_ENUM_USN_DATA - which returns one USN record for each record in the MFT. There are lots of records per each invocation - and after each invocation, you parameterize the next call in the loop with the first bit of info returned by the previous call.

BTW - I renamed the windows API calls in my code to make them look more .Net-like. I'm not sure I'd do that in the future.

Special note here: You're getting one record for each file - regardless of how many hard links there are - you have to do additional calls to enumerate the hard links.

The file system hierarchy is encoded in the FileReferenceNumber and ParentFileReferenceNumber of the structures you get back from the call. You'd nominally save off these usn records to a list, sorted by FileReferenceNumber and make a secondary index for the ParentFileReferenceNumber - or something like that. For the purpose of illustration, this code just dumps the MFT entries.

This example uses unsafe code - and fixes the location of the buffers containing the input and output. There are different ways to approach this - but this is nice and zippy. If you use this, you have to allow unsafe code in your project settings.

public unsafe static bool ReadMft( SafeHandle volume )
{
  var outputBufferSize = 1024 * 1024;
  var input = new NativeMethods.MFTEnumDataV0( );
  var usnRecord = new NativeMethods.UsnRecordV2( );

  var outputBuffer = new byte[ outputBufferSize ];

  var okay = true;
  var doneReading = false;

  try
  {
    fixed ( byte* pOutput = outputBuffer )
    {
      input.StartFileReferenceNumber = 0;
      input.LowUsn = 0;
      input.HighUsn = long.MaxValue;

      using ( var stream = new MemoryStream( outputBuffer, true ) )
      {
        while ( !doneReading )
        {
          var bytesRead = 0U;
          okay = NativeMethods.DeviceIoControl
          (
            volume.DangerousGetHandle( ),
            NativeMethods.DeviceIOControlCode.FsctlEnumUsnData,
            ( byte* ) &input.StartFileReferenceNumber,
            ( uint ) Marshal.SizeOf( input ),
            pOutput,
            ( uint ) outputBufferSize,
            out bytesRead,
            IntPtr.Zero
          );

          if ( !okay )
          {
            var error = Marshal.GetLastWin32Error( );
            okay = error == NativeMethods.ERROR_HANDLE_EOF;
            if ( !okay )
            {
              Console.WriteLine( "Crap! Windows error " + error.ToString( ) );
              break;
            }
            else
            {
              doneReading = true;
            }
          }

          input.StartFileReferenceNumber = stream.ReadULong( );
          while ( stream.Position < bytesRead )
          {
            usnRecord.Read( stream );

            //-->>>>>>>>>>>>>>>>> 
            //--> just an example of reading out the record...
            Console.WriteLine( "FRN:" + usnRecord.FileReferenceNumber.ToString( ) );
            Console.WriteLine( "Parent FRN:" + usnRecord.ParentFileReferenceNumber.ToString( ) );
            Console.WriteLine( "File name:" + usnRecord.FileName );
            Console.WriteLine( "Attributes: " + ( NativeMethods.EFileAttributes ) usnRecord.FileAttributes );
            Console.WriteLine( "Timestamp:" + usnRecord.TimeStamp );
            //-->>>>>>>>>>>>>>>>>>> 
          }
          stream.Seek( 0, SeekOrigin.Begin );
        }
      }
    }
  }
  catch ( Exception ex )
  {
    Console.Write( ex );
    okay = false;
  }
  return okay;
}

I do something probably kind of cheesy to save myself a lot of work - I add pseudo-serialization methods to windows API structures - so that they can read themselves out of streams. For example, the usnRecord used to read the buffer in the foregoing code is a windows API structure - but with a serialization interface implemented:

[StructLayout( LayoutKind.Sequential )]
internal struct UsnRecordV2: IBinarySerialize
{
  public uint RecordLength;
  public ushort MajorVersion;
  public ushort MinorVersion;
  public ulong FileReferenceNumber;
  public ulong ParentFileReferenceNumber;
  public long Usn;
  public long TimeStamp;
  public UsnReason Reason;
  public uint SourceInfo;
  public uint SecurityId;
  public uint FileAttributes;
  public ushort FileNameLength;
  public ushort FileNameOffset;
  public string FileName;

  /// <remarks>
  /// Note how the read advances to the FileNameOffset and reads only FileNameLength bytes.
  /// </remarks>
  public void Read( Stream stream )
  {
    var startOfRecord = stream.Position;
    RecordLength = stream.ReadUInt( );
    MajorVersion = stream.ReadUShort( );
    MinorVersion = stream.ReadUShort( );
    FileReferenceNumber = stream.ReadULong( );
    ParentFileReferenceNumber = stream.ReadULong( );
    Usn = stream.ReadLong( );
    TimeStamp = stream.ReadLong( );
    Reason = ( UsnReason ) stream.ReadUInt( );
    SourceInfo = stream.ReadUInt( );
    SecurityId = stream.ReadUInt( );
    FileAttributes = stream.ReadUInt( );
    FileNameLength = stream.ReadUShort( );
    FileNameOffset = stream.ReadUShort( );
    stream.Position = startOfRecord + FileNameOffset;
    FileName = Encoding.Unicode.GetString( stream.ReadBytes( FileNameLength ) );
    stream.Position = startOfRecord + RecordLength;

  }

  /// <summary>We never write instances of this structure</summary>
  void IBinarySerialize.Write( Stream stream )
  {
    throw new NotImplementedException( );
  }
}

...where IBinarySerialze is:

public interface IBinarySerialize
{
  /// <summary>Reads an object's data from a <see cref="Stream"/></summary>
  void Read( Stream stream );

  /// <summary>Writes an objects serializable content to a <see cref="Stream"/></summary>
  void Write( Stream stream );

}

There are stream extension methods used in this structure. Basically, they're lifted from BinaryReader. Why? Because in .Net 3.5 - where I had to write this originally - the BCL BinaryReader would close the stream you wrapped it around - and I had lots of places where that was just intolerable.

internal static class StreamingExtensions
{
  public static ushort ReadUShort( this Stream stream )
  {
    return BitConverter.ToUInt16( ReadBytes( stream, 2 ), 0 );
  }

  public static uint ReadUInt( this Stream stream )
  {
    return BitConverter.ToUInt32( ReadBytes( stream, 4 ), 0 );
  }

  public static long ReadLong( this Stream stream )
  {
    return BitConverter.ToInt64( ReadBytes( stream, 8 ), 0 );
  }

  public static ulong ReadULong( this Stream stream )
  {
    return BitConverter.ToUInt64( ReadBytes( stream, 8 ), 0 );
  }
  public static byte[ ] ReadBytes( this Stream stream, int length, bool throwIfIncomplete = false )
  {
    var bytes = new byte[ length ];
    var bytesRead = 0;
    var offset = 0;
    if ( length > 0 )
    {
      while ( offset < length )
      {
        bytesRead = stream.Read( bytes, offset, length - offset );
        if ( bytesRead == 0 )
        {
          if ( throwIfIncomplete ) throw new InvalidOperationException( "incomplete" );
          break;
        }
        offset += bytesRead;
      }
    }
    return bytes;
  }
}

And for completeness, here are the native methods, enums, constants, and noise. Most are from from PInvoke.net, but again...the names of many of these things were .Net-ified. Apologies to the purists.

internal class NativeMethods
{
  internal const int ERROR_HANDLE_EOF = 38;

  //--> Privilege constants....
  internal const UInt32 SE_PRIVILEGE_ENABLED = 0x00000002;
  internal const string SE_BACKUP_NAME = "SeBackupPrivilege";
  internal const string SE_RESTORE_NAME = "SeRestorePrivilege";
  internal const string SE_SECURITY_NAME = "SeSecurityPrivilege";
  internal const string SE_CHANGE_NOTIFY_NAME = "SeChangeNotifyPrivilege";
  internal const string SE_CREATE_SYMBOLIC_LINK_NAME = "SeCreateSymbolicLinkPrivilege";
  internal const string SE_CREATE_PERMANENT_NAME = "SeCreatePermanentPrivilege";
  internal const string SE_SYSTEM_ENVIRONMENT_NAME = "SeSystemEnvironmentPrivilege";
  internal const string SE_SYSTEMTIME_NAME = "SeSystemtimePrivilege";
  internal const string SE_TIME_ZONE_NAME = "SeTimeZonePrivilege";
  internal const string SE_TCB_NAME = "SeTcbPrivilege";
  internal const string SE_MANAGE_VOLUME_NAME = "SeManageVolumePrivilege";
  internal const string SE_TAKE_OWNERSHIP_NAME = "SeTakeOwnershipPrivilege";

  //--> For starting a process in session 1 from session 0...
  internal const int TOKEN_DUPLICATE = 0x0002;
  internal const uint MAXIMUM_ALLOWED = 0x2000000;
  internal const int CREATE_NEW_CONSOLE = 0x00000010;
  internal const uint TOKEN_ADJUST_PRIVILEGES = 0x0020;
  internal const int TOKEN_QUERY = 0x00000008;


  [DllImport( "advapi32.dll", SetLastError = true )]
  [return: MarshalAs( UnmanagedType.Bool )]
  internal static extern bool OpenProcessToken( IntPtr ProcessHandle, UInt32 DesiredAccess, out IntPtr TokenHandle );
  [DllImport( "kernel32.dll" )]
  internal static extern IntPtr GetCurrentProcess( );
  [DllImport( "advapi32.dll", SetLastError = true, CharSet = CharSet.Unicode )]
  [return: MarshalAs( UnmanagedType.Bool )]
  internal static extern bool LookupPrivilegeValue( string lpSystemName, string lpName, out LUID lpLuid );
  [DllImport( "advapi32.dll", SetLastError = true )]
  [return: MarshalAs( UnmanagedType.Bool )]
  internal static extern bool AdjustTokenPrivileges( IntPtr TokenHandle, [MarshalAs( UnmanagedType.Bool )]bool DisableAllPrivileges, ref TOKEN_PRIVILEGES NewState, Int32 BufferLength, IntPtr PreviousState, IntPtr ReturnLength );
  [DllImport( "kernel32.dll", ExactSpelling = true, SetLastError = true, CharSet = CharSet.Unicode )]
  [return: MarshalAs( UnmanagedType.Bool )]
  internal static unsafe extern bool DeviceIoControl( IntPtr hDevice, DeviceIOControlCode controlCode, byte* lpInBuffer, uint nInBufferSize, byte* lpOutBuffer, uint nOutBufferSize, out uint lpBytesReturned, IntPtr lpOverlapped );
  [DllImport( "kernel32.dll", SetLastError = true, CharSet = CharSet.Unicode )]
  internal static extern SafeFileHandle CreateFile( string lpFileName, EFileAccess dwDesiredAccess, uint dwShareMode, IntPtr lpSecurityAttributes, uint dwCreationDisposition, uint dwFlagsAndAttributes, IntPtr hTemplateFile );
  [DllImport( "kernel32.dll", SetLastError = true )]
  [return: MarshalAs( UnmanagedType.Bool )]
  internal static extern bool CloseHandle( IntPtr hObject );


  [Flags]
  internal enum EMethod: uint
  {
    Buffered = 0,
    InDirect = 1,
    OutDirect = 2,
    Neither = 3
  }

  [Flags]
  internal enum EFileAccess: uint
  {
    GenericRead = 0x80000000,
    GenericWrite = 0x40000000,
    GenericExecute = 0x20000000,
    GenericAll = 0x10000000,

    Delete = 0x10000,
    ReadControl = 0x20000,
    WriteDAC = 0x40000,
    WriteOwner = 0x80000,
    Synchronize = 0x100000,

    StandardRightsRequired = 0xF0000,
    StandardRightsRead = ReadControl,
    StandardRightsWrite = ReadControl,
    StandardRightsExecute = ReadControl,
    StandardRightsAll = 0x1F0000,
    SpecificRightsAll = 0xFFFF,

    AccessSystemSecurity = 0x1000000,
    MaximumAllowed = 0x2000000
  }


  [Flags]
  internal enum EFileDevice: uint
  {
    Beep = 0x00000001,
    CDRom = 0x00000002,
    CDRomFileSytem = 0x00000003,
    Controller = 0x00000004,
    Datalink = 0x00000005,
    Dfs = 0x00000006,
    Disk = 0x00000007,
    DiskFileSystem = 0x00000008,
    FileSystem = 0x00000009,
    InPortPort = 0x0000000a,
    Keyboard = 0x0000000b,
    Mailslot = 0x0000000c,
    MidiIn = 0x0000000d,
    MidiOut = 0x0000000e,
    Mouse = 0x0000000f,
    MultiUncProvider = 0x00000010,
    NamedPipe = 0x00000011,
    Network = 0x00000012,
    NetworkBrowser = 0x00000013,
    NetworkFileSystem = 0x00000014,
    Null = 0x00000015,
    ParallelPort = 0x00000016,
    PhysicalNetcard = 0x00000017,
    Printer = 0x00000018,
    Scanner = 0x00000019,
    SerialMousePort = 0x0000001a,
    SerialPort = 0x0000001b,
    Screen = 0x0000001c,
    Sound = 0x0000001d,
    Streams = 0x0000001e,
    Tape = 0x0000001f,
    TapeFileSystem = 0x00000020,
    Transport = 0x00000021,
    Unknown = 0x00000022,
    Video = 0x00000023,
    VirtualDisk = 0x00000024,
    WaveIn = 0x00000025,
    WaveOut = 0x00000026,
    Port8042 = 0x00000027,
    NetworkRedirector = 0x00000028,
    Battery = 0x00000029,
    BusExtender = 0x0000002a,
    Modem = 0x0000002b,
    Vdm = 0x0000002c,
    MassStorage = 0x0000002d,
    Smb = 0x0000002e,
    Ks = 0x0000002f,
    Changer = 0x00000030,
    Smartcard = 0x00000031,
    Acpi = 0x00000032,
    Dvd = 0x00000033,
    FullscreenVideo = 0x00000034,
    DfsFileSystem = 0x00000035,
    DfsVolume = 0x00000036,
    Serenum = 0x00000037,
    Termsrv = 0x00000038,
    Ksec = 0x00000039,
    // From Windows Driver Kit 7
    Fips = 0x0000003A,
    Infiniband = 0x0000003B,
    Vmbus = 0x0000003E,
    CryptProvider = 0x0000003F,
    Wpd = 0x00000040,
    Bluetooth = 0x00000041,
    MtComposite = 0x00000042,
    MtTransport = 0x00000043,
    Biometric = 0x00000044,
    Pmi = 0x00000045
  }

  internal enum EFileIOCtlAccess: uint
  {
    Any = 0,
    Special = Any,
    Read = 1,
    Write = 2
  }

  internal enum DeviceIOControlCode: uint
  {
    FsctlEnumUsnData = ( EFileDevice.FileSystem << 16 ) | ( 44 << 2 ) | EMethod.Neither | ( EFileIOCtlAccess.Any << 14 ),
    FsctlReadUsnJournal = ( EFileDevice.FileSystem << 16 ) | ( 46 << 2 ) | EMethod.Neither | ( EFileIOCtlAccess.Any << 14 ),
    FsctlReadFileUsnData = ( EFileDevice.FileSystem << 16 ) | ( 58 << 2 ) | EMethod.Neither | ( EFileIOCtlAccess.Any << 14 ),
    FsctlQueryUsnJournal = ( EFileDevice.FileSystem << 16 ) | ( 61 << 2 ) | EMethod.Buffered | ( EFileIOCtlAccess.Any << 14 ),
    FsctlCreateUsnJournal = ( EFileDevice.FileSystem << 16 ) | ( 57 << 2 ) | EMethod.Neither | ( EFileIOCtlAccess.Any << 14 )
  }

  /// <summary>Control structure used to interrogate MFT data using DeviceIOControl from the user volume</summary>
  [StructLayout( LayoutKind.Sequential )]
  internal struct MFTEnumDataV0
  {
    public ulong StartFileReferenceNumber;
    public long LowUsn;
    public long HighUsn;
  }


  /// <summary>A structure resurned form USN queries</summary>
  /// <remarks>
  /// FileName is synthetic...composed during a read of the structure and is not technically
  /// part of the Win32 API's definition...although the actual FileName is contained
  /// "somewhere" in the structure's trailing bytes, according to FileNameLength and FileNameOffset.
  /// 
  /// Alignment boundaries are enforced, and so, the RecordLength
  /// may be somewhat larger than the accumulated lengths of the members plus the FileNameLength.
  /// </remarks>
  [StructLayout( LayoutKind.Sequential )]
  internal struct UsnRecordV2: IBinarySerialize
  {
    public uint RecordLength;
    public ushort MajorVersion;
    public ushort MinorVersion;
    public ulong FileReferenceNumber;
    public ulong ParentFileReferenceNumber;
    public long Usn;
    public long TimeStamp;
    public UsnReason Reason;
    public uint SourceInfo;
    public uint SecurityId;
    public uint FileAttributes;
    public ushort FileNameLength;
    public ushort FileNameOffset;
    public string FileName;

    /// <remarks>Note how the read advances to the FileNameOffset and reads only FileNameLength bytes</remarks>
    public void Read( Stream stream )
    {
      var startOfRecord = stream.Position;
      RecordLength = stream.ReadUInt( );
      MajorVersion = stream.ReadUShort( );
      MinorVersion = stream.ReadUShort( );
      FileReferenceNumber = stream.ReadULong( );
      ParentFileReferenceNumber = stream.ReadULong( );
      Usn = stream.ReadLong( );
      TimeStamp = stream.ReadLong( );
      Reason = ( UsnReason ) stream.ReadUInt( );
      SourceInfo = stream.ReadUInt( );
      SecurityId = stream.ReadUInt( );
      FileAttributes = stream.ReadUInt( );
      FileNameLength = stream.ReadUShort( );
      FileNameOffset = stream.ReadUShort( );
      stream.Position = startOfRecord + FileNameOffset;
      FileName = Encoding.Unicode.GetString( stream.ReadBytes( FileNameLength ) );
      stream.Position = startOfRecord + RecordLength;

    }

    void IBinarySerialize.Write( Stream stream )
    {
      throw new NotImplementedException( );
    }
  }

  /// <summary>Structure returned from USN query that describes the state of the journal</summary>
  [StructLayout( LayoutKind.Sequential )]
  internal struct UsnJournalDataV1: IBinarySerialize
  {
    public ulong UsnJournalId;
    public long FirstUsn;
    public long NextUsn;
    public long LowestValidUsn;
    public long MaxUsn;
    public ulong MaximumSize;
    public ulong AllocationDelta;
    public ushort MinSupportedMajorVersion;
    public ushort MaxSupportedMajorVersion;

    public void Read( Stream stream )
    {
      UsnJournalId = stream.ReadULong( );
      FirstUsn = stream.ReadLong( );
      NextUsn = stream.ReadLong( );
      LowestValidUsn = stream.ReadLong( );
      MaxUsn = stream.ReadLong( );
      MaximumSize = stream.ReadULong( );
      AllocationDelta = stream.ReadULong( );
      MinSupportedMajorVersion = stream.ReadUShort( );
      MaxSupportedMajorVersion = stream.ReadUShort( );
    }

    void IBinarySerialize.Write( Stream stream )
    {
      throw new NotImplementedException( );
    }
  }

  [StructLayout( LayoutKind.Sequential )]
  internal struct LUID
  {
    public UInt32 LowPart;
    public Int32 HighPart;
  }


  [StructLayout( LayoutKind.Sequential )]
  internal struct LUID_AND_ATTRIBUTES
  {
    public LUID Luid;
    public UInt32 Attributes;
  }


  internal struct TOKEN_PRIVILEGES
  {
    public UInt32 PrivilegeCount;
    [MarshalAs( UnmanagedType.ByValArray, SizeConst = 1 )]      // !! think we only need one
    public LUID_AND_ATTRIBUTES[ ] Privileges;
  }

  [Flags]
  internal enum EFileAttributes: uint
  {
    /// <summary/>
    None = 0,

    //-->  these are consistent w/ .Net FileAttributes...
    Readonly = 0x00000001,
    Hidden = 0x00000002,
    System = 0x00000004,
    Directory = 0x00000010,
    Archive = 0x00000020,
    Device = 0x00000040,
    Normal = 0x00000080,
    Temporary = 0x00000100,
    SparseFile = 0x00000200,
    ReparsePoint = 0x00000400,
    Compressed = 0x00000800,
    Offline = 0x00001000,
    NotContentIndexed = 0x00002000,
    Encrypted = 0x00004000,

    //--> additional CreateFile call attributes...
    Write_Through = 0x80000000,
    Overlapped = 0x40000000,
    NoBuffering = 0x20000000,
    RandomAccess = 0x10000000,
    SequentialScan = 0x08000000,
    DeleteOnClose = 0x04000000,
    BackupSemantics = 0x02000000,
    PosixSemantics = 0x01000000,
    OpenReparsePoint = 0x00200000,
    OpenNoRecall = 0x00100000,
    FirstPipeInstance = 0x00080000
  }

  /// <summary>Reasons the file changed (from USN journal)</summary>
  [Flags]
  public enum UsnReason: uint
  {
    BASIC_INFO_CHANGE = 0x00008000,
    CLOSE = 0x80000000,
    COMPRESSION_CHANGE = 0x00020000,
    DATA_EXTEND = 0x00000002,
    DATA_OVERWRITE = 0x00000001,
    DATA_TRUNCATION = 0x00000004,
    EA_CHANGE = 0x00000400,
    ENCRYPTION_CHANGE = 0x00040000,
    FILE_CREATE = 0x00000100,
    FILE_DELETE = 0x00000200,
    HARD_LINK_CHANGE = 0x00010000,
    INDEXABLE_CHANGE = 0x00004000,
    NAMED_DATA_EXTEND = 0x00000020,
    NAMED_DATA_OVERWRITE = 0x00000010,
    NAMED_DATA_TRUNCATION = 0x00000040,
    OBJECT_ID_CHANGE = 0x00080000,
    RENAME_NEW_NAME = 0x00002000,
    RENAME_OLD_NAME = 0x00001000,
    REPARSE_POINT_CHANGE = 0x00100000,
    SECURITY_CHANGE = 0x00000800,
    STREAM_CHANGE = 0x00200000,
  
    None = 0x00000000
  }

  internal enum ECreationDisposition: uint
  {
    New = 1,
    CreateAlways = 2,
    OpenExisting = 3,
    OpenAlways = 4,
    TruncateExisting = 5
  }

}

Epistasis

Amazing answer. To anyone encountering error 0x5 (access denied) I had to use Volume_manager privilege (following msdn: learn.microsoft.com/en-us/windows-hardware/drivers/ifs/…) – Arm

@Fanchi, thanks - all this time, and no comments. ...and thanks for the hint on SeManageVolumePrivilege – Epistasis

Answer 2

You can use this https://sourceforge.net/projects/ntfsreader/ open source library that is written in C# by Danny Couture.

I tested it and its performance is good. It can parse an NTFS drive with more than 100000 entry(file and folder) in less than 2 seconds.

Dantzler

Answer 3

Using the https://sourceforge.net/projects/ntfsreader/ library by Danny Couture is quite straightforward:

There are two main activities in which I am interested. The first is reading the information for a given drive The second part is processing the information retrieved (Search for a given file/folder, etc, rename all files, find files with length greater than 260, etc).

All information is returned as nodes and these can be files or folders or some other strange NTFS stuff such as extensions, etc, which I currently do not need.
You can test for which types of nodes you are interested in by testing node.Attributes - for example:

if((node.Attributes & Attributes.Directory) != 0 ) { /*...*/ }  

On a 10 TB hard-drive (not particularly fast!) with 5 million files and 70k folders retrieving the nodes takes around 60 seconds using minimal retrieval mode - see later Searching the collection of Inode(s) - in my case searching for all folders with archive in the name - took less than 500ms.

Because of the huge difference in times for these two activities, I find it best to retrieve all nodes (60 seconds) once and then perform repeated searches (~500ms) on the node collection.
The node retrieval only needs to be re-run if folders or files have changed.

You have to add a reference to ntfsreader and an example of its use is shown in the following pseudocode:

using System.IO.Filesystem.Ntfs;
        
public IEnumerable<INode> nodes;
public void DoStuff
{
        
    // Obviously you can also use whatever you like to store this collection; array, dictionary, datatable, etc.        
    public List<string> foundObjects = new List<string>();
            
    // string drive is "c" or "d" or whatever.  Note that there is no "c:\" or "c:".  
    // Just "c" or "d" for the string drive argument.
    
    // Read the NTFS MFT
    RefreshNodes("f");
            
            
    // If, for example, you are looking for a folder with a given name then
    string strPattern="archive";
    foreach (INode node in nodes)
    {
        if ((node.Attributes & Attributes.Directory) != 0 && node.Name.Contains(strPattern))
        {
            foundObjects.Add(node.FullName.Replace(@"\\", @"\"));
        }
    }        
}

public void RefreshNodes(string drive)
{
    DriveInfo driveToAnalyze = new DriveInfo(drive);
    NtfsReader ntfsReader = new NtfsReader(driveToAnalyze, RetrieveMode.Minimal);
    nodes = ntfsReader.GetNodes(driveToAnalyze.Name);
}

The time taken to retrieve the nodes depends on the RetrieveMode you use. RetrieveMode.Minimal brings back most of the information I need and it is quicker.

Hope this helps and happy NTFSing!

Duodecimo