Accessing files beyond MAX_PATH in C#/.NET

Question

BACKGROUND

I need to write a tool using .NET version 2.0 at highest (using something off the shelf is not an option for this client for political, commercial, and confidentiality/trust reasons) to migrate files from one server to another over the network. The servers are file servers for local teams, and certain team folders need to be migrated to other servers to facilitate a reorganisation. The basic idea is we read each file and stream it over the network out of hours and after a few days the data will be migrated. File permissions need to be preserved. As this will take a few days (we are talking up to several gigabytes of data, for some teams), we need to iterate over the files each night and compare the modification dates and update those that have changed. The theory is that eventually the new server will have an up to date copy of the files and the users can switch over to the new server. It is of course not quite this simple, but we have a design we think should work :)

THE PROBLEM

So in theory we just open the file, stream it over the network, and write it at the other end, right? :)

Unfortunately, on the servers themselves, the file shares were created at folder paths such as:

D:\Data\Team Shares\DIVISION\DEPARTMENT\NAME OF TEAM - COULD BE FAIRLY LONG\

For each user, this path is mapped to a drive, for example it would be shared as \\SERVER\TEAMNAME and mapped to the T: drive.

This has caused the situation where the files as visible from the T: drive are within the MAX_PATH limitation, however when viewed locally on the server itself, they go way beyond it. We cannot access the files using the network shares as this tool needs to be generic, to run on hundreds of these servers, and there is no standard way to tell which file shares are ones we should be moving and those which are not - there is no naming convention standard even. Also, occasionally there are sub-shares of other shares and so we exceed the MAX_PATH limit twice over!

I am aware of the workaround to specify paths using the "\\?\" prefix, which treats the path as a UNC path and allows a theoretical maximum of 32k characters.

This workaround is implemented at the Win32 API level, the System.IO namespace is (mostly) basically just a thin wrapper around native Win32 API functions, however Microsoft have "helpfully" implemented extra (incorrect) validation before handing the call off to the API. In this case, the .NET Framework is rejecting the path because it claims that '?' is an invalid path character.

So my question is... is there a way I haven't thought of that will allow me to work around this without having to completely rewrite the almost the whole System.IO namespace, making a load of P/Invoke calls, just to remove this annoying validation?

Leshalesher

Answer 1

The BCL team did a 3 part series on exactly why these choices were made and what the work arounds are. If you haven't read that yet I suggest you do as it's a great source of information on the subject

• http://blogs.msdn.com/bclteam/archive/2007/02/13/long-paths-in-net-part-1-of-3-kim-hamilton.aspx

Storfer

Yeah I've seen this. Their recommended solution is what I'm trying to avoid to be honest. As we're dealing with file and directory metadata, permissions, and file contents, this hits a LOT of API calls, and I was really hoping someone would have a workable solution that would mean I wouldn't have to spend the next month on pinvoke.net ;) – Leshalesher

My own and other answers here suggest some wrapper libraries you can use for dealing with long paths. – Perpend

Answer 2

I ran into one third-party solution that may help: AlphaFS.

Kordofanian

Answer 3

It should be fairly easy to work around this limitation with a bit of platform invoke, assuming your software has the necessary permissions:

[DllImport("kernel32.dll", SetLastError = true)]
static extern SafeFileHandle CreateFile(string lpFileName, uint dwDesiredAccess,
  uint dwShareMode, IntPtr lpSecurityAttributes, uint dwCreationDisposition,
  uint dwFlagsAndAttributes, IntPtr hTemplateFile);

// Must close/dispose handle separately from FileStream since it's not owned by
// that object when passed to constructor.
using (SafeFileHandle h = CreateFile(longUncPath, GENERIC_WRITE, 0, IntPtr.Zero, 
       OPEN_EXISTING, 0, IntPtr.Zero))
{
    using (var fs = new FileStream(h, FileAccess.Read))
    {
        // operations with FileStream object
    }
}

Brookhouse

When my P/Invoke definitions class got to about 3000 lines of various imports, structs and constants and I still didn't have all the functionality I needed I thought I'd come here first and see if I was missing something obvious... it seems not :( – Leshalesher

Gotcha. And there's no existing third-party solution? I'm surprised! – Brookhouse

Answer 4

You could try shortening the path by mapping a parent directory using subst.exe (or whatever APIs it uses internally):

http://www.makeuseof.com/tag/how-to-map-a-local-windows-folder-to-a-drive-letter/

Ideally you'd map away as much as possible of the path.

Sanjak

Answer 5

I have had success deleting directory structures using below small script. pushd uses UNC format which gives you 32K instead of 260 limitation

set "folder=\\SERVER\SHARE\DIVISION\DEPARTMENT\NAME OF TEAM - COULD BE FAIRLY LONG\" 
pushd "%folder%"
for /d %%i in ("*") do rmdir "%%i" /s /q
popd

Strand