CreateFile with the FILE_FLAG_DELETE_ON_CLOSE flag

Question

Before I describe my problem, here is a description of the program (IHExplorer.exe) I'm writting:

This is a C++ application.

The IHExplorer application is to look as much like a Windows Explorer window as possible. With one exception, and that is that launching files from within this Explorer window will decrypt them first to the user's temp folder, then launch the app associated with the file extension and delete the file on close.

The problem i'm having is with the auto delete when the file is closed. Here's a scenario:

1. User double clicks an encrypted .txt file in IHExplorer.

2. IHExplorer decrypts the .txt file in memory, then writes it to %TEMP% using ::CreateFile which returns a HANDLE to the file (IHExplorer has to keep this handle open atleast until the .txt file is shell executed).

3. IHExplorer Shell Executes the .txt file (by calling ::ShellExecute) from it's temp location.

4. Now IHExplorer and notepad both have a handle to the file open.
5. The file must be auto deleted when both IHExplorer and notepad have both closed their handle to the file, even if IHExplorer closes first.

ok. that is a basical User Case that describes what I want to happen. The problem I have is when I ::ShellExecute(), notepad says "The process cannot access the file because it is being used by another process." (which would be IHExplorer). I need to get around this and have notepad open it even while I still have the handle open in IHExplorer.

Here's what my call to ::CreateFile looks like:

DWORD dwShareMode = FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE;
DWORD dwFlagsAndAttributes = FILE_ATTRIBUTE_TEMPORARY | FILE_FLAG_DELETE_ON_CLOSE;
HANDLE hFile = ::CreateFile(strTempFile.c_str(), GENERIC_WRITE, dwShareMode, &sa, CREATE_NEW, dwFlagsAndAttributes, NULL);

Notice I used FILE_SHARE_DELETE so that other processes (such as notepad) can open the file with delete access.

Notice that I used the FILE_ATTRIBUTE_TEMPORARY | FILE_FLAG_DELETE_ON_CLOSE attributes to indicicate the the file is temporary and should be deleted on close.

Also notice the &sa parameter. This is the SECURITY_ATTRIBUTES structure that I am using, and I feel (hope) this is where my problem lies. Here is the code again, this time I will post the entire function so you can see how I fill out the SECURITY_ATTRIBUTES structure:

int CIHExplorerDoc::OpenFile(std::string strFileName, bool bIsFullPath) {
    std::string strFullFilePath;
    if(bIsFullPath) {
        strFullFilePath = strFileName;
        strFileName = IHawk::RemovePath(strFileName);
    }else {
        strFullFilePath = m_strDirectory + strFileName;
    }

    if(!HasEncryptionFileExtension(strFullFilePath)) {
        LaunchFile(strFullFilePath);
    }else {
        //it's an encrypted file, so open it and copy unencrypted file to temp.
        IHawk::EncryptedFileHandle hEncryptedFile(strFullFilePath.c_str(), true, theApp.GetKeyServer());
        if(hEncryptedFile.IsValid()) {
            std::string strTempFile = g_strTempFolder + IHawk::ChangeFileExtension(strFileName, "");

            //TODO: Determine what the LPSECURITY_ATTRIBUTES should be.

            SECURITY_ATTRIBUTES sa;
            SECURITY_DESCRIPTOR sd;

            if(!InitializeSecurityDescriptor(&sd,SECURITY_DESCRIPTOR_REVISION)) {
                DWORD dwLastError = ::GetLastError();
                LOG4CPLUS_ERROR(m_Logger, "Cannot launch file '" << strFullFilePath << "'.  Failed to initialize security descriptor.  GetLastError=" << dwLastError);
                return dwLastError;
            }

            if(!SetSecurityDescriptorDacl(
                &sd,    // A pointer to the SECURITY_DESCRIPTOR structure to which the function adds the DACL
                TRUE,   // presence of a DACL in the security descriptor
                NULL,   // allows all access to the object
                FALSE   // DACL has been explicitly specified by a user
            )) 
            {
                DWORD dwLastError = ::GetLastError();
                LOG4CPLUS_ERROR(m_Logger, "Cannot launch file '" << strFullFilePath << "'.  Failed to set security descriptor DACL.  GetLastError=" << dwLastError);
                return dwLastError;
            }

            if(!SetSecurityDescriptorGroup(
                &sd,    // A pointer to the SECURITY_DESCRIPTOR structure whose primary group is set by this function
                NULL,   // no primary group
                FALSE   // Indicates whether the primary group information was derived from a default mechanism
            ))
            {
                DWORD dwLastError = ::GetLastError();
                LOG4CPLUS_ERROR(m_Logger, "Cannot launch file '" << strFullFilePath << "'.  Failed to set security descriptor primary group.  GetLastError=" << dwLastError);
                return dwLastError;
            }

            if(!SetSecurityDescriptorOwner(
                &sd,    // A pointer to the SECURITY_DESCRIPTOR structure whose owner is set by this function.
                NULL,   // If this parameter is NULL, the function clears the security descriptor's owner information. This marks the security descriptor as having no owner.
                FALSE   // Indicates whether the owner information is derived from a default mechanism.
            ))
            {
                DWORD dwLastError = ::GetLastError();
                LOG4CPLUS_ERROR(m_Logger, "Cannot launch file '" << strFullFilePath << "'.  Failed to set security descriptor owner information.  GetLastError=" << dwLastError);
                return dwLastError;
            }

            if(!SetSecurityDescriptorSacl(
                &sd,    // A pointer to the SECURITY_DESCRIPTOR structure to which the function adds the SACL
                FALSE,  // the security descriptor does not contain a SACL
                NULL,   // security descriptor has a NULL SACL
                FALSE   // A pointer to a flag that is set to the value of the SE_SACL_DEFAULTED flag in the SECURITY_DESCRIPTOR_CONTROL structure if a SACL exists for the security descriptor
            ))
            {
                DWORD dwLastError = ::GetLastError();
                LOG4CPLUS_ERROR(m_Logger, "Cannot launch file '" << strFullFilePath << "'.  Failed to set security descriptor SACL.  GetLastError=" << dwLastError);
                return dwLastError;
            }

            sa.nLength = sizeof(SECURITY_ATTRIBUTES);
            sa.lpSecurityDescriptor = &sd;
            sa.bInheritHandle = TRUE;

            DWORD dwShareMode = FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE;
//          DWORD dwFlagsAndAttributes = FILE_ATTRIBUTE_NORMAL;
            DWORD dwFlagsAndAttributes = FILE_ATTRIBUTE_TEMPORARY | FILE_FLAG_DELETE_ON_CLOSE;
            HANDLE hFile = ::CreateFile(strTempFile.c_str(), GENERIC_WRITE, dwShareMode, &sa, CREATE_NEW, dwFlagsAndAttributes, NULL);

            //verify we created the file.
            if(hFile == INVALID_HANDLE_VALUE) {
                DWORD dwLastError = ::GetLastError();
                return dwLastError;
            }

            //copy to temp
            char buffer[64*1024];
            size_t nBytesRead = hEncryptedFile.Read(buffer, sizeof(buffer));
            while(nBytesRead) {
                DWORD numBytesWritten;
                if(!::WriteFile(hFile, buffer, nBytesRead, &numBytesWritten, (LPOVERLAPPED) NULL)) {
                    DWORD dwLastError = ::GetLastError();
                    LOG4CPLUS_ERROR(m_Logger, "Failed to write file to %TEMP% folder.  GetLastError=" << dwLastError);
                    return dwLastError;
                }
                nBytesRead = hEncryptedFile.Read(buffer, sizeof(buffer));
            }
            hEncryptedFile.Close();

            //execute the file from temp.
            LaunchFile(strTempFile);
        }
    }
    return 0;
}

I think if I determine the correct SECURITY_DESCRIPTOR to pass to ::CreateFile it may work like I want it to. Please help.

btw, the LaunchFile function just ends up calling ::ShellExecute to launch the file.

Radu

After re-reading the msdn doc, I fear I have answered my own question. FILE_FLAG_DELETE_ON_CLOSE The file is to be deleted immediately after all of its handles are closed, which includes the specified handle and any other open or duplicated handles. If there are existing open handles to a file, the call fails unless they were all opened with the FILE_SHARE_DELETE share mode. Subsequent open requests for the file fail, unless the FILE_SHARE_DELETE share mode is specified. In my case, I doubt notepad is requesting the FILE_SHARE_DELETE permission, so it can't open the file. – Radu

Yup... you might want to grab a copy of FileMon or whatever the latest replacement is from technet.microsoft.com/en-us/sysinternals/bb896642.aspx. This will show you the flags that notepad is using when it opens the file. – Extrajudicial

I wrote a sample program that attempt to open the temp file using FILE_SHARE_DELETE and confirmed that is indeed the problem. Unless the other process requested FILE_SHARE_DELETE permission, it will get a access violation. So now I need to think of other ways to implement this. Any ideas? lol. – Radu

ah yeah ... I may look at filemon too, just to further confirm it. – Radu

Omit FILE_FLAG_DELETE_ON_CLOSE when creating the file, then use ShellExecuteEx() instead of ShellExecute() so you can get a HANDLE to the spawned process. Your app can then pass that HANDLE to any of the WaitFor...() functions to wait for Notepad (or whatever) to exit, before then calling DeleteFile(). – Unset

Good idea, but that would require my app still be running for the file to get deleted. I'm starting to think there's no way around that though. But if my app is going to manually delete them anyway, I will just try to delete each file on exit, and if it fails then too bad. Maybe I could set a registry key for the files I couldn't delete that will tell windows to delete it at next start up? – Radu

I have found that, if the child process doesn't close its own handle to the file before it exits, Windows might not close that handle until seconds after WinAPI reports that that process has exited. (This user reports the same: https://mcmap.net/q/1624624/-for-how-long-can-a-file-be-locked-in-windows-after-program-is-closed .) The only solution I've thought of (which will not help in all of your cases) is to wait in a loop after the child process has exited, repeatedly calling CreateFile with FILE_FLAG_DELETE_ON_CLOSE until it returns success, then closing the handle. :-( – Yehudit

Answer 1

After re-reading the msdn doc, I fear I have answered my own question. FILE_FLAG_DELETE_ON_CLOSE The file is to be deleted immediately after all of its handles are closed, which includes the specified handle and any other open or duplicated handles. If there are existing open handles to a file, the call fails unless they were all opened with the FILE_SHARE_DELETE share mode. Subsequent open requests for the file fail, unless the FILE_SHARE_DELETE share mode is specified. In my case, I doubt notepad is requesting the FILE_SHARE_DELETE permission, so it can't open the file

Radu

As an aside, you should NOT be specifying FILE_ATTRIBUTE_TEMPORARY! That is only for a file which you want the OS to avoid writing to disk at all (if possible). This is not the case in your scenario! – Salvation

@Mordachai: He very much wants to avoid having the decrypted data stored to disk, where it could be recovered by forensics. To be used by another application, it has to be part of the filesystem and part of the disk cache in RAM. But it does not need to be written to permanent storage. – Transit