CreateFile() returns INVALID_HANDLE_VALUE but GetLastError() is ERROR_SUCCESS

Question

I am opening a serial port using CreateFile(). I've got a testcase (too complicated to redistribute) that consistently causes CreateFile() to return INVALID_HANDLE_VALUE and GetLastError() to return ERROR_SUCCESS. By the looks of it, this bug only occurs if one thread opens the port at the exact same time that another port closes it. The thread opening the port runs across this problem.

I don't know if this makes a difference, but later on in the code I associate the port with a CompletionPort using CreateIoCompletionPort.

Here is my code:

HANDLE port = CreateFile(L"\\\\.\\COM1",
                         GENERIC_READ | GENERIC_WRITE,
                         0,                    // must be opened with exclusive-access
                         0,                    // default security attributes
                         OPEN_EXISTING,        // must use OPEN_EXISTING
                         FILE_FLAG_OVERLAPPED, // overlapped I/O
                         0);                   // hTemplate must be NULL for comm devices
if (port == INVALID_HANDLE_VALUE)
{
    DWORD errorCode = GetLastError();
    cerr << L"CreateFile() failed with error: " << errorCode << endl;
}

I'm pretty sure this sort of thing should not happen. Am I doing anything wrong? How do I get the API to return a correct result?

---

MORE DETAILS: This code is taken from a serial-port library I've developed: JPeripheral

Here is the actual (unsanitized) source-code:

JLong SerialChannel::nativeOpen(String name)
{
    cerr << "nativeOpen(" << name << ")" << endl;
    wstring nameWstring = name;
    HANDLE port = CreateFile((L"\\\\.\\" + nameWstring).c_str(),
        GENERIC_READ | GENERIC_WRITE,
        0,                                          // must be opened with exclusive-access
        0,                                          // default security attributes
        OPEN_EXISTING,                  // must use OPEN_EXISTING
        FILE_FLAG_OVERLAPPED,       // overlapped I/O
        0);                                         // hTemplate must be NULL for comm devices
    cerr << "nativeOpen.afterCreateFile(" << name << ")" << endl;
    cerr << "port: " << port << ", errorCode: " << GetLastError() << endl;
    if (port == INVALID_HANDLE_VALUE)
    {
        DWORD errorCode = GetLastError();

        switch (errorCode)
        {
            case ERROR_FILE_NOT_FOUND:
                throw PeripheralNotFoundException(jace::java_new<PeripheralNotFoundException>(name, Throwable()));
            case ERROR_ACCESS_DENIED:
            case ERROR_SHARING_VIOLATION:
                throw PeripheralInUseException(jace::java_new<PeripheralInUseException>(name, Throwable()));
            default:
            {
                throw IOException(jace::java_new<IOException>(L"CreateFile() failed with error: " +
                    getErrorMessage(GetLastError())));
            }
        }
    }

    // Associate the file handle with the existing completion port
    HANDLE completionPort = CreateIoCompletionPort(port, ::jperipheral::worker->completionPort, Task::COMPLETION, 0);
    if (completionPort==0)
    {
        throw AssertionError(jace::java_new<AssertionError>(L"CreateIoCompletionPort() failed with error: " +
            getErrorMessage(GetLastError())));
    }
    cerr << "nativeOpen.afterCompletionPort(" << name << ")" << endl;

    // Bind the native serial port to Java serial port
    SerialPortContext* result = new SerialPortContext(port);
    cerr << "nativeOpen.afterContext(" << name << ")" << endl;
    return reinterpret_cast<intptr_t>(result);
}

Here is the actual output I get:

nativeOpen(COM1)
nativeOpen.afterCreateFile(COM1)
port: 00000374, errorCode: 0
nativeOpen.afterCompletionPort(COM1)
nativeOpen.afterContext(COM1)
[...]
nativeOpen(COM1)
nativeOpen.afterCreateFile(COM1)
port: FFFFFFFF, errorCode: 0
java.io.IOException: CreateFile() failed with error: The operation completed successfully.

Alb

Are these lines the exact ones you're using in your test? Any simplifications (even apparently harmless ones) could hide the source of the problem. – Fulllength

What is the hardware you're accessing? – Endaendall

@Gabe: I'm accessing a embedded device we've developed in-house. It has a standard DB9 serial-port connection which I've connected to my PC (no USB-RS232 adapters here). – Alb

@David: I've posted the full source-code for your review. – Alb

You have to call GetLastError immediately after CreateFile faliure, without << in between. – Addressograph

@Gili: Yep, your simplification hid the problem. You're calling 'GetLastError' after calling '<<'. You need to call 'GetLastError' after calling 'CreateFile'. – Fulllength

@David and Roman, thanks for the head's up! – Alb

Answer 1

 HANDLE port = CreateFile(...);
 cerr << "nativeOpen.afterCreateFile(" << name << ")" << endl;
 cerr << "port: " << port << ", errorCode: " << GetLastError() << endl;
 if (port == INVALID_HANDLE_VALUE)
 {
    DWORD errorCode = GetLastError();

The output to cerr invokes winapi calls under the hood. Which will reset the thread error value returned by GetLastError(). Fix:

 HANDLE port = CreateFile(...);
 int err = GetLastError();
 // etc, use err instead...

Moynihan

I'm not convinced. The original code that had this problem did not contain any cerr instruction. I've only added that recently. Do you see anything wrong with the above code (minus cerr)? – Alb

The output trace you showed can only be generated by the code that does mess up the GetLastError value. I don't doubt it can fail, there are many reasons for opening a COM port to fail. – Moynihan

You were right. The original code invoked throw IOException(jace::java_new<IOException>(L"CreateFile() failed with error: " + getErrorMessage(GetLastError())). Notice how much code exists between the point of failure and the reading of GetLastError(). Reading the value sooner fixed the problem. Thanks! – Alb

I had something similar happen to me, even though the only thing between CreateFile2 and GetLastError() was "new wchar_t[messageLength]". That was sufficient to reset the error message returned by GetLastError(). – Pappas