When using IOCP, should I set WSAOVERLAPPED's hEvent to NULL or to a valid handle to a WSAEVENT object?

Question

According to MSDN:

hEvent: If an overlapped I/O operation is issued without an I/O completion routine (the operation's lpCompletionRoutine parameter is set to null), then this parameter should either contain a valid handle to a WSAEVENT object or be null.

As I'm using IOCP, when I call WSASend() or WSARecv() I pass NULL to their last parameter (i.e., lpCompletionRoutine):

WSASend(pIoRequest->GetSocket(), pIoRequest->GetWsaBuffer(), 1, NULL, pIoRequest->GetFlags(), pIoRequest, NULL);

WSARecv(pIoRequest->GetSocket(), pIoRequest->GetWsaBuffer(), 1, NULL, &(pIoRequest->GetFlags()), pIoRequest, NULL);

My "per I/O data" class (pIoRequest) looks something like:

class IoRequest : public WSAOVERLAPPED
{
public:
    IoRequest()
    {
        ...
        SecureZeroMemory(this, sizeof(WSAOVERLAPPED));
        hEvent = WSACreateEvent(); // A
    }
    ...
    void ResetForNextIoRequest()
    {
        WSACloseEvent(hEvent); // B
        SecureZeroMemory(this, sizeof(WSAOVERLAPPED));
        hEvent = WSACreateEvent(); // C
        ...
    }
    ...
    DWORD& GetFlags() { return m_dwFlags; }
    ...
private:
    ...
    DWORD m_dwFlags;
    ...
};

It doesn’t seem to make any difference to my program’s behaviour even if I comment out the lines marked A, B and C above.

So how do you decide when to call WSACreateEvent() or simply set hEvent to NULL?

Laudatory

Answer 1

If you're using IOCP, you don't need to pass event objects, because you'll be receiving completion notifications using GetQueuedCompletionStatus(). This will work assuming you have associated the socket with the completion port using CreateIoCompletionPort().

Yes, I/O on Windows is confusing. In particular, there are at least six different ways to use sockets. First the two you seem to have come across:

• Overlapped I/O using IOCP (CreateIoCompletionPort, GetQueuedCompletionStatus, WSASend etc.). This is probably the most efficient method. You can easily integrate into the event loop any kind of event which also use IOCP. For other events, you may be able to workaround using PostQueuedCompletionStatus. This is (AFAIK) the only method which scales for large numbers of sockets.

• Overlapped I/O without IOCP, that is using WSASend and friends with event objects, monitoring the events using e.g. WaitForMultipleObjects, and obtaining results using WSAGetOverlappedResult. This is relatively easy to integrate with any non-socket I/O which can also be mapped to HANDLE objects. However, WaitForMultipleObjects it is limited to monitoring no more than 64 handles at a time.

And also at least four more:

• Blocking calls (send, recv, and also the WSA* versions). If you do this, you're forced to use threads. This is both hard to implement correctly and likely inefficient.

• Non-blocking sockets using select(). This has the benefit that you can use similar code as on unix-like systems. However, it (AFAIK) cannot be integrated with I/O other than sockets.

• Non-blocking using WSAEventSelect. This is similar to the select() method, except that instead of using select() to get notifications, you map socket events to event objects, and monitor those using e.g. WaitForMultipleObjects. It is also similar to the overlapped without IOCP method, and suffers from the same limit of no more than 64 objects.

• Non-blocking using WSAAsyncSelect. This delivers socket notifications as messages to a window within a program using using the Windows message loop. This is easy to integrate into an application already using the message loop, such as many GUI applications.

Correct me if I left something out or if some of this doesn't actually work :).

Sleazy

Ambroz-Thanks for clarifying. May I ask one more question? Does it make sense to have "DWORD m_dwFlags" in "IoRequest" (see edited code above)? Can I instead create a stack based DWRD dwFlags in my worker thread and pass it to WSASend() and WSARecv()? – Laudatory

It seems that you can have it on the stack and let it go out of scope as the operation is in progress. From WSARecv page: "In this case, lpNumberOfBytesRecvd and lpFlags are not updated". WSASend does not have this issue, as it just takes a value and not a pointer. – Sleazy