Haskell concurrency over kqueue

Question

I wrote concurrent application and have caught the error:

buildFdSets: file descriptor out of range

I found out that it is the OS limit on the number of file descriptors in one process, in my FreeBSD it is 1024. It is the limit of select(). Also I have learned that there is another approach: kqueue().

My questions are:

• How to win limit on file descriptors?
• How to use kqueue() instead of select() in haskell programs?

Dakotadal

Answer 1

I believe that GHC 7 now has support for using kqueue() in it's back end:

link to paper, descriptive blog post

However the Haskell Platform is not yet out for GHC 7.

Alvie

Answer 2

You can simply upgrade to GHC 7, which is part of the Haskell Platform, which includes:

On POSIX platforms, there is a new I/O manager based on epoll/kqueue/poll, which allows multithreaded I/O code to scale to a much larger number (100k+) of threads

In particular:

Architecturally, our new I/O manager consists of two components. Our event notification library provides a clean and portable API, and abstracts the system-level mechanisms used to provide efficient event notifications (kqueue, epoll, and poll). We have also written a shim that implements the semi-public threadWaitRead and threadWaitWrite interfaces. This means that neither the core file or networking libraries, nor other low-level I/O libraries, require any changes to work with—and transparently benefit from the performance improvements of—our new code.

That is, just upgrade, and things work magically better.

Here's some reading material:

• Scalable Event Handling for GHC
• Scalable timeout support for GHC's I/O manager
• Playing with the new Haskell epoll event library

Idyll