Which green threads libraries are available for C that can match the performance and ease of use of Haskell's green threads? [closed]

Question

I am well used to relying on GHC's forkIO for portable lightweight threads when programming in Haskell.

What are equivalent libraries for C that can provide the same scalibility and ease of use?

Specifically I need C-equivalents of at least the following two functions.

forkIO     :: IO () -> IO ThreadId
killThread ::             ThreadId -> IO ()

I assume for my application, it would be enough if threads only switched on blocking operations rather than being forcefully suspended because all threads block highly frequently for network IO and I only use the splice system call to ask the Linux kernel to push data around between sockets.

---

Update

This paper has figures and tables comparing

• GNU Pth
• Protothreads
• PM2 Parcel

with results favoring Protothreads. As I have not used any and there may also be other libraries, I would love to hear from anyone who has used / developed such libraries.

Nata

Dare I say foreign export of libHSrts.a ... ? (i.e. just use the GHC runtime, which is a C library, after all). That said, the GHC runtime is 50k lines of C, and uses epoll for thread scheduling. You'd need an epoll wrapper. – Penitence

@DonStewart o__O wow ... what an unexpected answer to hear! You bring Haskell to my C :). The problem is libHSrts.a on Debian/MIPS(le) is 590KB! I was hoping the answers here might let me go embedded. And how could I foreign export the two forkIO and killThread as C functions? – Nata

As to you last question: write correct function prototypes and link with -lHSrts. – Overpass

Questions that ask "what is the best" are not appropriate here; they ask for discussion and speculation, and the FAQ specifically mentions them as being a poor fit here. Can you rephrase it so it doesn't ask for opinion and speculation? – Abruzzi

@KenWhite To be fair, the OP does point out criteria, or at least an entrance threshold: the performance and ease of use of Haskell's green threads. He proceeds to provide references to specific Haskell functions whose C equivalents are needed. Phrasing might be suboptimal, but the question as a whole is clearly above the "should I use Python or Ruby for my next web project" type speculation that is unwanted on SO. – Filigree

@user4815162342: I understand that, and didn't vote to close the question because of that; however, the subject (title) of the question asks "What is the best", which is a violation of the guidelines, and that's why I asked if it could be rephrased. :-) It could be as simple as replacing "What is the best" with "Is there a", for instance. – Abruzzi

I usually use libevent when I want this. – Chervonets

@KenWhite There's a small link labelled "edit" under the tags you can use to change the title as you suggest - probably the simplest solution to your concern. – Lens

@KenWhite I have rephrased the title as per your request. – Nata

@AndrewC: Um... Thanks. I'm quite aware of that link. I thought it might be better for the poster to phrase it more appropriately (and serve as useful information for future questions). (I've edited one or two questions before - I'm not exactly new here. <g>) – Abruzzi

@CetinSert: Much better. Thanks. :-) – Abruzzi

Take a look at #765868 swtch.com/libtask – Hippodrome

Answer 1

libMill is probably what you're searching for: http://libmill.org/

It implements user level threads in the Go-Lang channel style.

And it's being developed by the super smart Martin Sústrik, creator of ZeroMQ http://250bpm.com/. So it must be good ☺

Kilimanjaro

Answer 2

I no longer have the commenting for the following code, nor any examples - this is a portable ( pseudo ) thread library implemented as pre processor macros

     typedef struct
     {
     unsigned int magic;
     unsigned short ctx;
     unsigned char is_destroyed;
     }
     _run;

     typedef struct
     {
     unsigned int magic;
     unsigned int cnt;
     }
     _sem;


     #define aa_RUNNER_WAITING             0
     #define aa_RUNNER_YIELDED             1
     #define aa_RUNNER_EXITED              2
     #define aa_RUNNER_ENDED               3

     #define aaRunnerCreate(rp)            (rp)->magic='runr'; (rp)->ctx=0; (rp)->is_destroyed=NO
     #define aaRunnerDestroy(rp)           (rp)->is_destroyed=YES

     #define aaRunnerThread(args)          C args
     #define aaRunnerBegin(rp)             { C yflag=YES; if(yflag) {}  switch((rp)->ctx) { case 0:
     #define aaRunnerEnd(rp)               } yflag=NO; if(yflag) {}  aaRunnerCreate(rp); return aa_RUNNER_ENDED; }

     #define aaRunnerWaitUntil(rp,condx)   do  { (rp)->ctx=__LINE__; case __LINE__: if(!(condx))  { return aa_RUNNER_WAITING;  }  } while(0)
     #define aaRunnerWaitWhile(rp,condi)   aaRunnerWaitUntil((rp),!(condi))
     #define aaRunnerWaitThread(rp,thr)    aaRunnerWaitWhile((rp),aaRunnerSchedule(thr))
     #define aaRunnerWaitSpawn(rp,chl,thr) do { aaRunnerCreate((chl));  aaRunnerWaitThread((rp),(thr)); } while(0)

     #define aaRunnerRestart(rp)           do { aaRunnerCreate(rp); return aa_RUNNER_WAITING; } while(0)
     #define aaRunnerExit(rp)              do { aaRunnerCreate(rp); (rp)->magic=0; return aa_RUNNER_EXITED;  } while(0)

     #define aaRunnerSchedule(f)           ((f)<aa_RUNNER_EXITED)
     #define aaRunnerYield(rp)             do { yflag=NO; (rp)->ctx=__LINE__; case __LINE__: if(!yflag||!((rp)->is_destroyed))  { return aa_RUNNER_YIELDED;  }  } while(0)
     #define aaRunnerYieldUntil(rp,condi)  do { yflag=NO; (rp)->ctx=__LINE__; case __LINE__: if(!yflag||!(condi)) { return aa_RUNNER_YIELDED;   }   } while(0)

     #define aaRunnerSemInit(sp,c)         (sp)->magic='runs'; (sp)->cnt=c
     #define aaRunnerSemWait(rp,sp)        do { aaRunnerWaitUntil(rp,(sp)->cnt>0); --(sp)->cnt;  } while(0)
     #define aaRunnerSemSignal(rp,sp)      ++(sp)->cnt

Cenesthesia

Answer 3

Use POSIX threads. They are "green" on any modern implementation, not in the sense of "Green Threads", but in the sense of being lightweight and efficient. There is no portable way to roll your own threads on top of plain C or POSIX minus threads. As OP mentioned, there are some libraries implementing green-threads/co-routines in non-portable ways (often despite claiming portability).

The closest-to-portable approach is using makecontext/swapcontext, and unfortunately this cannot perform well because it has to make syscalls to save/restore the signal mask on each switch between "threads". This makes switching between "green" threads more expensive than a context switch between kernel-level threads on a "real" POSIX threads implementation, and basically negates any claimed benefit of "green threads".

Non-portable approaches that don't care about the signal mask could use machine-specific asm to do the context switches entirely in userspace, and in theory perform better than kernel-level threads, but performance would again fly out the window as soon as you introduce IO, since a thread that's about to perform IO would have to first do expensive tests to check if the operation will block, and if so, turn over control to a different thread.

I maintain my position that "green threads" are an idea whose time is long past. This also seems to be the position of the Austin Group (responsible for POSIX), who removed the ucontext functions in POSIX 2008 and recommended replacement with POSIX threads (which are now a mandatory feature).

Extractive

POSIX threads are decidedly not green threads as that term is generally understood. Linux implementation of POSIX threads was always creating OS-level threads in 1-to-1 correspondence with POSIX ones, and Solaris, FreeBSD, and NetBSD all switched to the 1-to-1 model at some point in their history. – Filigree

Of course. But the term "green threads" is misleading and obsolete. Especially with modern machines being multicore... – Extractive

en.wikipedia.org/wiki/Green_threads – Nata

... which tells you Haskell threads can be distributed across cores, so not obsolete for the reasons you assumed, @R.. , and certainly considerably more lightweight than OS threads. – Lens

Even if the green thread concept is obsolete (which is debatable), the term is well-defined, and the claim that "[POSIX threads] are 'green' on any modern implementation" is false. As I wrote, all modern implementation of POSIX threads are 1-to-1 mappings to OS LWPs, which is the very opposite of green threads. – Filigree

I have edited my answer to be more clear, avoid misleading use of "green", and to have reasonable justification behind the "inflammatory" parts. – Extractive

Can I launch 1M Posix threads on a reasonable laptop? – Penitence

As long as you first set the stack size attr I don't see why not, assuming 64 bit. On 32 bit, addr space is too small for 1M threads, green or not. – Extractive

@R.. hpaste.org/80868 creates 1M threads. With a 256byte thread stack, it allocates 300M total and completes in 2.7s. I would love to see the equivalent scaling with Posix threads. – Penitence

I don't see anywhere a stack size of 256 bytes is explicitly requested, so from what I can tell, (1) there's no guarantee they'll actually be that small, and (2) making them that small depends on powerful static analysis by the implementation. In C, I don't think there's any way you can expect a 256-byte stack to work, even if you could request one. You certainly can't expect to be able to call any function in the standard library, and it's doubtful you can reliably tell the compiler not to use more than that. (On archs with redzone, it's definitely impossible.) – Extractive

@R.. If you're really saying that Haskell's thread performance is genuinely impossible to replicate in C, it would be better to edit your question to emphasise this and your reasoning. Currently it still sounds a bit like a rant against the concept of green threads. (It's much better than draft 1 already.) – Lens

(256 bytes thread stack isn't explicitly requested in the source code because it's a runtime system option. You don't do explicit memory management in Haskell code.) – Lens

Yes, I'm saying it's impossible to replicate in portable C, in the sense of either plain C or C+POSIX portability, without just writing a virtual machine for it. The non-portability is extremely difficult to overcome if you want to have millions of threads, since it's impossible to get a reliable tiny bound on stack usage; the compiler and the ABI rules (e.g. redzone) have too much freedom to do things that would render false your assumptions that a tiny stack would work. You also have no way of knowing how much stack standard functions need. – Extractive

Really? Your contention is simultaneously 1) that green threads are not a useful idea, and that 2) the alternative you suggest can't hope to compete with established implementations of green threads? That's fascinating. You really should consider learning more about concurrency before posting authoritative-sounding answers. – Medawar

It's that it can't compete with a nonsensical benchmark. Try actually doing something useful with the threads... – Extractive

+1: I'm not sure all these downvotes are deserved. At least it contains very valuable technical details (if you add the comments) I hadn't heard of to make me rethink my problem. – Nela