fast user space inter-process notify approach on Linux?

Question

We have two Linux process communicate with domain socket, the performance goal is 5k iops with 4k request size through single domain socket connection, in order to reduce cpu cost we replace domain socket with a io-ring(shared memory based), but the critical "notify" step is still implemented by the domain socket, and seems the overhead is still too much.

PS: Currently io depth is usually 2-4, which indicates 1k iops for each "notify"

The semaphore, pthread_condvars, futex based locks, eventfd, inotify, which one do you prefer(or anyone else), I cares about performance and overhead very much. Thanks!

Boarer

While this question is probably quite broad in asking for the 'best' approach, I think it's specific enough (IPC, notify, Linux, comparison of listed mechanisms) to prevent closure. – Nonprofessional

Do you need to send each notify immediately? There is well known approach (used e.g. in interrupt generation by external devices) to suppress notify generation for a while after the previous one, so full rate doesn't exceed some maximal value. You can use e.g. 10 msec minimum interval which gives no more than 100 notifies per second but still provides reaction in 10 msecs after previous portion... – Tablespoon

well, latency is a primary goal, 10msecs is too big. – Boarer

Answer 1

You can use linux signals

Look at SIGALRM or you can use SIGUSR1

Vermis

Have you actually read the question? While signals are a form of inter-process notification... you've provided no benchmark data, no reason why the OP might consider this approach over others (where OP's primary concerns are bulk throughput as well as latency). – Nonprofessional