Thread per connection vs Reactor pattern (with a thread pool)?

Question

I want to write a simple multiplayer game as part of my C++ learning project.

So I thought, since I am at it, I would like to do it properly, as opposed to just getting-it-done.

If I understood correctly: Apache uses a Thread-per-connection architecture, while nginx uses an event-loop and then dedicates a worker [x] for the incoming connection. I guess nginx is wiser, since it supports a higher concurrency level. Right?

I have also come across this clever analogy, but I am not sure if it could be applied to my situation. The analogy also seems to be very idealist. I have rarely seen my computer run at 100% CPU (even with a umptillion Chrome tabs open, Photoshop and what-not running simultaneously)

Also, I have come across a SO post (somehow it vanished from my history) where a user asked how many threads they should use, and one of the answers was that it's perfectly acceptable to have around 700, even up to 10,000 threads. This question was related to JVM, though.

So, let's estimate a fictional user-base of around 5,000 users. Which approach should would be the "most concurrent" one?

1. A reactor pattern running everything in a single thread.
2. A reactor pattern with a thread-pool (approximately, how big do you suggest the thread pool should be?
3. Creating a thread per connection and then destroying the thread the connection closes.

I admit option 2 sounds like the best solution to me, but I am very green in all of this, so I might be a bit naive and missing some obvious flaw. Also, it sounds like it could be fairly difficult to implement.

PS: I am considering using POCO C++ Libraries. Suggesting any alternative libraries (like boost) is fine with me. However, many say POCO's library is very clean and easy to understand. So, I would preferably use that one, so I can learn about the hows of what I'm using.

Eunuchize

Answer 1

Reactive Applications certainly scale better, when they are written correctly. This means

• Never blocking in a reactive thread:
  • Any blocking will seriously degrade the performance of you server, you typically use a small number of reactive threads, so blocking can also quickly cause deadlock.
  • No mutexs since these can block, so no shared mutable state. If you require shared state you will have to wrap it with an actor or similar so only one thread has access to the state.
• All work in the reactive threads should be cpu bound
  • All IO has to be asynchronous or be performed in a different thread pool and the results feed back into the reactor.
  • This means using either futures or callbacks to process replies, this style of code can quickly become unmaintainable if you are not used to it and disciplined.
• All work in the reactive threads should be small
  • To maintain responsiveness of the server all tasks in the reactor must be small (bounded by time)
  • On an 8 core machine you cannot cannot allow 8 long tasks arrive at the same time because no other work will start until they are complete
  • If a tasks could take a long time it must be broken up (cooperative multitasking)

Tasks in reactive applications are scheduled by the application not the operating system, that is why they can be faster and use less memory. When you write a Reactive application you are saying that you know the problem domain so well that you can organise and schedule this type of work better than the operating system can schedule threads doing the same work in a blocking fashion.

I am a big fan of reactive architectures but they come with costs. I am not sure I would write my first c++ application as reactive, I normally try to learn one thing at a time.

If you decide to use a reactive architecture use a good framework that will help you design and structure your code or you will end up with spaghetti. Things to look for are:

• What is the unit of work?
• How easy is it to add new work? can it only come in from an external event (eg network request)
• How easy is it to break work up into smaller chunks?
• How easy is it to process the results of this work?
• How easy is it to move blocking code to another thread pool and still process the results?

I cannot recommend a C++ library for this, I now do my server development in Scala and Akka which provide all of this with an excellent composable futures library to keep the code clean.

Best of luck learning C++ and with which ever choice you make.

Huckleberry

If you are working on network application, ZMQ has its own reactor wrapper. A reference about CZMQ, which is a wrapper of ZMQ That could be one of your options. – Delorisdelorme

This answer is "best practices in reactor design pattern" The question is "reactor vs thread pool" – Rusk

@Rusk If you want to be pedantic the question is which would be the most concurrent 1) single threaded reactor 2) reactor with thread pool 3) thread per connection. The answer to this is very much depends on your workload, is there blocking IO? do requests have to coordinate access to state? is the state mutating? I clearly say reactive scales better when written correctly. So the answer is 1 or 2. Then give the asker enough information to decide if they between option 1 or 2 based on their workload. Also as important I help them decide if they could implement one correctly. – Huckleberry

Answer 2

Option 2 will most efficiently occupy your hardware. Here is the classic article, ten years old but still good.

http://www.kegel.com/c10k.html

The best library combination these days for structuring an application with concurrency and asynchronous waiting is Boost Thread plus Boost ASIO. You could also try a C++11 std thread library, and std mutex (but Boost ASIO is better than mutexes in a lot of cases, just always callback to the same thread and you don't need protected regions). Stay away from std future, cause it's broken:

http://bartoszmilewski.com/2009/03/03/broken-promises-c0x-futures/

The optimal number of threads in the thread pool is one thread per CPU core. 8 cores -> 8 threads. Plus maybe a few extra, if you think it's possible that your threadpool threads might call blocking operations sometimes.

Manners

std future is not broken!! C++11 is just lacking a feature to easily compose them. In other words, std future is just fine, but using them thoroughly and effectively might, in some cases, not be as easy as it should be. That said, +1 regardless :P – Maidstone

Answer 3

FWIW, Poco supports option 2 (ParallelReactor) since version 1.5.1

Succinct

Answer 4

I think that option 2 is the best one. As for tuning of the pool size, I think the pool should be adaptive. It should be able to spawn more threads (with some high hard limit) and remove excessive threads in times of low activity.

Rake

Answer 5

as the analogy you linked to (and it's comments) suggest. this is somewhat application dependent. now what you are building here is a game server. let's analyze that.

game servers (generally) do a lot of I/O and relatively few calculations, so they are far from 100% CPU applications. on the other hand they also usually change values in some database (a "game world" model). all players create reads and writes to this database. which is exactly the intersection problem in the analogy.

so while you may gain some from handling the I/O in separate threads, you will also lose from having separate threads accessing the same database and waiting for its locks.

so either option 1 or 2 are acceptable in your situation. for scalability reasons I would not recommend option 3.

Almanza