Website architecture design requiring real-time polling from external servers

Question

I have a game running in N ec2 servers, each with its own players inside (lets assume it a self-contained game inside each server).

What is the best way to develop a frontend for this game allowing me to have near real-time information on all the players on all servers.

My initial approach was:

• Have a common-purpose shared hosting php website polling data from each server (1 socket for each server). Because most shared solutions don't really offer permanent sockets, this would require me to create and process a connection each 5 seconds or so. Because there isn't cronjob with that granularity, I would end up using the requests of one unfortunate client to make this update. There's so many wrong's here, lets consider this the worst case scenario.

• The best scenario (i guess) would be to create small ec2 instance with some python/ruby/php web based frontend, with a server application designed just for polling and saving the data from the servers on the website database. Although this should work fine, I was looking for some solution where I don't need to spend that much money (even a micro instance is expensive for such pet project).

What's the best and cheap solution for this?

Pinson

If you want to poll more frequently than is practical from cron, why not write a small daemon to do it? – Deviation

Frank, that's my best scenario making this a more expensive design because I would require a server allowing daemons to run (VPS/Ec2) – Pinson

Answer 1

Is there a reason you can't have one server poll the others, stash the results in a json file, then push that file to the web server in question? The clients could then use ajax to update the listings in near real time pretty easily.

If you don't control the game servers I'd pass the work on updating the json off to one of the random client requests. it's not as bad as you think though.

Consider the following:

1. Deliver (now expired) data to client, including timestamp
2. call flush(); (test to make sure the page is fully rendered, you may need to send whitespace or something to fill the buffer depending on how the webserver is configured. appending flush(); sleep(4); echo "hi"; to a php script should be an easy way to test.
3. call ignore user abort (http://php.net/manual/en/function.ignore-user-abort.php) so your client will continue execution regardless of what the user does
4. poll all the servers, update your file
5. Client waits a suitable amount of time before attempting to update the updated stats via AJAX.

Yes that client does end up with the request taking a long time, but it doesn't affect their page load, so they might not even notice.

Noncooperation

Answer 2

You don't provide the information needed to make a decision on this. It depends on the number of players, number of servers, number of games, communication between players, amount of memory and cpu needed per game/player, delay and transfer rate of the communications channels, geographical distribution of your players, update rate needed, allowed movement of the players, mutual visibility. A database should not initially be part of the solution, as it only adds extra delay and complexity. Make it work real-time first.

Really cheap would be to use netnews for this.

Varityper