Is there any way Load balancing on a preforked multi process TCP server build by asyncio?

Thaks for guys in previous answer, now I can build a multi process TCP server with each process running a asynchronous server respectively but all binding to one port. (Could not use os.fork() bind several process to one socket server when using asyncio)

Theoretically ? this model would achieve its best performance when every process handling incoming message equally. Benefits may like lower latency or higher tps ? I'm not sure.

Here's the problem . I created a four process server ,and did statistics of how many tcp request would each process accept (by a while looped client who continuously making new connection request) .Result is like {p1: 20000 times , p2: 16000 times ,p3: 13000 times ,p4: 10000 times} <-- this. Probably not some kind of good result.

I'm figuring out if a lock would help (let the process who gets the lock accept the request ,instead of let processes competitively accepting request directly) .But it turns out in this way only the Parent process could get the lock while the other couldn't at all.

Trying to figure out solution ,need your help.

Here's a naive , sample server code (preforked model where processes competitively accepting request directly):

# sample_server.py
import asyncio
import os
from socket import *

def create_server():
    sock = socket(AF_INET , SOCK_STREAM)
    sock.setsockopt(SOL_SOCKET , SO_REUSEADDR ,1)
    sock.bind(('',25000))
    sock.listen()
    sock.setblocking(False)
    return sock

async def start_serving(loop , server):
    while True:
        client ,addr = await loop.sock_accept(server)
        loop.create_task(loop ,client)

async def handler(loop ,client):
    with client:
        while True:
            data = await loop.sock_recv(client , 64)
            if not data: break
            print(f"Incoming message {data} at pid {pid}")
            await loop.sock_sendall(client , data)

server = create_server()

for i in range(4 - 1):
    pid = os.fork()
    if pid <= 0:
        break
pid = os.getpid()

loop = asyncio.get_event_loop()
loop.create_task(start_serving(loop , server))
loop.run_forever()

Then we may redirect its output into a file like this:

python3 sample_server.py > sample_server.output

Next step maybe we roughly deal with these data:

import re
from collections import Counter

with open('./sample_server.output','r') as f:
    cont = file.read()

pat = re.compile('[\d]{4}')
res = pat.findall(cont)
print(Counter(res))

Get output like this (where key means port num while value means how many echos they handled):

Counter({'3788': 23136, '3789': 18866, '3791': 18263, '3790': 10817})

Inequalitiy.

Things went even more worse when I introduced multiprocessing Lock like this:

from multiprocessing import Lock
l = Lock()

async def start_serving(loop , server):
    while True:
        with l:
            client ,addr = await loop.sock_accept(server)
        loop.create_task(loop ,client)

↑ Then the only process can accept request is the parent process. while child process were totally blocked. Seems like if you acquired a lock before process was blocked ,then it will always behave like this . The interpreter is just faithfully doing what we told it to do.

In summary here's my two question:

1\ if there's any method let this preforked asynchronous server load balance ?
2\ Is there any way to introduce a lock help solving this problem?

Thanks!

PS: if anyone could tell me how to use uvloop drive eventloop in the interpreter of pypy? big thanks!

Recommended topics

Hot tags