Get worker id from Gunicorn worker itself

Question

I run multiple gunicorn workers with workers=4 setting. So as I understand I have 5 different processes: one gunicorn master process and 4 other worker processes. Can I get information about which worker is serving request at the moment? So can I get any worker id from inside worker itself like for each request send back a response with content: this request was served by worker:worker_id?

Thormora

Were you able to solve it? If yes, can you provide the example. – Overseas

@neel, so inside your worker code just simply call import os; print(os.getpid()) – Thormora

@MostWanted Please make this an answer – Querida

Answer 1

For debugging purpose, you may use post_request hook to log the worker pid

def post_response(worker, req, environ, resp):
    worker.log.debug("%s", worker.pid)

Restful

Thanks, that's what we actually do – Thormora

How to use it within the python file? How can we print worker id with each request? – Overseas

@neel, if you want to make that code working, place the snippet into e.g. gunicorn_config.py and then launch your code with gunicorn app_module:app --config gunicorn_config.py, so it will pick up this hook and will log the desired info – Thormora

The name of the hook is now post_request (Gunicorn 20.1.0) – Ulbricht

Answer 2

Within a worker code just use

import os
print(os.getpid())

Process id is a good enough identifier for such a case. Another option which is overkill obviously is to create a worker-id-file for each worker at this point https://docs.gunicorn.org/en/stable/settings.html?highlight=hooks#post-worker-init and read from it when needed. Do not forget to remove this file on exit https://docs.gunicorn.org/en/stable/settings.html?highlight=hooks#worker-exit

Thormora

Answer 3

from fastapi import FastAPI
import uvicorn
import os
import subprocess
import re
from contextlib import asynccontextmanager
import redis


# Initialize Redis client
redis_client = redis.Redis(host='localhost', port=6379, db=0)

def get_ipv4_addresses(all_interface):

    IS_WINDOWS = os.name == 'nt'
    if IS_WINDOWS:
        ipv4_addresses = '192.168.201.18'
        return ipv4_addresses
    else:
        def get_ipv4_addresses_inner(interface):
            # Execute 'ip addr show' command
            result = subprocess.run(['ip', 'addr', 'show', interface], stdout=subprocess.PIPE, text=True)
            # Convert the result to a string
            output = result.stdout
            # Extract IPv4 addresses using regular expression
            ipv4_pattern = re.compile(r'inet (\d+\.\d+\.\d+\.\d+)/\d+')
            ipv4_addresses_ = ipv4_pattern.findall(output)
            return ipv4_addresses_
        ipv4_addresses = []
        for interface in all_interface:
            ipv4_addresses.extend(get_ipv4_addresses_inner(interface))
        return ipv4_addresses

all_interface = ['ens33', 'lo']

# List of ipv4_add names
ipv4_add_names = get_ipv4_addresses(all_interface)
app = FastAPI()


@asynccontextmanager
async def lifespan(app: FastAPI):
    # Store ipv4_add names in Redis when the application starts
    if not redis_client.exists("ipv4_adds"):
        for ipv4_add in ipv4_add_names:
            redis_client.lpush("ipv4_adds", ipv4_add)
    
    # Each worker gets a unique ipv4_add name
    ipv4_add = redis_client.rpop("ipv4_adds")
    if ipv4_add:
        app.state.ipv4_add = ipv4_add.decode('utf-8')
        print(ipv4_add, "aohoiqdoqdwoqwdoqwdqowdnqu")

    else:
        app.state.ipv4_add = "No ipv4_add available"
        os._exit(0)


    yield
    
    redis_client.lpush("ipv4_adds", app.state.ipv4_add)

    """
    Also, you can implement queue-like behavior using lpush and rpop.
    By adding values at one end with lpush and removing them at the other end with rpop,
    you can process values in FIFO (First-In-First-Out) order.
    """

app = FastAPI(lifespan=lifespan)

@app.get("/")
async def root():
    return {"message": f"This worker's ipv4_add is {app.state.ipv4_add}"}

if __name__ == "__main__":
    all_interface = ['ens33', 'lo']
    ipv4_address_names = get_ipv4_addresses(all_interface)

    # Delete all data in Redis / If not deleted, duplicate values will occur
    redis_client.flushdb()

    uvicorn.run("main2:app", host="0.0.0.0", port=8000, workers=len(ipv4_address_names))

Convulsive

Remember that Stack Overflow isn't just intended to solve the immediate problem, but also to help future readers find solutions to similar problems, which requires understanding the underlying code. This is especially important for members of our community who are beginners, and not familiar with the syntax. Given that, can you edit your answer to include an explanation of what you're doing and why you believe it is the best approach? – Heinous