How to run gunicorn inside python not as a command line?

Question

I have a flask application.

I run it in production with this command:

python -m gunicorn -w 1 -b 0.0.0.0:5000 "path.to.wsgi:return_app()"

Instead, I want to run it inside a my_file.py

I need a function to run and it should accept the app object and port binding and number of workers

How can I do that?

I need something like this psudo code:

import gunicorn

app = return_app()

gunicorn(workers=1, ip="0.0.0.0", port=5000, app=app)

the most important part to me is the app=app part

the main point is that I want to use the app object as an instance of Flask(). I want to directly give app object to gunicorn not throough addressing it in a string

What I have tried: I have opened gunicorn library main.py file

from gunicorn.app.wsgiapp import run
run()

to see how it works but could not figure it out

def run():
    """\
    The ``gunicorn`` command line runner for launching Gunicorn with
    generic WSGI applications.
    """
    from gunicorn.app.wsgiapp import WSGIApplication
    WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()

Papoose

Answer 1

Something like this works for me. First I instantiate the BaseApplication class. It has a run() method. The details are on how to create a custom application in gunicorn documentation.

if platform.uname().system.lower()=='linux':
    print("Detected Linux, Preparing gunicorn")        
    import gunicorn.app.base
    class StandaloneApplication(gunicorn.app.base.BaseApplication):

        def __init__(self, app, options=None):
            self.options = options or {}
            self.application = app
            super().__init__()

        def load_config(self):
            config = {key: value for key, value in self.options.items()
                    if key in self.cfg.settings and value is not None}
            for key, value in config.items():
                self.cfg.set(key.lower(), value)

        def load(self):
            return self.application

if __name__ == "__main__":
    # Use a debugging session in port 5001    
    if platform.uname().system.lower()=='linux':
        print("Detected Linux, Running Gunicorn")
        options = {
            'bind': '%s:%s' % ('0.0.0.0', '5001'),
            'workers': number_of_workers(),
            # 'threads': number_of_workers(),
            'timeout': 120,
        }
        initialize()
        StandaloneApplication(app, options).run()
    else:
        print("Detected non Linux, Running in pure Flask")
        initialize()
        app.run(debug=True, host=socket.gethostbyname(socket.gethostname()), port=5001)

Manns

Your MWE is not really complete. e.g. number_of_workers() initialize() and some others are missing. It would be lovely if you would complete it, so one can run it self contained. – Oldcastle

Answer 2

My goals weren't exactly the same: I was fine with specifying the app as a string (same as you do with gunicorn on the command line) rather than passing a python app object. Actually I'm not sure if passing a single app object really makes sense, because shouldn't gunicorn have a different app object in each worker that it spawns?

My main concern was that I run gunicorn without the help of subprocess or similar. I also used FastAPI rather than Flask, and (per the current recommended prod FastAPI setup) told gunicorn to spawn uvicorn workers.

This is what I ended up going with (in myproject/web.py):

import multiprocessing

from gunicorn.app.wsgiapp import WSGIApplication


class StandaloneApplication(WSGIApplication):
    def __init__(self, app_uri, options=None):
        self.options = options or {}
        self.app_uri = app_uri
        super().__init__()

    def load_config(self):
        config = {
            key: value
            for key, value in self.options.items()
            if key in self.cfg.settings and value is not None
        }
        for key, value in config.items():
            self.cfg.set(key.lower(), value)


def run():
    options = {
        "bind": "0.0.0.0:8000",
        "workers": (multiprocessing.cpu_count() * 2) + 1,
        "worker_class": "uvicorn.workers.UvicornWorker",
    }
    StandaloneApplication("myproject.main:app", options).run()

My StandaloneApplication is very similar to (and is based on) the one in Zaero Divide's answer in this thread. However, I pass app_uri, and I inherit from WsgiApplication instead of from BaseApplication, which results in gunicorn starting up in basically the same way as when it's invoked from the command line.

Note: in myproject/main.py, I have app = FastAPI(). And in pyproject.toml, under [tool.poetry.scripts], I have web = "myproject.web:run" - so I can start gunicorn with poetry run web. I'm also building an artifact with shiv -c web myproject.whl -o web.pyz, so I can then just run /path/to/web.pyz to start gunicorn.

Eltonelucidate

Answer 3

insert the following inside your my_file.py

from subprocess import run
run("gunicorn -w 1 -b 0.0.0.0:5000 'path.to.wsgi:return_app()'".split(' '))

Moina

but I need to give it the app directly as a python object, not as a string. – Papoose

It still the same has limitations. My mindset is if I am running it inside python, I should be able to give the app directly to it. because gunicorn needs the app. and gunicorn is python libarray. I should be able to use it in python – Papoose

run("gunicorn -w 1 -b 0.0.0.0:5000 module_projet.wsgi".split(' '), cwd=path_to_wsgi) – Moina

@Amin Ba is it a django app or not? – Moina

look at the code I added – Papoose