How to use flask context with concurrent.futures.ThreadPoolExecutor

Question

I'm trying to make multiple requests async and get response back, I'm using concurrent.futures to do this, but inside my function using current_app which from flask and I always got this error:

RuntimeError: Working outside of application context.

I don't know how to resolve this. Can anyone please help?

Below are my code:

run.py:

import concurrent.futures
from flask import current_app
from http_calls import get_price, get_items

def init():
    with current_app._get_current_object().test_request_context():
        with concurrent.futures.ThreadPoolExecutor(max_workers=20) as executor:
            futs = []
            futs.append(executor.submit(get_price))
            futs.append(executor.submit(get_items))

            print([fut.result() for fut in concurrent.futures.as_completed(futs)])

init()

http_calls.py

from flask import current_app

def get_price():
    url = current_app.config['get_price_url']
    return requests.get(url).json()

def get_items():
    url = current_app.config['get_items_url']
    return requests.get(url).json()

Machmeter

Answer 1

You should import your Flask instance in your script. Use current_app under the app context.

import concurrent.futures
from your_application import your_app  # or create_app function to return a Flask instance
from flask import current_app
from http_calls import get_price, get_items

def init():
    with your_app.app_context():
        with concurrent.futures.ThreadPoolExecutor(max_workers=20) as executor:
            ...

Pooi

if i dont use current_app, there is no need of using app_context() ? – Lichtenfeld

Answer 2

I was running into similar issues around using concurrent.futures with Flask. I wrote Flask-Executor as a Flask-friendly wrapper for concurrent.futures to solve this problem. It may be an easier way for you to work with these two together.

Biform

This solution worked for me... although it's a shame to see that the code hasn't been maintained in 11 months. You made a great thing @Biform , I hope to see you continue to support it! – Fichtean

Answer 3

def with_app_context(cls, app, func):
    def wrapper(*args, **kwargs):
        with app.app_context():
            return func(*args, **kwargs)
    return wrapper

you can use wrapper for achieving this. you can call your function from thread pool like below

with concurrent.futures.ThreadPoolExecutor(max_workers=2) as executor:
                # download using wkhtmltopdf (helps to address access denied issues)
                res = executor.submit(with_app_context(app, cls.some_func))

Guardant