pyCUDA with Flask gives pycuda._driver.LogicError: cuModuleLoadDataEx

Question

I want to run a pyCUDA code on a flask server. The file runs correctly directly using python3 but fails when the corresponding function is called using flask.

Here is the relevant code:

cudaFlask.py:

import pycuda.autoinit
import pycuda.driver as drv
import numpy

from pycuda.compiler import SourceModule

def cudaTest():
    mod = SourceModule("""
        int x = 4;
    """)

    print ("done")
    return

if __name__ == "__main__":
    cudaTest()

server.py (only the part which calls the function):

@app.route('/bundle', methods=['POST'])
def bundle_edges():
    cudaTest()
    return "success"

On running python cudaFlask.py I get the output done as expected but on starting the server and doing POST request at website/bundle I get the following error on the flask console:

pycuda._driver.LogicError: cuModuleLoadDataEx failed: invalid device context - 

on the line mod = SourceModule...

Where am I going wrong? There is a similar question out there but it has not been answered yet.

Collotype

It may be a compatibility issue. Because now will run under a server mode and the module may be having issues with that – Prot

Answer 1

Solved the issue with lazy loading in flask and making the context manually (i.e. without pycuda.autoinit in PyCUDA.

Refer this for lazy loading in flask.

My views.py file:

import numpy as np
import pycuda.driver as cuda
from pycuda.compiler import SourceModule

def index():
    cuda.init()
    device = cuda.Device(0) # enter your gpu id here
    ctx = device.make_context()

    mod = SourceModule("""
        int x = 4;
    """)

    ctx.pop() # very important

    print ("done")
    return "success"

Collotype

Answer 2

PyCUDA may not be compatible with WSGI web server contexts. You could possibly make it work if you use some kind of message queue like Celery, where the HTTP request places a job on the queue and the worker on the other side of the queue runs the CUDA program.

Edit: A quick and easy way would be to use Python Subprocess check_output function

In the web request:

subprocess.check_output(['python', 'cudaFlask.py'])

Leannaleanne

I will check it out. Though, could there be a simpler solution? – Collotype

You could launch it using a python subprocess? Then block for the result and return that in your response? – Leannaleanne

Answer 3

According to your solution I changed my code from

def print_device_info(): 
    (free,total)=drv.mem_get_info()
    print("Global memory occupancy:%f%% free"%(free*100/total))

    for devicenum in range(cuda.Device.count()):
        device=drv.Device(devicenum)
        attrs=device.get_attributes()

        #Beyond this point is just pretty printing
        print("\n===Attributes for device %d"%devicenum)
        for (key,value) in attrs.iteritems():
            print("%s:%s"%(str(key),str(value)))

to

def print_device_info(): 
    drv.init()
    device = drv.Device(0) # enter your gpu id here
    ctx = device.make_context()

    (free,total)=drv.mem_get_info()
     print("Global memory occupancy:%f%% free"%(free*100/total))
     attrs=device.get_attributes()

     #Beyond this point is just pretty printing
     print("\n===Attributes for device %d"%0)
     for (key,value) in attrs.items():
        print("%s:%s"%(str(key),str(value)))
     ctx.pop()

and it works like a charm. Thank you so much for sharing your solution, this really made my day !

Kropp

Answer 4

Starting the Flask application in non-threaded mode:

app.run(host=HOST, port=PORT, debug=False,threaded=False)

Denier