Return value from thread

Question

How do I get a thread to return a tuple or any value of my choice back to the parent in Python?

Imparity

Answer 1

I suggest you instantiate a Queue.Queue before starting the thread, and pass it as one of the thread's args: before the thread finishes, it .puts the result on the queue it received as an argument. The parent can .get or .get_nowait it at will.

Queues are generally the best way to arrange thread synchronization and communication in Python: they're intrinsically thread-safe, message-passing vehicles -- the best way to organize multitasking in general!-)

Denitrify

before the thread finishes, it .puts the result on the queue it received as an argument you mean this will be done automatically by python? if not (meant as designing tip) then could you make it clear in the answer. – Budge

Its ugly to specialize an existing function for that; and the Queue has a lot of unnecessary overhead for a single result issue. More clearly and efficiently subclass threading.Thread and the new run() method simply stores the result as attribute like self.ret = ... (Much more comfortable would be a subclass of Thread which handles return values / exceptions of the custom target function. Indeed threading.Thread should be extended to offer that out of the box - as it would be compatible with the old behavior "return None".) – Baalman

Using Queue is the best answer, however, the post above poorly illustrates how to use Queue. See this answer that provides a basic example with Queue and return values. – Galliett

Answer 2

You should pass a Queue instance as a parameter then you should .put() your return object into the queue. You can gather the return value via queue.get() whatever object you put.

Sample:

queue = Queue.Queue()
thread_ = threading.Thread(
                target=target_method,
                name="Thread1",
                args=[params, queue],
                )
thread_.start()
thread_.join()
queue.get()

def target_method(self, params, queue):
 """
 Some operations right here
 """
 your_return = "Whatever your object is"
 queue.put(your_return)

Use for multiple threads:

#Start all threads in thread pool
    for thread in pool:
        thread.start()
        response = queue.get()
        thread_results.append(response)

#Kill all threads
    for thread in pool:
        thread.join()

I use this implementation and it works great for me. I wish you do so.

Cumuliform

Of course I start the thread I just miss to put the line here :) Thanks for notice. – Cumuliform

how would this look like if you had multiple threads? que.get() returns the result of one thread only for me? – Christelchristen

In the multiple threads case it seems like the response = queue.get() would raise the Empty exception if the thread wasn't finished yet and likely terminate with an handled exception. Even if it succeeded every time, that would mean each thread was finished and that little or no actual multi-threading ever occurred. – Hodometer

I believe it makes sense to get from the queue only after thread join, since that is when you know the result is added. – Utility

I am still not able to understand the correspondence between the sequence in which the queue will be populated here. Can we collect the return value in some sequence. – Fiddlefaddle

Answer 3

Use lambda to wrap your target thread function and pass its return value back to the parent thread using a queue. (Your original target function remains unchanged without extra queue parameter.)

Sample code:

import threading
import queue
def dosomething(param):
    return param * 2
que = queue.Queue()
thr = threading.Thread(target = lambda q, arg : q.put(dosomething(arg)), args = (que, 2))
thr.start()
thr.join()
while not que.empty():
    print(que.get())

Output:

4

Goins

Answer 4

If you were calling join() to wait for the thread to complete, you could simply attach the result to the Thread instance itself and then retrieve it from the main thread after the join() returns.

On the other hand, you don't tell us how you intend to discover that the thread is done and that the result is available. If you already have a way of doing that, it will probably point you (and us, if you were to tell us) to the best way of getting the results out.

Casias

you could simply attach the result to the Thread instance itself How do you pass Thread instance to the target it runs so the target could attach the result to this instance? – Hexamethylenetetramine

Piotr Dobrogost, if you're not subclassing Thread for your instance you could just using threading.current_thread() from the end of your target callable. I'd call that a tad ugly, but Alex' approach was always the more elegant one. This one is just more expedient in some cases. – Casias

It'd be nice if join() would just return the whatever the called method returns... seems silly that instead it returns None. – Hearken

Answer 5

I'm surprised nobody mentioned that you could just pass it a mutable:

from threading import Thread

def task(thread_return):
    thread_return['success'] = True

thread_return={'success': False}
Thread(target=task, args=(thread_return,)).start()

print(thread_return)
{'success': True}

perhaps this has major issues of which I'm unaware.

Chickie

This works perfectly! Would really like to hear some opinion about things missing with this approach if any. – Gambill

works. Just its ugly to specialize an existing function - and those many confusing things (readability) - see comment on 1st answer. – Baalman

Answer 6

Another approach is to pass a callback function to the thread. This gives a simple, safe and flexible way to return a value to the parent, anytime from the new thread.

# A sample implementation

import threading
import time

class MyThread(threading.Thread):
    def __init__(self, cb):
        threading.Thread.__init__(self)
        self.callback = cb

    def run(self):
        for i in range(10):
            self.callback(i)
            time.sleep(1)


# test

import sys

def count(x):
    print x
    sys.stdout.flush()

t = MyThread(count)
t.start()

Spurgeon

The problem with this is that the callback still runs in the child thread, rather than in the original thread. – Pilatus

@wilberforce could you please explain what problems it can cause? – Spurgeon

Ok. An example would be if the callback writes to a log file to which the parent thread also writes while the thread is running. Since the callback is running in the child thread there's a risk of the two writes happening at the same time and colliding - you could get garbled or interleaved output, or a crash if the logging framework did some internal bookkeeping. Using a thread-safe queue and having one thread do all the writing would avoid this. These kinds of problems can be nasty because they aren't deterministic - they might only show up in production, and can be difficult to reproduce. – Pilatus

Answer 7

You can use synchronised queue module.
Consider you need to check a user infos from database with a known id:

def check_infos(user_id, queue):
    result = send_data(user_id)
    queue.put(result)

Now you can get your data like this:

import queue, threading
queued_request = queue.Queue()
check_infos_thread = threading.Thread(target=check_infos, args=(user_id, queued_request))
check_infos_thread.start()
final_result = queued_request.get()

Vocalism

I keep getting the error : TypeError: square() takes 1 positional argument but 2 were given – Madagascar

Answer 8

For easy programs the above answeres look a little bit like overkill to me. I would en-nicen the mutable approach:

class RetVal:
 def __init__(self):
   self.result = None


def threadfunc(retVal):
  retVal.result = "your return value"

retVal = RetVal()
thread = Thread(target = threadfunc, args = (retVal))

thread.start()
thread.join()
print(retVal.result)

Run

Answer 9

POC:

import random
import threading

class myThread( threading.Thread ):
    def __init__( self, arr ):
        threading.Thread.__init__( self )
        self.arr = arr
        self.ret = None

    def run( self ):
        self.myJob( self.arr )

    def join( self ):
        threading.Thread.join( self )
        return self.ret

    def myJob( self, arr ):
        self.ret = sorted( self.arr )
        return

#Call the main method if run from the command line.
if __name__ == '__main__':
    N = 100

    arr = [ random.randint( 0, 100 ) for x in range( N ) ]
    th = myThread( arr )
    th.start( )
    sortedArr = th.join( )

    print "arr2: ", sortedArr

Halfcock

Answer 10

Well, in the Python threading module, there are condition objects that are associated to locks. One method acquire() will return whatever value is returned from the underlying method. For more information: Python Condition Objects

Elope

Answer 11

Based on jcomeau_ictx's suggestion. The simplest one I came across. Requirement here was to get exit status staus from three different processes running on the server and trigger another script if all three are successful. This seems to be working fine

  class myThread(threading.Thread):
        def __init__(self,threadID,pipePath,resDict):
            threading.Thread.__init__(self)
            self.threadID=threadID
            self.pipePath=pipePath
            self.resDict=resDict

        def run(self):
            print "Starting thread %s " % (self.threadID)
            if not os.path.exists(self.pipePath):
            os.mkfifo(self.pipePath)
            pipe_fd = os.open(self.pipePath, os.O_RDWR | os.O_NONBLOCK )
           with os.fdopen(pipe_fd) as pipe:
                while True:
                  try:
                     message =  pipe.read()
                     if message:
                        print "Received: '%s'" % message
                        self.resDict['success']=message
                        break
                     except:
                        pass

    tResSer={'success':'0'}
    tResWeb={'success':'0'}
    tResUisvc={'success':'0'}


    threads = []

    pipePathSer='/tmp/path1'
    pipePathWeb='/tmp/path2'
    pipePathUisvc='/tmp/path3'

    th1=myThread(1,pipePathSer,tResSer)
    th2=myThread(2,pipePathWeb,tResWeb)
    th3=myThread(3,pipePathUisvc,tResUisvc)

    th1.start()
    th2.start()
    th3.start()

    threads.append(th1)
    threads.append(th2)
    threads.append(th3)

    for t in threads:
        print t.join()

    print "Res: tResSer %s tResWeb %s tResUisvc %s" % (tResSer,tResWeb,tResUisvc)
    # The above statement prints updated values which can then be further processed

Gambill

Answer 12

The following wrapper function will wrap an existing function and return an object which points both to the thread (so that you can call start(),join(), etc. on it) as well as access/view its eventual return value.

def threadwrap(func,args,kwargs):
   class res(object): result=None
   def inner(*args,**kwargs): 
     res.result=func(*args,**kwargs)
   import threading
   t = threading.Thread(target=inner,args=args,kwargs=kwargs)
   res.thread=t
   return res

def myFun(v,debug=False):
  import time
  if debug: print "Debug mode ON"
  time.sleep(5)
  return v*2

x=threadwrap(myFun,[11],{"debug":True})
x.thread.start()
x.thread.join()
print x.result

It looks OK, and the threading.Thread class seems to be easily extended(*) with this kind of functionality, so I'm wondering why it isn't already there. Is there a flaw with the above method?

(*) Note that husanu's answer for this question does exactly this, subclassing threading.Thread resulting in a version where join() gives the return value.

Pendulum

Answer 13

Here is a code which implements multi-threading.

Thread 1 is adding numbers from 10 to 20. Thread 2 is adding numbers from 21 to 30.

Finally the output is returned to the main program where it can perform final addition. (not shown in this program) but you can use a numpy call.

import threading
import os
import queue

def task1(num, queue): 
    print("\n Current thread: {}".format(threading.current_thread().name)) 
    count = 0
    sum1 = 0
    while count <= 10:
        sum1 = sum1 + num
        num = num + 1
        count = count + 1
    print('\n'+str(sum1))
    queue.put(sum1)


if __name__ == "__main__":

    queue = queue.Queue()

    # print ID of current process 
    print("\n Process ID is: {}".format(os.getpid())) 

    # print name of main thread 
    print("\n Main thread is: {}".format(threading.main_thread().name)) 

    # creating threads 
    t1 = threading.Thread(target=task1, name='t1',args=[10,queue]) 
    t2 = threading.Thread(target=task1, name='t2',args=[21,queue])

    #Store thread names in a list
    pool = [t1,t2]

    #Used to store temporary values
    thread_results = []

    # starting threads
    #Start all threads in thread pool
    for thread in pool:
        thread.start()
        response = queue.get()
        thread_results.append(response)

    #Kill all threads
    for thread in pool:
        thread.join()

    print(thread_results)

Urbanus

Answer 14

I think the threading.Thread subclassing works in a more clear manner, given the fact that the result is tied only to the thread instance, without resorting to other objects (i.e. Queue) that may have other implications. Here's my 2 cents example:

class RThread(threading.Thread):
    def __init__(self, target, args, daemon: bool = False):
        super().__init__(group=None, target=target, args=args, daemon=daemon)
        self.target = target
        self.args = args
        self.daemon = daemon
        self.result = None

    def run(self):
        self.result = self.target(*self.args)

def foo(secs:int=10):
    print(f"Starting thread at {time.strftime('%X')}")
    time.sleep(secs)
    print(f"Ending  thread at {time.strftime('%X')}")
    return secs

t = RThread(target=foo, args=(random.randint(1, 10),), daemon=False)

def main():
    global t
    os.system("clear")
    try:
        name = input("Please input your name: ")
        t.start()
        print(f"Hi, {name}.")
        t.join()
        print(f"The result of thread is: {t.result}")
        print(f"Bye, {name}.")
    except KeyboardInterrupt:
        print("\nBye, bye.")

Key is run's method of subclassed thread, which links the return value of the target function to the 'result' attribute of the thread. Simple and straightforward.

Archine