AttributeError: '_MainProcess' object has no attribute '_exiting'

Question

I got an

AttributeError: '_MainProcess' object has no attribute '_exiting'

from a Python application. Unfortunately this code has to run Python 2.5 and therefore the processing module nowadays known as multiprocessing. What I was doing is to create a Process with a Queue and to put an item in the queue from the main process. Looking into the processing.queue code I can see that a feeder thread is started. This feeder thread will then check currentProcess()._exiting, but currentProcess() evaluates to a _MainProcess which does not have said attribute as can be seen in the processing.process module. How to solve this? Is it a bug in processing? If yes, can I simply monkeypatch it using currentProcess()._exiting = False?

Minimal example:

#!/usr/bin/python

import processing
import processing.queue

class Worker(processing.Process):
    def __init__(self):
        processing.Process.__init__(self)
        self.queue = processing.queue.Queue()

    def run(self):
        element = self.queue.get()
        print element

if __name__ == '__main__':
    w = Worker()
    w.start()
    # To trigger the problem, any non-pickleable object is to be passed here.
    w.queue.put(lambda x: 1)
    w.join()

Funicular

Can you post a snippet that reproduces the error? Also, I really like the word monkeypatch. Looking forward to using it soon :) – Febricity

I believe there is a right way to do what you want without patching. – Worsley

@Space_C0wb0y I'm sorry, I had a bit problems with reducing those 3k lines of code to a smaller example. :-/ – Funicular

@Helmut: In that case we are certainly not talking about a simple Python application. – Febricity

Now I got a: PicklingError: Can't pickle <type 'function'>: attribute lookup builtin.function failed. (I checked every possible code path to avoid this.) Debugging this is gonna be hard! – Funicular

This PicklingError is responsible for causing the AttributeError, so I better invest the time to fix the PicklingError. :-) – Funicular

Answer 1

I am not sure why you would want to pickle a function in this case, if you really want to do that have a look at this answer: Is there an easy way to pickle a python function (or otherwise serialize its code)?

otherwise, this works for python 2.6 (I know you are looking for 2.5 but I don't have 2.5). I have replaced your lambda function with a regular function and provide that to the processing constructor:

from multiprocessing import Process, Queue

def simple():
    return 1

class Worker(Process):
    def __init__(self, args):
        Process.__init__(self, args=args)
        self.queue = Queue()

    def run(self):
        element = self.queue.get()
        print element

if __name__ == '__main__':
    w = Worker(args=[simple])
    w.start()
    w.join()

Swingletree

I am sorry, but you completely missed the point. The problem is that it is not obvious from the error message that non-pickleable object was passed. The lambda is just an example for a non-pickleable object. – Funicular

but that's why i gave you the link to pickle a python function and you can put the pickled function in your queue, and when you get it from your queue then you can reconstruct the function – Swingletree

You still completely miss the point. The initial problem was the error message. Half of the solution was finding out it was caused by a non-pickleable object. Working around non-pickleable objects is trivial once you know what is the cause. The missing part is to fix the error message. – Funicular

Thanks for your constructive comments, I'll help you next time as well. – Swingletree