ipyparallel error when using decorator

Question

I am attempting to run a job using the ipyparallel system with lru_cache and am running into issues.

From the terminal:

ipcluster start -n 2

In an ipython notebook:

from ipyparallel import Client
clients = Client()

def world():
    return hello() + " World"

def hello():
    return "Hello"

world()
'Hello World'

Running it with ipyparallel requires this:

clients[:].push(dict(hello=hello))

Without the previous line the following fails, which is not unexpected, but works fine if it is run:

clients[:].apply_sync(world)
['Hello World', 'Hello World']

This all works as expected, however, with the lru_cache the parallel step generates errors

from ipyparallel import Client
from functools import lru_cache

clients = Client()

def world():
    return hello() + " World"

@lru_cache(maxsize=2048)
def hello():
    return "Hello"

clients[:].push(dict(hello=hello))
clients[:].apply_sync(world)

This fails with the following error:

  [0:apply]: 
---------------------------------------------------------------------------NameError                                 
Traceback (most recent call last)<string> in <module>()
<ipython-input-17-9ac5ef032485> in world()
NameError: name 'hello' is not defined

[1:apply]: 
---------------------------------------------------------------------------NameError                                 
Traceback (most recent call last)<string> in <module>()
<ipython-input-17-9ac5ef032485> in world()
NameError: name 'hello' is not defined

I realize this may be a namespace issue where lru_cache is using a closure, however I'm hoping there is a workaround so that I am able to use it. Or if someone can tell me that this simply isn't possible that would be useful as well, thanks.

I am using:
ipykernel (4.1.1)
ipyparallel (4.1.0)
ipython (4.0.1)
Python (3.5.1)

Cornea

I'm pretty sure it has to do with the decorator itself. If you replace lru_cache with any other decorator function you will have the same error. I don't know if it can be solved though.. – Koontz

Thanks, @imaluengo that was my initial thought as well, though I was able to get it working with other decorators. I debated about including an example with a working decorator in the original post though it was getting to be quite a long post. – Cornea

Ops, I tried with a test decorator in python 2.7 and it didn't work for me :/ – Koontz