Visibility of global variables in imported modules

Question

I've run into a bit of a wall importing modules in a Python script. I'll do my best to describe the error, why I run into it, and why I'm tying this particular approach to solve my problem (which I will describe in a second):

Let's suppose I have a module in which I've defined some utility functions/classes, which refer to entities defined in the namespace into which this auxiliary module will be imported (let "a" be such an entity):

module1:

def f():
    print a

And then I have the main program, where "a" is defined, into which I want to import those utilities:

import module1
a=3
module1.f()

Executing the program will trigger the following error:

Traceback (most recent call last):
  File "Z:\Python\main.py", line 10, in <module>
    module1.f()
  File "Z:\Python\module1.py", line 3, in f
    print a
NameError: global name 'a' is not defined

Similar questions have been asked in the past (two days ago, d'uh) and several solutions have been suggested, however I don't really think these fit my requirements. Here's my particular context:

I'm trying to make a Python program which connects to a MySQL database server and displays/modifies data with a GUI. For cleanliness sake, I've defined the bunch of auxiliary/utility MySQL-related functions in a separate file. However they all have a common variable, which I had originally defined inside the utilities module, and which is the cursor object from MySQLdb module. I later realised that the cursor object (which is used to communicate with the db server) should be defined in the main module, so that both the main module and anything that is imported into it can access that object.

End result would be something like this:

utilities_module.py:

def utility_1(args):
    code which references a variable named "cur"
def utility_n(args):
    etcetera

And my main module:

program.py:

import MySQLdb, Tkinter
db=MySQLdb.connect(#blahblah) ; cur=db.cursor()  #cur is defined!
from utilities_module import *

And then, as soon as I try to call any of the utilities functions, it triggers the aforementioned "global name not defined" error.

A particular suggestion was to have a "from program import cur" statement in the utilities file, such as this:

utilities_module.py:

from program import cur
#rest of function definitions

program.py:

import Tkinter, MySQLdb
db=MySQLdb.connect(#blahblah) ; cur=db.cursor()  #cur is defined!
from utilities_module import *

But that's cyclic import or something like that and, bottom line, it crashes too. So my question is:

How in hell can I make the "cur" object, defined in the main module, visible to those auxiliary functions which are imported into it?

Thanks for your time and my deepest apologies if the solution has been posted elsewhere. I just can't find the answer myself and I've got no more tricks in my book.

Vodka

Based on your update: You probably don't want a single shared cursor anyway. A single shared connection, yes, but cursors are cheap, and there are often good reasons to have multiple cursors alive at the same time (e.g., so you can iterate through two of them in lockstep instead of having to fetch_all and iterate through two lists instead, or just so you can have two different threads/greenlets/callback-chains/whatever using the database without conflicts). – Supposing

Anyway, whatever you want to share, I think the answer here is to move db (and cur, if you insist) into a separate module that both program and utilities_module import it from. That way you don't get circular dependencies (importing program from modules that program imports) and the confusion that comes with them. – Supposing

Answer 1

Globals in Python are global to a module, not across all modules. (Many people are confused by this, because in, say, C, a global is the same across all implementation files unless you explicitly make it static.)

There are different ways to solve this, depending on your actual use case.

---

Before even going down this path, ask yourself whether this really needs to be global. Maybe you really want a class, with f as an instance method, rather than just a free function? Then you could do something like this:

import module1
thingy1 = module1.Thingy(a=3)
thingy1.f()

---

If you really do want a global, but it's just there to be used by module1, set it in that module.

import module1
module1.a=3
module1.f()

---

On the other hand, if a is shared by a whole lot of modules, put it somewhere else, and have everyone import it:

import shared_stuff
import module1
shared_stuff.a = 3
module1.f()

… and, in module1.py:

import shared_stuff
def f():
    print shared_stuff.a

Don't use a from import unless the variable is intended to be a constant. from shared_stuff import a would create a new a variable initialized to whatever shared_stuff.a referred to at the time of the import, and this new a variable would not be affected by assignments to shared_stuff.a.

---

Or, in the rare case that you really do need it to be truly global everywhere, like a builtin, add it to the builtin module. The exact details differ between Python 2.x and 3.x. In 3.x, it works like this:

import builtins
import module1
builtins.a = 3
module1.f()

Supposing

Thanks for your answer. I will try the import shared_stuff approach, although I can't get over the fact that variables defined in the main module aren't really global - Isn't there any way to make a name truly accessible to everyone, from wherever in the program? – Vodka

Having something be "accessible to everyone, from whatever program" very much goes against the zen of python, specifically "Explicit is better than implicit". Python has a very well thought oo design, and if you use it well, you might manage to go the rest of your python career without ever needing to use the global keyword again. – Slesvig

UPDATE: THANKS! The shared_stuff approach worked just fine, with this I think I can work around any other issues. – Vodka

@DanielArmengod: I added the builtin answer, in case you really do need it. (And if you need more details, search SO; there are at least two questions on how to add stuff to builtins properly.) But, as Bi Rico says, you almost certainly don't really need it, or want it. – Supposing

@BiRico: On further thought, it's more "Namespaces are one honking great idea" than "Explicit is better than implicit". Each module having its own globals is the paradigmatic case of the honking great idea that everything else is built on (at least conceptually; in details, new-style classes are probably the best paradigm). – Supposing

This sentence was an epiphany: "Don't use a from import unless the variable is intended to be a constant. from shared_stuff import a would create a new a variable initialized to whatever shared_stuff.a referred to at the time of the import, and this new a variable would not be affected by assignments to shared_stuff.a." – Merrell

Answer 2

As a workaround, you could consider setting environment variables in the outer layer, like this.

main.py:

import os
os.environ['MYVAL'] = str(myintvariable)

mymodule.py:

import os

myval = None
if 'MYVAL' in os.environ:
    myval = os.environ['MYVAL']

As an extra precaution, handle the case when MYVAL is not defined inside the module.

Cherian

Answer 3

This post is just an observation for Python behaviour I encountered. Maybe the advices you read above don't work for you if you made the same thing I did below.

Namely, I have a module which contains global/shared variables (as suggested above):

#sharedstuff.py

globaltimes_randomnode=[]
globalist_randomnode=[]

Then I had the main module which imports the shared stuff with:

import sharedstuff as shared

and some other modules that actually populated these arrays. These are called by the main module. When exiting these other modules I can clearly see that the arrays are populated. But when reading them back in the main module, they were empty. This was rather strange for me (well, I am new to Python). However, when I change the way I import the sharedstuff.py in the main module to:

from globals import *

it worked (the arrays were populated).

Just sayin'

Armorial

Answer 4

A function uses the globals of the module it's defined in. Instead of setting a = 3, for example, you should be setting module1.a = 3. So, if you want cur available as a global in utilities_module, set utilities_module.cur.

A better solution: don't use globals. Pass the variables you need into the functions that need it, or create a class to bundle all the data together, and pass it when initializing the instance.

Philipphilipa

Instead of writing 'import module1', if the user had written 'from module1 import f', then f would come in global namespace of main.py. Now in main.py if we use f(), then since a=3 and f (function definition) are both in globalnamespace of main. Is this a solution? If I am wrong then please can you direct me to any article on this subject please – Highly

I used the approach above and passed the globals as arguments when instantiating the classes that are using the globals. This was OK, but sometime later we have activated a Sonarqube code check of the code and this complained that functions have too many arguments. So we had to look for another solution. Now we use environment variables that are read by each module that needs them. It's not really OOP compliant but that's it. This works only when the globals do not change during the code execution. – Gayomart

Answer 5

The easiest solution to this particular problem would have been to add another function within the module that would have stored the cursor in a variable global to the module. Then all the other functions could use it as well.

module1:

cursor = None

def setCursor(cur):
    global cursor
    cursor = cur

def method(some, args):
    global cursor
    do_stuff(cursor, some, args)

main program:

import module1

cursor = get_a_cursor()
module1.setCursor(cursor)
module1.method()

Clintclintock

Answer 6

Since globals are module specific, you can add the following function to all imported modules, and then use it to:

• Add singular variables (in dictionary format) as globals for those
• Transfer your main module globals to it .

addglobals = lambda x: globals().update(x)

Then all you need to pass on current globals is:

import module

module.addglobals(globals())

Bowra

Answer 7

Update

To test the theory, I created a module and put it on pypi. It all worked perfectly.

pip install superglobals

Short answer

This works fine in Python 2 or 3:

import inspect

def superglobals():
    _globals = dict(inspect.getmembers(
                inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
    return _globals

save as superglobals.py and employ in another module thusly:

from superglobals import *

superglobals()['var'] = value

Extended Answer

You can add some extra functions to make things more attractive.

def superglobals():
    _globals = dict(inspect.getmembers(
                inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
    return _globals

def getglobal(key, default=None):
    """
    getglobal(key[, default]) -> value
    
    Return the value for key if key is in the global dictionary, else default.
    """
    _globals = dict(inspect.getmembers(
                inspect.stack()[len(inspect.stack()) - 1][0]))["f_globals"]
    return _globals.get(key, default)

def setglobal(key, value):
    _globals = superglobals()
    _globals[key] = value

def defaultglobal(key, value):
    """
    defaultglobal(key, value)

    Set the value of global variable `key` if it is not otherwise st
    """
    _globals = superglobals()
    if key not in _globals:
        _globals[key] = value

Then use thusly:

from superglobals import *

setglobal('test', 123)
defaultglobal('test', 456)
assert(getglobal('test') == 123)

Justification

The "python purity league" answers that litter this question are perfectly correct, but in some environments (such as IDAPython) which is basically single threaded with a large globally instantiated API, it just doesn't matter as much.

It's still bad form and a bad practice to encourage, but sometimes it's just easier. Especially when the code you are writing isn't going to have a very long life.

Orthodontics

Hey! This is some great stuff, but I think it doesn't work with ipython, only with python. Do you have any ideas by any chance how to make this work with ipython as well in a robust manner? Thanks! :) – Borchardt

ipython is the devil. you might try changing the second line to _globals = globals() though. – Orthodontics

Answer 8

Since I haven't seen it in the answers above, I thought I would add my simple workaround, which is just to add a global_dict argument to the function requiring the calling module's globals, and then pass the dict into the function when calling; e.g:

# external_module
def imported_function(global_dict=None):
    print(global_dict["a"])


# calling_module
a = 12
from external_module import imported_function
imported_function(global_dict=globals())

>>> 12

Sheeting

Answer 9

The OOP way of doing this would be to make your module a class instead of a set of unbound methods. Then you could use __init__ or a setter method to set the variables from the caller for use in the module methods.

Tetramethyldiarsine

Answer 10

From inside the function that is being called from the parent module you can use...

import inspect


def caller_globals(key):
    return inspect.currentframe().f_back.f_globals.get(key, None)

---

---

I've played around with a few different approaches from the inspect module to do this something similar, and one thing I did discover is that most of it was useless unless it was used inside a function that is being called from the calling module.

Also, if you use this inside a module0.module1.module2.func and you want the globals from module0 then...

from inspect import currentframe
from types import FrameType


def caller_globals(f_back: FrameType) -> dict:
    if 'module0.py' not in f_back.f_globals['__file__']:
        f_back = caller_globals(f_back = f_back.f_back)

    return f_back.f_globals


f_globals = caller_globals(f_back = currentframe().f_back)

Etruria