Temporarily changing a variable's value in Python

Question

Python 3.4 provides this neat tool to temporarily redirect stdout:

# From https://docs.python.org/3.4/library/contextlib.html#contextlib.redirect_stdout
with redirect_stdout(sys.stderr):
    help(pow)

The code is not super-complicated, but I wouldn't want to write it over and over again, especially since some thought has gone into it to make it re-entrant:

class redirect_stdout:
    def __init__(self, new_target):
        self._new_target = new_target
        # We use a list of old targets to make this CM re-entrant
        self._old_targets = []

    def __enter__(self):
        self._old_targets.append(sys.stdout)
        sys.stdout = self._new_target
        return self._new_target

    def __exit__(self, exctype, excinst, exctb):
        sys.stdout = self._old_targets.pop()

I'm wondering if there's a general way to use the with statement to temporarily change the value of a variable. Two other use cases from sys are sys.stderr and sys.excepthook.

In a perfect world, something like this would work:

foo = 10
with 20 as foo:
    print(foo) # 20
print (foo) # 10

I doubt we can make that work, but maybe something like this is possible:

foo = 10
with temporary_set('foo', 20):
    print(foo) # 20
print (foo) # 10

I can sort of getting this working by rooting around in globals(), but it's nothing anyone would choose to use.

UPDATE: while I think my "foo = 10" examples clarified what I'm trying to do, they do not convey an actual use case. Here are two:

1. Redirect stderr, much like redirect_stdout
2. Temporarily change sys.excepthook. I do a lot of development interactively, and when I add something to excepthook (by wrapping the original function in one of my own, say, to log exceptions using the logging module), I generally want it to get removed at some point. That way I won't have more and more copies of my function wrapping itself. This question confronts a closely related problem.

Meyers

Can you motivate such an idiom? In which situation would it be useful? – Rheinlander

Answer 1

Building on the answer by @arthaigo, a more concise version is:

import contextlib

@contextlib.contextmanager
def temporary_assignment(object, new_value):
  old_value = eval(object)
  globals()[object] = new_value
  yield
  globals()[object] = old_value

Someone

Unfortunately, none of these solutions are able to work across different modules. So this temporary_assignment can't be placed in a library module and used elsewhere, because it won't have access to the globals() in the caller's module. See also #41455178 – Someone

Answer 2

I know this question is kind of old, but as I came around the same problem, here is my solution:

class test_context_manager():
    def __init__(self, old_object, new_object):
        self.new = new_object
        self.old = old_object
        self.old_code = eval(old_object)
    def __enter__(self):
        globals()[self.old] = self.new
    def __exit__(self, type, value, traceback):
        globals()[self.old] = self.old_code

It's not pretty as it makes heavy use of global variables, but it seems to work.

For example:

x = 5
print(x)
with test_context_manager("x", 7):
    print(x)

print(x)

Result:

5
7
5

or with functions:

def func1():
    print("hi")

def func2():
    print("bye")

x = 5
func1()
with test_context_manager("func1", func2):
    func1()

func1()

Result:

hi
bye
hi

Reclamation

Nice. I think you can get something a little more general if you use exec("{} = {}".format(self.old, repr(self.new)), globals()). With that change I can also handle strings. – Meyers

Nice. That's a useful addition. I only tested it with functions and integers. – Reclamation

By getting rid of exec completely it is now even more general. Now, a arbitrary secondary object can be passed. – Reclamation

Answer 3

I just found another clever way to do this, using a unittest.mock, documented here.

It's quite general, as one can specify a dictionary with many variables:

import unittest.mock
a = b = 1
with unittest.mock.patch.dict(locals(), a=2, b=3):
  print(a, b)  # shows 2, 3
print(a, b)  # shows 1, 1

Also, it works even if the variables are not previously defined in the current scope.

with unittest.mock.patch.dict(locals(), c=4):
  assert 'c' in locals()
assert 'c' not in locals()

Someone

Answer 4

I normally use this custom attr_as context manager:

from contextlib import contextmanager

@contextmanager
def attr_as(obj, field:str, value) -> None:
    old_value = getattr(obj, field)
    setattr(obj, field, value)
    yield
    setattr(obj, field, old_value)

You can then use it with the exact same set of arguments you'd use for setattr in attr_as.

class Foo:
    def __init__(self):
        self.x = 1

foo = Foo()
with attr_as(foo, 'x', 2):
    print(foo.x)

bar = 3
with attr_as(sys.modules[__name__], 'bar', 4):
    print(bar) 

Note, if you need to preserve the existence/nonexistence of the attribute rather than just the value, that's also possible with just a few more lines:

from contextlib import contextmanager

@contextmanager
def attr_as(obj, field:str, value) -> None:
    old_exists = hasattr(obj, field)
    if old_exists:
        old_value = getattr(obj, field)
    setattr(obj, field, value)
    yield
    if old_exists:
        setattr(obj, field, old_value)
    else:
        delattr(obj, field)

Riebling

Answer 5

How about a closure?

i.e.

#!/usr/bin/python

def create_closure(arg):
    var = arg

    def hide_me():
        return var

    return hide_me


x = 10
clo = create_closure(x)

x *= 2
x2 = clo()
x += 1

print "x: {0}".format(x)
print "x2: {0}".format(x2)

This yields:

x: 21
x2: 10

And since this is a closure, it could be substantially expanded to preserve a whole bunch of other variables and state, and then either used purely in the closure form going forward, or used as the slate from which to restore state later once you're done.

Faxon

Interesting. What does the closure achieve that you couldn't do with a temporary variable, as @arshajii suggests? In other words, instead of clo = create_closure(x) just set tmp = x? – Meyers

Well, being a closure it has the advantage of being able to be carried around everywhere: passed into other functions, scopes, etc. and it will always have that value. It's probably more useful if you use it for more than just a simple int. i.e. to store a state containing several different variables, structures, etc. You can also create more than one of them for different sets of values, which again may come in handy when storing states at different points and things of that nature. (My example above is contrived to illustrate the general concept.) – Faxon

Imagine that closure containing, say, 10 different variables representing a state, and say you wanted to have 4 different sets of those representing various points in your app. You could recall those at any point, pass them around, store them in a dict keyed by time or something, etc. All of that is much more expressive and more easily tracked than a bunch of separately declared local variables. The closure is essentially a stack frame of sorts in this type of usage. – Faxon