Python 'with' not deleting object

Question

Trying to properly delete a Python object. I'm creating an object and then supposedly deleting it with a 'with' statement. But when I do a print out after the 'with' statement is closed.... the object is still there:

class Things(object):
   def __init__(self, clothes, food, money):
       self.clothes = clothes
       self.food = food
       self.money = money

   def __enter__(self):
       return self

   def __exit__(self, exc_type, exc_val, exc_tb):
       print('object deleted')

with Things('socks','food',12) as stuff:
    greg = stuff.clothes
    print(greg)


print(stuff.clothes)

returns :

socks
object deleted
socks

Toffee

Alex Taylor's answer is right on the money. I want to add that since Python has automatic memory management (garbage collection) you don't need to worry about deleting objects that are no longer in use. So your use of a context manager for this purpose is pointless. Also the variable named "stuff" is created by the with statement but continues to exist until the end of the script, as you observed. If you have a line del stuff then the name "stuff" becomes undefined. It is rare that you have to do this. – Bureaucratize

Possible duplicate of Variable defined with with-statement available outside of with-block? – Anaesthesia

Answer 1

Python's with statement is not about deleting objects - it's about resource management. The __enter__ and __exit__ methods are for you to supply resource initialization and destruction code i.e. you may choose to delete something there, but there is no implicit deletion of objects. Have a read of this with article to get a better understanding of how to use it.

The object stays in scope after the with statement. You could call del on it if that's what you want. Since it's in scope you can query it after it's underlying resources have been closed. Consider this psuedo code:

class DatabaseConnection(object):
  def __init__(self, connection):
    self.connection = connection
    self.error = None

  def __enter__(self):
    self.connection.connect()

  def __exit__(self, exc_type, exc_val, exc_tb):
    self.connection.disconnect()

  def execute(self, query):
    try
      self.connection.execute(query)
    except e:
      self.error = e

with DatabaseConnection(connection) as db:
  db.execute('SELECT * FROM DB')
if db.error:
  print(db.error)

del db

We wouldn't want to keep a database connection hanging around longer than we need (another thread/client might need it), so instead we allow the resource to be freed (implicitly at the end of the with block), but then we can continue to query the object after that. I've then added an explicit del to tell the runtime that the code is finished with the variable.

Afterwards

Answer 2

with calls the __exit__ method of an object upon exiting the scope of the block. What your __exit__ method does is just print object_deleted. You have to actually put the code for destroying the object inside your __exit__ method (but note, this isn't good practice!).

What happens is:

with Things('socks','food',12) as stuff:
    # the __enter__() method is called and the returned object
    # is assigned to the variable "stuff"
    greg = stuff.clothes
    print(greg)
# when you've exited the block here, the __exit__() method is called.

Regarding your desire to explicitly delete the object, you should leave it to Python's garbage collector. Read this question, it will help. You can try using gc.collect() or the overriding the __del__ method. Here is another good discussion.

Whiplash

Well, it's not 100% correct really. You cannot delete self inside __exit__. – Explicate

yes please... in THIS case, show me the proper exit method to delete a Things object – Toffee

I've edited my answer recommending the OP to leave it to Python's garbage collector. – Whiplash

@user3583384 You can't do that. You're executing a method of an object which you're trying to delete. But your code is very simple, do you need to use the context manager at all? Why not: stuff = Things(...); greg = stuff.clothes; print(greg); del stuff ? – Explicate

That's what I'm saying in my answer - with doesn't do the deletion. You still have to call del if you want to try and 'force' a deletion. You could delete fields of the object inside __exit__, but the object is still in scope after the with block. You've seen that yourself with your example. – Afterwards

@ChuckLoganLim: You said "[...] leave it to Python's garbage collector.". In fact in this case the reference counting mechanism of Python would delete the object. The garbage collector wouldn't even see it. – Dorella

@AlexTaylor: del won't delete the object. del will delete the name which is bound to the object and after that Python might delete it. – Dorella

Answer 3

Trying to properly delete a Python object

This is your first mistake, interpreted languages explicitly free you from having to worry about deleting objects to free memory. What they do is delete them when there are no more references and the object is no longer in scope.

Secondly:

with Things('socks','food',12) as stuff:
    greg = stuff.clothes
    print(greg)

Using a with, despite indenting doesn't create a new scope.

Contrast your code with:

class Things(object):
   def __init__(self, clothes, food, money):
       self.clothes = clothes
       self.food = food
       self.money = money

   def __enter__(self):
       return self

   def __exit__(self, exc_type, exc_val, exc_tb):
       print('object deleted')

stuff = "no stuff"
def do_it():
    with Things('socks','food',12) as stuff:
        greg = stuff.clothes
        print(greg)
        print(stuff)

do_it()
print(stuff)

Which gives:

socks
<__main__.Things object at 0xb744370c>
object deleted
no stuff

Here the internal stuff inside do_it() no longer exists, so when the second print(stuff) is run there is "no stuff".

Ultimately, don't worry about deleting objects. It will happen, and its not your job to manage it.

Anaesthesia

Answer 4

So, thanks everyone for the good links. But I really wanted to actually delete an object and not wait for the fancy garbage collector. This worked:

class Things(object):
   def __init__(self,clothes, food, money):
       self.clothes = clothes
       self.food = food
       self.money = money

   def __enter__(self):
       return self

   def __exit__(self, exc_type, exc_val, exc_tb):
       print('nothing happens')



with Things('socks','food',12) as stuff:
    print(stuff.clothes)
    del stuff


print(stuff)

now returns:

socks
nothing happens
Traceback (most recent call last):
  File "/home/jeff/PycharmProjects/tensorflow_sandbox/main", line 36, in <module>
    print(stuff)
NameError: name 'stuff' is not defined

Hurray! it's deleted

Toffee

You might consider having a StuffManager that has an __enter__ method creating and returning a Stuff object and an __exit__ method deling it and using that class in you with statement. – Cicada

@Paul Evans That would not be a solution in this case, because there still is a reference to the object outside the scope, and del only removes the variable (in that case self) and decreases the reference count from two to one – Worley

Answer 5

In a context manager what you would like to achieve is possible by returning a weak reference to the instance of Things that does not increase the reference count of the object, and hence becomes an invalid reference once we __exit__ and the last reference of the object is not used anymore. Then you don't have to call delete manually after exiting the context manager.

In general this can be done simply by wrapping the return value of the __enter__ function:

import weakref

class Things(object):
   ...

   def __enter__(self):
       return weakref.proxy(self)

Your entire code snipplet looks as follows:

import weakref
class Things(object):
   def __init__(self, clothes, food, money):
       self.clothes = clothes
       self.food = food
       self.money = money

   def __enter__(self):
       return weakref.proxy(self)

   def __exit__(self, exc_type, exc_val, exc_tb):
       print('object deleted')

with Things('socks','food',12) as stuff:
    greg = stuff.clothes
    print(greg)


try:
    print(stuff.clothes)
except ReferenceError as e:
    print('Could not access stuff', e)

try:
    print(stuff)
except ReferenceError as e:
    print('Could not access stuff', e)

print(stuff is None, type(stuff))

The output of the program is

socks
object deleted
Could not access stuff weakly-referenced object no longer exists
Could not access stuff weakly-referenced object no longer exists
False <class 'weakproxy'>

As you see in the program's output, the only caveat of this solution is, that the Exception thrown when trying to access the variable stuff is different from the Exception that would be thrown if the object were deleted, and that stuff is of type weakproxy. But for all intents and purposes this solution does what it is supposed to do: raise an Exception when we try to access an object that is not supposed to be accessed anymore.

This solution works for both python2.7 and python3.x.

Worley