class Package:
def __init__(self):
self.files = []
# ...
def __del__(self):
for file in self.files:
os.unlink(file)
__del__(self) above fails with an AttributeError exception. I understand Python doesn't guarantee the existence of "global variables" (member data in this context?) when __del__() is invoked. If that is the case and this is the reason for the exception, how do I make sure the object destructs properly?
I'd recommend using Python's with statement for managing resources that need to be cleaned up. The problem with using an explicit close() statement is that you have to worry about people forgetting to call it at all or forgetting to place it in a finally block to prevent a resource leak when an exception occurs.
To use the with statement, create a class with the following methods:
def __enter__(self)
def __exit__(self, exc_type, exc_value, traceback)
In your example above, you'd use
class Package:
def __init__(self):
self.files = []
def __enter__(self):
return self
# ...
def __exit__(self, exc_type, exc_value, traceback):
for file in self.files:
os.unlink(file)
Then, when someone wanted to use your class, they'd do the following:
with Package() as package_obj:
# use package_obj
The variable package_obj will be an instance of type Package (it's the value returned by the __enter__ method). Its __exit__ method will automatically be called, regardless of whether or not an exception occurs.
You could even take this approach a step further. In the example above, someone could still instantiate Package using its constructor without using the with clause. You don't want that to happen. You can fix this by creating a PackageResource class that defines the __enter__ and __exit__ methods. Then, the Package class would be defined strictly inside the __enter__ method and returned. That way, the caller never could instantiate the Package class without using a with statement:
class PackageResource:
def __enter__(self):
class Package:
...
self.package_obj = Package()
return self.package_obj
def __exit__(self, exc_type, exc_value, traceback):
self.package_obj.cleanup()
You'd use this as follows:
with PackageResource() as package_obj:
# use package_obj
with Resource(param1, param2) as r: # ... –
Gib __init__ method to make my "interface" clearer. Is it more "Pythonic" or are there advantages I don't see in using *args and **kwargs? I understand it will only fail at interpretation time anyway, but perhaps having a clearer __init__ signature for the resource makes it easier to use for programmers with a nice IDE? –
Gib Package is an (abstract) base class? Is there a way to inherit from the inner class? –
Sybille PackageResource can be simplified using contextlib, see my answer –
Hypophysis close, simply use contextlib.closing –
Hypophysis __exit__ and __del__? –
Scrim with when the lifetime of some object exists outside of some syntactical frame. But instead is tied to some other event occurring. –
Ornithopter with when you have to use several objects with parameters in your code. The layered structure of with is good when there is 1 layer, but not 7! –
Salgado The standard way is to use atexit.register:
# package.py
import atexit
import os
class Package:
def __init__(self):
self.files = []
atexit.register(self.cleanup)
def cleanup(self):
print("Running cleanup...")
for file in self.files:
print("Unlinking file: {}".format(file))
# os.unlink(file)
But you should keep in mind that this will persist all created instances of Package until Python is terminated.
Demo using the code above saved as package.py:
$ python
>>> from package import *
>>> p = Package()
>>> q = Package()
>>> q.files = ['a', 'b', 'c']
>>> quit()
Running cleanup...
Unlinking file: a
Unlinking file: b
Unlinking file: c
Running cleanup...
with? did they explicitly call __enter__?) The downside is of course if you need the cleanup to happen before python exits, it won't work. In my case, I don't care if it is when the object goes out of scope or if it isn't until python exits. :) –
Ceres atexit.register(self.__exit__)? –
Anurous __exit__, and use a contextmanager? Also, __exit__ takes additional arguments (i.e. __exit__(self, type, value, traceback)), so you would need to accout for those. Either way, it sounds like you should post a separate question on SO, because your use-case appears unusual? –
Covarrubias A better alternative is to use weakref.finalize. See the examples at Finalizer Objects and Comparing finalizers with __del__() methods.
stop() method to close the ports and join() the processes. However, if the programs exits unexpectedly stop() is not called - I solved that with a finalizer. But in any case I call _finalizer.detach() in the stop method to prevent calling it twice (manually and later again by the finalizer). –
Congo As an appendix to Clint's answer, you can simplify PackageResource using contextlib.contextmanager:
@contextlib.contextmanager
def packageResource():
class Package:
...
package = Package()
yield package
package.cleanup()
Alternatively, though probably not as Pythonic, you can override Package.__new__:
class Package(object):
def __new__(cls, *args, **kwargs):
@contextlib.contextmanager
def packageResource():
# adapt arguments if superclass takes some!
package = super(Package, cls).__new__(cls)
package.__init__(*args, **kwargs)
yield package
package.cleanup()
def __init__(self, *args, **kwargs):
...
and simply use with Package(...) as package.
To get things shorter, name your cleanup function close and use contextlib.closing, in which case you can either use the unmodified Package class via with contextlib.closing(Package(...)) or override its __new__ to the simpler
class Package(object):
def __new__(cls, *args, **kwargs):
package = super(Package, cls).__new__(cls)
package.__init__(*args, **kwargs)
return contextlib.closing(package)
And this constructor is inherited, so you can simply inherit, e.g.
class SubPackage(Package):
def close(self):
pass
Package.__new__() method, however. Or maybe we can. We could probably define either a class decorator or metaclass genericizing that boilerplate for us. Food for Pythonic thought. –
Duodenary Package should also do this (though I haven't tested that yet), so no metaclass should be required. Though I have found some pretty curious ways to use metaclasses in the past... –
Hypophysis Package (or better a class named Closing) as your class parent instead of object. But don't ask me how multiple inheritance messes up with this... –
Hypophysis Here is a minimal working skeleton:
class SkeletonFixture:
def __init__(self):
pass
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, traceback):
pass
def method(self):
pass
with SkeletonFixture() as fixture:
fixture.method()
Important: return self
If you're like me, and overlook the return self part (of Clint Miller's correct answer), you will be staring at this nonsense:
Traceback (most recent call last):
File "tests/simplestpossible.py", line 17, in <module>
fixture.method()
AttributeError: 'NoneType' object has no attribute 'method'
Hope it helps the next person.
return self , a discussion is here #38282353 –
Tonkin I don't think that it's possible for instance members to be removed before __del__ is called. My guess would be that the reason for your particular AttributeError is somewhere else (maybe you mistakenly remove self.file elsewhere).
However, as the others pointed out, you should avoid using __del__. The main reason for this is that instances with __del__ will not be garbage collected (they will only be freed when their refcount reaches 0). Therefore, if your instances are involved in circular references, they will live in memory for as long as the application run. (I may be mistaken about all this though, I'd have to read the gc docs again, but I'm rather sure it works like this).
__del__ can be garbage collected if their reference count from other objects with __del__ is zero and they are unreachable. This means if you have a reference cycle between objects with __del__, none of those will be collected. Any other case, however, should be resolved as expected. –
Hint I think the problem could be in __init__ if there is more code than shown?
__del__ will be called even when __init__ has not been executed properly or threw an exception.
__del__ is to explicitly declare all members at class-level, ensuring that they always exist, even if __init__ fails. In the given example, files = () would work, though mostly you'd just assign None; in either case, you still need to assign the real value in __init__. –
Enchanter A good idea is to combine both approaches.
Implement a context manager for explicit life-cycle handling and handle automatic cleanup in case the user forgets it or it is not convenient to do so, like in an interpreter. Guaranteed cleanup at destruction is best done by weakref.finalize.
This is how many libraries actually do it. And some issue a warning if the context manager is not used, to encourage people to use it.
Once set up, a good thing about weakref.finalize is, that its logic is guaranteed to be called exactly once. So it can be used to trigger the cleanup in all cases.
import os
from typing import List
import weakref
class Package:
def __init__(self):
self.files = []
self._finalizer = weakref.finalize(self, self._cleanup_files, self.files)
@staticmethod
def _cleanup_files(files: List):
for file in files:
os.unlink(file)
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, traceback):
self._finalizer()
weakref.finalize returns a callable finalizer object which will be called when obj is garbage collected. Unlike an ordinary weak reference, a finalizer will always survive until the reference object is collected.
One downside is, that one has to implement the cleanup logic inside a staticmethod, as at that time the object is not alive anymore.
A benefit over atexit.register is that the object is not held in memory until the interpreter is shut down. The cleanup happens when the object gets garbage collected.
And unlike object.__del__, weakref.finalize is guaranteed to be called at interpreter shutdown, as the interpreter keeps special track of them.
Just wrap your destructor with a try/except statement and it will not throw an exception if your globals are already disposed of.
Edit
Try this:
from weakref import proxy
class MyList(list): pass
class Package:
def __init__(self):
self.__del__.im_func.files = MyList([1,2,3,4])
self.files = proxy(self.__del__.im_func.files)
def __del__(self):
print self.__del__.im_func.files
It will stuff the file list in the del function that is guaranteed to exist at the time of call. The weakref proxy is to prevent Python, or yourself from deleting the self.files variable somehow (if it is deleted, then it will not affect the original file list). If it is not the case that this is being deleted even though there are more references to the variable, then you can remove the proxy encapsulation.
It seems that the idiomatic way to do this is to provide a close() method (or similar), and call it explicitely.
atexit.register is the standard way as has already been mentioned in ostrakach's answer.
However, it must be noted that the order in which objects might get deleted cannot be relied upon as shown in example below.
import atexit
class A(object):
def __init__(self, val):
self.val = val
atexit.register(self.hello)
def hello(self):
print(self.val)
def hello2():
a = A(10)
hello2()
a = A(20)
Here, order seems legitimate in terms of reverse of the order in which objects were created as program gives output as :
20
10
However when, in a larger program, python's garbage collection kicks in object which is out of it's lifetime would get destructed first.
© 2022 - 2024 — McMap. All rights reserved.
__del__should not be used as the counterpart of__init__. (I.e., it is not a "destructor" in the sense that__init__is a constructor. – Sprat__init__succeeds. If that's not the case, I would consider it a bug. I don't see a big argument there, to be honest. – Globulin