Get object by id()? [duplicate]

Question

Let's say I have an id of a Python object, which I retrieved by doing id(thing). How do I find thing again by the id number I was given?

Calamanco

I'm curious: why do you want to do this? What is your objective? – Thrall

@Craig McQueen: #1400795 – Calamanco

I couldn't re-find the source of this but I thought that what is returned by id() is whatever the particular distribution wants it to be. Restated CPython may, at this time, return a memory-like address now, but other distributions could return different object types or ints that are not memory pointers. It would be nice if there was a built in function to obtain an object by what is returned by id(). Although other then persistence use-cases are hard to imagine. Also although *variable, like C seems to make sense; I love Python for it's lack of punctuation trickery like most other languages. – Elyn

Legitimate reason to do this: debugging. The default repr on objects includes their memory address. Sometimes when debugging (especially interactively), you want to be able to access that object without trying to dig into where it is defined. – Landmark

@asmeurer: I think there are other valid reasons for doing something like this (assuming the use of CPython). I've used it in at least a couple times of my answers to other questions here. – Toughie

@Toughie I'm by no means suggesting debugging is the only use case. It's definitely one that no one would argue with, though. – Landmark

By the way, this is my own answer that uses this functionality #44878245. – Landmark

@asmeurer: Very interesting non-debugging usage. There's a couple of my own here and here. Note they both use a function named di() I posted in my answer to question very similar question to this one. – Toughie

@CraigMcQueen I guess pickle would use something like this when unpickling a file, in order not to clone objects accidentally that were referenced several times in the object that was pickled? – Pestle

Answer 1

You'll probably want to consider implementing it another way. Are you aware of the weakref module?

(Edited) The Python weakref module lets you keep references, dictionary references, and proxies to objects without having those references count in the reference counter. They're like symbolic links.

Coset

Sometimes you can't create weak reference to the object, e.g: TypeError: cannot create weak reference to 'lxml.etree._Element' object – Heathen

Answer 2

If the object is still there, this can be done by ctypes:

import ctypes
a = "hello world"
print ctypes.cast(id(a), ctypes.py_object).value

output:

hello world

If you don't know whether the object is still there, this is a recipe for undefined behavior and weird crashes or worse, so be careful.

Thornton

In CPython, today, anyhow. :^) – Madness

This one is a perfect answer! – Ayurveda

@HamidFzM No, not really. If I have an ID, I maybe don't even know whether the object still exists or not. – Omaromara

repr outputs the hexadecimal representation of id(a), in order to use ctypes one must convert it back to decimal by using int(hexid, 0). just my two cents here – Broncho

Is there a way to check if a memory address exists first before doing this? If you pass an invalid value (say, because it's been garbage collected), the interpreter segfaults. – Landmark

@Broncho You don't convert it back to decimal; you convert it to Python's internal representation which is probably base 256 or base 65536... and you should be using int(hexid, 16) instead. – Merissameristem

I'm intrigued now—what could a program do that's worse than the things covered under "undefined behavior"? – Digenesis

@asmeurer: Nope. There are platform-specific ways you could check if the memory is still allocated, but even if a memory region is allocated and looks like it contains a valid object, it might have been passed back to the Python memory management system, or it might just be a random blob of bytes in the middle of some other data that just happens to look like an object. – Alidia

@Digenesis In general exposing a security hole in your application would be worse than those, but it seems to me OP was trying to say if you're down this rabbit hole, you're already digging under the hood in e.g. gc output and trying to find a leaked resource or something. OP's just warning that this wouldn't be something to rely on for some hacky feature or workaround in production code. But in a pdb session, anything that can shine a light on an obscure bug is fair game. – Shank

Answer 3

You can use the gc module to get all the objects currently tracked by the Python garbage collector.

import gc

def objects_by_id(id_):
    for obj in gc.get_objects():
        if id(obj) == id_:
            return obj
    raise Exception("No found")

Cauvery

This has an aliasing issue: an ID obtained at an arbitrary point in the past may now refer to a different object. – Carpophore

As long as you've maintained a reference to the object, that won't happen. Just the same, this is generally a bad idea. – Mensurable

Answer 4

Short answer, you can't.

Long answer, you can maintain a dict for mapping IDs to objects, or look the ID up by exhaustive search of gc.get_objects(), but this will create one of two problems: either the dict's reference will keep the object alive and prevent GC, or (if it's a WeakValue dict or you use gc.get_objects()) the ID may be deallocated and reused for a completely different object.

Basically, if you're trying to do this, you probably need to do something differently.

Carpophore

+1: Agree: Don't do this. Simply create a proper dictionary of objects with proper keys -- you'll be a lot happier. – Quoits

Answer 5

Just mentioning this module for completeness. This code by Bill Bumgarner includes a C extension to do what you want without looping throughout every object in existence.

The code for the function is quite straightforward. Every Python object is represented in C by a pointer to a PyObject struct. Because id(x) is just the memory address of this struct, we can retrieve the Python object just by treating x as a pointer to a PyObject, then calling Py_INCREF to tell the garbage collector that we're creating a new reference to the object.

static PyObject *
di_di(PyObject *self, PyObject *args)
{
    PyObject *obj;
    if (!PyArg_ParseTuple(args, "l:di", &obj))
        return  NULL;

    Py_INCREF(obj);
    return obj;
}

If the original object no longer exists then the result is undefined. It may crash, but it could also return a reference to a new object that's taken the location of the old one in memory.

Millennium

Answer 6

eGenix mxTools library does provide such a function, although marked as "expert-only": mx.Tools.makeref(id)

Malikamalin

Answer 7

This will do:

a = 0
id_a = id(a)
variables = {**locals(), **globals()}
for var in variables:
    exec('var_id=id(%s)'%var)
    if var_id == id_a:
        exec('the_variable=%s'%var)
print(the_variable)
print(id(the_variable))

But I suggest implementing a more decent way.

Menagerie

This absolutely does not do. This only suffices for objects directly residing in the current lexical scope (either local or global); this fails to suffice for objects defined in other scopes (e.g., other modules, packages, and scripts). – Fevre