boost python return same instance with make_constructor

Question

I'm trying to make so this code returns the same instance in both the init function and the callback function

test1.py

import test1

c = test1.C()
print 'init:', c

def func(c):
    print 'func:', c

test1.register_callback(func)

test1.cpp

#include <iostream>
#include <vector>

#include <boost/python.hpp>

using namespace boost::python;

class C;
std::vector<boost::shared_ptr<C>> v;

class C
: public boost::noncopyable
{
public:
    C() {
        std::cout << "C()" << std::endl;
    }

    ~C() {
        std::cout << "~C()" << std::endl;
    }
};

boost::shared_ptr<C> create() {
    C *c = new C();
    auto ptr = boost::shared_ptr<C>(c);
    v.push_back(ptr);
    return ptr;
}

void register_callback(object func) {
    func(v[0]);
}

BOOST_PYTHON_MODULE(test1)
{
    class_<C, boost::shared_ptr<C>, boost::noncopyable>("C", no_init)
        .def("__init__", make_constructor(&create))
    ;

    def("register_callback", register_callback);
}

The output I get now is:

init: <test1.C object at 0x7f62181bd5d0>
func: <test1.C object at 0x7f62181c1848>

And what I'm trying to get is:

init: <test1.C object at 0x7f62181bd5d0>
func: <test1.C object at 0x7f62181bd5d0>

is this possible and how?

Nominative

Answer 1

When Python constructs an object, the __init__ will be invoked with the first argument representing the object being constructed, often called self. Boost.Python tries to hide this argument as much as possible, only exposing it to an init-expression under certain conditions. The callable Python object returned from boost::python::make_constructor() is aware of this first argument, but there are no customization points to have it forward the argument to wrapped function. One solution is to expose a C++ function as __init__ with boost::python::make_function() that accepts all of the arguments to be provided from Python, including self, then delegate to the functor returned from boost::python::make_constructor():

...
std::vector<boost::python::object> v;

void create(boost::python::object self)
{
  // Create a constructor object.  In this case, a lambda
  // casted to a function is being used.
  auto constructor = boost::python::make_constructor(+[]() {
    return boost::make_shared<C>();
  });

  // Invoke the constructor.
  constructor(self);

  // If construction does not throw, then store a reference to self.
  v.push_back(self);
}

...

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;
  python::class_<C, boost::shared_ptr<C>, boost::noncopyable>(
      "C", python::no_init)
    .def("__init__", python::make_function(&create))
    ;

  ...
}

---

Here is a complete example demonstrating this approach:

#include <boost/python.hpp>
#include <vector>
#include <boost/make_shared.hpp>

class C: public boost::noncopyable {};
std::vector<boost::python::object> v;

template <typename ...Args>
void create(boost::python::object self, Args&&... args)
{
  // Create a constructor object.
  auto constructor = boost::python::make_constructor(
    +[](Args&&...args) {
    return boost::make_shared<C>(std::forward<Args>(args)...);
  });

  // Invoke the constructor.
  constructor(self, std::forward<Args>(args)...);

  // If construction does not throw, then store a reference to self.
  v.push_back(self);
}

void register_callback(boost::python::object func)
{
  func(v[0]);
}

BOOST_PYTHON_MODULE(example)
{
  namespace python = boost::python;
  python::class_<C, boost::shared_ptr<C>, boost::noncopyable>(
      "C", python::no_init)
    .def("__init__", python::make_function(&create<>))
    ;

  python::def("register_callback", &register_callback);
}

Interactive usage:

>>> import example
>>> c1 = example.C()
>>> print 'init:', c1
init: <example.C object at 0x7f12f425d0a8>
>>> c2 = None
>>> def func(c):
...     global c2
...     print 'func:', c
...     c2 = c
...
>>> example.register_callback(func)
func: <example.C object at 0x7f12f425d0a8>
>>> assert(c1 is c2)

Lineolate

Answer 2

It already is the same C instance inside of the same shared_ptr<C>. You can easily verify this.

The issue is that you're creating two different py::objects based on this shared_ptr<C>: one from create() and one from register_callback(). Those are necessarily going to be different objects, they have nothing to do with each other.

The simple solution is to just not use make_constructor. Create a factory function:

py::object makeC() {
    C *c = new C();
    auto ptr = py::object{boost::shared_ptr<C>(c)};
    v.push_back(ptr); // store as objects, since you need them to be the same
    return ptr;
}

void register_callback(object func) {
    func(v[0]);
}

BOOST_PYTHON_MODULE(test1)
{
    class_<C, boost::shared_ptr<C>, boost::noncopyable>("C", no_init)
        /* actually no_init for real */
    ;

    def("makeC", makeC);
    def("register_callback", register_callback);
}

Campagna

Yes I know its the same instance in the vector, but how do I return the same py::object for both ways? – Nominative

@Nominative Why do you need that? – Campagna

It's for a plugin based system, plugin can create a instance in python for example c = C(); c.some_variable = 10; then another plugin gets this instance in a registered callback, if it's not the same object, the newly created some_variable is not present obviously. I had it fixed by a ugly hack, but it didn't make a clean good api, so I prefer that C() returns the same object, or rather that I can return an object, save it somewhere and reuse the same object when I do the callback, but make_constructor cannot return an py::object – Nominative

@Nominative So just don't use make_constructor. – Campagna

Now your solution is no better than my previous hack, and the plugin can't do c = C(). but instead c = makeC() which is not a good API either. – Nominative