boost-python when C++ method returns std::map<string,X*>

Question

I'm exposing an API to Python, written in C++ that I have no access to change, using Boost Python.

I have successfully exposed methods returning references to a std:map where the key,value pairs are value types - eg:

class_< std::map<std::string, std::string> >("StringMap")
    .def(map_indexing_suite< std::map<std::string, std::string>, true >());

This works seamlessly. But when trying to achieve a similar result where the map values are pointers to classes I've exposed within the API doesn't work:

struct X_wrap : X, wrapper<X>
{
     X_wrap(int i): X(i) {}
    // virtual methods here, omitted for brevity - as unlikely to be the issue
}

BOOST_PYTHON_MODULE(my_py_extension)
{

    class_< std::map<std::string, X*> >("XPtrMap")
        .def(map_indexing_suite< std::map<std::string, X*> >());

    class_<X_wrap, boost::noncopyable, bases<XBase> >("X", init<int>())

   // other definitions omitted
}

Error seen in g++ 7.3.0:

/usr/include/boost/python/detail/caller.hpp:100:98: error: ‘struct boost::python::detail::specify_a_return_value_policy_to_wrap_functions_returning<X*>’ has no member named ‘get_pytype’

I understand why the compiler is complaining - the X* in the map needs to be wrapped in a call policy so that it can be returned to Python, just like with a basic method that returns a raw pointer.

My question is what is the best way to do this?

From Googling it strikes that I can perhaps specify a DerivedPolicies child class of map_indexing_suite that will overload the necessary parts to wrap the X* in an appropriate return_value_policy. However so far I've be unsuccessful in putting anything together that the compiler doesn't bawk at!

I also suspect I can literally copy-and-paste the whole map_indexing_suite and rename it, and make the changes therein to produce a new indexing_suite with the right return_value_policy, but this seems ugly compared to the solution using DerviedPolicies - assuming I'm right that DeriviedPolicies can be used at all!

Any help, pointers, or examples gratefully received!

EDIT

I have proved that the cut-and-paste option works with a single trivial change of is_class to is_pointer. It's curious that is_pointer is not allowed in the original as the target policy can handle pointers. I'm yet to convince myself that it's an object lifetime restriction that means pointers are not allowed in the original?

The whole class is public so I suspect it's possible to avoid the full cut-and-paste by simply inheriting from map_indexing_suite or perhaps by using the mysterious DerivedPolicies parameter?

    extension_def(Class& cl)
    {
        //  Wrap the map's element (value_type)
        std::string elem_name = "mapptr_indexing_suite_";
        object class_name(cl.attr("__name__"));
        extract<std::string> class_name_extractor(class_name);
        elem_name += class_name_extractor();
        elem_name += "_entry";

        typedef typename mpl::if_<
            mpl::and_<is_pointer<data_type>, mpl::bool_<!NoProxy> >
          , return_internal_reference<>
          , default_call_policies
        >::type get_data_return_policy;

        class_<value_type>(elem_name.c_str())
            .def("__repr__", &DerivedPolicies::print_elem)
            .def("data", &DerivedPolicies::get_data, get_data_return_policy())
            .def("key", &DerivedPolicies::get_key)
        ;
    }

EDIT 2

Now see answer

When I have a c++ class object that I need to use from python code, I provide an identifier string, and pass that string to a module function which does whatever I need it to do in c++. So instead of returning a map, I'll just have the python code pass in the map key, so the c++ function can look it up in the map (internal to the c++ extension module) and do whatever it needs to do. (sometimes the identifier is just the pointer, but python doesn't know or care about that) — Kenny Ostrom
– Kenny Ostrom, Commented Mar 27, 2019 at 19:38
Thanks. So you wrap the C++ method that returns the map with a C++ function taking the key and returning the value only? This is how my original code worked. You can even have your C++ helper return a dict of the whole map. Nothing wrong with this. My reason for looking beyond is to minimize any C++ middleware component I'll need to maintain between the 3rd party library and my python, also for debugging I like to be able to iterate over the whole map in pdb in python without worrying (so much) if there is a bug in my middleware... and lastly, I was just curious to see if it was possible :) — Phil
– Phil, Commented Mar 27, 2019 at 20:52

Phil · Accepted Answer · 2019-03-29 02:35:51Z

Slightly cleaner implementation from the cut-and-paste is to inherit map_indexing_suite - a few tweaks are needed to make this work.

This seems reasonably sensible - if someone chimes in with a neater solution or can better explain DerivedPolicies then great, otherwise I'll accept the below as the answer in a few days or so...

using namespace boost;
using namespace boost::python;

//Forward declaration
template <class Container, bool NoProxy, class DerivedPolicies>
class mapptr_indexing_suite;

template <class Container, bool NoProxy>
class final_mapptr_derived_policies
    : public mapptr_indexing_suite<Container,
        NoProxy, final_mapptr_derived_policies<Container, NoProxy> > {};

template <
    class Container,
    bool NoProxy = false,
    class DerivedPolicies
        = final_mapptr_derived_policies<Container, NoProxy> >
class mapptr_indexing_suite
    : public map_indexing_suite<
    Container,
    NoProxy,
    DerivedPolicies
    >
{
public:
    // Must be explicit if the compiler is
    // going to take from the base class
    using typename map_indexing_suite<
        Container,NoProxy,DerivedPolicies>::data_type;
    using typename map_indexing_suite<
        Container,NoProxy,DerivedPolicies>::value_type;

    // Only one class needs to be overridden from the base
    template <class Class>
    static void
    extension_def(Class& cl)
    {
        //  Wrap the map's element (value_type)
        std::string elem_name = "mapptr_indexing_suite_";
        object class_name(cl.attr("__name__"));
        extract<std::string> class_name_extractor(class_name);
        elem_name += class_name_extractor();
        elem_name += "_entry";

        // use of is_pointer here is the only
        // difference to the base map_indexing_suite
        typedef typename mpl::if_<
            mpl::and_<std::is_pointer<data_type>, mpl::bool_<!NoProxy> >
            , return_internal_reference<>
            , default_call_policies
            >::type get_data_return_policy;

        class_<value_type>(elem_name.c_str())
            .def("__repr__", &DerivedPolicies::print_elem)
            .def("data", &DerivedPolicies::get_data, get_data_return_policy())
            .def("key", &DerivedPolicies::get_key)
            ;
    }
};

Collectives™ on Stack Overflow

boost-python when C++ method returns std::map<string,X*>

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related