Boost::Python- possible to automatically convert from dict --> std::map?

Question

I've got a C++ class, with a member function that can take a small-to-large number of parameters. Lets name those parameters, a-f. All parameters have default values. As a part of the python project I am working on, I want to expose this class to python. Currently, the member function looks something like this:

class myClass {
    public:
    // Constructors - set a-f to default values.

    void SetParameters(std::map<std::string, double> &);
    private:
    double a, b, c, d, e, f;
}

void myClass::SetParameters(std::map<std::string, double> const& params) {
    // Code to iterate over the map, and set any found key/value pairs to their
    // corresponding variable.  i.e.- "a" --> 2.0, would set myClass::a to 2.0
}

Ideally, in Python, I would like to accomplish this using a dict:

>>> A = myModule.myClass();
>>> A.SetParameters({'a': 2.2, 'd': 4.3, b: '9.3'})

In this way, the user could enter the values in any order, and enter any number of them to be over-ridden. Any thoughts on how this could be accomplished in boost::python? It seems to me that I can do this via changing the map input to a boost::python object, and using the extract functions. However, this would require me to change the interface of my library (I'd prefer to keep the std::map interface, and have some intermediary/auto conversion technique for the python version). Thoughts?

Aleksey Vitebskiy · Accepted Answer · 2011-05-25 02:40:28Z

25

I think there's a couple of ways that are easier to accomplish than writing your own converter. You can use boost::python's map_indexing_suite to do the conversion for you, or you can use keyword arguments in python. I personally prefer keyword arguments, as this is the more "Pythonic" way to do this.

So this is your class (I added a typedef for the map):

typedef std::map<std::string, double> MyMap;

class myClass {
public:
    // Constructors - set a-f to default values.

    void SetParameters(MyMap &);
private:
    double a, b, c, d, e, f;
};

Example using map_indexing_suite:

#include <boost/python/suite/indexing/map_indexing_suite.hpp>

using boost::python;

BOOST_PYTHON_MODULE(mymodule)
{
    class_<std::map<std::string, double> >("MyMap")
        .def(map_indexing_suite<std::map<std::wstring, double> >() );

    class_<myClass>("myClass")
        .def("SetParameters", &myClass::SetParameters);
}

Example using keyword arguments. This requires using a raw_function wrapper:

using namespace boost::python;

object SetParameters(tuple args, dict kwargs)
{
    myClass& self = extract<myClass&>(args[0]);

    list keys = kwargs.keys();

    MyMap outMap;
    for(int i = 0; i < len(keys); ++i) {
        object curArg = kwargs[keys[i]];
        if(curArg) {
            outMap[extract<std::string>(keys[i])] = extract<double>(kwargs[keys[i]]);
        }               
    }
    self.SetParameters(outMap);

    return object();
}

BOOST_PYTHON_MODULE(mymodule)
{
    class_<myClass>("myClass")
        .def("SetParameters", raw_function(&SetParameters, 1));
}

this allows you to write stuff like this in Python:

A.SetParameters(a = 2.2, d = 4.3, b = 9.3)

edited May 25, 2011 at 2:40

answered May 25, 2011 at 1:44

Aleksey Vitebskiy

2,1271 gold badge15 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

MarkD Over a year ago

@aleksey- the keyword argument method looks optimal for what I'd like to do. However, I am having some issues with the code you posted. It looks like raw_function only likes the specified function to have as inputs a tuple and a dict. Is there a separate, similar construct for class member functions?

Aleksey Vitebskiy Over a year ago

oh, I see. I think I got around this by using a wrapper class that inherits from the class I'm trying to wrap.

Aleksey Vitebskiy Over a year ago

also, the first argument in the args tuple should be the reference to your class. You can omit the self argument and extract myClass& from args[0]. See if that works.

MarkD Over a year ago

Worked great! Thanks aleksey!

YvesgereY Over a year ago

For the record, map_indexing_suite solution doesn't work, since no implicit "dict->std::map" from_python converter will be applied. Hence, you'll get an ArgumentError at runtime stating "myClass.SetParameters(dict) did not match C++ signature : SetParameters(std::map [...]".

|

Ray · Accepted Answer · 2011-05-24 21:32:32Z

11

This blog post has a pretty clear description of how to write these converters. The basic pattern is to define a class that has the form:

struct SomeType_from_PyObject
{
    SomeType_from_PyObject();
    static void* convertible(PyObject* obj_ptr);
    static void construct(PyObject* obj_ptr,
                          converter::rvalue_from_python_stage1_data* data);
};

Where the constructor is responsible for adding this converter to Boost.Python's registry:

SomeType_from_PyObject::SomeType_from_PyObject()
{
    converter::registry::push_back(&convertible,
                                   &construct,
                                   type_id<SomeType>());
}

The function convertible tells Boost whether or not this converter can convert the specified Python object:

void* SomeType_from_PyObject::convertible(PyObject* obj_ptr)
{
    if (PyMapping_Check(obj_ptr)) {
        return obj_ptr;
    } else {
        return NULL;
    }
}

The construct function actually creates an object of the conversion type:

void SomeType_from_PyObject::construct(PyObject* obj_ptr,
                                       converter::rvalue_from_python_stage1_data* data)
{
    typedef converter::rvalue_from_python_storage<SomeType> storage_t;
    storage_t* the_storage = reinterpret_cast<storage_t*>(data);
    void* memory_chunk = the_storage->storage.bytes;
    object obj(handle<>(borrowed(obj_ptr)));
    SomeType* output = new (memory_chunk) SomeType();
    // Use the contents of obj to populate output, e.g. using extract<>
    data->convertible = memory_chunk;
}

and then in your inside your BOOST_PYTHON_MODULE, include the line

SomeType_from_PyObject();

answered May 24, 2011 at 21:32

Ray

4,7311 gold badge26 silver badges33 bronze badges

3 Comments

MarkD Over a year ago

Thank you very much for this great post Ray. I started going down this route (and will most likely complete what I was working on to learn a bit more about the Python/C API. I think aleksay's keyword argument method will be how I'll go if I can get it working, but this is very valuable stuff here. +1

Aleksey Vitebskiy Over a year ago

I agree, while converters is not the way I would solve Mark's problem, I do appreciate a nice post on boost::python converters.

user1221647 Jul 14 at 12:39

Can you help me understand how is the data->convertible being memory managed? How is the convertible pointer being deallocated after the construct?

totowtwo · Accepted Answer · 2011-05-24 21:26:52Z

0

I'm just getting my toes wet with boost::python so can't completely answer your question. But the first roadblock I see is guaranteeing that the py dict's keys are all strings. Python dicts can also be keyed on tuples (and I assume more types).

answered May 24, 2011 at 21:26

totowtwo

2,1871 gold badge15 silver badges22 bronze badges

1 Comment

Wichert Akkerman Over a year ago

Anything that is hashable can be a key, and a single dictionary can have many different types of keys.

Collectives™ on Stack Overflow

Boost::Python- possible to automatically convert from dict --> std::map?

3 Answers 3

6 Comments

3 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

6 Comments

3 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related