How to convert NumPy ndarray to C++ vector with Boost.Python and back?

Question

I am working on a project where I need to convert an ndarray in Python to a vector in C++ and then return a processed vector from C++ back to Python in an ndarray. I am using Boost.Python with its NumPy extension. My problem specifically lies in converting from ndarray to vector, as I am using an extended class of vector:

class Vector
{
   public:
      Vector();
      Vector(double x, double y, double z);
      /* ... */
      double GetLength(); // Return this objects length.
      /* ... */
      double x, y, z;
};

The ndarray I receive is nx2 and filled with x,y data. Then I process the data in C++ with a function, which returns an std::vector<Vector>. This vector then should be returned to Python as an ndarray, BUT only with the x and y values.

I have written the following piece of code, with inspiration from "how to return numpy.array from boost::python?" and the gaussian.cpp from the Boost NumPy examples.

#include <vector>
#include "Vector.h"
#include "ClothoidSpline.h"

#include <boost/python/numpy.hpp>

namespace py = boost::python;
namespace np = boost::python::numpy;

std::vector<Vector> getFineSamples(std::vector<Vector> data)
{
    /* ... */
}

np::ndarray wrapper(np::ndarray const & input)
{
    std::vector<Vector> data;

    /* Python ndarray --> C++ Vector */
    Py_intptr_t const* size = input.get_shape();
    Py_intptr_t const* strides = input.get_strides();

    double x;
    double y;
    double z = 0.0;

    for (int i = 0; i < size[0]; i++)
    {
        x = *reinterpret_cast<double const *>(input.get_data() + i * strides[0] + 0 * strides[1]);
        y = *reinterpret_cast<double const *>(input.get_data() + i * strides[0] + 1 * strides[1]);
        data.push_back(Vector::Vector(x,y,z));
    }

    /* Run Algorithm */
    std::vector<Vector> v = getFineSamples(data);

    /* C++ Vector --> Python ndarray */
    Py_intptr_t shape[1] = { v.size() };
    np::ndarray result = np::zeros(2, shape, np::dtype::get_builtin<std::vector<Vector>>());
    std::copy(v.begin(), v.end(), reinterpret_cast<double*>(result.get_data()));

    return result;
}

EDIT: I am aware that this is a (possibly) failed attempt, and I am more interested in a better method to solve this problem, than edits to my code.

So to sum up:

How do I convert an boost::python::numpy::ndarray to a std::vector<Vector>?
How do I convert a std::vector<Vector> to an boost::python::numpy::ndarray, returning only x and y?

As a last note: I know almost nothing about Python, and I am beginner/moderate in C++.

fabda01 · Accepted Answer · 2019-02-22 17:56:14Z

7

I will consider the title of your question to give a more generalized answer to whoever finds this post.

You have a boost::python::numpy::ndarray called input that contains doubles and you want to convert it a std::vector<double> called v:

int input_size = input.shape(0);
double* input_ptr = reinterpret_cast<double*>(input.get_data());
std::vector<double> v(input_size);
for (int i = 0; i < input_size; ++i)
    v[i] = *(input_ptr + i);

Now, you have a std::vector<double> called v and you want to convert it back to boost::python::numpy::ndarray of doubles called output:

int v_size = v.size();
py::tuple shape = py::make_tuple(v_size);
py::tuple stride = py::make_tuple(sizeof(double));
np::dtype dt = np::dtype::get_builtin<double>();
np::ndarray output = np::from_data(&v[0], dt, shape, stride, py::object());

Supposing you are wrapping this function, don't forget that you need to create a new reference to this array before returning it to python:

np::ndarray output_array = output.copy();

answered Feb 22, 2019 at 17:56

fabda01

3,8433 gold badges37 silver badges45 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

T-800 Over a year ago

why do we have to create a new reference to this array? Will the existing one be deleted? I cannot see the exact reason and I would appreciate if you could explain the reason for that

fabda01 Over a year ago

@T-800 Yes, I used to have issues with python deleting the dynamic array when it went out of scope. Have you tried not creating a new reference to see if you still have this problem? You can check more here Memory Management

T-800 Over a year ago

Well, I have had another problem: when I used your proposed implementation, the first element of the vector is always empty. The pointer probably does not exist, but I don't know why. Copying the output array was not a remedy. Therefore, I created an empty ndarray and filled the values in loop.

Elad Maimoni Over a year ago

@yellow01 this will not work if boost::python::numpy::ndarray referes to a numpy array that is not contiguous in memory.

Collectives™ on Stack Overflow

How to convert NumPy ndarray to C++ vector with Boost.Python and back?

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related