exchange numpy array with boost.python: pyublas or boost.numpy?

Question

I am interfacing a C++ data intense library with Python by py++/boost.python. After profiling my program, I find 70% of the run time is spent on code like this:

ni = range(v2o.getHits())
tau = np.array([v2o.TofCorrectedTime[i] for i in ni])
q = [v2o.getCharge()[i] for i in ni]

v2o.TofCorrectedTime is typed __array_1_float_2368 from py++. v2o.getCharge() is typed _impl_details_range_iterator_ from py++, too. Size being about 2000, the convertion from these py++ array wrappers to numpy is slow:

In [42]: timeit np.array(v2o.TofCorrectedTime)
100 loops, best of 3: 2.52 ms per loop

In [43]: timeit np.array(v2o.getCharge())
100 loops, best of 3: 4.94 ms per loop

In [44]: timeit np.array([0]*2368)
1000 loops, best of 3: 310 µs per loop

In [45]: timeit np.array(np.zeros(2368))
100000 loops, best of 3: 4.41 µs per loop

I searched the web for a solution. The candidates are:

Questions and Answers (updated):

Is cython/memoryview easy to be integrated with boost.python and py++? I want to keep the rest of the library wrapper.

No. (Jim's answer)

cython c++ wrapper and boost.python have intrinsicly different infrastructures. It's hard for them to talk to each other. (Although in principle, we could teach py++ to output cython code. But that's another story.)

Extending the present wrapper with Boost.NumPy is the most manageable way.
Which one best suites my problem in terms of convertion overhead?

(No definite answer yet.)

Thanks

jbosch · Accepted Answer · 2013-12-04 16:47:12Z

5

(Disclaimer: I'm the primary author of Boost.NumPy.)

I'm afraid none of these options are particularly great. Here's how I think the pro/con analysis goes:

Cython has a large number of users and developers, and hence you'll have a lot more support if you go with that option. It's not at all integrated with Boost.Python, however, and I think it'd be a tremendous amount of work to make Cython objects talk to Boost.Python, let alone Py++; you'd probably need to gain a pretty solid understanding of the low-level implementation details of both Cython and Boost.Python to get that going. You'd probably be better off scrapping your Py++/Boost.Python wrappers if you want to use Cython.
Boost.NumPy has a much smaller community, and support resources are hence more limited, but it's much better suited to the code you already have. Py++ knows nothing about Boost.NumPy, so it won't automatically generate code that uses it (it might be that you could teach Py++ about Boost.NumPy; I'm not familiar enough with Py++ to know), but it's very straightforward to add custom Boost.Python code (and hence Boost.NumPy code) to a Py++ project.

answered Dec 4, 2013 at 16:47

jbosch

9916 silver badges8 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

heroxbd Over a year ago

Good to see you Jim! I will update my strategy in the question. What's your opinion on PyuBlas?

jbosch Over a year ago

I've never used pyublas myself, so I don't have much of an opinion there. I imagine it'd be a good choice if you're using Boost.UBlas in C++ already, and perhaps not otherwise.

Collectives™ on Stack Overflow

exchange numpy array with boost.python: pyublas or boost.numpy?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related