cannot import module using boost.python

Question

I'm trying to build a simple program using boost.python.
I have the following code:

//greet.cpp
#include <iostream>
#include <boost/python.hpp>

void greet()
{
    std::cout << "hello world!" << std::endl;
}

BOOST_PYTHON_MODULE(greet)
{
    using namespace boost::python;
    def("greet", greet);
}

and the follwing makefile:

PYTHON_VERSION := 2.7

PYTHON_INC := /usr/include/python$(PYTHON_VERSION)
PYTHON_LIB_LOCATION := /usr/lib/python${PYTHON_VERSION}
PYTHON_LIB_FILE := python${PYTHON_VERSION}

BOOST_INC := ~/boost_1_54_0
BOOST_LIB_LOCATION := /home/elyashiv/boost_1_54_0/stage/lib
BOOST_LIB_FILE := boost_python

CC := gcc

CFLAGS := -c -fPIC
CInc := -I ${BOOST_INC} -I ${PYTHON_INC}

CLinkFlags = -shared -Wl,-soname,$@ -L${BOOST_LIB_LOCATION} -l${BOOST_LIB_FILE} -L${PYTHON_LIB_LOCATION} -l${PYTHON_LIB_FILE}

greet.o: greet.cpp

%.so: %.o
    gcc ${CLinkFlags} -o $@ $^

%.o: %.cpp
    ${CC} ${CFLAGS} ${CInc} $^

running make greet.so runs with just a few warnings (redefinition in some boost files).

when I try to import the module in python I get the following:

Python 2.7.3 (default, Apr 10 2013, 05:46:21) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import greet
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: ./greet.so: undefined symbol: _ZNK5boost6python7objects21py_function_impl_base9max_arityEv

what I have done wrong and how to fix it?

edit

the output of ldd greet.so:

linux-gate.so.1 =>  (0x001ee000)
libgcc_s.so.1 => /lib/i386-linux-gnu/libgcc_s.so.1 (0x0055d000)
libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0x0058e000)
/lib/ld-linux.so.2 (0x003a2000)

If you are on Linux, could you also paste the output of ldd greet.so? Does that show your boost_python library linked in? — André Anjos
– André Anjos, Commented Jul 16, 2013 at 11:57
Where is libboost_python? It is not getting linked in, therefore, you have the missing symbol. Make sure your compilation is working as expected. — André Anjos
– André Anjos, Commented Jul 16, 2013 at 14:44
@AndréAnjos as I wrote, I don't get any compilation/linking errors. If some thing is wrong with the linking process, I will like to point it out, because I can't spot it. — elyashiv
– elyashiv, Commented Jul 16, 2013 at 16:15
I just added a full explanation of your problem below, please upvote if you think it fixes your problem. — André Anjos
– André Anjos, Commented Jul 17, 2013 at 10:29

Community · Accepted Answer · 2017-05-23 12:21:04Z

Please note that order is important when you are linking binaries with gcc. The order in which you pass binaries to the linker should be such that the first unit (e.g. your object file) should resolve using the following units (other object files or libraries). In your example, you are linking greet.so wrongly:

%.so: %.o
    gcc ${CLinkFlags} -o $@ $^

This is going to generate a compilation line like this:

gcc -shared -Wl,-soname,greet.so -L/usr/lib -lboost_python -L/usr/lib/python2.7 -lpython2.7 -o greet.so greet.o

Notice that the unit greet.o, which depends on symbols defined in libboost_python.so and libpython2.7.so is coming last and therefore, by the time gcc's linker gets to it, it cannot resolve the undefined symbols anymore. Unfortunately, this is not an error, because the linker cannot know if you want this or not (for example, in Python, libpythonX.Y will be loaded before you can import your code in and it may be skipped therefore - you can dump that library from your command line altogether). So, the default is to ignore all undefined symbols.

You can change that behavior by setting a couple of flags to force undefined symbol detection:

CLinkFlags += -Wl,--unresolved-symbols=report-all

Will report all unresolved symbols as errors and:

CLinkFlags += -Wl,--unresolved-symbols=report-all -Wl,--warn-unresolved-symbols

Will report all unresolved symbols but will still link the binary. You have other options explained on this SO thread. Be warned: this is not what you normally want. Things like libpythonX.Y are never explicitly linked for example, but they are still available at runtime. In practice, you will still get a bunch of fake undefined references that are not worth persuing. The best is to fix your Makefile and make sure your object code comes before the libraries.

To fix your example, just move $^ to the begin of the linking like:

gcc $^ ${CLinkFlags} -o $@

After you compile, and you run ldd, you should see that now libboost_python (and python since you are explicitly linking it) will be linked against greet.so and the loading should work as you expect. I tested it myself locally.

As a rule of thumb, if you have undefined references with gcc and you are confident they should be present on any of the linked code, double-check the order.

Here is a fully working/minimal version of your Makefile that will do the job for your specific case (note we also set the runtime path to your private boost library, so you don't need to set the LD_LIBRARY_PATH as indicated by other answers - see details below):

PYTHON_VERSION := 2.7
PYTHON_INC := /usr/include/python$(PYTHON_VERSION)
BOOST_INC := /home/elyashiv/boost_1_54_0
BOOST_LIB_LOCATION := /home/elyashiv/boost_1_54_0/stage/lib
BOOST_LIB_FILE := boost_python

CC := gcc

CFLAGS := -c -fPIC
CInc := -I${BOOST_INC} -I${PYTHON_INC}

CLinkFlags = -shared -Wl,-soname,$@ -Wl,-rpath,${BOOST_LIB_LOCATION} -L${BOOST_LIB_LOCATION} -l${BOOST_LIB_FILE}

greet.so: greet.o

%.so: %.o
  gcc $^ ${CLinkFlags} -o $@

%.o: %.cpp
  ${CC} ${CFLAGS} ${CInc} $^

By setting the runtime path on the library itself, the runtime linker automatically looks there first before trying the LD_LIBRARY_PATH. If you decide not to do so, then you must set the environment variable LD_LIBRARY_PATH as indicated by other answers.

Your answer is great, but it still not the end of it. merging your answer with doctorloves answer gt rid from all the errors.
Ok, I just added a comment indicating how you can do that using the linker itself.

doctorlove · Accepted Answer · 2013-07-16 08:48:45Z

1

Boost python requires the boost python so file. You can add it to your path when you run the python in a variety of ways. I am using

export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH:../ThirdParty/boost_1_52_0/lib/linux64/

answered Jul 16, 2013 at 8:48

doctorlove

19.4k3 gold badges49 silver badges65 bronze badges

3 Comments

elyashiv Over a year ago

It doesn't seem to work. I run export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH:~/boost_1_54_0/stage/lib and got the same error. I'm using the wrong dir?

doctorlove Over a year ago

I take it /boost_1_54_0/stage/lib has the boost python so?

elyashiv Over a year ago

does that mean I cannot use the extension on a machine that doesn't have the boost libs? if not - how do I do it?

Fr34K · Accepted Answer · 2013-07-17 11:29:03Z

0

Try these commands. These worked for me.

g++ -c -I/usr/include/python2.7 -fPIC hello.cpp -o hello.o 
g++ -shared -Wl,-soname,"hello.so" -L/usr/local/lib hello.o -lboost_python -fPIC -o hello.so

answered Jul 17, 2013 at 11:29

Fr34K

5446 silver badges19 bronze badges

Collectives™ on Stack Overflow

cannot import module using boost.python

3 Answers 3

3 Comments

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related