Tensorflow custom op with OpenCV undefined symbol

Question

I'm writing a custom operation for Tensorflow that is supposed to load a video. For this, I need to include OpenCV.

For now, the operation simply tries to open a VideoCapture and returns an empty tensor.

Here's the C++ code:

#include "opencv2/opencv.hpp"
#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"


#include "tensorflow/core/framework/op.h"
#include "tensorflow/core/framework/shape_inference.h"
#include "tensorflow/core/framework/op_kernel.h"

#include <iostream>

using namespace tensorflow;
using namespace cv;
using namespace std;

using shape_inference::ShapeHandle;
using shape_inference::DimensionHandle;

REGISTER_OP("LoadVideo")
    .Input("filename: string")
    .Output("frame: float32")
    .SetShapeFn([](::tensorflow::shape_inference::InferenceContext* c) {
        TensorShape outputTensorShape({224, 224, 3});
        ShapeHandle outputShapeHandle;
        c->MakeShapeFromTensorShape(outputTensorShape, &outputShapeHandle);
        c->set_output(0, outputShapeHandle);
        return Status::OK();
    });

class LoadVideoOp : public OpKernel {
 public:
  explicit LoadVideoOp(OpKernelConstruction* context) : OpKernel(context) {}

  void Compute(OpKernelContext* context) override {
    // Grab the input tensor
    const Tensor& input_tensor = context->input(0);
    auto input = input_tensor.flat<string>();
    string filename = input(0);

    VideoCapture cap = VideoCapture("data/0eRkpTGq5pA.mp4");

    Tensor* output_tensor = NULL;
    OP_REQUIRES_OK(context, context->allocate_output(0, {224, 224, 3}, &output_tensor));

  }
};

REGISTER_KERNEL_BUILDER(Name("LoadVideo").Device(DEVICE_CPU), LoadVideoOp);

Then, I use the following command to compile the code:

g++ -std=c++11 -shared -fPIC \
-I /home/master/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/include \
-I ~/anaconda3/envs/tf/include/opencv2/ -I ~/anaconda3/envs/tf/include/opencv/ -O2 \
-L ~/anaconda3/envs/tf/lib \
load_video.cc -o load_video.so \
-lopencv_core -lopencv_videoio -lopencv_highgui \
-lopencv_imgproc -lopencv_video -lopencv_objdetect

When I load the compiled code into a Python script (using tf.load_op_library) and try to run the op I get the following error:

tensorflow.python.framework.errors_impl.NotFoundError: lib/ops/load_video.so: undefined symbol: _ZN2cv12VideoCaptureC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE

It looks like the compiled C++ code cannot access the appropriate OpenCV object. I don't know much about C++ compilation and linking, so the problem probably is that I'm compiling the custom op in a wrong way.

Could you please help me compile the op in such way, that it can be successfully loaded and run by tensorflow?

EDIT 1:

This is the Python script I use to load the custom op:

import tensorflow as tf
load_video_module = tf.load_op_library('lib/ops/load_video.so')
with tf.Session():
  x = load_video_module.load_video("data/0eRkpTGq5pA.mp4").eval()
  print(x)

The error happens on the line 2 (i.e. when trying to load the compiled C++ code).

Solution:

I managed to successfully compile and run the custom tensorflow op after rebuilding OpenCV. The compilation command is:

g++ -std=c++11 -ggdb -shared -I`python -c 'import tensorflow as tf; print(tf.sysconfig.get_include())'` `pkg-config --cflags opencv` -o load_video.so load_video.cc `pkg-config --libs opencv` -fPIC

nessuno · Accepted Answer · 2017-07-30 19:11:25Z

2

You can check if your library requires some missing library using ldd.

Just check ldd load_video.so.

However, probably you're not linking some shared library that some OpenCV method you're using requires.

To be sure to link and include every library needed you can use pkg-config.

Remove your manually -I and -l flags that point to OpenCV libs and just add pkg-config --libs --cflags opencv that do the complete work (of including and linking libraries) for you

answered Jul 30, 2017 at 19:11

nessuno

27.1k5 gold badges87 silver badges78 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Nikash Walia · Accepted Answer · 2017-07-30 17:05:45Z

1

If you are running on Linux, cd into the directory that contains your video file and then perform, without the parentheses:

sudo chmod 777 (the name of your video file)

This should give your program access to the video file. I don't know much about C++, but TensorFlow often throws this error when it's denied permission, so give it a shot and good luck!

answered Jul 30, 2017 at 17:05

Nikash Walia

315 bronze badges

11 Comments

ondrejba Over a year ago

I tried it and it didn't help. I think the problem occurs before OpenCV even tries to access the video file.

Nikash Walia Over a year ago

Does the stack trace indicate a certain line in your code where the error is happening? For TensorFlow, you would probably find this at the top of the stack trace.

ondrejba Over a year ago

Yes, see Edit 1.

Nikash Walia Over a year ago

TensorFlow works only with Python, Java, C, and Go, so that may be an issue. OpenCV works with Python, Java, C, and C++, so I would recommend that for something like this, if you want to combine OpenCV and TensorFlow, I would recommend you write everything in Python, Java, or C. Unfortunately, I cannot help in any other way, but I can guarantee that your code should be able to integrate if you write everything in Python- it worked for me!

Nikash Walia Over a year ago

Ok, let me know when you finish that and if it makes any difference.

|

Collectives™ on Stack Overflow

Tensorflow custom op with OpenCV undefined symbol

2 Answers 2

Comments

11 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

11 Comments

Your Answer

Sign up or log in

Post as a guest

Related