Building libtorch from source gives unexpected contents

Question

I'm trying to build libtorch (i.e. the headers and shared object files for pytorch) from source, but get an unexpected result. I expected something that looks like the libtorch downloads from pytorch.org:

bin/
include/
lib/
share/

but instead got this:

abi-check    Caffe2Config.cmake         cmake_uninstall.cmake   empty.cpp             modules/               TorchConfig.cmake
aten/        Caffe2ConfigVersion.cmake  compile_commands.json   FXdiv/                nccl/                  TorchConfigVersion.cmake
bin/         CMakeCache.txt             confu-deps/             include/              nccl_external-prefix/
build.ninja  CMakeFiles/                CTestTestfile.cmake     install_manifest.txt  sleef/
caffe2/      cmake_install.cmake        detect_cuda_version.cc  lib/                  third_party/

It also contains lots of .o files and other junk, which I didn't expect. And it's missing many of the headers, such as the contents of torch/csrc/autograd/.

I followed the instructions for setting up my environment with Conda. Here's the approximate command I used to build:

export BUILD_CAFFE2=1 # probably don't need this
export BUILD_TEST=0
export CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
export DEBUG=0
export MAX_JOBS=16
export USE_CUDA=1
export USE_CUDNN=1
export CUDA_HOME=...
export NVCC_EXECUTABLE=...
export CUDNN_LIB_DIR=...
export CUDNN_INCLUDE_DIR=...
export CUDNN_LIBRARY=...

python ../tools/build_libtorch.py

I'm doing this on Linux with Python 3.8.

How can I get it to build the usual libtorch package structure?

mwlon · Accepted Answer · 2022-12-30 19:42:21Z

1

I found a way to create the intended libtorch structure by directly using cmake, following this document:

cmake \
  -DBUILD_SHARED_LIBS:BOOL=ON \
  -DCMAKE_BUILD_TYPE:STRING=Release \
  -DPYTHON_EXECUTABLE:PATH=`which python3` \
  -DCMAKE_INSTALL_PREFIX:PATH=../pytorch-install \
  -DBUILD_CAFFE2=1 \
  -DCMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"} \
  -DUSE_CUDA=1 \
  -DUSE_CUDNN=1 \
  -DCUDNN_INCLUDE_DIR=... \
  -DCUDNN_LIBRARY=... \
  ../pytorch

cmake --build . --target install -- -j 16

This produces libtorch in ../pytorch-install.

answered Dec 30, 2022 at 19:42

mwlon

1,0961 gold badge10 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Building libtorch from source gives unexpected contents

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related