6

❓ Problem with building PyTorch from source

Hello everyone,

I have problem with building PyTorch from source. I followed the official build instructions. I use Anaconda Python 3.7.1 (version 2018.12, build py37_0). I installed all neccessary dependencies using conda and issued python setup.py install command to build it. It builds all files successfully but then it fails at the installation step saying:

build/temp.linux-x86_64-3.7/torch/csrc/stub.o: file not recognized: file format not recognized

I tried building with gcc/g++ versions 5,6,7 but it did not help. Can you help me resolve this problem?

Build output in terminal

This is the output of python setup.py install with the error I am getting:

[manjaro-pc pytorch]# python setup.py install
Building wheel torch-1.1.0a0+44809fd
-- Building version 1.1.0a0+44809fd
[0/1] Install the project...
-- Install configuration: "Release"
running install
running build
running build_py
copying torch/version.py -> build/lib.linux-x86_64-3.7/torch
copying caffe2/proto/caffe2_pb2.py -> build/lib.linux-x86_64-3.7/caffe2/proto
copying caffe2/proto/caffe2_legacy_pb2.py -> build/lib.linux-x86_64-3.7/caffe2/proto
copying caffe2/proto/predictor_consts_pb2.py -> build/lib.linux-x86_64-3.7/caffe2/proto
copying caffe2/proto/metanet_pb2.py -> build/lib.linux-x86_64-3.7/caffe2/proto
copying caffe2/proto/torch_pb2.py -> build/lib.linux-x86_64-3.7/caffe2/proto
copying caffe2/proto/prof_dag_pb2.py -> build/lib.linux-x86_64-3.7/caffe2/proto
copying caffe2/proto/hsm_pb2.py -> build/lib.linux-x86_64-3.7/caffe2/proto
running build_ext
-- Building with NumPy bindings
-- Not using cuDNN
-- Not using MIOpen
-- Not using CUDA
-- Not using MKLDNN
-- Not using NCCL
-- Building without distributed package

Copying extension caffe2.python.caffe2_pybind11_state
Copying caffe2.python.caffe2_pybind11_state from torch/lib/python3.7/site-packages/caffe2/python/caffe2_pybind11_state.cpython-37m-x86_64-linux-gnu.so to /home/manjaro/Downloads/pytorch/build/lib.linux-x86_64-3.7/caffe2/python/caffe2_pybind11_state.cpython-37m-x86_64-linux-gnu.so
building 'torch._C' extension
gcc -pthread -B /opt/anaconda/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/anaconda/include/python3.7m -c torch/csrc/stub.cpp -o build/temp.linux-x86_64-3.7/torch/csrc/stub.o -std=c++11 -Wall -Wextra -Wno-strict-overflow -Wno-unused-parameter -Wno-missing-field-initializers -Wno-write-strings -Wno-unknown-pragmas -Wno-deprecated-declarations -fno-strict-aliasing -Wno-missing-braces
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
g++ -pthread -shared -B /opt/anaconda/compiler_compat -L/opt/anaconda/lib -Wl,-rpath=/opt/anaconda/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/torch/csrc/stub.o -L/home/manjaro/Downloads/pytorch/torch/lib -lshm -ltorch_python -o build/lib.linux-x86_64-3.7/torch/_C.cpython-37m-x86_64-linux-gnu.so -Wl,-rpath,$ORIGIN/lib
/opt/anaconda/compiler_compat/ld: build/temp.linux-x86_64-3.7/torch/csrc/stub.o: unable to initialize decompress status for section .debug_info
/opt/anaconda/compiler_compat/ld: build/temp.linux-x86_64-3.7/torch/csrc/stub.o: unable to initialize decompress status for section .debug_info
/opt/anaconda/compiler_compat/ld: build/temp.linux-x86_64-3.7/torch/csrc/stub.o: unable to initialize decompress status for section .debug_info
/opt/anaconda/compiler_compat/ld: build/temp.linux-x86_64-3.7/torch/csrc/stub.o: unable to initialize decompress status for section .debug_info
build/temp.linux-x86_64-3.7/torch/csrc/stub.o: file not recognized: file format not recognized
collect2: error: ld returned 1 exit status
error: command 'g++' failed with exit status 1

From this output, I assume the error was caused by this line:

g++ -pthread -shared -B /opt/anaconda/compiler_compat -L/opt/anaconda/lib -Wl,-rpath=/opt/anaconda/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.7/torch/csrc/stub.o -L/home/manjaro/Downloads/pytorch/torch/lib -lshm -ltorch_python -o build/lib.linux-x86_64-3.7/torch/_C.cpython-37m-x86_64-linux-gnu.so -Wl,-rpath,$ORIGIN/lib

My environment

PyTorch version: 1.0
Is debug build: N/A
CUDA used to build PyTorch: 10

OS: Manjaro Linux
GCC version: (GCC) 6.4.1 20171002
CMake version: version 3.12.2

Python version: 3.7
Is CUDA available: N/A
CUDA runtime version: 10.0.130
GPU models and configuration: GPU 0: GeForce GTX 650
Nvidia driver version: 415.27
cuDNN version: Probably one of the following:
/usr/lib/libcudnn.so.7.4.1
/usr/lib/libcudnn_static_v7.a

Versions of relevant libraries:
[pip] Could not collect
[conda] blas                      1.0                         mkl  
[conda] magma-cuda100             2.4.0                         1    pytorch
[conda] mkl                       2019.1                      144  
[conda] mkl-include               2019.1                      144  
[conda] mkl-service               1.1.2            py37he904b0f_5  
[conda] mkl_fft                   1.0.6            py37hd81dba3_0  
[conda] mkl_random                1.0.2            py37hd81dba3_0  
[conda] mkldnn                    0.16.1                        0    mingfeima

My CMake configuration printed before building PyTorch


Building wheel torch-1.1.0a0+44809fd
-- Building version 1.1.0a0+44809fd
['cmake',
 '/home/manjaro/Downloads/pytorch',
 '-GNinja',
 '-DBLAS=MKL',
 '-DBUILDING_WITH_TORCH_LIBS=ON',
 '-DBUILD_BINARY=False',
 '-DBUILD_CAFFE2_OPS=False',
 '-DBUILD_PYTHON=True',
 '-DBUILD_SHARED_LIBS=ON',
 '-DBUILD_TEST=False',
 '-DBUILD_TORCH=ON',
 '-DCAFFE2_STATIC_LINK_CUDA=False',
 '-DCMAKE_BUILD_TYPE=Release',
 '-DCMAKE_CXX_FLAGS= ',
 '-DCMAKE_C_FLAGS= ',
 '-DCMAKE_EXE_LINKER_FLAGS=',
 '-DCMAKE_INSTALL_PREFIX=/home/manjaro/Downloads/pytorch/torch',
 '-DCMAKE_PREFIX_PATH=/opt/anaconda/bin/../',
 '-DCMAKE_SHARED_LINKER_FLAGS=',
 '-DINSTALL_TEST=False',
 '-DNCCL_EXTERNAL=False',
 '-DNUMPY_INCLUDE_DIR=/opt/anaconda/lib/python3.7/site-packages/numpy/core/include',
 '-DONNX_NAMESPACE=onnx_torch',
 '-DPYTHON_EXECUTABLE=/opt/anaconda/bin/python',
 '-DPYTHON_INCLUDE_DIR=/opt/anaconda/include/python3.7m',
 '-DPYTHON_LIBRARY=/opt/anaconda/lib/libpython3.7m.so.1.0',
 '-DTHD_SO_VERSION=1',
 '-DTORCH_BUILD_VERSION=1.1.0a0+44809fd',
 '-DUSE_CUDA=False',
 '-DUSE_DISTRIBUTED=False',
 '-DUSE_FBGEMM=False',
 '-DUSE_FFMPEG=False',
 '-DUSE_LEVELDB=False',
 '-DUSE_LMDB=False',
 '-DUSE_MKLDNN=False',
 '-DUSE_NNPACK=False',
 '-DUSE_NUMPY=True',
 '-DUSE_OPENCV=False',
 '-DUSE_QNNPACK=False',
 '-DUSE_ROCM=False',
 '-DUSE_SYSTEM_EIGEN_INSTALL=OFF',
 '-DUSE_SYSTEM_NCCL=False',
 '-DUSE_TENSORRT=False']
{'BLAS': 'MKL',
 'BUILD_CAFFE2_OPS': '0',
 'BUILD_TEST': '0',
 'CMAKE_PREFIX_PATH': '/opt/anaconda/bin/../',
 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus',
 'HG': '/usr/bin/hg',
 'HOME': '/root',
 'LANG': 'en_US.UTF-8',
 'LC_ADDRESS': 'sk_SK.UTF-8',
 'LC_IDENTIFICATION': 'sk_SK.UTF-8',
 'LC_MEASUREMENT': 'sk_SK.UTF-8',
 'LC_MONETARY': 'sk_SK.UTF-8',
 'LC_NAME': 'sk_SK.UTF-8',
 'LC_NUMERIC': 'sk_SK.UTF-8',
 'LC_PAPER': 'sk_SK.UTF-8',
 'LC_TELEPHONE': 'sk_SK.UTF-8',
 'LC_TIME': 'sk_SK.UTF-8',
 'LD_LIBRARY_PATH': ':/usr/local/lib64:/usr/local/lib:/opt/cuda/lib64',
 'LOGNAME': 'manjaro',
 'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.pdf=00;32:*.ps=00;32:*.txt=00;32:*.patch=00;32:*.diff=00;32:*.log=00;32:*.tex=00;32:*.doc=00;32:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.axa=00;36:*.oga=00;36:*.spx=00;36:*.xspf=00;36:',
 'MAIL': '/var/spool/mail/manjaro',
 'MAX_JOBS': '4',
 'MOZ_PLUGIN_PATH': '/usr/lib/mozilla/plugins',
 'NO_CAFFE2_OPS': '1',
 'NO_CUDA': '1',
 'NO_CUDNN': '1',
 'NO_DISTRIBUTED': '1',
 'NO_FBGEMM': '1',
 'NO_MIOPEN': '1',
 'NO_MKLDNN': '1',
 'NO_NNPACK': '1',
 'NO_QNNPACK': '1',
 'NO_TEST': '1',
 'OLDPWD': '/home/manjaro/Downloads',
 'PATH': '/opt/anaconda/bin:/opt/anaconda/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/cuda/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/usr/local/bin:/opt/cuda/bin:/usr/local/bin:/opt/cuda/bin',
 'PWD': '/home/manjaro/Downloads/pytorch',
 'PYTHONPATH': ':/usr/local/lib/python3.7/site-packages/:/usr/local/lib/python3.7/site-packages/',
 'QT_LINUX_ACCESSIBILITY_ALWAYS_ON': '1',
 'SHELL': '/bin/bash',
 'SHLVL': '2',
 'SSH_CLIENT': '62.197.243.30 45360 44444',
 'SSH_CONNECTION': '62.197.243.30 45360 192.168.1.147 44444',
 'SSH_TTY': '/dev/pts/4',
 'TERM': 'xterm-256color',
 'TORCH_CUDA_ARCH_LIST': '3.0',
 'USER': 'manjaro',
 'USE_CUDA': '0',
 'USE_CUDNN': '0',
 'USE_DISTRIBUTED': '0',
 'USE_FBGEMM': '0',
 'USE_MIOPEN': '0',
 'USE_MKLDNN': '0',
 'USE_NNPACK': '0',
 'USE_QNNPACK': '0',
 'XDG_DATA_DIRS': '/home/manjaro/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share',
 'XDG_RUNTIME_DIR': '/run/user/1000',
 'XDG_SESSION_ID': 'c8',
 'YAOURT_COLORS': 'nb=1:pkg=1:ver=1;32:lver=1;45:installed=1;42:grp=1;34:od=1;41;5:votes=1;44:dsc=0:other=1;35',
 '_': '/opt/anaconda/bin/python'}
-- The CXX compiler identification is GNU 5.4.1
-- The C compiler identification is GNU 5.4.1
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Performing Test COMPILER_WORKS
-- Performing Test COMPILER_WORKS - Success
-- Performing Test SUPPORT_GLIBCXX_USE_C99
-- Performing Test SUPPORT_GLIBCXX_USE_C99 - Success
-- Performing Test CAFFE2_EXCEPTION_PTR_SUPPORTED
-- Performing Test CAFFE2_EXCEPTION_PTR_SUPPORTED - Success
-- std::exception_ptr is supported.
-- Performing Test CAFFE2_IS_NUMA_AVAILABLE
-- Performing Test CAFFE2_IS_NUMA_AVAILABLE - Success
-- NUMA is available
-- Performing Test CAFFE2_NEED_TO_TURN_OFF_DEPRECATION_WARNING
-- Performing Test CAFFE2_NEED_TO_TURN_OFF_DEPRECATION_WARNING - Success
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX2_EXTENSIONS
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX2_EXTENSIONS - Success
-- Current compiler supports avx2 extension. Will build perfkernels.
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX512_EXTENSIONS
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX512_EXTENSIONS - Success
-- Current compiler supports avx512f extension. Will build fbgemm.
-- Performing Test COMPILER_SUPPORTS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_SUPPORTS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_SUPPORTS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_SUPPORTS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_SUPPORTS_RDYNAMIC
-- Performing Test COMPILER_SUPPORTS_RDYNAMIC - Success
-- Building using own protobuf under third_party per request.
-- Use custom protobuf build.
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Caffe2 protobuf include directory: $<BUILD_INTERFACE:/home/manjaro/Downloads/pytorch/third_party/protobuf/src>$<INSTALL_INTERFACE:include>
-- Trying to find preferred BLAS backend of choice: MKL
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of void*
-- Check size of void* - done
-- Checking for [mkl_intel_lp64 - mkl_gnu_thread - mkl_core - gomp - pthread - m - dl]
--   Library mkl_intel_lp64: /opt/anaconda/lib/libmkl_intel_lp64.so
--   Library mkl_gnu_thread: /opt/anaconda/lib/libmkl_gnu_thread.so
--   Library mkl_core: /opt/anaconda/lib/libmkl_core.so
-- Found OpenMP_C: -fopenmp (found version "4.0") 
-- Found OpenMP_CXX: -fopenmp (found version "4.0") 
-- Found OpenMP: TRUE (found version "4.0")  
--   Library gomp: -fopenmp
--   Library pthread: /usr/lib/libpthread.so
--   Library m: /usr/lib/libm.so
--   Library dl: /usr/lib/libdl.so
-- Looking for cblas_sgemm
-- Looking for cblas_sgemm - found
-- MKL library found
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Caffe2: Found gflags with new-style gflags target.
-- Caffe2: Cannot find glog automatically. Using legacy find.
-- Found glog: /usr/include  
-- Caffe2: Found glog (include: /usr/include, library: /usr/lib/libglog.so)
-- Found Numa: /usr/include  
-- Found Numa  (include: /usr/include, library: /usr/lib/libnuma.so)
-- Downloading PSimd to /home/manjaro/Downloads/pytorch/build/confu-srcs/psimd (define PSIMD_SOURCE_DIR to avoid it)
-- Configuring done
-- Generating done
-- Build files have been written to: /home/manjaro/Downloads/pytorch/build/confu-deps/psimd-download
[1/9] Creating directories for 'psimd'
[2/9] Performing download step (git clone) for 'psimd'
Cloning into 'psimd'...
Already on 'master'
Your branch is up to date with 'origin/master'.
[3/9] No patch step for 'psimd'
[4/9] Performing update step for 'psimd'
Current branch master is up to date.
[5/9] No configure step for 'psimd'
[6/9] No build step for 'psimd'
[7/9] No install step for 'psimd'
[8/9] No test step for 'psimd'
[9/9] Completed 'psimd'
-- Using third party subdirectory Eigen.
Python 3.7.1
-- Found PythonInterp: /opt/anaconda/bin/python (found suitable version "3.7.1", minimum required is "2.7") 
-- Found PythonLibs: /opt/anaconda/lib/libpython3.7m.so.1.0 (found suitable version "3.7.1", minimum required is "2.7") 
-- Found PythonInterp: /opt/anaconda/bin/python (found version "3.7.1") 
-- Found PythonLibs: /opt/anaconda/lib/libpython3.7m.so.1.0
-- System pybind11 found
-- pybind11 include dirs: /usr/include;/opt/anaconda/include/python3.7m
CMake Warning at cmake/Dependencies.cmake:805 (message):
  Not using CUDA, so disabling NCCL.  Suppress this warning with
  -DUSE_NCCL=OFF.
Call Stack (most recent call first):
  CMakeLists.txt:219 (include)


CMake Warning at cmake/Dependencies.cmake:950 (message):
  Metal is only used in ios builds.
Call Stack (most recent call first):
  CMakeLists.txt:219 (include)


-- 
-- ******** Summary ********
--   CMake version         : 3.12.2
--   CMake command         : /opt/anaconda/bin/cmake
--   System                : Linux
--   C++ compiler          : /usr/bin/c++
--   C++ compiler version  : 5.4.1
--   CXX flags             :   -fvisibility-inlines-hidden -Wnon-virtual-dtor
--   Build type            : Release
--   Compile definitions   : TH_BLAS_MKL
--   CMAKE_PREFIX_PATH     : /opt/anaconda/bin/../
--   CMAKE_INSTALL_PREFIX  : /home/manjaro/Downloads/pytorch/torch
--   CMAKE_MODULE_PATH     : /home/manjaro/Downloads/pytorch/cmake/Modules;/usr/share/cmake/pybind11
-- 
--   ONNX version          : 1.4.1
--   ONNX NAMESPACE        : onnx_torch
--   ONNX_BUILD_TESTS      : OFF
--   ONNX_BUILD_BENCHMARKS : OFF
--   ONNX_USE_LITE_PROTO   : OFF
--   ONNXIFI_DUMMY_BACKEND : OFF
-- 
--   Protobuf compiler     : 
--   Protobuf includes     : 
--   Protobuf libraries    : 
--   BUILD_ONNX_PYTHON     : OFF
-- Found gcc >=5 and CUDA <= 7.5, adding workaround C++ flags
-- Could not find CUDA with FP16 support, compiling without torch.CudaHalfTensor
-- Removing -DNDEBUG from compile flags
-- Found OpenMP_C: -fopenmp (found version "4.0") 
-- Found OpenMP_CXX: -fopenmp (found version "4.0") 
-- Compiling with OpenMP support
-- MAGMA not found. Compiling without MAGMA support
-- Could not find hardware support for NEON on this machine.
-- No OMAP3 processor on this machine.
-- No OMAP4 processor on this machine.
-- Looking for cpuid.h
-- Looking for cpuid.h - found
-- Performing Test HAVE_GCC_GET_CPUID
-- Performing Test HAVE_GCC_GET_CPUID - Success
-- Performing Test NO_GCC_EBX_FPIC_BUG
-- Performing Test NO_GCC_EBX_FPIC_BUG - Success
-- Performing Test C_HAS_AVX_1
-- Performing Test C_HAS_AVX_1 - Failed
-- Performing Test C_HAS_AVX_2
-- Performing Test C_HAS_AVX_2 - Success
-- Performing Test C_HAS_AVX2_1
-- Performing Test C_HAS_AVX2_1 - Failed
-- Performing Test C_HAS_AVX2_2
-- Performing Test C_HAS_AVX2_2 - Success
-- Performing Test CXX_HAS_AVX_1
-- Performing Test CXX_HAS_AVX_1 - Failed
-- Performing Test CXX_HAS_AVX_2
-- Performing Test CXX_HAS_AVX_2 - Success
-- Performing Test CXX_HAS_AVX2_1
-- Performing Test CXX_HAS_AVX2_1 - Failed
-- Performing Test CXX_HAS_AVX2_2
-- Performing Test CXX_HAS_AVX2_2 - Success
-- AVX compiler support found
-- AVX2 compiler support found
-- Performing Test HAS_C11_ATOMICS
-- Performing Test HAS_C11_ATOMICS - Failed
-- Performing Test HAS_MSC_ATOMICS
-- Performing Test HAS_MSC_ATOMICS - Failed
-- Performing Test HAS_GCC_ATOMICS
-- Performing Test HAS_GCC_ATOMICS - Success
-- Atomics: using GCC intrinsics
-- Performing Test BLAS_F2C_DOUBLE_WORKS
-- Performing Test BLAS_F2C_DOUBLE_WORKS - Failed
-- Performing Test BLAS_F2C_FLOAT_WORKS
-- Performing Test BLAS_F2C_FLOAT_WORKS - Success
-- Performing Test BLAS_USE_CBLAS_DOT
-- Performing Test BLAS_USE_CBLAS_DOT - Success
-- Found a library with BLAS API (mkl).
-- Found a library with LAPACK API (mkl).
disabling CUDA because NOT USE_CUDA is set
-- CuDNN not found. Compiling without CuDNN support
disabling ROCM because NOT USE_ROCM is set
-- MIOpen not found. Compiling without MIOpen support
disabling MKLDNN because USE_MKLDNN is not set
-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - found
-- Looking for mmap
-- Looking for mmap - found
-- Looking for shm_open
-- Looking for shm_open - found
-- Looking for shm_unlink
-- Looking for shm_unlink - found
-- Looking for malloc_usable_size
-- Looking for malloc_usable_size - found
-- Performing Test C_HAS_THREAD
-- Performing Test C_HAS_THREAD - Success
-- GCC 5.4.1: Adding gcc and gcc_s libs to link line
-- NUMA paths:
-- /usr/include
-- /usr/lib/libnuma.so
disabling CUDA because USE_CUDA is set false
-- Check size of long double
-- Check size of long double - done
-- Performing Test COMPILER_SUPPORTS_LONG_DOUBLE
-- Performing Test COMPILER_SUPPORTS_LONG_DOUBLE - Success
-- Performing Test COMPILER_SUPPORTS_FLOAT128
-- Performing Test COMPILER_SUPPORTS_FLOAT128 - Success
-- Performing Test COMPILER_SUPPORTS_SSE2
-- Performing Test COMPILER_SUPPORTS_SSE2 - Success
-- Performing Test COMPILER_SUPPORTS_SSE4
-- Performing Test COMPILER_SUPPORTS_SSE4 - Success
-- Performing Test COMPILER_SUPPORTS_AVX
-- Performing Test COMPILER_SUPPORTS_AVX - Success
-- Performing Test COMPILER_SUPPORTS_FMA4
-- Performing Test COMPILER_SUPPORTS_FMA4 - Success
-- Performing Test COMPILER_SUPPORTS_AVX2
-- Performing Test COMPILER_SUPPORTS_AVX2 - Success
-- Performing Test COMPILER_SUPPORTS_SVE
-- Performing Test COMPILER_SUPPORTS_SVE - Failed
-- Performing Test COMPILER_SUPPORTS_AVX512F
-- Performing Test COMPILER_SUPPORTS_AVX512F - Failed
-- Found OpenMP_C: -fopenmp (found version "4.0") 
-- Found OpenMP_CXX: -fopenmp (found version "4.0") 
-- Performing Test COMPILER_SUPPORTS_OPENMP
-- Performing Test COMPILER_SUPPORTS_OPENMP - Success
-- Performing Test COMPILER_SUPPORTS_WEAK_ALIASES
-- Performing Test COMPILER_SUPPORTS_WEAK_ALIASES - Success
-- Performing Test COMPILER_SUPPORTS_BUILTIN_MATH
-- Performing Test COMPILER_SUPPORTS_BUILTIN_MATH - Success
-- Configuring build for SLEEF-v3.2
   Target system: Linux-4.14.94-1-MANJARO
   Target processor: x86_64
   Host system: Linux-4.14.94-1-MANJARO
   Host processor: x86_64
   Detected C compiler: GNU @ /usr/bin/cc
-- Using option `-Wall -Wno-unused -Wno-attributes -Wno-unused-result -Wno-psabi -ffp-contract=off -fno-math-errno -fno-trapping-math` to compile libsleef
-- Building shared libs : OFF
-- MPFR : /opt/anaconda/lib/libmpfr.so
-- MPFR header file in /opt/anaconda/include
-- GMP : /opt/anaconda/lib/libgmp.so
-- RUNNING_ON_TRAVIS : 0
-- COMPILER_SUPPORTS_OPENMP : 1
-- /usr/bin/c++ /home/manjaro/Downloads/pytorch/torch/abi-check.cpp -o /home/manjaro/Downloads/pytorch/build/abi-check
-- Determined _GLIBCXX_USE_CXX11_ABI=1
-- NCCL operators skipped due to no CUDA support
-- Excluding ideep operators as we are not using ideep
-- Excluding image processing operators due to no opencv
-- Excluding video processing operators due to no opencv
-- MPI operators skipped due to no MPI support
-- Include Observer library
-- Using lib/python3.7/site-packages as python relative installation path
CMake Warning at CMakeLists.txt:416 (message):
  Generated cmake files are only fully tested if one builds with system glog,
  gflags, and protobuf.  Other settings may generate files that are not well
  tested.


-- 
-- ******** Summary ********
-- General:
--   CMake version         : 3.12.2
--   CMake command         : /opt/anaconda/bin/cmake
--   System                : Linux
--   C++ compiler          : /usr/bin/c++
--   C++ compiler version  : 5.4.1
--   BLAS                  : MKL
--   CXX flags             :   -fvisibility-inlines-hidden -D_FORCE_INLINES -D_MWAITXINTRIN_H_INCLUDED -D__STRICT_ANSI__ -fopenmp -O2 -fPIC -Wno-narrowing -Wall -Wextra -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -Wno-unused-but-set-variable -Wno-maybe-uninitialized
--   Build type            : Release
--   Compile definitions   : TH_BLAS_MKL;ONNX_NAMESPACE=onnx_torch;USE_GCC_ATOMICS=1;HAVE_MMAP=1;_FILE_OFFSET_BITS=64;HAVE_SHM_OPEN=1;HAVE_SHM_UNLINK=1;HAVE_MALLOC_USABLE_SIZE=1
--   CMAKE_PREFIX_PATH     : /opt/anaconda/bin/../
--   CMAKE_INSTALL_PREFIX  : /home/manjaro/Downloads/pytorch/torch
-- 
--   TORCH_VERSION         : 1.1.0
--   CAFFE2_VERSION        : 1.1.0
--   BUILD_ATEN_MOBILE     : OFF
--   BUILD_ATEN_ONLY       : OFF
--   BUILD_BINARY          : False
--   BUILD_CUSTOM_PROTOBUF : ON
--     Link local protobuf : ON
--   BUILD_DOCS            : OFF
--   BUILD_PYTHON          : True
--     Python version      : 3.7.1
--     Python executable   : /opt/anaconda/bin/python
--     Pythonlibs version  : 3.7.1
--     Python library      : /opt/anaconda/lib/libpython3.7m.so.1.0
--     Python includes     : /opt/anaconda/include/python3.7m
--     Python site-packages: lib/python3.7/site-packages
--   BUILD_CAFFE2_OPS      : False
--   BUILD_SHARED_LIBS     : ON
--   BUILD_TEST            : False
--   USE_ASAN              : OFF
--   USE_CUDA              : False
--   USE_ROCM              : False
--   USE_EIGEN_FOR_BLAS    : 
--   USE_FBGEMM            : OFF
--   USE_FFMPEG            : False
--   USE_GFLAGS            : ON
--   USE_GLOG              : ON
--   USE_LEVELDB           : False
--   USE_LITE_PROTO        : OFF
--   USE_LMDB              : False
--   USE_METAL             : OFF
--   USE_MKL               : ON
--   USE_MKLDNN            : OFF
--   USE_NCCL              : OFF
--   USE_NNPACK            : False
--   USE_NUMPY             : ON
--   USE_OBSERVERS         : ON
--   USE_OPENCL            : OFF
--   USE_OPENCV            : False
--   USE_OPENMP            : OFF
--   USE_PROF              : OFF
--   USE_QNNPACK           : False
--   USE_REDIS             : OFF
--   USE_ROCKSDB           : OFF
--   USE_ZMQ               : OFF
--   USE_DISTRIBUTED       : False
--   Public Dependencies  : Threads::Threads;caffe2::mkl;glog::glog
--   Private Dependencies : cpuinfo;/usr/lib/libnuma.so;fp16;onnxifi_loader;rt;gcc_s;gcc;dl
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    NCCL_EXTERNAL
    THD_SO_VERSION

1 Answer 1

5

Problem solved.

I found what was wrong and I fixed it. The whole problem lies in the fact that Anaconda distribution comes with its own ld linker that is located in /opt/anaconda/compiler_compat/ and it overshadows system ld residing at /usr/bin.

To fix my error I ran python setup.py clean and then I temporarily renamed Anaconda's ld linker to ld-old to make it invisible during PyTorch installation.

Sign up to request clarification or add additional context in comments.

1 Comment

Yes, and the linker is not compatible with Manjaro's elf format. Same here, only my first build failed because of a build error in MKL-DNN. Still waiting for the effects of my second try.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.