Create a Pip Wheel for OpenCV Built from Source to Prevent Overwriting with Library Dependencies

I have a Dockerfile that builds OpenCV from source with cuda. The build itself succeeds, but pip doesn’t recognize this custom installation. As a result, when I later install a Python package that depends on OpenCV, pip fetches a prebuilt opencv-python wheel, which then overrides (or hides) my source build. According to this suggestion (https://stackoverflow.com/a/62642547/13045595 by @jkr), the proper fix is to package my custom build as a wheel and install it so pip treats the dependency as satisfied. I tried that, and although the wheel build script finishes without errors, the installed wheel doesn’t actually work. Could you review the wheel-building script and/or the Dockerfile and let me know what needs to be adjusted or optimized?

# syntax=docker/dockerfile:1
# Requires Docker BuildKit for cache mounts (build with DOCKER_BUILDKIT=1)
FROM nvidia/cuda:12.1.0-cudnn8-devel-ubuntu22.04

# Environment setup
ENV DEBIAN_FRONTEND=noninteractive \
    LANG=en_US.UTF-8 \
    LC_ALL=en_US.UTF-8 \
    PYTHONIOENCODING=UTF-8 \
    NVIDIA_DRIVER_CAPABILITIES=all \
    CUDA_HOME=/usr/local/cuda \
    PATH=/usr/local/cuda/bin:$PATH \
    LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH \
    CCACHE_DIR=/ccache

# Create ccache directory
RUN mkdir -p /ccache

# Set locale
RUN apt-get update && apt-get install -y --no-install-recommends \
    locales \
    && locale-gen en_US.UTF-8 \
    && update-locale LANG=en_US.UTF-8

# Essential system dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
    # Build tools
    build-essential \
    cmake \
    pkg-config \
    ninja-build \
    ccache \
    # GCC-10 for CUDA 12.x compatibility
    gcc-10 \
    g++-10 \
    # Version control
    git \
    # Utilities
    wget \
    curl \
    unzip \
    vim \
    nano \
    htop \
    software-properties-common \
    && add-apt-repository ppa:deadsnakes/ppa -y \
    && apt-get update \
    && apt-get install -y --no-install-recommends \
    python3.9 \
    python3.9-dev \
    python3.9-venv \
    python3.9-distutils \
    python3-pip \
    # OpenCV dependencies
    libgtk-3-dev \
    libgtk2.0-dev \
    libavcodec-dev \
    libavformat-dev \
    libswscale-dev \
    libswresample-dev \
    libtbb-dev \
    libjpeg-dev \
    libpng-dev \
    libtiff-dev \
    libwebp-dev \
    libopenexr-dev \
    libdc1394-dev \
    libgstreamer1.0-dev \
    libgstreamer-plugins-base1.0-dev \
    libv4l-dev \
    libxvidcore-dev \
    libx264-dev \
    libfdk-aac-dev \
    libmp3lame-dev \
    libtheora-dev \
    libvorbis-dev \
    libxine2-dev \
    libopencore-amrnb-dev \
    libopencore-amrwb-dev \
    # Math libraries
    libopenblas-dev \
    liblapack-dev \
    libatlas-base-dev \
    libeigen3-dev \
    libhdf5-dev \
    # OpenGL support
    libgl1-mesa-glx \
    libglu1-mesa-dev \
    libglew-dev \
    # Qt5 for OpenCV GUI
    qtbase5-dev \
    qtchooser \
    qt5-qmake \
    qtbase5-dev-tools \
    # Additional utilities
    libprotobuf-dev \
    protobuf-compiler \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

# Set Python 3.9 as default
RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.9 1 \
    && update-alternatives --install /usr/bin/python python /usr/bin/python3.9 1 \
    && python3.9 -m pip install --upgrade pip setuptools wheel

# Install numpy first (required for OpenCV Python bindings)
# Also install wheel building tools
RUN python3 -m pip install --no-cache-dir \
    numpy==1.26.4 \
    wheel \
    setuptools \
    build \
    auditwheel \
    patchelf

# Remove any pre-existing OpenCV packages that might conflict
RUN pip3 uninstall -y opencv-python opencv-python-headless opencv-contrib-python opencv-contrib-python-headless || true

# Set GCC-10 as default for OpenCV build
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-10 100 \
    && update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-10 100

# Configure ccache
RUN ccache --set-config=cache_dir=/ccache \
    && ccache --set-config=max_size=15G \
    && ccache --set-config=compression=true \
    && ccache --set-config=compression_level=6


# OpenCV 4.8.0 with CUDA support and NONFREE modules
ARG OPENCV_VERSION=4.8.0
ARG CUDA_ARCH_BIN="7.5;8.0;8.6;8.9"

# opencv and opencv-contrib :
# including NONFREE code -could be used or not-
# Use BuildKit cache mount for ccache to speed up rebuilds
RUN --mount=type=cache,target=/ccache \
    cd /opt/ &&\
    wget https://github.com/opencv/opencv/archive/${OPENCV_VERSION}.zip -O opencv.zip &&\
    unzip -qq opencv.zip &&\
    rm opencv.zip &&\
    wget https://github.com/opencv/opencv_contrib/archive/${OPENCV_VERSION}.zip -O opencv-co.zip &&\
    unzip -qq opencv-co.zip &&\
    rm opencv-co.zip &&\
    mkdir /opt/opencv-${OPENCV_VERSION}/build && cd /opt/opencv-${OPENCV_VERSION}/build &&\
    # Configure OpenCV with proper CUDA linking:\
    # - Use stubs for linking but set RPATH to real CUDA libs for runtime\
    # - This prevents runtime failures from stub library references\
    cmake \
    -D BUILD_opencv_java=OFF \
    -D WITH_CUDA=ON \
    -D BUILD_opencv_dnn=ON \
    -D CUDA_ARCH_BIN="${CUDA_ARCH_BIN}" \
    -D WITH_CUBLAS=ON \
    -D WITH_CUDNN=ON \
    -D OPENCV_DNN_CUDA=ON \
    -D ENABLE_FAST_MATH=ON \
    -D CUDA_FAST_MATH=ON \
    -D WITH_CUFFT=ON \
    -D WITH_OPENGL=ON \
    -D WITH_QT=ON \
    -D WITH_IPP=ON \
    -D WITH_TBB=ON \
    -D WITH_EIGEN=ON \
    -D WITH_OPENEXR=ON \
    -D BUILD_TESTS=OFF \
    -D BUILD_PERF_TESTS=OFF \
    -D BUILD_DOCS=OFF \
    -D BUILD_EXAMPLES=OFF \
    -D WITH_OPENCL=ON \
    -D WITH_OPENMP=ON \
    -D WITH_FFMPEG=ON \
    -D WITH_V4L=ON \
    -D WITH_GSTREAMER=ON \
    -D CMAKE_BUILD_TYPE=RELEASE \
    -D CMAKE_C_COMPILER_LAUNCHER=ccache \
    -D CMAKE_CXX_COMPILER_LAUNCHER=ccache \
    -D CMAKE_CUDA_COMPILER_LAUNCHER=ccache \
    -D OPENCV_EXTRA_MODULES_PATH=/opt/opencv_contrib-${OPENCV_VERSION}/modules \
    -D CMAKE_INSTALL_PREFIX=/usr/local \
    -D PYTHON3_EXECUTABLE=$(which python3) \
    -D PYTHON3_INCLUDE_DIR=$(python3 -c "import sysconfig; print(sysconfig.get_paths()['include'])") \
    -D PYTHON3_LIBRARY=$(python3 -c "import sysconfig; cfg=sysconfig.get_config_vars(); print(cfg['LIBDIR'] + '/' + cfg['LDLIBRARY'])") \
    -D PYTHON3_PACKAGES_PATH=$(python3 -c "import sysconfig; print(sysconfig.get_paths()['platlib'])") \
    -D PYTHON3_NUMPY_INCLUDE_DIRS=$(python3 -c "import numpy; print(numpy.get_include())") \
    -D BUILD_opencv_python3=ON \
    -D CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda \
    -D CMAKE_LIBRARY_PATH=/usr/local/cuda/lib64/stubs \
    -D CMAKE_INSTALL_RPATH=/usr/local/cuda/lib64 \
    -D CMAKE_BUILD_RPATH=/usr/local/cuda/lib64 \
    -D CMAKE_INSTALL_RPATH_USE_LINK_PATH=ON \
    -D OPENCV_ENABLE_NONFREE=ON \
    .. &&\
    make -j$(nproc) && \
    make install && \
    ldconfig

# Copy constraints file to prevent PyPI OpenCV packages from overwriting our build
COPY constraints.txt /tmp/constraints.txt

# Build and install OpenCV as a proper Python wheel
COPY build_opencv_wheel.py /tmp/
RUN cd /tmp && \
    python3 build_opencv_wheel.py ${OPENCV_VERSION} --output-dir /tmp/opencv_wheel --install && \
    # Save the wheel for potential reuse
    cp /tmp/opencv_wheel/dist/*.whl /tmp/ 2>/dev/null || true && \
    # Clean up build directories
    rm -rf /opt/opencv-${OPENCV_VERSION} && \
    rm -rf /opt/opencv_contrib-${OPENCV_VERSION} && \
    rm -rf /tmp/opencv_wheel && \
    rm /tmp/build_opencv_wheel.py

# Quick verification that OpenCV is installed
RUN python3 -c "import cv2; print(f'OpenCV installed: {cv2.__version__}')" && \
    pip3 list | grep opencv

build_opencv_wheel.py

#!/usr/bin/env python3
"""
Build a proper Python wheel for OpenCV after compilation.
This script creates a wheel package from the compiled OpenCV libraries.
"""

import os
import sys
import shutil
import subprocess
from pathlib import Path
from setuptools import setup, find_packages
import sysconfig

def find_cv2_module():
    """Find the compiled cv2 module."""
    # Check both platlib and purelib
    for key in ('platlib', 'purelib'):
        site_packages = Path(sysconfig.get_paths()[key])
        cv2_path = site_packages / 'cv2'
        if cv2_path.exists() and cv2_path.is_dir():
            return cv2_path
    
    # Common locations where cv2.so might be after make install
    possible_paths = [
        Path('/usr/local/lib/python3.9/dist-packages/cv2'),
        Path('/usr/local/lib/python3.9/site-packages/cv2'),
        Path('/usr/lib/python3/dist-packages/cv2'),
        Path('/usr/lib/python3.9/dist-packages/cv2'),
        Path('/usr/lib/python3.9/site-packages/cv2'),
    ]
    
    for path in possible_paths:
        if path.exists() and path.is_dir():
            return path
    
    # Try to find cv2.*.so files
    for path in [Path('/usr/local/lib'), Path('/usr/lib'), Path('/usr/local')]:
        cv2_files = list(path.glob('**/cv2*.so'))
        if cv2_files:
            return cv2_files[0].parent
    
    raise FileNotFoundError("Could not find compiled cv2 module")

def create_wheel(opencv_version='4.8.0', output_dir='/tmp/opencv_wheel'):
    """Create a wheel from the compiled OpenCV."""
    
    # Find the cv2 module
    cv2_path = find_cv2_module()
    print(f"Found cv2 module at: {cv2_path}")
    
    # Create temporary build directory
    build_dir = Path(output_dir)
    build_dir.mkdir(parents=True, exist_ok=True)
    
    # Copy cv2 module directly to build directory (top-level package)
    cv2_dest = build_dir / 'cv2'
    if cv2_dest.exists():
        shutil.rmtree(cv2_dest)
    shutil.copytree(cv2_path, cv2_dest)
    
    # Ensure cv2 has proper version info
    version_file = cv2_dest / '__version__.py'
    version_file.write_text(f'__version__ = "{opencv_version}+cuda12.1"\n')
    
    # Create setup.py
    setup_py = build_dir / 'setup.py'
    setup_content = f'''
from setuptools import setup, find_packages

setup(
    name='opencv-contrib-python',
    version='{opencv_version}+cuda12.1',
    description='OpenCV Python bindings with contrib modules and CUDA support (custom build)',
    long_description='Custom build of OpenCV {opencv_version} with CUDA support and contrib modules',
    author='OpenCV Team',
    author_email='',
    url='https://opencv.org',
    license='Apache 2.0',
    packages=['cv2', 'cv2.data', 'cv2.misc', 'cv2.mat_wrapper', 'cv2.utils'],
    package_data={{
        'cv2': [
            '*.so', 
            '*.pyd', 
            'config*.py',
            '__version__.py',
            'data/*.xml',
            'data/*.dat', 
            'misc/**/*.json',
            'mat_wrapper/*.json',
            'utils/**/*.py'
        ]
    }},
    include_package_data=True,
    install_requires=['numpy>=1.19.3'],
    python_requires='>=3.6',
    classifiers=[
        'Development Status :: 5 - Production/Stable',
        'Intended Audience :: Developers',
        'Intended Audience :: Science/Research',
        'License :: OSI Approved :: Apache Software License',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.9',
        'Programming Language :: C++',
        'Topic :: Scientific/Engineering',
        'Topic :: Scientific/Engineering :: Image Recognition',
        'Topic :: Software Development :: Libraries :: Python Modules',
    ],
    zip_safe=False,
)
'''
    setup_py.write_text(setup_content)
    
    # Create MANIFEST.in to include all necessary files
    manifest = build_dir / 'MANIFEST.in'
    manifest.write_text('''
recursive-include cv2 *.so *.pyd *.py
recursive-include cv2/data *
recursive-include cv2/misc *
recursive-include cv2/mat_wrapper *
recursive-include cv2/utils *
''')
    
    # Build the wheel using modern build module
    os.chdir(build_dir)
    result = subprocess.run(
        [sys.executable, '-m', 'build', '--wheel', '--outdir', 'dist'],
        capture_output=True,
        text=True
    )
    
    if result.returncode != 0:
        print(f"Error building wheel: {result.stderr}")
        # Fallback to legacy method if build module fails
        print("Falling back to legacy setup.py bdist_wheel...")
        result = subprocess.run(
            [sys.executable, 'setup.py', 'bdist_wheel'],
            capture_output=True,
            text=True
        )
        if result.returncode != 0:
            print(f"Error with fallback build: {result.stderr}")
            return None
    
    # Find the built wheel
    dist_dir = build_dir / 'dist'
    wheels = list(dist_dir.glob('*.whl'))
    
    if not wheels:
        print("No wheel file found after build")
        return None
    
    wheel_file = wheels[0]
    print(f"Successfully built wheel: {wheel_file}")
    
    # Skip auditwheel for internal use (often fails with CUDA libs)
    # Uncomment if you need manylinux compatibility for distribution
    # try:
    #     repaired_dir = build_dir / 'wheelhouse'
    #     repaired_dir.mkdir(exist_ok=True)
    #     
    #     result = subprocess.run(
    #         ['auditwheel', 'repair', str(wheel_file), 
    #          '--plat', 'manylinux_2_17_x86_64',
    #          '-w', str(repaired_dir)],
    #         capture_output=True,
    #         text=True
    #     )
    #     
    #     if result.returncode == 0:
    #         repaired_wheels = list(repaired_dir.glob('*.whl'))
    #         if repaired_wheels:
    #             print(f"Repaired wheel: {repaired_wheels[0]}")
    #             return repaired_wheels[0]
    #     else:
    #         print(f"Auditwheel repair failed: {result.stderr}")
    #         print("Using original wheel")
    # except Exception as e:
    #     print(f"Auditwheel not available or failed: {e}")
    #     print("Using original wheel")
    
    return wheel_file

def main():
    import argparse
    
    parser = argparse.ArgumentParser(
        description='Build a Python wheel for compiled OpenCV'
    )
    parser.add_argument(
        'version',
        nargs='?',
        default='4.8.0',
        help='OpenCV version without +cuda suffix (default: 4.8.0)'
    )
    parser.add_argument(
        '--output-dir',
        default='/tmp/opencv_wheel',
        help='Output directory for wheel build (default: /tmp/opencv_wheel)'
    )
    parser.add_argument(
        '--install',
        action='store_true',
        help='Install the wheel after building'
    )
    
    args = parser.parse_args()
    
    # Build the wheel
    wheel_file = create_wheel(args.version, args.output_dir)
    
    if wheel_file and args.install:
        print(f"Installing wheel: {wheel_file}")
        result = subprocess.run(
            [sys.executable, '-m', 'pip', 'install', 
             str(wheel_file), '--force-reinstall'],
            capture_output=True,
            text=True
        )
        if result.returncode == 0:
            print("Wheel installed successfully")
            # Verify installation
            subprocess.run([sys.executable, '-c', 
                          'import cv2; print(f"OpenCV {cv2.__version__} installed")'])
        else:
            print(f"Installation failed: {result.stderr}")
            return 1
    
    return 0

if __name__ == '__main__':
    sys.exit(main())

constraints.txt

# Block all PyPI OpenCV packages - we use our custom build
opencv-python==99.99.99  # Impossible version to prevent installation
opencv-python-headless==99.99.99  # Impossible version to prevent installation
opencv-contrib-python==99.99.99  # Impossible version to prevent installation
opencv-contrib-python-headless==99.99.99  # Impossible version to prevent installation