How to import an editable package?

One depends on the other.

Better to un-ask the question. Cyclic deps and cyclic imports can in principle be hacked together, but they're no fun, and you would just be leaving a minefield for the next maintainer.

What you really want is to identify the core functionality that both need, and break it out as a third item on its own. Then the problem becomes trivial, because now you're using the machinery in the intended way.

This goes for small scale import of a third module. And it goes for the larger scale of taking a dependency on a third package.

Sorry this is not a cyclic dependency; what I meant is that one of the packages is a dependency to the other. I reworded this in my question.

Even if I install one of the packages as editable and the other as a regular dependency, the editable package cannot be imported. So this may not have anything to do with the project's structure but something cold be wrong with my venv. I rewrote the question.

It looks like you have had a botched install of luma.lcd at some point. editable___luma_lcd_2_11_0_finder.py should have an extra two underscores at the front (ie. __editable___luma_lcd_2_11_0_finder.py) and if should contain a lot more code. Specifically, it should contain a function called install(). And __editable__.luma_lcd-2.11.0.pth should look like:

import __editable___luma_lcd_2_11_0_finder; __editable___luma_lcd_2_11_0_finder.install()

I would recommend recreating your venv, installing everything from scratch, and updating pip before you install anything else. You should also ensure that the branch of luma.lcd you have checked out is working and installable. Maybe try a normal install first and see if it's importable (ie. pip install . rather than an editable install).

Alternatively

As luma uses namespace packages and the install of luma.lcd seems relatively simple, You might find it easier to not install it. Instead, install its single dependency and make it available on PYTHONPATH eg.

pwd
pip install 'luma.core>=2.4.1' # only requirement of luma.lcd on main
export PYTHONPATH=luma.lcd
python -c 'from luma import core, lcd; print(core); print(lcd)'

Gives:

/home/dunes/luma-test
<module 'luma.core' from '/home/dunes/luma-test/.venv/lib/python3.11/site-packages/luma/core/__init__.py'>
<module 'luma.lcd' from '/home/dunes/luma-test/luma.lcd/luma/lcd/__init__.py'>

Sorry, I botched the filename when copy/pasting it. The filename is __editable___luma_lcd_2_11_0_finder.py

These are the full contents of it:

from __future__ import annotations
import sys
from importlib.machinery import ModuleSpec, PathFinder
from importlib.machinery import all_suffixes as module_suffixes
from importlib.util import spec_from_file_location
from itertools import chain
from pathlib import Path

MAPPING: dict[str, str] = {'luma': '/home/user/luma/luma.lcd/luma'}
NAMESPACES: dict[str, list[str]] = {'luma': ['/home/user/luma/luma.lcd/luma']}
PATH_PLACEHOLDER = '__editable__.luma_lcd-2.11.0.finder' + ".__path_hook__"


class _EditableFinder:  # MetaPathFinder
    @classmethod
    def find_spec(cls, fullname: str, path=None, target=None) -> ModuleSpec | None:  # type: ignore
        # Top-level packages and modules (we know these exist in the FS)
        if fullname in MAPPING:
            pkg_path = MAPPING[fullname]
            return cls._find_spec(fullname, Path(pkg_path))

        # Handle immediate children modules (required for namespaces to work)
        # To avoid problems with case sensitivity in the file system we delegate
        # to the importlib.machinery implementation.
        parent, _, child = fullname.rpartition(".")
        if parent and parent in MAPPING:
            return PathFinder.find_spec(fullname, path=[MAPPING[parent]])

        # Other levels of nesting should be handled automatically by importlib
        # using the parent path.
        return None

    @classmethod
    def _find_spec(cls, fullname: str, candidate_path: Path) -> ModuleSpec | None:
        init = candidate_path / "__init__.py"
        candidates = (candidate_path.with_suffix(x) for x in module_suffixes())
        for candidate in chain([init], candidates):
            if candidate.exists():
                return spec_from_file_location(fullname, candidate)
        return None


class _EditableNamespaceFinder:  # PathEntryFinder
    @classmethod
    def _path_hook(cls, path) -> type[_EditableNamespaceFinder]:
        if path == PATH_PLACEHOLDER:
            return cls
        raise ImportError

    @classmethod
    def _paths(cls, fullname: str) -> list[str]:
        paths = NAMESPACES[fullname]
        if not paths and fullname in MAPPING:
            paths = [MAPPING[fullname]]
        # Always add placeholder, for 2 reasons:
        # 1. __path__ cannot be empty for the spec to be considered namespace.
        # 2. In the case of nested namespaces, we need to force
        #    import machinery to query _EditableNamespaceFinder again.
        return [*paths, PATH_PLACEHOLDER]

    @classmethod
    def find_spec(cls, fullname: str, target=None) -> ModuleSpec | None:  # type: ignore
        if fullname in NAMESPACES:
            spec = ModuleSpec(fullname, None, is_package=True)
            spec.submodule_search_locations = cls._paths(fullname)
            return spec
        return None

    @classmethod
    def find_module(cls, _fullname) -> None:
        return None


def install():
    if not any(finder == _EditableFinder for finder in sys.meta_path):
        sys.meta_path.append(_EditableFinder)

    if not NAMESPACES:
        return

    if not any(hook == _EditableNamespaceFinder._path_hook for hook in sys.path_hooks):
        # PathEntryFinder is needed to create NamespaceSpec without private APIS
        sys.path_hooks.append(_EditableNamespaceFinder._path_hook)
    if PATH_PLACEHOLDER not in sys.path:
        sys.path.append(PATH_PLACEHOLDER)  # Used just to trigger the path hook

I found this issue that led me to reinstall the package using an older version of setuptools that doesn’t create the file __editable__.luma_lcd-2.11.0.finder but relies on some older mechanics. The import now works in VS Code. Still not sure what’s wrong with the newer setuptools.