3

I am trying to include a python file in the build/lib directory created when running

python setup.py install

In particular, I would like to include a simple configuration file ('definitions.py') that defines a ROOT_DIR variable, which is then used by subpackages. The 'definitions.py' file contains:

import os
ROOT_DIR  = os.path.dirname(os.path.abspath(__file__))

My goal is to have configuration files within each subpackage ('config.py') call ROOT_DIR to build their own absolute paths:

from definitions import ROOT_DIR
PACKAGE_DIR = os.path.join(ROOT_DIR, 'package1/')

The idea is drawn from this stackoverflow answer: https://stackoverflow.com/a/25389715.
However, this 'definitions.py' file never shows up in the build directory when running 'setup.py install'.

Here is the directory structure of the project:

project
|
├── setup.py
|
├── definitions.py
|
├── package1
|   ├── __init__.py
|   ├── config.py
|   └── ...
|
├── package2
|   ├── __init__.py
|   └── ...
└── ...

My multiple attempts have failed (trying, e.g. the suggestions offered in https://stackoverflow.com/a/11848281). As far as I can tell, it's because definitions.py is in the top-level of my project structure (which lacks an __init__.py file).

I have tried:

1) ...using the 'package-data' variable in setuptools.setup() 

package_data={'package': ['./definitions.py']}

but definitions.py does not show up in the build
(I think because definitions.py is not in a 'package' that has an __init__.py?).

2) ...using a MANIFEST.in file, but this also does not work
(I think because MANIFEST does not work with .py files?)

My question:
Is there a way to include definitions.py in the build directory?
Or, is there a better way to provide access to absolute paths built from the top-level directory for multiple sub-packages?

Improve this question
10
  • Relevant context is missing. 1. Are you packaging an app or library code? 2. What is actually in definitions.py? Is it basically just a configuration file, or is there logic? Perhaps edit the question to include the contents. Commented Oct 22, 2018 at 21:10
  • edited to include context Commented Oct 22, 2018 at 21:36
  • Would it be fair to say your goal is for the package to know the path of where it has been installed? Note that in the example you're following, the definitions.py module is contained within the installed project dir, so it's quite a different context to what you seem to be attempting here. Commented Oct 22, 2018 at 21:42
  • yes, "goal is for the package to know the path of where it has been installed". Sorry for not being clearer... my current understanding is that the build/lib/ directory is where the package is installed, that's why I phrased it that way in the question. My thinking was that the most straightforward way to access absolute paths for each sub-package was to maintain the same structure in the installed package as in the development structure. Commented Oct 22, 2018 at 21:46
  • 1
    Why does the code need to know where the code is installed? Is it in order to build a relative path to some data or resource files? If so, there are much better options available here. Commented Oct 22, 2018 at 21:47

1 Answer 1

Reset to default
2

If you are looking for a way to access a non-python data file in the installed module like in the question you've linked (a configuration file in the top-level package that should be accessible in subpackages), use pkg_resources machinery instead of inventing a custom path resolution. An example project structure:

project
├── setup.py
└── root
    ├── __init__.py
    ├── config.txt
    ├── sub1
    │   └── __init__.py
    └── sub2
        └── __init__.py

setup.py:

from setuptools import setup

setup(
    name='myproj',
    ...,
    packages=['root', 'root.sub1', 'root.sub2'],  # or setuptools.find_packages()
    package_data={'root': ['config.txt']}
)

Update:

As pointed out by wim in the comments, there's now a backport for importlib.resources (which is only available in Python 3.7 and onwards) - importlib_resources, which offers a modern resource machinery that utilizes pathlib:

# access the filepath
importlib_resources.path('root', 'config.txt')

# access the contents as string
importlib_resources.read_text('root', 'config.txt')

# access the contents as file-like object
importlib_resources.open_binary('root', 'config.txt')

Original answer

Using pkg_resources, you can access the root/config.txt from any spot of your package without having to perform any path resolution at all:

import pkg_resources

# access the filepath:
filepath = pkg_resources.resource_filename('root', 'config.txt')

# access the contents as string:
contents = pkg_resources.resource_string('root', 'config.txt')

# access the contents as file-like object:
contents = pkg_resources.resource_stream('root', 'config.txt')

etc.

Improve this answer
Sign up to request clarification or add additional context in comments.

9 Comments

This will put definitions.py directly in site-packages dir. Unlikely what OP wanted, IMO. It should be namespaced in the project somehow!
While I agree that the use case of py_modules is rather when you are packaging a single-module distribution, there are many packages that define a top-level module for various reasons; colorama uses it for constant definition, pytest for defining non-private imports etc. I don't do that, but it's debatable whether it's a good practice or not.
Given the context the OP has now edited into the question, I have to give this a firm -1. It's not a good way for a package to discover the installed location.
Hmm, didn't see that.
pkg_resources is a part of setuptools (installer) and is sometimes a bit heavy/slow for such simple library task such as runtime resource management. It also creates an undesirable runtime dependency on setuptools. For a more modern approach: importlib.resources
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.