add data files to python projects setup.py

Question

I have a project like this:

├── CHANGES.txt
├── LICENSE
├── MANIFEST.in
...
├── docs
│   └── index.rst
├── negar
│   ├── Negar.py
│   ├── Virastar.py
│   ├── Virastar.pyc
│   ├── __init__.py
│   ├── data
│   │   ├── __init__.py
│   │   └── untouchable.dat
│   ├── gui.py
│   ├── gui.pyc
│   ├── i18n
│   │   ├── fa_IR.qm
│   │   └── fa_IR.ts
│   └── negar.pro
├── setup.py
...

and inside that my file Virastar.py need some data from data.untouchable.dat. it works fine until I install the project with this setup.py:

setup(
    ...
    include_package_data=True,
    packages = find_packages() + ['negar'],
    package_dir={'negar': 'negar'},
    package_data={'negar': ['data/*.dat']},
    entry_points={
        'console_scripts': [
            'negar = negar.Negar:main',
        ],
    }, 
    ...  
)

after that when I start my program and when it needs that data file it return this error:

IOError: [Errno 2] No such file or directory: 'data/untochable.dat'

even in my egg-info sources I can't find any data file:

...
negar/Negar.py
negar/Virastar.py
negar/__init__.py
negar/gui.py
negar/data/__init__.py

have I missed something here?

Thank you all.

EDIT: Do I have to add any special thing in init.py?

and I have to add this: I used untouchable.dat just like this:

f = codecs.open('data/untouchable.dat', encoding="utf-8")

in python 2.7 they changed the way to include files to MANIFEST.in or something - I'm not sure and didn't use it but it could be a direction — zenpoy
– zenpoy, Commented Aug 30, 2012 at 15:20
I'm in python 2.6! and for some reasons I can't upgrade now! — Shahin
– Shahin, Commented Aug 30, 2012 at 15:39

dfrankow · Accepted Answer · 2013-03-10 14:19:35Z

23

I used data_files

data_files = [('', ['negar/data/untouchable.dat'])],

answered Mar 10, 2013 at 14:19

dfrankow

21.7k44 gold badges167 silver badges246 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Jari Turkia Over a year ago

For those wondering where the file will go: installation destination will be sys.prefix. Suggest replacing '' with a dir-name to make some sense for the destination.

dfrankow Over a year ago

@JariTurkia Ah. So, are you suggesting ('negar/data', ['untouchable.dat'])?

Jari Turkia Over a year ago

On the tuple, left side is destination, right side is source file. By having the destination as empty, during setup the file will be placed in the root of your Python's prefix. My suggestion is to have a directory like this: [('negar/data', ['negar/data/untouchable.dat'])]

Community · Accepted Answer · 2017-05-23 12:10:30Z

13

The first problem is that I didn't import my data file into the package with MANIFEST.in file. I imported it like this:

include negar/data/*.dat

After that my data file already imported with my package install. but because I had mistakes in open my data files, python couldn't find it. this question helped me to find the right way Python Access Data in Package Subdirectory and now I use something like this:

import os
this_dir, this_filename = os.path.split(__file__)
DATA_PATH = os.path.join(this_dir, "data", "data.txt")
print open(DATA_PATH).read()

edited May 23, 2017 at 12:10

CommunityBot

11 silver badge

answered Aug 31, 2012 at 5:56

Shahin

1,4754 gold badges22 silver badges34 bronze badges

1 Comment

Lucas Over a year ago

in which file did you add the lines import os ...?

schacki · Accepted Answer · 2012-08-30 07:38:20Z

3

Maybe try:

package_data={'negar/data': ['data/*.dat']},

answered Aug 30, 2012 at 7:38

schacki

9,5335 gold badges31 silver badges32 bronze badges

Comments

nyxgear · Accepted Answer · 2022-09-08 13:13:34Z

A solution that does not require any of:

MANIFEST.in
include_package_data=True
package_dir={}
and neither the __init__.py file in the folder!

Having the project like:

├── CHANGES.txt
├── LICENSE
...
├── negar
│   ├── Negar.py
│   ├── Virastar.py
│   ├── Virastar.pyc
│   ├── __init__.py
│   ├── data
│   │   ├── __init__.py
│   │   └── untouchable.dat
│   ├── subfolder
│   │   ├── __init__.py
│   │   ├── data_NOT_included
│   │   │   └── garbage.toml
│   │   └── data_with_no_init_file
│   │       ├── config.ini
│   │       └── options.yml
│   ├── gui.py
│   ├── gui.pyc
│   ├── i18n
│   │   ├── fa_IR.qm
│   │   └── fa_IR.ts
│   └── negar.pro
├── setup.py
...

The setup file:

setup(
    ...
    packages = find_packages()
    package_data= {
        # all .dat files at any package depth
        '': ['**/*.dat'],

        # into the data folder (being into a module) but w/o the init file
        'negar.subfolder': [ '**/*.ini', '**/*.yml', ]
        
    },
    entry_points={
        'console_scripts': [
            'negar = negar.Negar:main',
        ],
    }, 
    ...
)

This worked great in my case and I avoided maintaining an additional MANIFEST.in file.

caveat: the "negar.subfolder" has to be a python module.

To the access the file:

from importlib.resources import files
config = files('negar.subfolder').joinpath('data_with_no_init_file').joinpath('config.ini').read_text()

from accessing-data-files-at-runtime

Collectives™ on Stack Overflow

add data files to python projects setup.py

4 Answers 4

3 Comments

1 Comment

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

3 Comments

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related