1

I am having following structure for my project

Object_Detection/
                 setup.py
                 setup.cfg
                 requirement.txt
                 object_detection/
                                  models
                                  __init__.py #contains from . import models
                 tests/ 
                       # inside tests dir
                       test_utils_image.py
                       __init__.py #empty
                 utils/
                      # inside utils dir
                      __init__.py #inside 
                      utils_image_preprocess.py
                      utils_image.py
                      utils_tfrecord.py

Now init.py inside utils directory contains the following code.

# inside __init__.py
from . import utils_image_preprocess
from . import utils_image
from . import utils_tfrecord

Running above init.py files gives me an error:

ImportError: attempted relative import with no known parent package

test_utils.py inside tests dir contains the following code

# inside test_utils.py
from object_detection.utils import utils_image

While running test_utils.py I got the following error

ImportError: cannot import name 'utils_image' from 'object_detection.utils'

I have gone through this and this and tried to follow every aspect mentioned there but details about what to put inside init.py is not clear.

This problem seems to be associated with the structuring of init.py in different dir.

I have gone through various and got to know that if we keep even an empty init.py file then things will work out but I am not sure about my understanding.

Please let me know

  1. what I am missing here and whether I am following the correct structure for packaging my code or not?
  2. How to resolve these two errors?
  3. Is this issue related to setting up source in IDE as I am using Vscode and I have also seen this has been mentioned at many places. See here? (But also tried the same code with PyCharm and encountered same error )
Improve this question
1
  • For me, I have a package named: object_detection installed. add sys.path.insert(0, 'the file dir'), not sys.path.append(), solved. Commented Nov 1, 2023 at 12:15

3 Answers 3

Reset to default
6
+50

If you want to be able to say ...

from object_detection.utils import utils_image

... then clearly the utils directory must be a subdirectory of the object_detection directory and not a sibling directory, i.e. at the same level.

Now for your other error:

ImportError: attempted relative import with no known parent package

You did not really specify under what circumstances you get this error other than saying "Running above init.py files gives me an error:". But how are you "running" these py files or what does that even mean?

If you are executing a script when this occurs (how else would you be getting this error?), the script must be invoked as a module (because scripts cannot have relative imports -- see below) as follows (we will assume that the script you are trying to execute is test_utils_image.py):

First, the parent directory of object_detection, which is Object_Detection, must be in the system path of directories to be searched for finding modules and packages referenced in import statements. In general, this can be accomplished several ways, for instance

  1. The script you are executing is in Object_Detection (the directory of the script is automatically added to the sys.path list of directories to be searched by the interpreter).
  2. Dynamically appending Object_Detection to the sys.path list of directories at runtime by your script.
  3. Appending Object_Detection to the PYTHONPATH environment variable.

Item 1 above would not be applicable for this specific case since the module we are executing by definition is not in the Object_Detection directory.

Note that if your classes will eventually be installed with pip, then site-packages will be the parent directory of object_detection, which is already in sys.path.

Then you can execute your script as:

python -m tests.test_utils_image

If you want to execute this .py file as a script, for example by right-mouse clicking on it is VS Code, then see Relative imports for the billionth time, in particular the section Scripts can't import relative, which says it all -- it cannot work!

To invoke this as a script, just convert the relative imports to absolute imports. In fact, the PEP 8 Style Guide says:

Absolute imports are recommended, as they are usually more readable and tend to be better behaved (or at least give better error messages) if the import system is incorrectly configured (such as when a directory inside a package ends up on sys.path):

Improve this answer
Sign up to request clarification or add additional context in comments.

12 Comments

Actually, Object_Detection and object_detection` are not the same names (at least not if you are running under Linux/Unix). But no matter if you are running under say Windows. And if you want to have test at the same level as object_detection, that's OK with me. So as far as your first comment goes, I don't believe there are any issues.
As I said in my answer, once this package is installed, there will be a new directory object_detection under directory python_install_directory/lib/site_packages (if it is installed globally), which is in the sys.path list (that is how all packages installed with pip are found) and so there is nothing necessary for you to do. The simplest thing to test this would be for you to just install this. Or you can in your script: import sys; sys.path.append(full-path-to-Object_Detection) and then you can do your imports.
No, I can't say it's usual to update sys.path dynamically at runtime. While you are testing just set PYTHONPATH or if you are using a development environment that has a facility for specifying directories to search, add the parent directory there. Or simply install the package with pip (you can install it locally with the --user option).
But If you ever weren't going to be installing your package using pip and if all your scripts were in a common directory and you knew where the location of the parent directory for you package was relative to your scripts, it would be relatively simple to compute the absolute path to that parent directory and update sys.path as the very first thing the scripts do. For example, if the directory to be added was in the grandfather directory: import sys; import os; sys.path.insert(0, os.path.abspath(os.path.dirname(__file__ ) + '/../../Object_Detection'))
You do not run __init__.py files. They exist to signify a package as a opposed to just a module and its purpose is to initialize the package when you import the package in a script (they can be empty). That script is what you run from outside the package even if it is one of the member .py files in the package. From within the package directories you have to run it as a module, i.e. python -m tests.script_name. If you must run it as a script from within the package (by right-clicking?), then you will have to get rid of the relative imports; but they are not that necessary.
|
0

Have you tried to do the following?

inside your utils __init__.py import your modules as follows:

from .utils_image_preprocess import <func1>... <rest of functions/classes you want to import>
from .utils_image import <func1>... <rest of functions/classes you want to import>
from .utils_tfrecord import <func1>... <rest of functions/classes you want to import>

And then in your test file do:

from object_detection.utils.utils_image import *

OR

from object_detection.utils.utils_image import <func1>,...

Also, make sure you don't have any circular dependencies in your modules. for example importing of function from your tests to your util module and vise versa

Improve this answer

1 Comment

I followed your advice and call all the function inside init.py . Then in test_utils_image.py I called from object_detection.utils.utils_image import * It is giving ModuleNotFoundError: No module named 'object_detection.utils.utils_image'
0

Python3 has two types of packages

  1. Regular Packages
  2. Namespace Packages

Regular packages contains init.py and namespace packages don't need to have init.py

Regarding your folder structure, it is correct, no change needed. You just need to import like this

from utils import utils_image

without mentioning the objects_detection as objects_detection is just a namespace package.

So it would be usefull when you would refer to the utils module from outside the objects_detection folder.

Here what python docs say about the namespace packages:

A namespace package is a composite of various portions, where each portion contributes a subpackage to the parent package.

enter image description here

Improve this answer

6 Comments

It is saying ModuleNotFoundError: No module named 'utils'. I tried this with init.py and using the structure mentioned by @Joseph in his answer and with empty init.py file. This is not working.
Interesting, It worked on my local machine. Could you also tell us which directory do you run this module from?
Does that change anything as long as internal structure and references are preserved? I am running using virtual env and directly by going into the test_utils_image.py file. I also tried it by restarting my ide but things are the same.
Alright, I am sorry but It is hard to see what the error is, I have checked on my local machine everything seems to be fine. Regardless could you open your terminal, go to directory of objects_detection and run this command python3 then inside the shell from tests imort test_utils.image
Can you please have a look now? I have updated the entire project structure. There are two directories with the name object_detection. I guess that is the requirement to make a package in python. Is that what caused the error? Can you please validate this at your end and let me know whether things are working fine at your end or not?
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.