Python reload() not re-loading class implementation?

Question

I have a module structure like the following:

.
└── testmodule
    ├── __init__.py
    └── submodule
        ├── __init__.py
        └── implementation.py

2 directories, 3 files

Here are the contents of each file

# testmodule/__init__.py
import submodule

# testmodule/submodule/__init__.py
from implementation import *

# testmodule/submodule/implementation.py
class Car(object):
    def __init__(self):
        self.doors = 2
        self.color = 'red'

Why does the class implementation not "reload" when I use reload() as in the following test?

Python 2.7.12 (default, Oct 11 2016, 05:24:00)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import testmodule
>>> car = testmodule.submodule.Car()
>>> car.doors
2
>>> # I edit the file and change self.doors = 2 to self.doors = 4
>>> reload(testmodule)
<module 'testmodule' from 'testmodule/__init__.pyc'>
>>> car = testmodule.submodule.Car()
>>> car.doors
2
>>> # no more edits made before the Python REPL is restarted

me@laptop # python
Python 2.7.12 (default, Oct 11 2016, 05:24:00)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import testmodule
>>> car = testmodule.submodule.Car()
>>> car.doors
4

Update

By @wim's logic, I would have to reload testmodule.submodule.implementation, then reload testmodule.submodule to make this "work", and indeed that is the case. See these tests:

Python 2.7.12 (default, Oct 11 2016, 05:24:00)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import testmodule
>>> c = testmodule.submodule.Car()
>>> c.doors
2
>>> # I edit the file and change self.doors = 2 to self.doors = 4
>>> reload(testmodule.submodule.implementation)
<module 'testmodule.submodule.implementation' from 'testmodule/submodule/implementation.py'>
>>> c = testmodule.submodule.Car()
>>> c.doors
2
>>> reload(testmodule.submodule)
<module 'testmodule.submodule' from 'testmodule/submodule/__init__.pyc'>
>>> c = testmodule.submodule.Car()
>>> c.doors
4

me@laptop # python
Python 2.7.12 (default, Oct 11 2016, 05:24:00)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import testmodule
>>> c = testmodule.submodule.Car()
>>> c.doors
2
>>> # I edit the file and change self.doors = 2 to self.doors = 4
>>> reload(testmodule.submodule)
<module 'testmodule.submodule' from 'testmodule/submodule/__init__.pyc'>
>>> c = testmodule.submodule.Car()
>>> c.doors
2
>>> reload(testmodule.submodule.implementation)
<module 'testmodule.submodule.implementation' from 'testmodule/submodule/implementation.py'>
>>> c = testmodule.submodule.Car()
>>> c.doors
2
>>> reload(testmodule.submodule)
<module 'testmodule.submodule' from 'testmodule/submodule/__init__.pyc'>
>>> c = testmodule.submodule.Car()
>>> c.doors
4

reload is far less thorough than people usually expect. This is only one of the ways in which it doesn't reload everything you'd want it to reload. It's usually better to just restart Python. — user2357112
– user2357112, Commented Feb 1, 2017 at 19:34
By the way, it's unrelated to your issue, but the line from implementation import *, and also the line import submodule, are very bad practice. These are implicit relative imports, and your code will stop working properly on Python 3. — wim
– wim, Commented Feb 1, 2017 at 20:09

wim · Accepted Answer · 2017-02-01 19:51:06Z

4

TL;DR that's expected behaviour of reload.

Why it happens?

Importing testmodule actually loads the subpackages into sys.modules:

>>> import testmodule
>>> [m for m in sys.modules if m.startswith('testmodule')]
['testmodule.submodule', 'testmodule.submodule.implementation', 'testmodule']

Now, when you reload testmodule, it will pickup changes in the implementation of testmodule directly - i.e. any lines changed in testmodule/__init__.py.

However, you didn't change any lines there, and the submodule name which testmodule holds a reference will still point to the old unchanged submodule.

Basically, the meta-answer is that you overestimated how smart "reload" is about things. You can implement a deep-reload which hooks into subpackages, but it's pretty hard to do correctly, and I would advise you not to bother.

What can you do about it?

IPython made some attempts in that direction, if you're interested:

# Python 2
import __builtin__
from IPython.lib import deepreload
__builtin__.reload = deepreload.reload

(see IPython.lib.deepreload for more information).

edited Feb 1, 2017 at 19:51

answered Feb 1, 2017 at 19:45

wim

368k114 gold badges681 silver badges816 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

OregonTrail Over a year ago

Thank you for the information, see my edits to the question.

OregonTrail Over a year ago

As I use the IPython REPL, starting up and then running from IPython.lib.deepreload import reload is a good solution for me, and actually fixes the issue that I was having in this case.

OregonTrail Over a year ago

I've already run into an issue caused by a stateful __init__.py when using this technique. So, I'm going back to restarting Python, which is a pain.

wim Over a year ago

Yep. There's a reason no one uses reload...it sucks! ;)

Collectives™ on Stack Overflow

Python reload() not re-loading class implementation?

1 Answer 1

Why it happens?

What can you do about it?

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Why it happens?

What can you do about it?

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related