Import module dynamically

Question

I have a program that supports different communication protocols. The idea is, that the user chooses the protocol during startup and then the module supporting the respective protocol is imported.

The function below populates the protocol module with all functions and variables from the user-chosen protocol.

protocol.py:

def provide_protocol( name ):
    for key, val in vars( __import__( name ) ).items():
        if key.startswith( "__" ) and key.endswith( "__" ):
            continue
        vars()[key] = val

    importlib.invalidate_caches()

name is provided by the user at program startup.

The issue is, that protocol.py is imported before this function is run and the program is not able to access the newly populated functions in this previously empty module.

Using invalidate_caches() from the importlib module does not change this.

I also tried, to remove protocol.py and create the whole module dynamically using importlib, but this also did not succeed:

__init__.py:

protocol = None
def provide_protocol( name ):
    protocol = importlib.import_module( name )
    importlib.invalidate_caches()

The problem seems to be identical, because if I import protocol, it seems to be None.

Both of your functions do nothing useful with the module you import, and it is simply discarded. vars()[key] = val in a function just assigns to a dict object returned by locals() which has no effect on anything. — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Jun 28, 2022 at 8:28

juanpa.arrivillaga · Accepted Answer · 2022-06-28 08:35:22Z

Both of your functions don't don't return anything useful (they both return None). Your first function tries to modify vars, but that won't do anything useful. From the docs:

Without an argument, vars() acts like locals(). Note, the locals dictionary is only useful for reads since updates to the locals dictionary are ignored.

Note, if you had done that in a global scope, locals() is globals(), so those modifications would take effect, but since you are doing it in a function, nothing happens.

In your second function, you assign to a local variable protocol then don't do anything with it. The global variable protocol will remain None. You should just return protocol then use it like:

def provide_protocol( name ):
    protocol = importlib.import_module( name )
    importlib.invalidate_caches()
    return protocol


protocol = provide_protocol(name)

Note, in both cases, the module would still be in the cache, so:

import sys
protocol = sys.modules[name]

Would have retrieved it. But you should just explicitly return and assign it to whatever name you want.

Robin Dillen · Accepted Answer · 2022-06-28 08:35:02Z

1

This seems to solve your issue(note that there is no need to invalidate the caches in this case):

protocol = None
def provide_protocol(name):
    global protocol
    protocol = importlib.import_module(name)

answered Jun 28, 2022 at 8:35

Robin Dillen

7125 silver badges11 bronze badges

Collectives™ on Stack Overflow

Import module dynamically

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related