List classes in directory (Python)

Question

I'm developing a Python 2.6 package in which I would like to fetch a list of all classes in a certain directory (within the package) in order to then perform introspection on the class objects.

Specifically, if the directory containing the currently executing module has a sub-dir called 'foobar' and 'foobar' contains .py files specifying class Foo(MyBase), class Bar(MyBase), and class Bar2, I want to obtain a list of references to the class objects that inherit from MyBase, i.e. Foo and Bar, but not Bar2.

I'm not sure if this task actually need involve any dealing with the filesystem or if the modules in the sub-dir are automatically loaded and just need to be listed via introspection somehow. Any ideas here please? Example code is much appreciated, since I'm pretty new to Python, in particular introspection.

Community · Accepted Answer · 2020-01-04 04:55:41Z

11

Modules are never loaded automatically, but it should be easy to iterate over the modules in the directory and load them with the __import__ builtin function:

import os
from glob import glob
for file in glob(os.path.join(os.path.dirname(os.path.abspath(__file__))), "*.py"):
    name = os.path.splitext(os.path.basename(file))[0]
    # add package prefix to name, if required
    module = __import__(name)
    for member in dir(module):
        # do something with the member named ``member``

edited Jan 4, 2020 at 4:55

CommunityBot

11 silver badge

answered Jul 5, 2010 at 9:20

Philipp

50.1k12 gold badges88 silver badges112 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Noldorin Over a year ago

Ah, the __import__ function looks very helpful, thanks for that.

mtnpaul Over a year ago

Not all the members are necessarily classes.

Kafka4PresidentNow Over a year ago

glob takes only one argument, the correct loop should be for file in glob(os.path.join(os.path.dirname(os.path.abspath(__file__)), "*.py"))

krakover · Accepted Answer · 2014-07-10 10:21:43Z

9

I wanted to do the same thing, this is what I ended up with:

import glob
import importlib
import inspect
import os

current_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)))
current_module_name = os.path.splitext(os.path.basename(current_dir))[0]
for file in glob.glob(current_dir + "/*.py"):
     name = os.path.splitext(os.path.basename(file))[0]

     # Ignore __ files
     if name.startswith("__"):
         continue
     module = importlib.import_module("." + name,package=current_module_name)

     for member in dir(module):
         handler_class = getattr(module, member)

         if handler_class and inspect.isclass(handler_class):
             print member

Hope it helps..

answered Jul 10, 2014 at 10:21

krakover

3,0592 gold badges31 silver badges29 bronze badges

Comments

culebrón · Accepted Answer · 2010-07-05 09:33:35Z

5

Option 1: grep for "^class (\a\w+)\(Myclass" regexp with -r parameter.

Option 2: make the directory a package (create an empty __init__.py file), import it and iterate recursively over its members:

import mymodule
def itermodule(mod):
    for member in dir(mymod):
        ...

itermodule(mymodule)

edited Jul 5, 2010 at 9:33

answered Jul 5, 2010 at 9:18

culebrón

36.9k21 gold badges78 silver badges117 bronze badges

2 Comments

Noldorin Over a year ago

Not sure how grep-ing helps, though I'm sure it's just my ignorance. How can I do this within Python, and then get the class objects? Second solution looks good.

Mark Over a year ago

I think grep is meant to find all the files that contain 'class ' (the ones with classes in them), e.g. egrep -ir --include=*.py "class " . gives a list of these files, which you can convert to Python imports

Jossef Harush Kadouri · Accepted Answer · 2016-05-22 13:35:22Z

3

Dealt with it myself, this is my version (forked @krakover snippet):

Iterate directory and import each script placed there
- Filter out abstract classes
- Filter out classes that not inherit a base class
- New instance for each iterated class (change it if you don't find it useful)

import importlib
import inspect
import os
import glob


def import_plugins(plugins_package_directory_path, base_class=None, create_instance=True, filter_abstract=True):

    plugins_package_name = os.path.basename(plugins_package_directory_path)

    # -----------------------------
    # Iterate all python files within that directory
    plugin_file_paths = glob.glob(os.path.join(plugins_package_directory_path, "*.py"))
    for plugin_file_path in plugin_file_paths:
        plugin_file_name = os.path.basename(plugin_file_path)

        module_name = os.path.splitext(plugin_file_name)[0]

        if module_name.startswith("__"):
            continue

        # -----------------------------
        # Import python file

        module = importlib.import_module("." + module_name, package=plugins_package_name)

        # -----------------------------
        # Iterate items inside imported python file

        for item in dir(module):
            value = getattr(module, item)
            if not value:
                continue

            if not inspect.isclass(value):
                continue

            if filter_abstract and inspect.isabstract(value):
                continue

            if base_class is not None:
                if type(value) != type(base_class):
                    continue

            # -----------------------------
            # Instantiate / return type (depends on create_instance)

            yield value() if create_instance else value

Usage:

SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
plugins_directory_path = os.path.join(SCRIPT_DIR, 'plugins')
plugins = import_plugins(plugins_directory_path, base_class=BasePlugin)

for plugin in plugins:
    plugin.foo()

imagine there's a sub directory called plugins contains implementations of a BasePlugin class

edited May 22, 2016 at 13:35

answered May 19, 2016 at 17:24

Jossef Harush Kadouri

34.6k10 gold badges143 silver badges133 bronze badges

1 Comment

Jonathan Over a year ago

Although a very nice solution there are two issues with this solution. 1. I think the last filtering on base_class will not work find classes that inherit from the base class as type will never evaluate to the base_class. Using above you thus will only find the real bases classes, not the classes inherited from it. 2. When create_instance is True this will fail as soon as constructor is instantiated that has mandatory arguments, e.g anything that does not have a __init__(self), but a __init__(self, argument1) or something like __init__(self, argument1, argument2='').

Mark · Accepted Answer · 2013-07-20 22:49:51Z

On platforms that have egrep:

from subprocess import Popen, PIPE
from re import search

def get_classes(directory):
    job = Popen(['egrep', '-ir', '--include=*.py', 'class ', str(directory), ], stdout=PIPE)
    fileout, fileerr = job.communicate()
    if fileerr:
        raise Exception(fileerr)
    while directory[-1] == '/':
        directory = directory[:-1]
    found = []
    for line in fileout.split('\n'):
        match = search('^([^:]+).py:\s*class\s*(\S+)\s*\((\S+)\):', line)
        if match:
            pypath = match.group(1).replace(directory, '').replace('/', '.')[1:]
            cls = match.group(2)
            parents = filter(lambda x: x.strip, match.group(3).split())
            found.append((pypath, cls, parents, ))
    return found

For get_classes('.'), egrep returns something like:

./helpers/action.py:class Action(object):
./helpers/get_classes.py:    job = Popen(['egrep', '-ir', '--include=*.py', 'class ', str(directory), ], stdout=PIPE) # this is the get_classes script; not a valid result
./helpers/options.py:class Option(object):

which is converted into tuples of the path, class name and direct ancestors:

[('helpers.action', 'Action', ['object']), ('helpers.options', 'Option', ['object'])]

If you just want the paths, that's [item[0] for item in get_classes('.')].

Collectives™ on Stack Overflow

List classes in directory (Python)

5 Answers 5

3 Comments

Comments

2 Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

3 Comments

Comments

2 Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related