python logging using filename and line number

Question

I'm using the basic logging python module to add log.

I decide to create a log class, as follow, named Log.py :

class Log:
    def __init__(self, path):
        self.LOGGER = logging.getLogger('Open-Capture')
        if self.LOGGER.hasHandlers():
            self.LOGGER.handlers.clear() # Clear the handlers to avoid double logs
        logFile = RotatingFileHandler(path, mode='a', maxBytes=5 * 1024 * 1024,
                            backupCount=2, encoding=None, delay=0)
        formatter = logging.Formatter('[%(threadName)-14s] [%(filename)s:%(lineno)-15s] %(asctime)s %(levelname)s %(message)s', datefmt='%d-%m-%Y %H:%M:%S')
        logFile.setFormatter(formatter)
        self.LOGGER.addHandler(logFile)
        self.LOGGER.setLevel(logging.DEBUG)

    def info(self, msg):
        self.LOGGER.info(msg)

    def error(self, msg):
        self.LOGGER.error(msg)

As you can see, I have the %(filename)s:%(lineno) vars to add more lisibility. But in my log file I have Log.py:34 instead of filename and line number of the source file because when I log I call it like this :

Log = Log(Config.cfg['GLOBAL']['logfile'])
Log.info('test log')

Is there any way to had the source file on my log file instead of Log.py ?

Thanks in advance

There are multiple python file using the Log class. FindContact.py, FindDate.py etc.... — Nathan Cheval
– Nathan Cheval, Commented Jan 17, 2020 at 14:26
I'm confused because it's working as how it should. What do you mean by "But in my log file I have Log.py:34 instead of filename and line number" isn't Log.py is your filename and 34 is the line number? — Amit Tripathi
– Amit Tripathi, Commented Jan 17, 2020 at 14:34

Community · Accepted Answer · 2020-06-20 09:12:55Z

4

Yes it is possible, althrough I'm quite sure the implementation I'm posting here is not 100% safe and sure there is a better / more elegant implementation, take this as a hint.

To get the filename and line# of the caller you can use the inspect module, to add those custom infos to your logs you can add a custom Filter:

log.py

import logging
from logging.handlers import RotatingFileHandler
from inspect import getframeinfo, stack


class CallerFilter(logging.Filter):
    """ This class adds some context to the log record instance """
    file = ''
    line_n = ''

    def filter(self, record):
        record.file = self.file
        record.line_n = self.line_n
        return True

def caller_reader(f):
    """This wrapper updates the context with the callor infos"""
    def wrapper(self, *args):
        caller = getframeinfo(stack()[1][0])
        self._filter.file = caller.filename
        self._filter.line_n = caller.lineno
        return f(self, *args)
    return wrapper

class Log:

    def __init__(self, path):
        self.LOGGER = logging.getLogger('Open-Capture')
        if self.LOGGER.hasHandlers():
            self.LOGGER.handlers.clear() # Clear the handlers to avoid double logs
        logFile = RotatingFileHandler(path, mode='a', maxBytes=5 * 1024 * 1024,
                            backupCount=2, encoding=None, delay=0)
        formatter = logging.Formatter('[%(threadName)-14s] [%(file)s:%(line_n)-15s] %(asctime)s %(levelname)s %(message)s', datefmt='%d-%m-%Y %H:%M:%S')
        logFile.setFormatter(formatter)
        self.LOGGER.addHandler(logFile)
        # Here we add the Filter, think of it as a context
        self._filter = CallerFilter()
        self.LOGGER.addFilter(self._filter)
        self.LOGGER.setLevel(logging.DEBUG)

    @caller_reader
    def info(self, msg):
        self.LOGGER.info(msg)

    @caller_reader
    def error(self, msg):
        self.LOGGER.error(msg)

script_file.py

from log import Log

log = Log('l.log')
log.info('LOG MESSAGE')
log.error('LOG MESSAGE 2 ')
log.info('LOG MESSAGE 2 ')

output:

[MainThread    ] [script_file.py:4              ] 17-01-2020 16:34:36 INFO LOG MESSAGE
[MainThread    ] [script_file.py:5              ] 17-01-2020 16:34:36 ERROR LOG MESSAGE 2
[MainThread    ] [script_file.py:6              ] 17-01-2020 16:34:36 INFO LOG MESSAGE 2

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Jan 17, 2020 at 15:39

Hrabal

2,5332 gold badges24 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Nathan Cheval Over a year ago

Thanks a lot, it works well. But, why this isn't safe ?

Hrabal Over a year ago

Each call to the log "debug" and "error" methods I am storing the caller on CallerFilter class attributes, which are shared between different logger instances. This is safe for "normal" code, but say you are using this in a web app, there may be concurrency problems (depending on the app structure) that will lead to a log written with filename and number of a concurrent request... This is just theory though, if you use this on a script with no use of concurrency you'd be allright. (maybe better add a reset of "_filter" attributes after the call, now that I think about it... )

Nathan Cheval Over a year ago

So if I use logger.filters.clear() at each Log Class call, before adding filter, it could be better ?

Hrabal Over a year ago

Yes, I guess so.

Moises Marques Over a year ago

Great solution!

Collectives™ on Stack Overflow

python logging using filename and line number

1 Answer 1

log.py

script_file.py

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

log.py

script_file.py

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related