How to distinguish between files and submodules in Python logging?

Question

I wrote a submodule where I define logging there since I mainly use my submodule to reuse code in my different rest api projects. How can I set up my custom logging record so that logging knows which code (main.py or sub.py) called the logging? I tried using __file__, but then it would always say "sub.py".

My submodule:

# sub.py
import logging
from secrets import token_urlsafe

# https://stackoverflow.com/a/57820456/1150923
def record_factory(*args, **kwargs):
    record = old_factory(*args, **kwargs)
    record.session_id = session_id

    # What do I do here?
    # This doesn't work:
    record.src = __file__

    return record

session_id = token_urlsafe(8)

logger = logging.getLogger(__name__)
old_factory = logging.getLogRecordFactory()
logging.setLogRecordFactory(record_factory)

# Always prepend session_id and src to logs
format = logging.Formatter("%(asctime)s %(session_id) %(src) %(message)s", datefmt="%Y-%m-%d %H:%M:%S")
handler = logging.StreamHandler()
handler.setFormatter(format)
logger.addHandler(handler)

logger.info("Hello!") # Should also print out "sub.py".

My main script:

# main.py
import logging
import sub

logger = logging.getLogger("sub")
logger.info("Hi!") # Should also print out "main.py".

Dan D. · Accepted Answer · 2019-12-03 08:53:07Z

1

Logging provides an attribute for the module name, the same value as __file__, under the name pathname.

Replace %(src)s with %(pathname)s.

The way the Logger does it is via stack inspection in the Logger.findCaller method. It walks up the stack until it finds a frame that isn't in the logging module. And then extracts that frame's filename, lineno, and function name.

See Is module __file__ attribute absolute or relative? for the issue with __name__.

edited Dec 3, 2019 at 8:53

answered Dec 3, 2019 at 2:11

Dan D.

75k15 gold badges111 silver badges129 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

hobbes3 Over a year ago

Any idea why it only says ./main.py for the main file, but the full path like /Users/hobbes3/sub.py for the submodule? Ideally, I would like it to say the full path for both.

squareRoot17 · Accepted Answer · 2019-12-03 02:13:42Z

1

You can use the inspect module in this case.

import inspect as insp

def record_factory(*args, **kwargs):
    record = old_factory(*args, **kwargs)
    record.session_id = session_id

    (filename, line_number,func_name, lines, index) = insp.getframeinfo(insp.currentframe().f_back)

    #Do whatever with this values.
    return record

The way currentframe() works is it returns the frame object for the caller’s stack frame. This can be passed into the getframeinfo() method.

Find more details here : inspect Module

answered Dec 3, 2019 at 2:13

squareRoot17

9106 silver badges16 bronze badges

Collectives™ on Stack Overflow

How to distinguish between files and submodules in Python logging?

2 Answers 2

1 Comment

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related