Python logging: create log if not exists or open and continue logging if it does

Question

I'm writing some code that uses the python logging system. The idea is that if the LOG doesn't already exist create the log but if it does then get the log and resume logging to that file. Here is my code:

import logging
import os

log_filename='Transactions.log')
if os.path.isfile(log_filename)!=True:
    LOG = logging.getLogger('log_filename')
    LOG.setLevel(logging.DEBUG)
    # create file handler which logs even debug messages
    fh = logging.FileHandler('log_filename')
    fh.setLevel(logging.DEBUG)
    # create console handler with a higher log level
    ch = logging.StreamHandler()
    ch.setLevel(logging.DEBUG)
    # create formatter and add it to the handlers
    formatter = logging.Formatter('-->%(asctime)s - %(name)s:%(levelname)s - %(message)s')
    fh.setFormatter(formatter)
    ch.setFormatter(formatter)
    # add the handlers to the logger
    LOG.addHandler(fh)
    LOG.addHandler(ch)
else:
    LOG=logging.getLogger()

I suspect the problem is with my else block but I don't know how to fix. Could anybody shed some light on this situation.

set up your logger once and then import LOG from any place you want, no need to check if the log file exists — mic4ael
– mic4ael, Commented Jan 20, 2017 at 13:36

hjpotter92 · Accepted Answer · 2017-01-20 13:37:27Z

45

The logging module's FileHandler takes care of that for you. No need for complexity.

The handler takes an optional mode parameter, to specify whether it starts writing or appending data to it.

From the docs:

class logging.FileHandler(filename, mode='a', encoding=None, delay=False)

The specified file is opened and used as the stream for logging. If mode is not specified, 'a' is used.

answered Jan 20, 2017 at 13:37

hjpotter92

81.1k36 gold badges148 silver badges188 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

mightypile Over a year ago

If the directory does not exist, you will still get an exception. Logging is happy to create a new file for you, but not a whole directory structure.

UserK Over a year ago

Which mode should @CiaranWelsh specify in order to create the file if not present?

hjpotter92 Over a year ago

@UserK if file is not present, either of a or w would be fine. a would append to the same file in next run, whereas w mode will delete the old file and create an entirely new one. Also read what @mightypile commented above.

user8491363 Over a year ago

On Python 2.7, it doesn't seem to work. FileHandler can't find the path and it returns error unless you explicitly make a path using os.makedirs()

KhoPhi · Accepted Answer · 2024-03-20 20:29:55Z

41

For anyone who was trying to create a new directory structure like logs/mylogfile.log, as @mightypile mentioned, FileHandler will not create a new directory structure for you. I used os.makedirs to ensure the directory structure.

import os
import logging

log_filename = "logs/output.log"
os.makedirs(os.path.dirname(log_filename), exist_ok=True)
file_handler = logging.FileHandler(log_filename, mode="w", encoding=None, delay=False)

edited Mar 20, 2024 at 20:29

KhoPhi

9,57718 gold badges88 silver badges139 bronze badges

answered Mar 19, 2019 at 19:24

lotrgollum87

4104 silver badges7 bronze badges

1 Comment

hackinghorn Over a year ago

Thanks! I believe it's log_filename on the last line

Angelo · Accepted Answer · 2018-07-24 12:49:56Z

0

When you run

LOG = logging.getLogger('log_filename')

for the first time a global variable is created. Hence, you could also add the following code to the script above:

global LOG
if LOG is not None:
    print("found logger !")
else:
    ("no global variable logger found")

answered Jul 24, 2018 at 12:49

Angelo

1,8335 gold badges27 silver badges62 bronze badges

Comments

loop · Accepted Answer · 2024-03-17 20:21:34Z

0

You might want to use absolute path.

logging.FileHandler(filename="/home/user/PycharmProjects/project/logs/log_file.log")

answered Mar 17, 2024 at 20:21

loop

517 bronze badges

Comments

Victor Badenas · Accepted Answer · 2024-07-02 16:01:25Z

0

Based on @KhoPhi and @loop 's answers, when using a logging config dict you might want to use a class as a handler that you can use in the config. For that you can use the following:

from pathlib import Path
from logging.handlers import RotatingFileHandler as _RotatingFileHandler

class RotatingFileHandler(_RotatingFileHandler):
    def __init__(self, filename: str, *args, **kwargs):
        Path(filename).absolute().parent.mkdir(exist_ok=True, parents=True)
        super().__init__(filename, *args, **kwargs)

you can then use all the examples above with your own handler:

RotatingFileHandler("logs/my_file.log")

or use it in the config file as:

...
handlers:
  my_handler:
    class: your_module.your_submodule.RotatingFileHandler
    filename: logs/my_file.log
...

edited Jul 2, 2024 at 16:01

answered Jul 2, 2024 at 15:44

Victor Badenas

11 bronze badge

Collectives™ on Stack Overflow

Python logging: create log if not exists or open and continue logging if it does

5 Answers 5

4 Comments

1 Comment

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

4 Comments

1 Comment

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related