0

In a main.py file I have following code:

import time

from v1 import log, ServiceLogger
from logging.handlers import MemoryHandler


class Service:
    h = MemoryHandler(1000)
    id = None
    log = None

    def __init__(self, id: int):
        self.id = id
        self.log = ServiceLogger(log)
        log.addHandler(self.h)

    def do(self):
        print("Do some job per service!")


def service_exec():
    service = Service(id=4)
    service.do()


if __name__ == '__main__':
    while True:
        for i in range(10):
            service_exec()
            time.sleep(1)

And then v1.py has following content:

import logging

log = logging.getLogger('.'.join(__name__.split('.')[:-1]))


class ServiceLogger(logging.LoggerAdapter):
    def log(self, level, msg, *args, **kwargs):
        print("Custom logger hello!")

The problem is that over a time (~15-30 mins) I noticed that my computer memory increases over a period, and continue to grow forever with around 15-30 mins intervals. I used top -p <PID> to track the "VIRT RES SHR" resources, in particular that "RES". However, if I remove this log.addHandler(self.h) code snippet, then memory doesn't grow (so I left computer for couple hours and no grow has been noticed). Could somebody hint me what kind of problem could this be and how to solve it?

UPD: moving logging handler into global variable space, doesn't solve an issue, memory still grows:

h = MemoryHandler(1000)
log.addHandler(h)

class Service:
    id = None
    log = None

    def __init__(self, id: int):
        self.id = id
        self.log = ServiceLogger(log)
8
  • You’re adding a new handler every time Service is constructed and never removing it. This causes the logger’s handlers list to grow indefinitely, causing the memory leak Commented Aug 11 at 11:04
  • @aran there is a tiny problem, when I move these h = MemoryHandler(1000) and log.addHandler(self.h) into the global variable space, i.e. declare them before class Service declaration, then memory still continue to grow, how about that? Commented Aug 11 at 11:33
  • @aran and no, handlers are not indefinitely added, since "h" is initialized once Commented Aug 11 at 11:41
  • How much does it increase? Commented Aug 11 at 11:48
  • @DenisProt handlers are added each time a service is created. Commented Aug 11 at 11:49

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.