14

I know how to assert that a log message was generated, but I can't seem to figure out how to assert that a log message was not generated. Here's the unit test I have now (sanitized). Note that XYZ class takes the logger as a param, and test_check_unexpected_keys_found passes as expected.

import unittest
import logging

class TestXYZ(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        cls.test_logger = logging.getLogger('test_logger')
        cls.test_logger.addHandler(logging.NullHandler())

    def test_check_unexpected_keys_found(self):
        test_dict = {
            'unexpected': 0,
            'expected1': 1,
            'expected2': 2,
        }
        xyz = XYZ(self.test_logger)
        with self.assertLogs('test_logger', level='WARNING'):
            xyz._check_unexpected_keys(test_dict)

    def test_check_unexpected_keys_none(self):
        test_dict = {
            'expected1': 1,
            'expected2': 2,
        }
        xyz = XYZ(self.test_logger)
        xyz._check_unexpected_keys(test_dict)
        # assert that 'test_logger' was not called ??

I tried using unittest.patch like so:

with patch('TestXYZ.test_logger.warning') as mock_logwarn:
    xyz._check_unexpected_keys(test_dict)
    self.assertFalse(mock_logwarn.called)

But I get

ImportError: No module named 'TestXYZ'
I tried some variants on that as well, but got nowhere.

Anyone know how to handle this?

8
  • 1
    I'm confused - if you're injecting the logger into XYZ, why not just pass in a mock? Why do you need to patch, or create a real logger? Commented Mar 8, 2016 at 15:49
  • Yes, jonsharpe is correct. You should mock out the logger with respect to where you are testing and assert that the method is called or not called, determine whether you have certain parts of the message contained or not contained, etc...Mock is your friend here Commented Mar 8, 2016 at 15:51
  • Also, in case you didn't know. Mock is packaged in to Python 3. Commented Mar 8, 2016 at 15:52
  • @jonrsharpe Oops! Post as an answer, with code example and I'll accept that. Thanks. Commented Mar 8, 2016 at 15:56
  • @JimWood Did you ever find a solution? Commented Jul 8, 2019 at 13:53

3 Answers 3

15

A new assertNoLogs method is added in Python 3.10.
Until then, here is a workaround: add a dummy log, and then assert it is the only log.

with self.assertLogs(logger, logging.WARN) as cm:
    # We want to assert there are no warnings, but the 'assertLogs' method does not support that.
    # Therefore, we are adding a dummy warning, and then we will assert it is the only warning.
    logger.warn("Dummy warning")
    # DO STUFF

self.assertEqual(
    ["Dummy warning"],
    cm.output,
)

If you need to do this is more than once, then to avoid duplication you can do the following. Assuming you have a base class from which all your test classes inherit, override assertLogs in that class as follows:

class TestBase(TestCase):
    def assertLogs(self, logger_to_watch=None, level=None) -> 'CustomAssertLogsContext':
        """
        This method overrides the one in `unittest.case.TestCase`, and has the same behavior, except for not causing a failure when there are no log messages.
        The point is to allow asserting there are no logs.
        Get rid of this once this is resolved: https://github.com/python/cpython/pull/18067
        """
        return CustomAssertLogsContext(self, logger_to_watch, level)

class CustomAssertLogsContext(_AssertLogsContext):    
    def __exit__(self, exc_type, exc_val, exc_tb) -> Optional[bool]:
        # Fool the original exit method to think there is at least one record, to avoid causing a failure
        self.watcher.records.append("DUMMY")
        result = super().__exit__(exc_type, exc_val, exc_tb)
        self.watcher.records.pop()
        return result
Sign up to request clarification or add additional context in comments.

2 Comments

As of 2020-05-04 this PR is not get merged, if it's not merged within the next two weeks (feature freeze at 2020-05-18 with 3.9.0 beta 1) we'll have to wait for Python 3.10 for this.
I suggest asserting the python version assert sys.version_info < (3, 10). That way you can't forget about changing the function to assertNoLogs when you mention it in the log as well.
1

Building on Joe's answer, here is an implementation of an assertNoLogs(...) routine, as a mixin class, which can be used until a formal version is released in Python 3.10:

import logging
import unittest

def assertNoLogs(self, logger, level):
    """ functions as a context manager.  To be introduced in python 3.10
    """

    class AssertNoLogsContext(unittest.TestCase):
        def __init__(self, logger, level):
            self.logger = logger
            self.level = level
            self.context = self.assertLogs(logger, level)

        def __enter__(self):
            """ enter self.assertLogs as context manager, and log something
            """
            self.initial_logmsg = "sole message"
            self.cm = self.context.__enter__()
            self.logger.log(self.level, self.initial_logmsg)
            return self.cm

        def __exit__(self, exc_type, exc_val, exc_tb):
            """ cleanup logs, and then check nothing extra was logged """
            # assertLogs.__exit__ should never fail because of initial msg
            self.context.__exit__(exc_type, exc_val, exc_tb)
            if len(self.cm.output) > 1:
                """ override any exception passed to __exit__ """
                self.context._raiseFailure(
                    "logs of level {} or higher triggered on : {}"
                    .format(logging.getLevelName(self.level),
                            self.logger.name, self.cm.output[1:]))

    return AssertNoLogsContext(logger, level)

To use it, just start your test case with

class Testxxx(unittest.TestCase, AssertNoLog):
    ...

The following test case shows how it works:

import unittest
import logging
class TestAssertNL(unittest.TestCase, AssertNoLog):

    def test_assert_no_logs(self):
        """ check it works"""
        log = logging.getLogger()
        with self.assertNoLogs(log, logging.INFO):
            _a = 1
            log.debug("not an info message")

    @unittest.expectedFailure
    def test2_assert_no_logs(self):
        """ check it records failures """
        log = logging.getLogger()
        with self.assertNoLogs(log, logging.INFO):
            _a = 1
            log.info("an info message")

    def test3_assert_no_logs_exception_handling(self):
        log = logging.getLogger()
        with self.assertRaises(TypeError):
            with self.assertNoLogs(log, logging.INFO):
                raise TypeError('this is not unexpected')

    def test4_assert_no_logs_exception_handling(self):
        """ the exception gets reported as the failure.
        This matches the behaviour of assertLogs(...) """
        log = logging.getLogger()
        with self.assertRaises(AssertionError):
            with self.assertNoLogs(log, logging.INFO):
                log.info("an info message")
                raise TypeError('this is not unexpected')

1 Comment

I think the top code block is missing a class AssertNoLog: wrapping the top def into a class to be used as a mix-in.
0

A simple and slightly hacky way to check that no logging has happened at all:

with self.assertRaises(AssertionError) as ar, \
        self.assertLogs('test_logger', level='WARNING'):
    do_something_that_may_produce_logs()
self.assertEqual('no logs of level WARNING or higher triggered on test_logger', str(ar.exception))

Or a check for not logging a specific log:

with self.assertLogs('test_logger', level='WARNING'):
    do_something_that_may_produce_logs()
    self.assertNotIn(f'very specific log', ';'.join(cm.output))

Contrarily, a check that something specific has been logged:

with self.assertLogs('test_logger', level='WARNING'):
    do_something_that_may_produce_logs()
    self.assertIn(f'very specific log', ';'.join(cm.output))

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.