Make a Python log file only when there are errors (using logging module)

I'd like to use the "logging" module in Python to write errors to a log file. However, I want the file to only be created when there are errors. I use the following code:

import logging

f = 'test.conf'

logger = logging.getLogger("test_logger")
logger.setLevel(logging.INFO)

ch_file = logging.FileHandler("test_logger.conf")
ch_file.setLevel(logging.ERROR)

logger.addHandler(ch_file)

ch_file.close()

ch = logging.StreamHandler()
ch.setLevel(logging.INFO)

formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")

ch.setFormatter(formatter)


logger.addHandler(ch)

logger.info("info")
logger.warn("warning")
#logger.error("error")

When logger.error("error") is uncommented, I expect the file "test_logger.conf" to be made with the error in it. However, when the line is commented out, I find that the test_logger.conf file is still made and is empty. How can I make it so this file is NOT made unless there are errors to report?

Thanks.