Testing Python Click Command Exceptions

Question

I am trying to test the raising of exceptions by a command implemented with the Click package.

This is my command:

@click.option(
    '--bucket_name',
...)
@click.option(
    '--group_id',
...)
@click.option(
    '--artifact_id',
...)
@click.option(
    '--version',
...)
@click.option(
    '--artifact_dir',
    required=False,
    default='downloads/artifacts/',
...)
@click.command()
def download_artifacts(
    bucket_name,
    group_id, artifact_id, version,
    artifact_dir
):
    logger.info(
        f"bucket_name: {bucket_name}, "
        f"group_id: {group_id}, "
        f"artifact_id: {artifact_id}, "
        f"version: {version}, "
        f"artifact_dir: {artifact_dir}, "
        )

    if not artifact_dir.endswith('/'):
        raise ValueError(
            "Enter artifact_dir ending with '/' ! artifact_dir: "
            f"{artifact_dir}")
...

This is my test code with assertRaises that doesn't work:

def test_download_artifacts_invalid_dir(
        self,
    ):
        runner = CliRunner()
        with self.assertRaises(ValueError):
            result = runner.invoke(
                download_artifacts,
                '--bucket_name my_bucket \
                --group_id gi \
                --artifact_id ai \
                --version 1.0.0 \
                --artifact_dir artifact_dir'.split(),
                input='5')

The assert fails and it gives E AssertionError: ValueError not raised instead.

I have found this way of testing, which passes, but it doesn't seem very elegant:

def test_download_artifacts_invalid_dir(
        self,
    ):
        runner = CliRunner()
        result = runner.invoke(
            download_artifacts,
            '--bucket_name my_bucket \
            --group_id gi \
            --artifact_id ai \
            --version 1.0.0 \
            --artifact_dir artifact_dir'.split(),
            input='5')
        print(f"result.exception: {result.exception}")
        assert "Enter artifact_dir ending" in str(result.exception)

Stephen Rauch · Accepted Answer · 2020-12-21 14:42:51Z

10

Two ways to test for exceptions with click.CliRunner()

The first method is hinted out in the DOCS:

Basic Testing

The basic functionality for testing Click applications is the CliRunner which can invoke commands as command line scripts. The CliRunner.invoke() method runs the command line script in isolation and captures the output as both bytes and binary data.

The return value is a [Result] object, which has the captured output data, exit code, and optional exception attached.

result = runner.invoke(throw_value_error)
assert isinstance(result.exception, ValueError)

The second method is to set the catch_exceptions=False parameter on CliRunner.invoke()

runner.invoke(..., catch_exceptions=False)

Test Code

import click.testing
import pytest

@click.command()
def throw_value_error():
    raise ValueError("This is My Message!")

def test_catch_value_error():
    """Read the CliRunner exception report"""
    runner = click.testing.CliRunner()
    result = runner.invoke(throw_value_error)
    assert isinstance(result.exception, ValueError)
    assert 'My Message' in str(result.exception)

def test_throw_value_error():
    """Have the CliRunner not catch my exception"""
    runner = click.testing.CliRunner()
    with pytest.raises(ValueError):
        runner.invoke(throw_value_error, catch_exceptions=False)

Test Results

============================= test session starts ==============================
platform linux -- Python 3.7.7, pytest-6.2.1 -- /usr/bin/python
collecting ... collected 2 item

tests/test_api_authz.py::test_catch_value_error PASSED                   [ 50%]
tests/test_api_authz.py::test_throw_value_error PASSED                   [100%]

============================== 2 passed in 0.05s ===============================

edited Dec 21, 2020 at 14:42

answered Dec 21, 2020 at 14:33

Stephen Rauch♦

50.1k32 gold badges118 silver badges143 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

yuletide Over a year ago

I'm trying to replicate this using built in click exceptions but pytest only sees it as a SystemError<2>. Any ideas? raise click.UsageError("End datetime must be after start datetime") assert isinstance(result.exception, click.exceptions.UsageError) ``` E AssertionError: assert False E + where False = isinstance(SystemExit(2), <class 'click.exceptions.UsageError'>) E + where SystemExit(2) = <Result SystemExit(2)>.exception ```

Stephen Rauch Over a year ago

@yuletide All click exceptions get translated to SystemExit unless you use standalone_mode. One example: stackoverflow.com/q/34286165/7311767

Collectives™ on Stack Overflow

Testing Python Click Command Exceptions

1 Answer 1

Two ways to test for exceptions with click.CliRunner()

Basic Testing

Test Code

Test Results

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Two ways to test for exceptions with click.CliRunner()

Basic Testing

Test Code

Test Results

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related