4

I have a Python script that accepts one input (a text file): ./myprog.py file.txt. The script outputs a string based on the given input.

I have a set of test files that I would like to test my program with. I know the expected output for each file and want to make sure my script produces the correct output for each file.

What is the generally accepted way to do this type of testing?

I was thinking of using Python's unittest module as the testing framework, and then running my script through subprocess.check_output(stderr=subprocess.STDOUT), capturing stdout and stderr, and then doing a unittest assertEqual to compare the actual and expected strings. I want to make sure I'm not missing some nicer solution.

3
  • Unit testing is usually done on individual functions or classes (and doesn't usually involve writing output to files or running things with subprocess). I can't recommend anything else but to look at a python unit testing tutorial, and that would probably render this question off-topic. Commented Jan 23, 2017 at 0:18
  • 1
    Its not technically a unit test because it changes things on the hard drive, but its okay to do it anyway. Consider refactoring your code by moving the functionality inside a function that takes a file-like object for input and output. The regular code would pass in an open file handle and sys.stdout. But your unit test code would use StringIO buffers. There are differences when writing to stringio instead of a pipe attached to a terminal so this is not always the best approach. You'll have to decide. Commented Jan 23, 2017 at 0:23
  • you can always run ./myprog.py file.txt > result.txt and later use tools to compare files - ie. diff result.txt expected.txt - but it is not so usefull as test - even if you put all in bash/batch script. Commented Jan 23, 2017 at 0:30

2 Answers 2

2

There's two problems here. Testing a program, as opposed to a library of functions, and testing something that prints, as opposed to values returned from a function. Both make testing more difficult, so it's best to side step these problems as much as possible.

The usual technique is to create a library of functions and then have your program be a thin wrapper around that. These functions return their results, and only the program does the printing. This means you can use normal unit testing techniques for most of the code.

You can have a single file which is both a library and a program. Here's a simple example as hello.py.

def hello(greeting, place):
    return greeting + ", " + place + "!"

def main():
    print(hello("Hello", "World"))

if __name__ == '__main__':
    main()

That last bit is how a file can tell if it was run as a program or if it was imported as a library. It allows access to the individual functions with import hello, and it also allows the file to run as a program. See this answer for more information.

Then you can write a mostly normal unit test.

import hello
import unittest
import sys
from StringIO import StringIO
import subprocess

class TestHello(unittest.TestCase):
    def test_hello(self):
        self.assertEqual(
             hello.hello("foo", "bar"),
            "foo, bar!"
        )

    def test_main(self):
        saved_stdout = sys.stdout
        try:
            out = StringIO()
            sys.stdout = out
            hello.main()
            output = out.getvalue()
            self.assertEqual(output, "Hello, World!\n")
        finally:
            sys.stdout = saved_stdout

    def test_as_program(self):
        self.assertEqual(
            subprocess.check_output(["python", "hello.py"]),
            "Hello, World!\n"
        )

if __name__ == '__main__':
    unittest.main()

Here test_hello is unit testing hello directly as a function; and in a more complicated program there would be more functions to test. We also have test_main to unit test main using StringIO to capture its output. Finally, we ensure the program will run as a program with test_as_program.

The important thing is to test as much of the functionality as functions returning data, and to test as little as possible as printed and formatted strings, and almost nothing via running the program itself. By the time we're actually testing the program, all we need to do is check that it calls main.

Sign up to request clarification or add additional context in comments.

4 Comments

Great explanation. I think the module should be named example.py as we are importing it as import example
@KshitijSaraogi Just an oversight. I renamed it from example.py to hello.py while I was working on the code and didn't recopy the tests.
That's good to know. I learnt some new things about unit testing through your example.
Thanks for taking the time to illustrate different ways to test.
0
public class CriminalRecordApp {
    
    private String criminalName;
    private String criminalID;
    private String crimeDetails;
    private String criminalImage;
    
    public void editData(String name, String ID, String details) {
        criminalName = name;
        criminalID = ID;
        crimeDetails = details;
    }
    
    public void sendWhatsAppMessage(String phoneNumber) {
        // code to send WhatsApp message to given phone number
    }
    
    public void generatePDFReport() {
        // code to generate a PDF report with all data and image in A4 size jpg format
    }
}

1 Comment

As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.