How to catch the errors of a child process using Python subprocess?

Question

I have the following Python(2.7) code:

try:    
  FNULL = open(os.devnull,'w')
  subprocess.check_call(["tar", "-czvf", '/folder/archive.tar.gz', '/folder/some_other_folder'], stdout=FNULL, stderr=subprocess.STDOUT)
except Exception as e:
  print str(e)

The problem which I face is that, when there is no more space for the archive, print str(e) prints Command '['tar', '-czvf', '/folder/archive.tar.gz', '/folder/some_other_folder']' returned non-zero exit status 1, which is true, but I want to catch the real error here, that is gzip: write error: No space left on device (I got the this error when I ran the same tar comand manually). Is that possible somehow? I assume that gzip is another process within tar. Am I wrong? Please keep in mind that upgrading to Python 3 is not possible.

EDIT: I also tried to use subprocess.check_output() and print the contents of e.output but that also didn't work

In general, "exit status 1" is the only thing that's an actual unambiguous machine-readable error. stderr content can contain any diagnostic logging or human-readable informational content, not just errors -- so following the answer @ShadowRanger has written, while not wrong, can cause log messages that precede an error to be treated as if they're part of the error themselves. — Charles Duffy
– Charles Duffy, Commented Nov 10, 2020 at 16:19
...so, f/e, when you write if tar -czvf folder.archive.gz /folder; then ...; else echo "There was an error" >&2; fi in shell, it's the exit status that the shell is relying on to know that something went wrong and follow the else branch. — Charles Duffy
– Charles Duffy, Commented Nov 10, 2020 at 16:19
@CharlesDuffy: My answer only echoes stderr when the exit status is non-zero, so sure, you might see diagnostic output when an error occurs, but it'll be silent under normal (success) conditions. Definitely not relying on whether stderr is empty/non-empty. — ShadowRanger
– ShadowRanger, Commented Nov 10, 2020 at 16:22
@ShadowRanger, if you were checking empty/nonempty stderr you'd have a downvote; my position is rather one of cautious, qualified support. — Charles Duffy
– Charles Duffy, Commented Nov 10, 2020 at 16:23

ShadowRanger · Accepted Answer · 2020-11-10 16:24:39Z

Python 3 solution for sane people

On Python 3, the solution is simple, and you should be using Python 3 for new code anyway (Python 2.7 ended all support nearly a year ago):

The problem is that the program is echoing the error to stderr, so check_output doesn't capture it (either normally, or in the CalledProcessError). The best solution is to use subprocess.run (which check_call/check_output are just a thin wrapper over) and ensure you capture both stdout and stderr. The simplest approach is:

try:
    subprocess.run(["tar", "-czvf", '/folder/archive.tar.gz', '/folder/some_other_folder'],
                   check=True, stdout=subprocess.DEVNULL, stderr=subprocess.PIPE)
                             # ^ Ignores stdout           ^ Captures stderr so e.stderr is populated if needed
except CalledProcessError as e:
    print("tar exited with exit status {}:".format(e.returncode), e.stderr, file=sys.stderr)

Python 2 solution for people who like unsupported software

If you must do this on Python 2, you have to handle it all yourself by manually invoking Popen, as none of the high level functions available there will cover you (CalledProcessError didn't spawn a stderr attribute until 3.5, because no high-level API that raised it was designed to handle stderr at all):

with open(os.devnull, 'wb') as f:
    proc = subprocess.Popen(["tar", "-czvf", '/folder/archive.tar.gz', '/folder/some_other_folder'],
                 stdout=f, stderr=subprocess.PIPE)
    _, stderr = proc.communicate()
if proc.returncode != 0:
    # Assumes from __future__ import print_function at top of file
    # because Python 2 print statements are terrible
    print("tar exited with exit status {}:".format(proc.returncode), stderr, file=sys.stderr)

Collectives™ on Stack Overflow

How to catch the errors of a child process using Python subprocess?

1 Answer 1

Python 3 solution for sane people

Python 2 solution for people who like unsupported software

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Python 3 solution for sane people

Python 2 solution for people who like unsupported software

Comments

Your Answer

Sign up or log in

Post as a guest

Related