Using Python subprocess to call Python command with whitespace separated list

Question

I would like to call a python subprocess which should execute something in the structure of

script.py -m opt -o oFile -i file1 file2 file3 ...

I'm trying to build the command with code like:

inputfileList = "file1,file2,file3"
cmd = ['script.py', '-m opt', '-o oFile', '-i']
for file in inputfileList.split(','):
   cmd.append(file)
subprocess.call(cmd)

This does, however, not work properly as it somehow interprets it as -i 'file1 file2 file3' instead of -i file1 file2 file3. How can I get the expected behavior?

Your shell command line operates like '-o', 'oFile' -- not '-o oFile'. To get the same behavior as your current Python, the shell would need to be script.py -m opt "-o oFile" ... — Charles Duffy
– Charles Duffy, Commented Feb 28, 2020 at 14:00
BTW, are you sure it's actually interpreting the list as -i 'file1 file2 file3'? If that's your working theory about why you're getting a file not found exception, but you aren't actually checking that the working directory matches the location the input files are in... well, it's probably worth going back and testing those assumptions. — Charles Duffy
– Charles Duffy, Commented Feb 28, 2020 at 14:20

Charles Duffy · Accepted Answer · 2020-02-28 14:14:27Z

The sensible thing to do to debug here is to replace script.py with something that prints your argument list unambiguously, so you can adjust that list until it exactly matches what you want. For example, replacing it with printf '%s\n':

inputfileList = "file1,file2,file3"
cmd = ['printf', r'%s\n', 'script.py', '-m opt', '-o oFile', '-i']
for file in inputfileList.split(','):
   cmd.append(file)
subprocess.call(cmd)

...we get output of:

script.py
-m opt
-o oFile
-i
file1
file2
file3

The file1/file2/file3 list part of that output is perfectly fine. What's wrong is -m and -o, which you can see if you compare to the output of your shell command modified the same way:

printf '%s\n' script.py -m opt -o oFile -i file1 file2 file3

...which outputs:

script.py
-m
opt
-o
oFile
-i
file1
file2
file3

...thus showing the output you'd need to make your Python match to have it behave like the stated line of shell.

Side note: %s\n isn't the best format string for this, it's just the best one /usr/bin/printf is guaranteed to support. If you want to display nonprintable characters, I'd actually suggest using %q instead, being the ksh/bash equivalent of Python's %r. It takes more work to use it from Python, though:

inputfileList = "file1,file2,file3"
cmd = ['bash', '-c', r'''printf '%q\n' "$@"''', 'script.py', '-m opt', '-o oFile', '-i']
for file in inputfileList.split(','):
   cmd.append(file)
subprocess.call(cmd)

...shows the arguments your Python code is generating in the same format you'd need to put them on a shell command line to have the same effect. Thus, in this case:

-m\ opt
-o\ oFile
-i
file1
file2
file3

...showing that you'd need to escape your spaces to make your shell behave like your current Python (or move the argument values into separate strings from the arguments themselves, to make your Python behave like your current shell).

Collectives™ on Stack Overflow

Using Python subprocess to call Python command with whitespace separated list

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related