capture Python script output

Question

I have written very small script and try to capture output of the script. I have written multiple time similar way but never had issue. Could I have input. I think I am doing very silly mistake

numpy_temp = """
import numpy
import sys
a, b, c = numpy.polyfit(%s,%s, 2)
print a, b, c""" %(x, y)
fp_numpy = open("numpy_temp.py", "w")
fp_numpy.write(numpy_temp)
cmd = "/remote/Python-2.7.2/bin/python numpy_temp.py "
proc = subprocess.Popen(cmd, stdout = subprocess.PIPE,
            stderr = subprocess.PIPE, shell = True)
out, err = proc.communicate()
print "out", out

Maybe you are actually getting stderr response. Try print out, err and see if you have anything. — sberry
– sberry, Commented Oct 17, 2013 at 18:14

abarnert · Accepted Answer · 2013-10-17 18:21:21Z

2

You're never actually closing fp_numpy, so the script can be empty or incomplete at the time you try to run it.

It isn't guaranteed to be empty, but it's very likely. When I try this on two different *nix computers with 7 different versions of Python, it's empty every time… (The fact that, after your script finishes, the file gets closed, and therefore flushed, makes the problem harder to debug.)

The best way to fix this is to use a with statement, so it's impossible to forget to close the file:

with open("numpy_temp.py", "w") as fp_numpy:
    fp_numpy.write(numpy_temp)

But beyond that, you've got another problem. If the generated script raises an exception, it will print nothing to stdout, and will dump a traceback to stderr, which you will read and ignore. It's very hard to debug problems when you're ignoring the errors… I don't know what you're passing for x and y, but if you're, say, passing a numpy.array instead of a string that evaluates to one, you could easily get an exception and never see it. Either send stderr to stdout, or print "err", err at the end.

And finally, you really shouldn't be using a command string and shell=True here, because you end up with an extra level of indirection for no good reason, which can also make things harder to debug. Just do this:

cmd = ["/remote/Python-2.7.2/bin/python", "numpy_temp.py"]
proc = subprocess.Popen(cmd, stdout = subprocess.PIPE,
                        stderr = subprocess.PIPE)

edited Oct 17, 2013 at 18:21

answered Oct 17, 2013 at 18:15

abarnert

368k54 gold badges626 silver badges691 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

sberry Over a year ago

Indeed, the file is not closed, and is subsequently empty.

dav · Accepted Answer · 2013-10-17 18:09:54Z

0

use print out not print "out".

answered Oct 17, 2013 at 18:09

dav

1,2191 gold badge13 silver badges19 bronze badges

Comments

iChux · Accepted Answer · 2013-10-17 18:14:00Z

0

Check out this method

see.py

for line in range(6):
    print line

me@pubx:~$ python see.py > all

answered Oct 17, 2013 at 18:14

iChux

2,42627 silver badges39 bronze badges

Comments

axblount · Accepted Answer · 2013-10-17 18:15:16Z

0

I don't know why you'd want to write a script to capture the output of a script (maybe you have a good reason). this sort of task is much easier using the shell.

Create your numpy_temp.py and then pipe it's output to a file.

/remote/Python-2.7.2/bin/python numpy_temp.py > output.txt

To append the data to an existing file use

/remote/Python-2.7.2/bin/python numpy_temp.py >> output.txt

answered Oct 17, 2013 at 18:15

axblount

2,65924 silver badges28 bronze badges

Collectives™ on Stack Overflow

capture Python script output

4 Answers 4

1 Comment

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

1 Comment

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related