19

I'd like to execute multiple commands in a standalone application launched from a python script, using pipes. The only way I could reliably pass the commands to the stdin of the program was using Popen.communicate but it closes the program after the command gets executed. If I use Popen.stdin.write than the command executes only 1 time out of 5 or so, it does not work reliable. What am I doing wrong?

To elaborate a bit :

I have an application that listens to stdin for commands and executes them line by line. I'd like to be able to run the application and pass various commands to it, based on the users interaction with a GUI. This is a simple test example:

import os, string
from subprocess import Popen, PIPE

command = "anApplication" 
process = Popen(command, shell=False, stderr=None, stdin=PIPE)

process.stdin.write("doSomething1\n")
process.stdin.flush()
process.stdin.write("doSomething2\n")
process.stdin.flush()

I'd expect to see the result of both commands but I don't get any response. (If I execute one of the Popen.write lines multiple times it occasionally works.)

And if I execute:

process.communicate("doSomething1")

it works perfectly but the application terminates.

Improve this question
9
  • 5
    Please include a code snippet. What do you mean by "multiple commands"? Commented Jan 29, 2010 at 11:37
  • 2
    You question is not clear at all. What are 'multiple commands'? To whose stdin are you trying to pass 'commands'?. As Adam mentioned, please include a code snippet. Commented Jan 29, 2010 at 11:45
  • Sorry guys, I've made a few changes to the post to make it clear what I'd like to do. Commented Jan 29, 2010 at 14:12
  • Are you sure that the arguments you're giving to Popen are stderr=None, stdin=PIPE? Might you be giving to Popen for stdin, stderr, or stdout be a bit different from what you've put here? Commented Jan 29, 2010 at 15:14
  • 1
    Does your application prints something to stdout? If so, you can try to use pexpect Commented Apr 8, 2015 at 20:41

5 Answers 5

Reset to default
1

If I understand your problem correctly, you want to interact (i.e. send commands and read the responses) with a console application.

If so, you may want to check an Expect-like library, like pexpect for Python: http://pexpect.sourceforge.net

It will make your life easier, because it will take care of synchronization, the problem that ddaa also describes. See also: http://www.noah.org/wiki/Pexpect#Q:_Why_not_just_use_a_pipe_.28popen.28.29.29.3F

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

0

The real issue here is whether the application is buffering its output, and if it is whether there's anything you can do to stop it. Presumably when the user generates a command and clicks a button on your GUI you want to see the output from that command before you require the user to enter the next.

Unfortunately there's nothing you can do on the client side of subprocess.Popen to ensure that when you have passed the application a command the application is making sure that all output is flushed to the final destination. You can call flush() all you like, but if it doesn't do the same, and you can't make it, then you are doomed to looking for workarounds.

Improve this answer

1 Comment

subprocess' stdout is not redirected in this case. Therefore the buffering is the same as if the app is run from the command line (line-buffered if it is a stdio-based program). Buffering of stdin on the parent side could be controlled using bufsize (on Python 2) or explicit process.stdin.flush(). Unless there is a bug (such as the read-ahead bug in Python 2 that could be workarounded using an explicit .readline() call); the child process should be able to receive the command immediately.
0

Your code in the question should work as is. If it doesn't then either your actual code is different (e.g., you might use stdout=PIPE that may change the child buffering behavior) or it might indicate a bug in the child application itself such as the read-ahead bug in Python 2 i.e., your input is sent correctly by the parent process but it is stuck in the child's internal input buffer.

The following works on my Ubuntu machine:

#!/usr/bin/env python
import time
from subprocess import Popen, PIPE

LINE_BUFFERED = 1

#NOTE: the first argument is a list
p = Popen(['cat'], bufsize=LINE_BUFFERED, stdin=PIPE,
          universal_newlines=True)
with p.stdin:
    for cmd in ["doSomething1\n", "doSomethingElse\n"]:
        time.sleep(1) # a delay to see that the commands appear one by one
        p.stdin.write(cmd)
        p.stdin.flush() # use explicit flush() to workaround
                        #   buffering bugs on some Python versions
rc = p.wait()
Improve this answer

Comments

-1

It sounds like your application is treating input from a pipe in a strange way. This means it won't get all of the commands you send until you close the pipe.

So the approach I would suggest is just to do this:

process.stdin.write("command1\n")
process.stdin.write("command2\n")
process.stdin.write("command3\n")
process.stdin.close()

It doesn't sound like your Python program is reading output from the application, so it shouldn't matter if you send the commands all at once like that.

Improve this answer

2 Comments

The problem with this solution, is that it executes all three commands at once, and not one by one, based on the users interaction with a GUI. Simply executing the commands work well with Popen.communicate but I'd like to be able to run them separately.
@sz, I suspect this is the application's problem. And I suspect that in order to fix it you'd have to go to the trouble of opening a pseudo-terminal, which is a type of virtual device in Unix/Linux. It allows one program to pretend to be a terminal for another program. If you are on Unix/Linux, you can test to make sure it's the application's problem and not Python's be having the application be "cat -u" and seeing if the lines come out when Python writes them.
-1

Must define stdout and use communicate() to execute

import os, string
from subprocess import Popen, PIPE

command = "anApplication" 
process = Popen(command, stdout=PIPE, stderr=PIPE, stdin=PIPE, shell=True)

nl = b'\x0a'

process.stdin.write(b'def sayHi():' + nl) 
process.stdin.write(b'  a = "Hello"' + nl) 
process.stdin.write(b'  print(a)' + nl)

process.communicate(b'sayHi()' + nl)

#Expected result:

Hello
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.