Multiple shell commands in python (Windows)

Question

I'm working on a windows machine and I want to set a variable in the shell and want to use it with another shell command, like:

set variable = abc
echo %variable%

I know that I could do this using os.system(com1 && com2) but I also know, that this is considered 'bad style' and it should be possible by using the subprocess module, but I don't get how. Here is what I got so far:

proc = Popen('set variable=abc', shell=True, stdin=PIPE, stdout=PIPE, stderr=STDOUT)
proc.communicate(input=b'echo %variable%)

But neither line seems to work, both commands don't get executed. Also, if I type in nonexisting commands, I don't get an error. How is the proper way to do it?

The shell started by Popen isn't waiting on its standard input; the set command runs and then the shell exits without trying to read anything from standard input. Your command in this case (from the Python script's perspective) is a single shell script that happens to consist of two shell commands, not a pair of shell commands. — chepner
– chepner, Commented Aug 22, 2018 at 15:20
I'd strongly suggest making this more explicitly about Windows in the title and tagging. You'd get a very different answer on UNIX. — Charles Duffy
– Charles Duffy, Commented Aug 22, 2018 at 15:37
...in general, though, the thing you're relying on (shell variables being set and expanded) relies on having a shell, so in the UNIX world, best practice would be to rework what you're doing entirely, to where it doesn't depend on shell variables at all. — Charles Duffy
– Charles Duffy, Commented Aug 22, 2018 at 15:38

Giacomo Alzetta · Accepted Answer · 2018-08-23 07:29:58Z

1

Popen can only execute one command or shell script. You can simply provide the whole shell script as single argument using ; to separate the different commands:

proc = Popen('set variable=abc;echo %variable%', shell=True)

Or you can actually just use a multiline string:

>>> from subprocess import call
>>> call('''echo 1
... echo 2
... ''', shell=True)
1
2
0

^{The final 0 is the return-code of the process}

The communicate method is used to write to the stdin of the process. In your case the process immediately ends after running set variable and so the call to communicate doesn't really do anything.

You could spawn a shell and then use communicate to write the commands:

>>> proc = Popen(['sh'], stdin=PIPE, stdout=PIPE, stderr=PIPE)
>>> proc.communicate('echo 1; echo 2\n')
('1\n2\n', '')

Note that communicate also closes the streams when it is done, so you cannot call it mulitple times. If you want an interactive session you hvae to write directly to proc.stdin and read from proc.stdout.

By the way: you can specify an env parameter to Popen so depending on the circumstances you may want to do this instead:

proc = Popen(['echo', '%variable%'], env={'variable': 'abc'})

Obviously this is going to use the echo executable and not shell built-in but it avoids using shell=True.

edited Aug 23, 2018 at 7:29

answered Aug 22, 2018 at 15:21

Giacomo Alzetta

2,47911 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

AracKnight Over a year ago

This doesn't work (for me). If I type: proc = Popen('echo abc;echo 123', shell=True) output=proc.communicate()[0] print(output) I get abc;echo123, so it doesn't read it as 2 lines/commands

Giacomo Alzetta Over a year ago

@AracKnight >>> from subprocess import call >>> call('echo 1; echo 2', shell=True) 1 2 It works for me. You should specify exactly what code you are using in your question. Also, are you on Windows or linux? The ; works on linux, I'm not sure whether powershell/batch files use a different separator. But then: you can also use a multiline string as a command call('echo 1\necho 2', shell=True).

AracKnight Over a year ago

Im working on Windows 10 64bit. I'm setting a password for a Database connection in the first line as an environment variable so I can call psql in the next line and connect to the database. With using call('echo abc\necho 123', shell=True) I get abc as output, so the second command is still not working.

Giacomo Alzetta Over a year ago

@AracKnight If that is your use case I believe you should do what I describe in the end: provide the env variable by the env parameter instead of hacking using the shell.

AracKnight Over a year ago

When I specify an env parameter to Popen I can't call psql anymore without knowing the exact path of the psql.exe, since I usually run it from the PATH Variable, wich gets ignored by Popen if its passed an env. Also however I might be able to work around my current problem, I'd really like to know how such purposes are achieved in general, since I might run into similar problems in the future.

|

Collectives™ on Stack Overflow

Multiple shell commands in python (Windows)

1 Answer 1

6 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related