Running a Bash command on files with Python

Question

So I'm creating a script that needs to go through all the files on a server and run each of those files names followed by the command "ll" and then take the output of that command and print it to a txt file.

example:

folder/filename.txt ll

output: SoMETHINGSomethingSomethingother - This is sent to a output.txt file

folder/subfolder/filename3.txt ll

output: SoMETHINGSomethingSomethingother - This is sent to a output.txt file

This is what I have so far:

import os

with open("output.txt", "w") as a:
    for path, subdirs, files in os.walk(r'C:\Users\user\Desktop\Test_Py'):
       for filename in files:
         f = os.path.join(filename)
         m = f + ' ll'

         a.write(str(m) + os.linesep)

So what I'm trying to figure out now is how to make the file names that are printed out run with the "ll" command. So far this code will write the names of all the files in that folder and its subfolders in to my output.txt file.

Does anybody have any ideas?

check subprocess module docs.python.org/library/subprocess.html — avasal
– avasal, Commented Aug 31, 2012 at 8:33
possible duplicate of Running shell command from python and capturing the output — Shawn Chin
– Shawn Chin, Commented Aug 31, 2012 at 8:34
It's worth noting that in Perl you can write to a pipeline by opening the file (eg open(my $pipeline, "| command | command > file");), but Python doesn't allow this. — Eliot Ball
– Eliot Ball, Commented Aug 31, 2012 at 8:45
@Eliot. Python performs that function. It would be odd if Python used the same syntax as Perl don't you think? — Paddy3118
– Paddy3118, Commented Aug 31, 2012 at 10:46
@Paddy3118 yes, I'm just pointing out for general knowledge :) — Eliot Ball
– Eliot Ball, Commented Aug 31, 2012 at 11:08

Community · Accepted Answer · 2017-05-23 12:12:20Z

1

Use os.system():

import os

with open("output.txt", "w") as a:
    for path, subdirs, files in os.walk(r'C:\Users\user\Desktop\Test_Py'):
        for filename in files:
            f = os.path.join(filename)
            m = f + ' ll > output.txt'

            os.system(m)

This will send only the standard output to the output.txt file. If you want to send error messages to output.txt as well, use m = f + ' ll > output.txt 2>&1' instead.

Explanation: os.system(command_string) will execute the command command_string in your system as if you typed that command into a terminal. The > operator is standard in Windows and Linux to redirect standard output from a command into a file. The 2>&1 extra argument at the end is the only not-so-clear part: it redirects standard error to the same place where standard output is going. See more about this last part here.

edited May 23, 2017 at 12:12

CommunityBot

11 silver badge

answered Aug 31, 2012 at 8:37

HerrKaputt

2,60420 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Adilicious Over a year ago

Well it turns out I can't test this out until Thursday but I want to first say thank you for answering. I believe this will work. Also thank you for explaining it its really helped. Once I check on Thursday I will let you know if it worked.

Dr. Jan-Philip Gehrcke Over a year ago

We should replace os.system calls by their subprocess counterparts. Therefore a downvote.

HerrKaputt Over a year ago

@user1636295: subprocess is more general than os.system. However, os.system is not being deprecated, so there's no big problem in using it, even though the Python docs suggest [1] that people use subprocess. You can replace the os.system(m) line with subprocess.call(m,shell=True) [1]. I prefer system for these cases because it is less verbose. [1]docs.python.org/library/subprocess.html

consumer · Accepted Answer · 2012-08-31 08:41:52Z

0

In order to run the files with the "ll" command you can use subprocess module available in python.

Your revised code will be:-

import os
import subprocess
import shlex

with open("output.txt", "w") as a:
    for path, subdirs, files in os.walk(r'C:\Users\user\Desktop\Test_Py'):
       for filename in files:
         f = os.path.join(filename)
         m = f + ' ll'

         cmd_args = shlex.split(m)
         output = subprocess.check_output(cmd_args)
         a.write(output + os.linesep)

edited Aug 31, 2012 at 8:41

answered Aug 31, 2012 at 8:35

consumer

7093 gold badges7 silver badges19 bronze badges

Collectives™ on Stack Overflow

Running a Bash command on files with Python

2 Answers 2

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related