how to find number of processes running particular command in python

Question

The output of

ps uaxw | egrep 'kms' | grep -v 'grep'

yields:

user1  8148  0.0  0.0 128988  3916 pts/8    S+   18:34   0:00 kms 
user2 11782  0.7  0.3 653568 56564 pts/14   Sl+  20:29   0:01 kms

Clearly two processes running the program. I want to store this number (2 here) as a variable. Any suggestions on how to do this in python?

I tried the following:

procs = subprocess.check_output("ps uaxw | egrep 'kmns' |grep -v 'grep'",shell=True)

But i get the following (I think when the jobs are not currently running, so number of processes running the jobs is zero):

Traceback (most recent call last): File "", line 1, in File "/usr/lib64/python2.7/subprocess.py", line 573, in check_output raise CalledProcessError(retcode, cmd, output=output) subprocess.CalledProcessError: Command 'ps uaxw | egrep 'kmns' |grep -v 'grep'' returned non-zero exit status 1

How do I get around this?

Btw, here is the function I wrote to detect if my system was busy (which means if the number of cpus > total installed, and if load avg > 0.9 per cpu):

def busy():
    import subprocess
    output = subprocess.check_output("uptime", shell=False)
    words = output.split()
    sys.stderr.write("%s\n"%(output)) 
    procs = subprocess.check_output("ps uaxw | egrep '(kmns)' | grep -v 'grep'", shell=True)
    kmns_wrds = procs.split("\n")
    wrds=words[9]
    ldavg=float(wrds.strip(','))+0.8
    sys.stderr.write("%s %s\n"%(ldavg,len(kmns_wrds)))
    return max(ldavg, len(kmns_wrds)) > ncpus

The above is called by:

def wait_til_free(myseconds):
    while busy():
        import time
        import sys
        time.sleep(myseconds)
        """ sys.stderr.write("Waiting %s seconds\n"%(myseconds)) """

which basically tells the system to wait while all cpus are taken.

Any suggestions?

Many thanks!

Why are you using egrep 'kms' instead of just grep kms? Also, why are you passing flags to ps whose only purpose is to add extra information to the output when you don't care about the output? — abarnert
– abarnert, Commented May 10, 2015 at 22:09

abarnert · Accepted Answer · 2015-05-10 22:14:31Z

8

If you're going to do this all with a big shell command, just add the -c argument to grep, so it gives you a count of lines instead of the actual lines:

$ ps uaxw |grep python |grep -v grep
abarnert         1028   0.0  0.3  2529488  55252 s000  S+    9:46PM   0:02.80 /Library/Frameworks/Python.framework/Versions/3.4/Resources/Python.app/Contents/MacOS/Python /Library/Frameworks/Python.framework/Versions/3.4/bin/ipython3
abarnert         9639   0.0  0.1  2512928  19228 s002  T     3:06PM   0:00.40 /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python /usr/local/bin/ipython2
$
$ ps uaxw |grep python |grep -c -v grep
2

Of course you could make this more complicated by adding a | wc -l to the end, or by counting the lines in Python, but why?

Alternatively, why even involve the shell? You can search within Python just as easily as you can run grep—and then you don't have the problem that you've accidentally created a grep process that ps will repeat as matching your search and then need to grep -v it back out:

procs = subprocess.check_output(['ps', 'uaxw']).splitlines()
kms_procs = [proc for proc in procs if 'kms' in proc]
count = len(kms_procs)

Or, even more simply, don't ask ps to give you a whole bunch of information that you don't want and then figure out how to ignore it, just ask for the information you want:

procs = subprocess.check_output(['ps', '-a', '-c', '-ocomm=']).splitlines()
count = procs.count('kms')

Or, even more more simplierly, install psutil and don't even try to run subprocesses and parse their output:

count = sum(1 for proc in psutil.process_iter() if proc.name() == 'kms')

edited May 10, 2015 at 22:14

answered May 10, 2015 at 22:08

abarnert

368k54 gold badges626 silver badges691 bronze badges

Sign up to request clarification or add additional context in comments.

11 Comments

user3236841 Over a year ago

Thanks! What if there are multiple commands I am looking for, such as suppose I have another command called AdjR and another called BHI. etc? I guess i could put them all in the if command, but is there another way? That is why I was using egrep, etc.

abarnert Over a year ago

@user3236841: Each of these different solutions can be adapted pretty easily. For example, in the first one, you'd replace the grep kms; for the penultimate one, sum(procs.count(p) for p in ('kms', 'AdjR', 'BHI')); for the last one, … if proc.name() in ('kms', 'AdjR', 'BHI'); etc.

user3236841 Over a year ago

Thanks! I decided to stick with subprocess (is there an advantage to psutil?) so I went ahead with option 2. I modified it and it appears to work (no error). Thanks!

abarnert Over a year ago

@user3236841: Well, doing things directly in Python is generally simpler than using a subprocess and parsing its output. Plus, it's portable to any platform psutil works on (which includes Windows, old versions of Solaris, etc.) instead of depending on having a ps tool that handles the mix of old-style and new-style arguments you used, an egrep tool, etc. And you'll get better error handling if something goes wrong, because you don't have all those intermediate layers to try to parse the information out of.

abarnert Over a year ago

@user3236841: Of course the disadvantage is that you add an external dependency (not usually a problem if you're going to put your tool on PyPI or deploy it to a bunch of machines that you control, but it can be a problem in other cases obviously), and you have to learn psutil (which is pretty simple and well-documented, but if you already know ps like the back of your hand, nothing is easier to learn than what you already learned:).

|

Padraic Cunningham · Accepted Answer · 2015-05-10 22:43:48Z

1

If you want to simulate pipes you can use Popen:

p1 = Popen(["ps", "uaxw"], stdout=PIPE)
p2 = Popen(["grep", 'kms'], stdout=PIPE, stdin=p1.stdout)
p1.stdout.close()

out,_ = p2.communicate()
print(len(out.splitlines()))

Or use pgrep if it is available:

count  = check_output(["pgrep", "-c", "kms"])

You may get different output from both as pgrep only gets the executable's names but so will ps -aux vs ps -a.

edited May 10, 2015 at 22:43

answered May 10, 2015 at 22:09

Padraic Cunningham

181k30 gold badges264 silver badges327 bronze badges

5 Comments

abarnert Over a year ago

Sure, but if he wants to do this in Python instead of a big shell pipeline, he doesn't even need grep (and therefore doesn't need the extra grep -v grep at the end…), he can just search each line or the whole output of ps from within Python.

Padraic Cunningham Over a year ago

@abarnert, to be honest I did not really pay much attention to the command, I just prefer using Popen when piping but yes you are right.

abarnert Over a year ago

I don't think pgrep is guaranteed by POSIX. But it might be guaranteed by whatever OS he's using (I think Linux, FreeBSD, and OS X all have it in the core userland?), so maybe that's a good improvement anyway.

Padraic Cunningham Over a year ago

@abarnert, yes, not available on all but if it is, it will make life easier.

abarnert Over a year ago

From a quick check, it's part of FreeBSD base, the "standard" it's part of is NetBSD 1.6+, and it's based on the function from Solaris 7. And GNU includes it in core, and it's based on the function from Solaris 7. So, I think it's pretty close to available everywhere that he's likely to care about.

Collectives™ on Stack Overflow

how to find number of processes running particular command in python

2 Answers 2

11 Comments

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

11 Comments

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related