I am trying to get the memory usage of an external program within my python script. I have tried using the script http://code.activestate.com/recipes/286222/ as follows:
m0 = memory()
subprocess.call('My program')
m1 = memory(m0)
print m1
But this seems to be just giving me the memory usage of the python script rather than 'My program'. Is there a way of outputting the memory usage of the program for use within the python script?
Try using Psutil
import psutil
import subprocess
import time
SLICE_IN_SECONDS = 1
p = subprocess.Popen('calling/your/program')
resultTable = []
while p.poll() == None:
resultTable.append(psutil.get_memory_info(p.pid))
time.sleep(SLICE_IN_SECONDS)
while p.poll() == None: would be better. Popen.poll() returns Popen.returncode which is None until the process terminated.If you look at the recipe you will see the line:
_proc_status = '/proc/%d/status' % os.getpid()
I suggest you replace the os.getpid() with the process id of your child process. As @Neal said, as I was typing this you need to use Popen and get the pid attribute of the returned object.
However, you have a possible race condition because you don't know at what state the child process is at, and the memory usage will vary anyway.
psutil is possibly a better solution - I was trying to reuse your code. You might be better monitoring it internally in the child program itself using a profiler or something like getrusage (although that can change the thing you are trying to measure). Did you say which language the child is written in?You may want to check out the psutil module: http://code.google.com/p/psutil/. The Process Management section on the homepage gives you examples of getting memory usage for a running process specified by the pid.
Do you want to spawn the process you are monitoring in your script as well? If so, you probably don't want to use subprocess.call as this will wait for the program to exit and you won't be able to monitor it while it's running. If you want to spawn the process then monitor it, you probably want to use Popen http://docs.python.org/library/subprocess.html#subprocess.Popen. This will allow you to spawn the process, get the pid, hand the pid to psutil, then monitor the memory usage.
I know this is an older post, but it's the only one that appears when I google this issue, so, I want to add the updated version of this:
import psutil
import humanfriendly
proc = subprocess.Popen("...Your process...")
SLICE_IN_SECONDS = 1
while proc.poll() is None:
p = psutil.Process(proc.pid)
mem_status = "RSS {}, VMS: {}".format(humanfriendly.format_size(p.memory_info().rss),
humanfriendly.format_size(p.memory_info().vms))
time.sleep(SLICE_IN_SECONDS)
print(mem_status)
I used humanfriendly here, to make the values more readable, but it's not required.
The RSS and VMS values are on all os, and there may be other values depending on the os you're using: https://psutil.readthedocs.io/en/latest/#psutil.Process.memory_info
call()waits for the process to exit. Thus, you cannot measure it's memory usage from a python script that usescall().