A module to profile peak memory usage of Python code

Question

Currently, I tried to use the memory_profiler module to get the used memory like the following code:

from memory_profiler import memory_usage
memories=[]
def get_memory(mem,ended):
  if ended:
    highest_mem=max(mem)
    print highest_mem
  else:
  memories.append(mem)

def f1():
  #do something
  ended=False
  get_memory(memory_usage(),ended)
  return #something
def f2():
  #do something
  ended=False
  get_memory(memory_usage(),ended)
  return #something

#main
f1()
f2()
ended=True
get_memory(memory_usage(),ended) #code end

>>>#output
# output 
# highest memory

however, it did not successfully execute. It got stuck when ended=True and sent the value of memory_usage() and ended to the function of get_memory. It did not show any error as well., just waiting for long long time, then I force to stop executing. Anyone knows the better way or the solution?

Also is there some reason you're not just using the decorators or the default profiler report? The use of the lower level api just seems to complicate things in your case and you're not really doing anything with it. — pvg
– pvg, Commented Dec 22, 2016 at 6:31
I would change the title of the question, since what is "the best" is primarly opinion based, which is a reason to close question. — abukaj
– abukaj, Commented Dec 22, 2016 at 6:55

Nathan Thompson · Accepted Answer · 2022-04-11 23:09:18Z

9

An easy way to use memory_usage to get the peak / maximum memory from a block of code is to first put that code in a function, and then pass that function - without the () call - to memory_usage() as the proc argument:

from memory_profiler import memory_usage

def myfunc():
  # code
  return
    
mem = max(memory_usage(proc=myfunc))

print("Maximum memory used: {} MiB".format(mem))

Other arguments allow you to collect timestamps, return values, pass arguments to myfunc, etc. The docstring seems to be the only complete source for documentation on this: https://github.com/fabianp/memory_profiler/blob/master/memory_profiler.py

https://github.com/fabianp/memory_profiler/blob/4089e3ed4d5c4197925a2df8393d4cbfca745ae5/memory_profiler.py#L244

edited Apr 11, 2022 at 23:09

answered Feb 14, 2017 at 23:24

Nathan Thompson

3341 silver badge10 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Mike JS Choi · Accepted Answer · 2016-12-22 06:49:16Z

0

I mainly use Heapy because it's really easy to use.

Just type the following code where you want to test for memory usage.

from guppy import hpy
hp = hpy()
print hp.heap()

answered Dec 22, 2016 at 6:49

Mike JS Choi

1,17110 silver badges19 bronze badges

Collectives™ on Stack Overflow

A module to profile peak memory usage of Python code

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related