27 questions
2
votes
0
answers
67
views
How can I dynamically trace and visualize memory allocation patterns of nested Python coroutines in real-time?
I'm trying to understand how memory is used in my async Python application. I have multiple coroutines running, and I want to see how much memory each one is using, especially when they are nested or ...
- 218
1
vote
0
answers
263
views
Why does my python app use far more memory than tracemalloc suggests? [duplicate]
I trying to debug a memory leak in my python ML app. When I run it locally, the memory usage on my local machine increases gradually by 700mb. So I put a breakpoint in when memory use is around its ...
- 45
1
vote
1
answer
1k
views
Tracking Memory Usage in Python
Python code:
def my_function():
my_list = [i for i in range(1_000_000)]
return my_list
my_list = my_function()
Memory usage reported by different libraries:
memory_profiler: 82MB
tracemalloc:...
- 139
2
votes
1
answer
591
views
ResourceWarning: unclosed socket <zmq.Socket(zmq.PUSH) when Threading
So I have an issue with the import of a driver, which apparently leads to almost all threads that I open to end in a Resourcewarning. (python version 3.9.16 on Windows in Anaconda environment; nidaqmx ...
- 21
1
vote
0
answers
430
views
Can I use tracemalloc to line-by-line profile a subprocess?
I am attempting to write a generic,python script that memory profiles a target script, but I can't figure out how to get line-by-line memory stats from the script being run by subprocess.
# ...
- 11
0
votes
0
answers
431
views
sort tracemalloc statistics by size
I have this code:
def print_statistics(s: Snapshot, s2: Snapshot):
stats = s2.compare_to(s, 'lineno')
for stat in stats[:20]:
print(stat)
def some_func()
tracemalloc.start(50)
...
- 437
1
vote
1
answer
1k
views
long-running python program ram usage
I am currently working on a project where a python program is supposed to be running for several days, essentially in an endless loop until an user intervenes.
I have observed that the ram usage (as ...
- 31
0
votes
0
answers
379
views
How to trace memory allocations in Apache httpd server?
I am running apache benchmark (ab) with server [httpd2.4.52] running locally. I want to track how many memory allocations and what size allocations does the server make.
I run 'valgrind --trace-malloc=...
- 31
1
vote
0
answers
463
views
Understanding tracemalloc output for small lists
I'm running the following simple script to get some familiarity with tracemalloc:
import tracemalloc
import linecache
def display_custom(snapshot):
... # skipped for shorter code
tracemalloc....
- 43
1
vote
0
answers
1k
views
memory leak that persists after colab cell executes
I'm encountering a subtle memory leak, and unable to determine the source using tracemalloc. I run the following code in google colab, which is meant to optimize hyperparameters for a custom ppo agent....
2
votes
1
answer
2k
views
Python tracemalloc, what is size and count?
I am using the tracemalloc library to pinpoint memory concerns in my application.
This is the code that I am using.
tracemalloc.start()
snapshot = tracemalloc.take_snapshot()
...
- 149
1
vote
1
answer
983
views
Python tracemalloc's "compare_to" function delivers always "StatisticDiff" objects with len(traceback)=1
Using Python's 3.5 tracemalloc module as follows
tracemalloc.start(25) # (I also tried PYTHONTRACEMALLOC=25)
snapshot_start = tracemalloc.take_snapshot()
... # my code is running
snapshot_stop = ...
1
vote
1
answer
4k
views
ResourceWarning for a file that is unclosed but UnitTest is throwing it
I have a function like this:
def init_cars(self, directory=''):
#some_code
cars = set([line.rstrip('\n') for line in open(directory + "../myfolder/all_cars.txt")])
#some_more_code
I am ...
- 1,013
-1
votes
1
answer
453
views
Serialize to JSON string tracemalloc in Python 3.7 [closed]
I have to serialize to JSON string the result of tracemalloc.
current_mem, peak_mem = tracemalloc.get_traced_memory()
overhead = tracemalloc.get_tracemalloc_memory()
stats = tracemalloc.take_snapshot(...
- 16.3k