I am using Python (Canopy) extensively for Earth science application. Because my application is memory consuming, I am trying find way to erase variable that I don't need any more in my programs, I tried to use del command to erase the variable memory, but I found that space used by Canopy is still the same. Any ideas about how to erase variable completely from the memory. thanks
2 Answers
You can't manually nuke an object from your memory in Python!
The Python Garbage Collector (GC) will automatically free up memory of objects that have no existing references any more (implementation details differ per interpreter). It's periodically checking for abandoned objects in background without your interaction.
So to get an object recycled, you have to eliminate all references to it by assigning a different value (e.g. None) to all variables that pointed to the object. You can also delete a variable name using the del statement, but as you already noticed, this only deletes the name with the reference, but not the object and its data itself. Only the GC can do that.
5 Comments
PM 2Ring
And even then, the reclaimed memory isn't returned back to the OS immediately. Like many sophisticated programs the CPython interpreter has its own internal memory management for efficiency reasons. You can read about Python's memory arenas in the link in my comment on the question, and in the docs.
Jonathan March
All the responses and comments above are correct. To sum up: deleting all Python references to an object (including implicit references in the IPython kernel), and then forcing Python garbage collector to run will make the memory eligible for re-use in the same Python process, and will often but not always make it for re-use by the OS. But even then it won't necessarily show up in OS stats.
Kernel
All, thanks a lot for your help. It is clear now that erasing variable from memory is not straight forward, I think that I can apply some technique like shown here by deleting all python reference and then calling the GC, or by creating subprocess as mention above. Yet, I am using my script on different platform, thus I am not sure whether such techniques will work effectively across different platform. I have to test it. Thanks again.
Kernel
Byte Commander, did you meant in your title that ** I cannot automatically …….**
ShadowRanger
You're overstating the benefit of triggering a garbage collection run. CPython only needs the GC to finish collecting memory when there are reference cycles involved. If no reference cycles are involved, the memory is in fact deleted deterministically when the last reference is released, with or without a run of the cyclic garbage collector.
Now, it is possible to do it manually. To do that, you need to do it in two steps:
- use the command del to remove the variable from the workspace
- use the comand gc.collect to run the garbage collector and clear the ram memory
A code example:
import numpy as np
import gc
a = np.array([1,2,3])
del a
gc.collect()
More info: link
2 Comments
Chris Wolf
Doesn't always work. If object reference count is not zero, it won't be garbage collected.
ShadowRanger
You're overstating the benefit of triggering a garbage collection run. CPython only needs the GC to finish collecting memory when there are reference cycles involved. If no reference cycles are involved, the memory is in fact deleted deterministically when the last reference is released, with or without a run of the cyclic garbage collector.
__import__("gc").collect().deldoes not delete objects, it removes references to them. Usesys.getrefcount()orweakrefsto see whether there's something that is keeping an object alive