I have many hundreds of python functions I'm evaluating for memory usage, and many of them differ simply by the usage of intermediate variable assignments. I would like a way, if possible, to automatically refactor these functions to remove such intermediate variable assignments. This way, the memory differences between the functions would be due to algorithm logic instead of just additional variable assignments.
For example, consider the two functions
def se1(vals1, vals2):
diff = [(v1 - v2) for v1, v2 in zip(vals1, vals2)]
diff_sq = [d**2 for d in diff]
return(sum(diff_sq))
def se2(vals1, vals2):
return(sum([(x-y)**2 for x,y in zip(vals1, vals2)]))
When I measure memory usage for them using tracemalloc:
import numpy as np
import tracemalloc
listA = np.random.randint(0, 1000, 1000)
listB = np.random.randint(0, 1000, 1000)
tracemalloc.start()
se1(listA, listB)
current, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()
tracemalloc.reset_peak()
print("Peak memory usage of se1: ", peak)
tracemalloc.start()
se2(listA, listB)
current, peak = tracemalloc.get_traced_memory()
tracemalloc.stop()
tracemalloc.reset_peak()
print("Peak memory usage of se2: ", peak)
the usages are vastly different:
Peak memory usage of se1: 66722
Peak memory usage of se2: 34229
Even though they are producing the same answer using the same logic. I feel like there should be a way to remove intermediate variables using successive substitution within the function body, but I can't think of a general and streamlined way to do it.
dis. I'd suggest seeing if you can find and fix the optimizations using that. Alternately, it is possible thatpypydoes this kind of optimization for you. Unfortunately its debugging tools aren't as rich. But if you simply want speed...disand while it helps to do an initial partitioning of the functions, I’m not clear how to use it to automatically rewrite functions. Preferably I would want something like an operator that takes in the source code of a function and returns it with all intermediate values collapsed.