python will store all strings in athene_pool and will never release them,so bigtext prcocessing bothered me recently. I save article contents to redis,and fetch them from redisclient,and send them to http response.while serialization may will cause too much memory,is there any good idea to resolve things like this? I wonder if django generated too many responses,how it releases them to keep memory low?
1 Answer
Python does not pool all strings, see Are strings pooled in Python.
For example, this 10K string is not pooled:
>>> a = '5' * 10000
>>> b = '5' * 10000
>>> a is b
False
But a short 5 character string is pooled:
>>> a = '5' * 5
>>> b = '5' * 5
>>> a is b
True
Ergo: you don't need to worry about long strings being kept in memory longer than necessary.
2 Comments
Ondergetekende
About the 'is' test: the python 'is' operator tests if two objects are really the same object. In this case I'm abusing the operator to check if two strings got merged into one string.
Even short strings are only pooled if they occur as string literals in the source code (or are explicitly interned via some function
sys IIRC). The compiler constant-folds the 5 * 5` in your snippet into '55555' as you'll see if you put it into a function and run dis.dis over it.