I have an application which keeps appending data for a wide range of sessionIDs and expiring them in a few hours, currently we use Redis List data structure and having the SessionID as the key.
For each sessionID, we have a set of attributes such as IP_address, user_agent, timestamp, etc. that we want to keep track of as a event sequence. For example, for sessionID_1, we would have a list of strings each representing the attributes for that particular visit: "["1.0.0.0", "Mozilla/5.0", "1234567890"]", "["2.0.0.0", "Mozilla/3.0", "2134567890"]", etc.
We are not sure if storing them as plain strings are the most efficient way and are looking for potential optimizations to reduce the storage/memory requirement as much as possible.
We explored data compression with Snappy, Deflater, GZIP (+ base64 encoding to stored them as strings required by Redis) which showed some marginal improvement due to limited repeated values in each string representation of the visit attributes. I am asking here for any idea that we could try to cut down the storage size, either by modifying how our application works or by leveraging Redis features if any. Appreciate your time.
