Redis-python add multiple values of type set to a redis hash

Question

Redis n00b here so don't shoot!

I need to store a collection of sets in redis indexed by key - I can:

import redis
r = redis.Redis()

r.sadd("A:B:123", *{456, 789})
r.sadd("A:B:124", *{666, 777})

but then if I want the collection "A:B" I have to collect the keys manually as in:

{k.rsplit(b':', 1)[-1]: r.smembers(k) for k in r.scan_iter("A:B:*") }
# {b'124': {b'666', b'777'}, b'123': {b'456', b'789'}}

This seems awfully slow (note also the rsplit)

I have been trying to use hmset to do the trick:

r.hmset("A:B", mapping={123: 'X', 124: 'Z'})

but I can't find a way of substituting my sets for 'X' and 'Z'.

(note that ideally set elements should be of type int as passed in - note also that those collections are meant to be read only, so I want to optimize lookup not insertion time)

sonus21 · Accepted Answer · 2021-01-20 19:17:07Z

2

import redis
r = redis.Redis()

r.sadd("A:B:123", *{456, 789})
r.sadd("A:B:124", *{666, 777})

New code

import redis
r = redis.Redis()

r. hmset("A:B", {"123": ",".join(map(str, {456, 789}))})
r. hmset("A:B", {"124": ",".join(map(str, {666, 777}))})

Print All Elements

print(dict([(k,set(map(int, v.split(b",")))) for k,v in r.hgetall("A:B").items()]))

Use a single map to store related keys and in a given key store concatenated values as a string.

NOTE: Adding element(s) to set is not atomic since a procedure has to read first then perform deserialization and store it back unless LUA script is used.

Using SUNION

r.sadd("A:B:123", *{456, 789})
r.sadd( "A:B:members", "A:B:123")
r.sadd("A:B:124", *{666, 777})
r.sadd( "A:B:members", "A:B:124")

Print Method

r.sunion(r.smembers("A:B:members"))

edited Jan 20, 2021 at 19:17

answered Jan 20, 2021 at 18:48

sonus21

5,4082 gold badges27 silver badges52 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Mr_and_Mrs_D Over a year ago

Thanks! but is there no way of retrieving the whole dict at once? The sets are big and reconstructing the dict on every query is not an option. I knew I could store strings but I what I want is to get the sets back as stored - not do string operations that are super slow for my needs

sonus21 Over a year ago

Which version of Redis you have?

Mr_and_Mrs_D Over a year ago

Using github.com/andymccurdy/redis-py 3.5.1

sonus21 Over a year ago

see the edited answer, you can use SUNION if you're using Redis 6.2+

Mr_and_Mrs_D Over a year ago

Yes thanks tried - what this essentially adds is storing the keys itself in a set so I avoid scanning for them - so the code in OP becomes dict((k, r.smembers(k)) for k in r.smembers("A:B:members")) -> {b'A:B:123': {b'789', b'456'}, b'A:B:124': {b'666', b'777'}} - I still have to manually split the keys and reconstruct the collection

|

Collectives™ on Stack Overflow

Redis-python add multiple values of type set to a redis hash

1 Answer 1

7 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

7 Comments

Your Answer

Sign up or log in

Post as a guest

Related