redis: set if exist - via python

Question

I've got a huge list of keys and one value (the same for all keys) that should be written to a redis instance. This should only be happen, if an entry with this key already exists.

There is the MSETNX command which is doing the opposite: putting in the entry, when the key does NOT exist.

Sure, i can just check before via EXIST and set it depending on the result, but how do i ensure that this atomic. And as this list can be really huge, how do i do this the pipeline-based?

i'am using redis-py as wrapper and there's no Lua-Scripting support available on the server.

Use SETNX to create locks. This will make all your operations atomic. — freakish
– freakish, Commented Aug 6, 2012 at 13:21

Didier Spezia · Accepted Answer · 2012-08-06 14:09:51Z

4

This is not extremely convenient to do without Lua, but it can still be done using WATCH/MULTI/EXEC commands. The idea is to partition your huge list of keys in blocks of n items (n=20 for instance), and then apply the following function to each block:

def process_block( redis_client, block, value ):
    with redis_client.pipeline() as pipe:
        while True:
            try:
                pipe.watch( block )
                to_set = [ for x in block if pipe.exists(x) ]
                if to_set:
                    pipe.multi()
                    for x in to_set:
                        pipe.set( x, value )
                    pipe.execute()
                break
            except redis.WatchError:
                continue
            finally:
                pipe.reset()

Atomicity is guaranteed by the WATCH/MULTI/EXEC commands, and you will pipeline 1 WATCH, 1 MULTI, up to n SET, and 1 EXEC commands. You will pay for the EXISTS roundtrips though. Choosing the correct value for n is a tradeoff between the pipelining factor and the risk of concurrent interactions. It should not be set too high.

edited Aug 6, 2012 at 14:09

answered Aug 6, 2012 at 13:55

Didier Spezia

73.6k12 gold badges194 silver badges157 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

kraiz Over a year ago

not what i expected but as long as i don't want to update, i'll give it a try. thx!

Collectives™ on Stack Overflow

redis: set if exist - via python

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related