Redis, only allow operation on existing keys

Question

I am using the python package (redis-py) to operate the redis database. I have a bunch of clients that set keys and values of a hash in redis. I want they set keys and values only when the hash exists. If the hash doesn't exist, setting keys and values will create the hash, which is not what I want to do.

In the redis-py page (https://github.com/andymccurdy/redis-py), the author suggested a way to do atomic operation on client side. So I wrote a similar function:

    with r.pipeline() as pipe:
        while True:
            try:
                pipe.watch("a_hash")
                if pipe.exists("a_hash"):
                    pipe.hset("a_hash", "key", "value")                  
                break
            except redis.WatchError:
                continue
            finally:
                pipe.reset()

However, this seems does not work. After I delete the hash from another client, this hash still gets created by this piece of code, so I guess this piece of code is not atomic operation. Could someone help me to identify what's the problem with this code? Or is there a better to achieve this purpose?

Appreciate your help!

Didier Spezia · Accepted Answer · 2012-06-11 22:27:50Z

5

I would suggest to read the definition of a WATCH/MULTI/EXEC block as explained in the Redis documentation.

In such block, only the commands between MULTI and EXEC are actually processed atomically (and conditionally, with an all-or-nothing semantic depending on the watch).

In your example, the EXISTS and HSET commands are not executed atomically. Actually, you don't need this atomicity: what you want is the conditional execution.

This should work better:

with r.pipeline() as pipe:
    while True:
        try:
            pipe.watch("a_hash")
            if pipe.exists("a_hash"):
                pipe.multi()
                pipe.hset("a_hash", "key", "value")
                pipe.execute()
            break
        except redis.WatchError:
            continue
        finally:
            pipe.reset()

If the key is deleted after the EXISTS but before the MULTI, the HSET will not be executed, thanks to the watch.

With Redis 2.6, a Lua server-side script is probably easier to write, and more efficient.

edited Jun 11, 2012 at 22:27

answered Jun 11, 2012 at 22:21

Didier Spezia

73.6k12 gold badges194 silver badges157 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

qkhhly Over a year ago

Thanks Didier. One more question, what if the key is deleted after multi? Since redis is queueing commands after multi, is hset still executed in this situation?

Didier Spezia Over a year ago

If the key is deleted between the MULTI and HSET or even between the HSET and EXEC, then the MULTI/EXEC block is aborted, and the HSET not applied.

Joe Jasinski Over a year ago

Is the reset() needed in this example? The docs say: "If the Pipeline is used as a context manager reset() will be called automatically."

Didier Spezia Over a year ago

It will be called automatically - I still prefer to make it explicit.

Collectives™ on Stack Overflow

Redis, only allow operation on existing keys

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related