Redis - handling changes to data structures

Question

I have been experimenting with Redis, and I really like the scalability that it brings to the table. However, I'm wondering how to handle changes to data structures for a system that's already in production.

For example, let me say that I am collecting information about a user, and I use the user_id as a key, and dumping the other data about the user as comma separated values.

user_id: name, email, etc.

Now, say after about 100,000 records, I realise that I needed to query by email - how would I now take a snapshot of the existing data and create a new index for it?

Tom Clarkson · Accepted Answer · 2011-01-06 03:28:12Z

1

Using csv is not a great idea if you want to support changes. You need to use a serializer that handles missing/new values if everything is in one key, or you can use a redis hash, which gives you named subkeys. Either way you can add/remove fields with the only requirement being that your code knows what to do if it reads a record without the new value.

To allow lookup by email you need to add an index - basically a key (or list) for each email with the user id as the value. You will need to populate this index by getting all keys once, then making sure you update it when emails change.

You could iterate over all keys and store them with a different id, but that is probably more trouble than it is worth.

answered Jan 6, 2011 at 3:28

Tom Clarkson

16.2k2 gold badges45 silver badges52 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Colum · Accepted Answer · 2011-01-05 21:34:42Z

0

From my understanding of Redis, this would require something which Redis is not designed to do. You would have to loop though all your records (using keys *) and then change the order of the data and make a new key. I, personally, would recommend using a list instead of a comma separated string. In a list, you can reorder it from inside redis. A Redis List looks like the following:

"Colum" => [0] [email protected]
            [1] password
            [2] Something

I am building an app in which I encountered the same problem. I solved it by having a list for all the user's info, and then have a key with the user's email with a value of the user's id. So my database would something like this:

"Colum" => [0] [email protected]
            [1] password
            [2] Something
"[email protected]" => "Colum"

So I could query the ID or the Email and still get the information I needed.

Sorry that I was not able to directly answer your question. Just hope this helped.

answered Jan 5, 2011 at 21:34

Colum

3,9241 gold badge25 silver badges26 bronze badges

2 Comments

jeffreyveon Over a year ago

Yes, thanks, using lists seems to be the golden rule here. Any form of CSV would make future changes very difficult.

Colum Over a year ago

Glad I could help. I found this out the hard way as well when I was starting off in Redis

Collectives™ on Stack Overflow

Redis - handling changes to data structures

2 Answers 2

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related