Select all values by pattern from redis server using node js

Question

I had a multiple chanel like this

client.HMSET('live:user:1', {
    "a": "1",
    "b": "1"
});
client.HMSET('live:user:2', {
    "a": "2",
    "b": "2"
});
client.HMSET('live:user:3', {
    "a": "3",
    "b": "3"
});

client.HMSET('otherchanel:user:4', {
    "a": "4",
    "b": "4"
});
client.HMSET('otherchanel:user:5', {
    "a": "5",
    "b": "5"
});
client.HMSET('otherchanel:user:6', {
    "a": "6",
    "b": "6"
});

How can i select all values by key pattern "live:*"? I need it to retern JSON object to my client app like this:

[{user:1, a:1, b:1}, {user:2, a:2, b:2}, {user:3, a:3, b:3}]

did you try client.get('live:*')?

Mukesh Soni
– Mukesh Soni

2014-03-13 06:18:20 +00:00
Commented Mar 13, 2014 at 6:18 — Mukesh Soni
– Mukesh Soni, Commented Mar 13, 2014 at 6:18

Santosh Joshi · Accepted Answer · 2015-09-20 05:59:30Z

8

To get all keys with a pattern use

client.keys('live:*', function (err, keys) {
    if (err) return console.log(err);

    for(var i = 0, len = keys.length; i < len; i++) {
        console.log(keys[i]);
    }
});

edited Sep 20, 2015 at 5:59

answered Mar 13, 2014 at 6:37

Santosh Joshi

3,3405 gold badges39 silver badges49 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

NiCk Newman Over a year ago

You forgot a closing ) :)

Liviu B · Accepted Answer · 2014-03-13 14:08:32Z

It's simple, but you have to also fetch all the keys and combine them. I suggest using async to make the calls asynchronous:

function getAllUsersInChannel_suboptimal(channelName, functionCallback) {
client.keys(channelName + ":user:*", function(err, userKeys) {
    if (!err) {
        async.map(userKeys, 
            function(userKey, callback) {
                client.hgetall(userKey, function(err, user) {
                    if (!err) {
                        callback(null, user);
                    }
                    else {
                        callback(err);
                    }
                });
            }, 
            function(err, userList) {
                if (!err) {
                    functionCallback(null, userList);
                }
                else {
                    functionCallback(err);
                }
            });
    }
    else {
        functionCallback(err);
    }
});
}

Using a pattern match can drastically reduce performance even on medium sized databases. I suggest you also index the id's of the users using a set.

Create the counter key if it does not exist and set its value to 0:

SETNX <channel_name>:user:counter 0

When you add a new user, first increment the counter key, which also fetches its value:

INCR <channel_name>:user:counter => <new_id>

This will yield the new id for the user. Now you have to save a new member in the "user:index" set like so:

SADD <channel_name>:user:index <new_id>

You can now add the new user key like so:

HMSET <channel_name>:user:<new_id> <list of values>

To fetch data for all users you would first get a the list of existing users like so:

SMEMBERS <channel_name>:user:index => <list of all user ids>

For each id you would have to fetch the hash values:

asynchronous each <list of all user ids> as <user_id>
HGETALL channel_name>:user:<user_id> => all values of key
end asynchronous each

In Node.js, considering that you've already used the techniques above to index users, you retrieve all users in a channel using something like this:

function getAllUsersInChannel_optimal(channelName, functionCallback) {
client.smembers(channelName + ":user:index", function(err, userIds) {
    if (!err) {
        async.map(userIds, 
            function(userId, callback) {
                client.hgetall(channelName + ":user:" + userId, function(err, user) {
                    if (!err) {
                        callback(null, user);
                    }
                    else {
                        callback(err);
                    }
                });
            }, 
            function(err, userList) {
                if (!err) {
                    functionCallback(null, userList);
                }
                else {
                    functionCallback(err);
                }
            });
    }
    else {
        functionCallback(err);
    }
});
}

Piyush Beli · Accepted Answer · 2017-01-11 10:00:00Z

4

Using client.keys() is very inefficient and raises CPU spikes, it is not advised to use it in the production environment. If there is a lot of data stored in the Redis you should consider client.scan() method which will fetch the keys in batches. It will also reduce memory usages if the number of the keys are a lot.

answered Jan 11, 2017 at 10:00

Piyush Beli

1932 silver badges8 bronze badges

Collectives™ on Stack Overflow

Select all values by pattern from redis server using node js

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related