Query() DynamoDB for each value in array on Lambda NodeJS

Question

I have the following code in a NodeJS 12x Lambda function:

targetedAliases = ["abc", "efg"]; //Not used yet. But want to replace alias with this array.
alias = "abc" //using this one aka targetedAliases[0].

const dynamoDB = new AWS.DynamoDB({
    apiVersion: "2012-10-08"
});
var aliasScores;


const params = {
    TableName: 'a-table',
    FilterExpression: '#alias = :alias',
    ExpressionAttributeNames: {
        '#alias': 'alias',
    },
    ExpressionAttributeValues: {
        ':alias': {
            S: alias
        },
    }
};

aliasScores = await dynamoDB.scan(params).promise();

console.log("====> ", aliasScores);

The function as is prints the content of aliasScores as expected but I need to execute this n times for each item on the targetedAliases array.

Is there a way I can use a Array.forEach or something similar to execute a query/scan iteratively for each item in an array? Alternatively, can I use an array on the FilterExpression to execute the query/scan just once?

I want store each query/scan result in the aliasScores variable as a concatenation of all returned objects to use it further below.

Ashish Modi · Accepted Answer · 2020-02-26 20:29:11Z

3

You can use Promise.all with .map to execute these in parallel and get the results in array. Something like this

const targetedAliases = ["abc", "efg"]; //Not used yet. But want to replace alias with this array.
alias = "abc" //using this one aka targetedAliases[0].

const dynamoDB = new AWS.DynamoDB({
    apiVersion: "2012-10-08"
});
var aliasScores;

const result = await Promise.all(targetedAliases.map(alias => {
  const params = {
    TableName: 'a-table',
    FilterExpression: '#alias = :alias',
    ExpressionAttributeNames: {
        '#alias': 'alias',
    },
    ExpressionAttributeValues: {
        ':alias': {
            S: alias
        },
    }
};

return dynamoDB.scan(params).promise();
}));

console.log(result);

edited Feb 26, 2020 at 20:29

answered Feb 26, 2020 at 19:49

Ashish Modi

7,7902 gold badges23 silver badges36 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

Ashish Modi Over a year ago

changing alias to array will result in this error as it expects it to be string. You need to iterate through array and for each item send the query. The way I have done

10110 Over a year ago

Yes sorry I noticed that... I do get an error on const aliasScores = await dynamoDB.Scan(params).promise(); ... It says "unexpected token dynamoDB", if I remove await error goes away. Are we missing an async?

jarmod Over a year ago

Why await dynamoDB.scan() within the map when all of the promises are later awaited via Promise.all? Presumably you should just return dynamoDB.scan(params).promise().

10110 Over a year ago

The scan runs successfully but seems like aliasScores is being overwritten instead of getting items appended to it. targetedAliases[0] returns 57 items and targetedAliases[1] returns 20 items for a total of 77. But I get just 20. If I leave just targetedAliases[0] I get 57.

Ashish Modi Over a year ago

Just use the result variable which will have results from both calls. see my updated answer.

|

starpebble · Accepted Answer · 2020-02-28 20:24:59Z

1

Your options kinda depend on whether the attribute alias is a primary key or not. Just knowing this difference will help a lot.

Promise.all() executes one operation for each alias. Consider the batch approach. A single batch for executes in one API invocation which will reduce the number of operations and reduce your DynamoDB API calls. Batching is fun.

When the attribute key 'alias' is the primary key, BatchGetItem a is very efficient operation for up to a dozen or so operations where each operation is a query for one alias. Just create a single BatchGetItem with one operation for each entry in the array. The response is a single json dictionary.

When the attribute key 'alias' is not the primary key then it's different because the table must be scanned. A DynamoDB scan() is different than a DynamoDB query(). scan can find items by any attribute and can filter by the value of 'alias.' The FilterExpression supports OR and it is possible to filter the scan. The scan will check every item and the filter just reduces the returned result set. The response is a single json dictionary.

answered Feb 28, 2020 at 20:24

starpebble

5443 silver badges7 bronze badges

2 Comments

10110 Over a year ago

Thanks for the info! I ended up changing it to a query but i will definitely look into batch operations!

starpebble Over a year ago

Sounds good. query() is goodness. The forEach approach certainly keeps the query() responses in order so that you can create a concatenated result array aliasScore. BatchGetItem will order results too however a single scan operation with a FilterExpression will not.

Collectives™ on Stack Overflow

Query() DynamoDB for each value in array on Lambda NodeJS

2 Answers 2

7 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

7 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related