In MongoDb is there a way to get last 1000 array elements from multiple documents

Question

Given the following Mongo document model:

Each document in the collection represents 1 hour of resource monitoring. In each document, there is a collection of summaries. There is also a count of the number of summaries as an integer, as it may make life easier.

Is there an efficient way to query the collection and return either just the recent most 1000 summaries as an aggregated list?

Or an efficient way to query the collection and return the number of documents that contain the recent most 1000 summaries?

The number of summaries in each document will differ, but the number of summaries in one single document will never equal more than 1000.

EDIT: I should mention I am using mongo with the .NET driver so have LINQ available to me.

I have a couple of questions : Which version of MongoDB are you running ? And are you filtering the request on a certain date ? (ie: summaries for 26/9/2017) — Nicolas Ducom
– Nicolas Ducom, Commented Sep 26, 2017 at 18:40
I believe I am using version 3.4.7 and I am using csharp driver v2.4.4.0. I generally do filtering on the timestamp and have that indexed, I figured I could probably get a number of hours a check if there is 1000 summaries, but I was wondering if there was a more efficient way. — gcoulby
– gcoulby, Commented Sep 26, 2017 at 19:52

Nicolas Ducom · Accepted Answer · 2017-09-26 19:59:26Z

1

Have you looked at Mongo aggregation ? If you want to return the 1000 most recent summaries you could go with an $unwind followed by a $replaceRoot operation, here's the shell query I tried :

db.getCollection('test').aggregate([
{$match : {your timestamp match query here}},
{$sort : {"timestamp": -1}},
{$unwind : "$summaries"},
{$limit : 1000},
{$replaceRoot: {newRoot: "$summaries"}}
])

The match operation at the beginning of your aggregation pipeline is important as indexes are only used at the first step. If you unwind your whole collection your performance might drop drastically.

answered Sep 26, 2017 at 19:59

Nicolas Ducom

2482 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Nicolas Ducom Over a year ago

Regarding your second question, what exactly do you want to return, the ids of the documents containing the first 1000 timestamps or just the number of documents ?

gcoulby Over a year ago

Ultimately I want a list of the recent most 1000 summaries.

gcoulby Over a year ago

I will give it a try tomorrow. Thanks for helping

gcoulby Over a year ago

OK just tested this in Robo and it works great... Now to figure out how to do it with the driver. Thanks for the help

Collectives™ on Stack Overflow

In MongoDb is there a way to get last 1000 array elements from multiple documents

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related