MongoDB $slice (embedded array paging)

Question

I have this schema:

article: {
    subject,
    comments: []
}

if I have 8 comments, and query

 article.find({}, {
     comments: {
         $slice: [ -10, 5 ]
     }
 });

And I get comments from index 0 to index 4,
but I only want comments from index 0 to index 2 to be returned because of paging.
(page 1 $slice[ -5, 5 ] from index 3 to index 7, page 2 $slice[ -10, 5 ] from index 0 to index 2)

now I have to pass another parameter "lastId" to compare each comments and remove that "_id" < "lastId", but I think it is a little hacky.

Any one has a good solution for this?

christkv · Accepted Answer · 2012-05-03 09:54:18Z

16

So I'm going to say that you should switch your schema to leave comments as separate documents as this is an unbound array and it will make your queries more efficient. I'll explain it.

When you add embedded documents to an array that is not a fixed size mongoDB will potentially need to move the document around as it grows, changing the padding factor and causing fragmentation (the padding factor is a guess from mongodb's side off how big your document will grow, it preallocates more space for that case).

You are also limited to 16MB pr document so imagine if you get a crazy popular thread or you decide to extends comments with other metadata it's feasible that you'll break that barrier. Retrieving a big document is also expensive and time consuming.

In general embedded documents are great if they are not unbound arrays. So keeping a list of the top 10 comments will work great but keeping 1000+ comments is bad.

There are some good presentation under

http://www.10gen.com/presentations/mongodb-berlin/2012/10-key-performance-indicators http://www.10gen.com/presentations/mongosv-2011/schema-design-by-example

I think there is more work coming soon on schema design that will more helpful in the long term. I think it's the hardest bit to be honest. I know, it took me awhile to wrap my head around the differences from relational models.

answered May 3, 2012 at 9:54

christkv

4,44025 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Kevin Over a year ago

the comments are not long text and large amount, so 16MB per document is totally enough. and what I really worry about is flexibility and efficiency. so it is really really a bad way to store comments(or any other unbound array) as embedded document, right?

christkv Over a year ago

well some sort of queries are very difficult to do with embedded docs, such as give me all the comments by a specific user. So in general this is a good case for denormalization.

Kevin Over a year ago

I agree you. But I also think that embedded or not depends on what the app needs, if I don't need query comments by other fields but only by the article(in other words, comments always appear with article), I prefer to embed the comments array in article document.

christkv Over a year ago

Yeah I think it's a valid case I just wanted to make sure you understand the tradeoff and potential problems related to growing embedded documents. As long as you know that and you make the tradeoff knowingly :).

Collectives™ on Stack Overflow

MongoDB $slice (embedded array paging)

1 Answer 1

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related