How to control the projection definition in MongoDb using C#

Question

I have a domain class like this.

public class Thing
{
    public Guid Id { get; set; }
    public string Name { get; set; }
    public Dictionary<string, string> Stuff { get; set; }
}

I'm retrieving it from my DB using the following.

return await _db.Things
    .Find(x => x.Name == name)
    .SingleOrDefaultAsync(token);

As I noticed, there might be huge chunks of unnecessary data so I used a projection like this.

ProjectionDefinition<Thing> projection = Builders<Thing>
    .Projection
    .Include(a => a.Id)
    .Include(a => a.Name);

BsonDocument projected = await _dbContext.Things
    .Find(x => x.Name == name)
    .Project(projection)
    .SingleOrDefaultAsync(token);

This works but, naturally, cuts of all the dictionary contents. I'd like to alter the definition of projection to include the field but perform a filtration of the constituting elements. Suppose I'd only want to bring in the keys of said dictionary that start with duck. An attempt might be something like this.

ProjectionDefinition<Thing> projection = Builders<Thing>
    .Projection
    .Include(a => a.Id)
    .Include(a => a.Name)
    .Include(a => a.Stuff.Where(b => b.Key.StartsWith("duck")));

That resulted in exception as follows.

System.NotSupportedException: The expression tree is not supported: {document}{configuration}

Given my ignorance with MongoDb, I have no idea if I should add something, remove something or forget the idea all together. I also tried to work with the original type to be able to filter the stuff that way but the only solution I've got was post-fetch, basically trumming the retrieved data. I'd like to lower the payload from the DB to my service.

Thing projected = await _dbContext.Things
    .Find(x => x.Name == name)
    .Project<Thing>(projection)
    .SingleOrDefaultAsync(token);

Is it doable at all and if so, how (or at least what to google for)?

Proof of effort: code examples, general operations, tutorials, wrong answers etc. It might be out there somewhere but I failed to find it (or recognize if found).

Finally, I landed into the following - God forgive me for I know not what I'm doing. Is this at all in the right direction or is a gang of crazy donkeys going to bite me in the lower back for this?!

ProjectionDefinition<Thing, Thing> wtf = Builders<Thing>.Projection
    .Expression(a => new Thing
    {
        Id = a.Id,
        Name = a.Name,
        Stuff = a.Stuff
            .Where(b => b.Key == scope)
            .ToDictionary(b => scope, b => b.Value)
    });

This seems a bit tricky and I don't have much idea but .NET things. But I found something for you that might help you - mongodb.github.io/mongo-csharp-driver/2.7/reference/driver/… — thelovekesh
– thelovekesh, Commented Jun 17, 2021 at 5:06
Also see this question about the error you are getting - stackoverflow.com/questions/55597781/… — thelovekesh
– thelovekesh, Commented Jun 17, 2021 at 5:07
Please post a sample document with the stuff field and what you want in your projected stuff. — prasad_
– prasad_, Commented Jun 17, 2021 at 11:53
@prasad_ Not sure what you're requesting. A sample document? The class describing it is shown in the first sample at the top. The field Stuff is a dictionary of string as a key and string as a value. I want to be able to retrieve the document but only pull a single element of the dictionary part, instead of all of them. Please elaborate if I missed something. — Konrad Viltersten
– Konrad Viltersten, Commented Jun 17, 2021 at 20:50
Generally, the issue of filtering a dictionary (in MongoDB its an object or embedded document) based upon the key name is solved using an aggregate query. You can convert the dictionary's key-value pairs to an array (using the aggregate operator $objectToArray) and filter(in your case can use a regex) the array by the key name . — prasad_
– prasad_, Commented Jun 18, 2021 at 4:56

prasad_ · Accepted Answer · 2021-06-18 08:50:51Z

1

+50

This is a mongo shell query with MongoDB v4.2.8.

Consider this input document:

{
        "_id" : 1,
        "name" : "john",
        "stuff" : {
                "mycolor" : "red",
                "fruit" : "apple",
                "mybook" : "programming gems",
                "movie" : "star wars"
        }
}

The goal is to project the name and stuff fields, but stuff with only field names starting with "my".

The aggregation query:

db.test.aggregate([
  { 
      $project: { 
          _id: 0, 
          name: 1, 
          stuff: { 
              $filter: { 
                  input: { $objectToArray: "$stuff" }, 
                  as: "stf", 
                  cond: { $regexMatch: { input: "$$stf.k" , regex: "^my" } }
              }
          }
      }
  },
  { 
      $addFields: { stuff: { $arrayToObject: "$stuff" } } 
  }
])

And, the projected output:

{
        "name" : "john",
        "stuff" : {
                "mycolor" : "red",
                "mybook" : "programming gems"
        }
}

answered Jun 18, 2021 at 8:50

prasad_

14.4k2 gold badges27 silver badges44 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Konrad Viltersten Over a year ago

What is the input to the regex (i.e. $$stf.k)? I'm guessing that it relates to the filter output somehow but it's not apparent to be. Also, the .k tells me nothing in the context. Am I missing the point or is it some special mango-script syntax?

prasad_ Over a year ago

The result of { $objectToArray: "$stuff" } (see $objectToArray); its an array. E.g., each array element is like {"k" : "mycolor", "v" : "red"} and corresponds to key-value "mycolor" : "red". Next, iterate the array, using $filter and match with the regex. In the following $addFields, convert that filtered array back into key-value dictionary. "stf" represents each array element used in the $filter.

Konrad Viltersten Over a year ago

Got it, now. Based on the trickyness of the solution, I'll change my dictionary to an array. It's not my decision but I'm going to press my dev-lead to make the decision to accept array instead by threatening them to do it themselves otherwise. I'm assuming that you have little to contribute with when it comes to actual C# syntax in the Mongo driver. If I'm wrong on this point, feel free to shoot in some .NET'y code. Otherwise, thanks for the stuff anyway. It was big help.

Konrad Viltersten Over a year ago

Remind me in a day or two so I don't forget to award you the bounty. Important for the credit to go when the credit is due.

prasad_ Over a year ago

"Based on the trickyness of the solution, I'll change my dictionary to an array." I think its good design decision. Also see this: MongoDB Blog Post - Attribute Pattern. It may also allow you to create indexes for an efficient search (indexes on array fields are called as Multikey Index).

Collectives™ on Stack Overflow

How to control the projection definition in MongoDb using C#

1 Answer 1

5 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

5 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related