Exact search in array object type using elasticsearch

Question

I'm looking for a way to do exact array matches in elastic search. Let's say these are my documents:

{"id": 1, "categories" : ["c", "d"]}
{"id": 2, "categories" : ["b", "c", "d"]}
{"id": 3, "categories" : ["c", "d", "e"]}
{"id": 4, "categories" : ["d"]}
{"id": 5, "categories" : ["c", "d"]}

Is there a way to search for all document's that have exactly the categories "c" and "d" (documents 1 and 5), no more or less?

As a bonus: Searching for "one of these" categories should still be possible as well (for example you could search for "c" and get 1, 2, 3 and 5)

Any clever way to tackle this problem?

femtoRgon · Accepted Answer · 2014-08-06 05:17:53Z

20

If you have a discrete, known set of categories, you could use a bool query:

"bool" : {
    "must" : {
        "terms" : { "categories" : ["c", "d"],
             minimum_should_match : 2
         }
    },
    "must_not" : {
        "terms" : { "categories" : ["a", "b", "e"],
             minimum_should_match : 1
         }
    }
}

Otherwise, Probably the easiest way to accomplish this, I think, is to store another field serving as a categories keyword.

{"id": 1, "categories" : ["c", "d"], "categorieskey" : "cd"}

Something like that. Then you could easily query with a term query for precisely the results you want, like:

term { "categorieskey" : "cd" }

And you could still search non-exclusively, as;

term { "categories" : "c" }

Querying for two categories that must both be present is easy enough, but then preventing any other potential categories from being present is a bit harder. You could do it, probably. You'dd probably want to write a query to find records with both, then apply a filter to it eliminating any records with categories other than the ones specified. It's not really a sort of search that Lucene is really designed to handle, to my knowledge.

Honestly I'm having a bit of trouble coming up with a good filter to use here. You might need a script filter, or you could filter the results after they have been retrieved.

edited Aug 6, 2014 at 5:17

answered Oct 1, 2012 at 16:38

femtoRgon

33.4k7 gold badges67 silver badges90 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

phoet Over a year ago

funny, that's exactly what i told him :)

Conrad.Dean Over a year ago

This query won't run. minimum_match doesn't appear to be a valid parameter to a TermsFilter.

femtoRgon Over a year ago

@Conrad.Dean Who said anything about using a filter?

Conrad.Dean Over a year ago

@femtoRgon oh woops. i was way too zoomed in. the syntax is identical if that's wrapped with a "filter":{...} instead of just a "query":{...}. minimum_should_match is a valid parameter in a terms query elasticsearch.org/guide/en/elasticsearch/reference/current/…

femtoRgon Over a year ago

Ah, you're right, probably should be minimum_should_match. Not sure whether that's a change in ElasticSearch, or a mistake, but certainly doesn't hurt to update. Thanks.

|

Lucas Holt · Accepted Answer · 2013-03-21 14:44:33Z

I found a solution for our usage case that appears to work. It relies on two filters and the knowledge of how many categories we want to match against. We make use of a terms filter and a script filter to check the size of the array. In this example, marketBasketList is similar to your categories entry.

{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "siteId": 4
          }
        },
        {
          "match": {
            "marketBasketList": {
              "query": [
                10,
                11
              ],
              "operator": "and"
            }
          }
        }
      ]
    },
    "boost": 1,
    "filter": {
      "and": {
        "filters": [
          {
            "script": {
              "script": "doc['marketBasketList'].values.length == 2"
            }
          },
          {
            "terms": {
              "marketBasketList": [
                10,
                11
              ],
              "execution": "and"
            }
          }
        ]
      }
    }
  }
}

Collectives™ on Stack Overflow

Exact search in array object type using elasticsearch

2 Answers 2

7 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

7 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related