Installed mapper-size plugin and added it to index mapping but no _size field is showing up in indexes

Question

We are trying to figure out which documents in our Elasticsearch (version 7.0.1) index are consuming the most disk space. We found the mapper-size plugin provided by Elastic. We installed the plugin on all Elasticsearch data/master nodes, and restarted the ES service on each one. We also added the _size field to the index pattern mapping. However, the _size field is not showing up. This index is fed by several Filebeat services running on our application servers, and the index rolls over each night.

We tried creating a brand new index that matches the index pattern. The _size field was present in the mapping:

"application_log_test" : {
"mappings" : {
  "_size" : {
    "enabled" : true
  }

After adding a few test documents, however, the _size field did not show up in the queried documents. We verified that all Elasticsearch nodes came up with the plugin loaded:

[2019-09-16T15:10:45,103][INFO ][o.e.p.PluginsService     ] [node-name-1] loaded plugin [mapper-size]

We are expecting any document added to the index to calculate and display a _size metadata field. This field doesn't display in our output.

Val · Accepted Answer · 2019-09-18 15:00:52Z

5

The _size field is not added to your source document. You can query it, aggregate it, sort on it, but to actually see its value, you need to do it through script fields. Try to run the query below and you'll see:

GET application_log_test/_search
{
  "query": {
    "range": {
      "_size": { 
        "gt": 10
      }
    }
  },
  "aggs": {
    "sizes": {
      "terms": {
        "field": "_size", 
        "size": 10
      }
    }
  },
  "sort": [
    {
      "_size": { 
        "order": "desc"
      }
    }
  ],
  "script_fields": {
    "size": {
      "script": "doc['_size']"  
    }
  }
}

answered Sep 18, 2019 at 15:00

Val

218k14 gold badges377 silver badges384 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Linden X. Quan · Accepted Answer · 2023-06-01 14:42:45Z

0

For ElasticSearch 7.10 You can use docvalue_fileds parameter to return the _size in the research response.

GET index/_search
{ 
  "docvalue_fields": ["_size"]
}

answered Jun 1, 2023 at 14:42

Linden X. Quan

8508 silver badges20 bronze badges

Collectives™ on Stack Overflow

Installed mapper-size plugin and added it to index mapping but no _size field is showing up in indexes

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related