JSON formatting in Elasticsearch

Question

Bit confused.

I'm using jackson to parse JSON strings into maps that will then be accepted in elasticsearch.

That works fine.

However due to the JSON specifications, all integers and other formats need to be sent through as a string - aka - "key":"value". But then elasticSearch sees them as strings, not numbers. Which makes totaling values like user ratings not possible when it should be trivial.

How can I use the JSON format to input numbers/integers into elasticsearch?

Is there a way around this?

Sure. the first part of JSON input {"positive score":"0",,"publishedAt":"2017-09-07T15:35:34.000Z"} is read as String:String and not String:Object which is accepted by elasticsearch. When I change it to "score":0 I get a parse error by jackson - com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot deserialize instance of java.lang.String out of START_ARRAY token... So how can I use the JSON format to input numbers/integers into elasticsearch? — vachina
– vachina, Commented Sep 14, 2017 at 22:14
the code i'm using to send the json data to jackson is try (BufferedReader br = new BufferedReader(new FileReader(addedDir))){ for (String line; (line = br.readLine())!= null;) { System.out.println("#" + this.counter + " processing " + line); this.storeMap(this.convertToMap(line)); this.counter++; } } — vachina
– vachina, Commented Sep 14, 2017 at 22:18
and the double comma in the example JSON above was my pasting mistake in this stackoverflow textbox, not in the original code — vachina
– vachina, Commented Sep 14, 2017 at 22:22
I found the solution. I was sending object to jackson but using hm = mapper.readValue(json, new TypeReference<HashMap<String, String>>() {}); where i should have used hm = mapper.readValue(json, new TypeReference<HashMap<String, Object>>() {}); - chaging the 2nd string inthe typerference to ype object. — vachina
– vachina, Commented Sep 14, 2017 at 22:55

denov · Accepted Answer · 2017-09-16 20:45:34Z

3

You may need to create a mapping for your index.

Mapping is the process of defining how a document, and the fields it contains, are stored and indexed. For instance, use mappings to define:

which string fields should be treated as full text fields.
which fields contain numbers, dates, or geolocations.
whether the values of all fields in the document should be indexed into the catch-all _all field.
the format of date values.
custom rules to control the mapping for dynamically added fields.

https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html

If you already have a some existing data that you don't want to lose you'll need to reindex after updating your mapping. Due to how ES indexes data you can not change the type after it's been created.

https://www.elastic.co/blog/changing-mapping-with-zero-downtime https://www.elastic.co/guide/en/elasticsearch/reference/5.6/docs-reindex.html

edited Sep 16, 2017 at 20:45

answered Sep 15, 2017 at 1:28

denov

12.9k2 gold badges32 silver badges45 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

vachina Over a year ago

Thanks for this. It goes above and beyond my question by providing understanding that should hoepfully help prevent future issues. Cheers!

Collectives™ on Stack Overflow

JSON formatting in Elasticsearch

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related