Elasticsearch multiple JSON insert bulk

Question

I am trying to insert multiple JSON documents in Elastic search. I have done with the single document as the following curl sample

curl --request POST \
  --url 'http://localhost:9200/articles/_doc/?pretty=' \
  --header 'Content-Type: application/json' \
  --data '{
    "topic":"python",
    "title": "python tuples",
    "description": "practical operations with python tuples",
    "author": "test",
    "date": "1-1-2019",
    "views" : "100"
}'

When I tried to insert a bulk JSON array as the following CURL

curl --request POST \
  --url 'http://localhost:9200/articles/_bulk/?pretty=' \
  --header 'Content-Type: application/json' \
  --data '[{
        "topic":"python",
        "title": "python tuples",
        "description": "practical operations with python tuples",
        "author": "test",
        "date": "1-1-2019",
        "views" : "100"
        },
        {
        "topic":"python",
        "title": "python tuples",
        "description": "practical operations with python tuples",
        "author": "test2",
        "date": "1-1-2019",
        "views" : "100"
}]'

I am getting the following error

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Malformed action/metadata line [1], expected START_OBJECT but found [START_ARRAY]"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "Malformed action/metadata line [1], expected START_OBJECT but found [START_ARRAY]"
  },
  "status": 400
}

Jozef - Spatialized.io · Accepted Answer · 2021-02-08 16:19:49Z

4

The Bulk API requires the application/x-ndjson header and thus the payload to be a newline-delimited JSON. So use this instead:

curl -X POST "localhost:9200/articles/_bulk?pretty" -H 'Content-Type: application/x-ndjson' -d'
{ "index" : {  } }
{"topic":"python","title":"python tuples","description":"practical operations with python tuples","author":"test","date":"1-1-2019","views":"100"}
{ "index" : {  } }
{"topic":"python","title":"python tuples","description":"practical operations with python tuples","author":"test2","date":"1-1-2019","views":"100"}
'

BTW there's a nodejs cmd utility called json-to-es-bulk that'll generate such payloads for you.

answered Feb 8, 2021 at 16:19

Jozef - Spatialized.io

17k4 gold badges29 silver badges79 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Elasticsearch multiple JSON insert bulk

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related