Missing documents while connecting Mongodb and elasticsearch via mongo-connector

Question

I am trying to transfer documents from MongoDB 4.4.5 to Elasticsearch7.12 via mongo-connector 3.1.1

I start by using:

sudo mongo-connector -m 127.0.0.1:27017 -t 127.0.0.1:9200 -d elastic2_doc_manager -n Data.myData --continue-on-error --batch-size=1000 --only-dump --batch-size=1000 --no-dump --tz-aware

at the beginning of the log, I can see the total number of the documents in MongoDB which is 454

when the process is done I get

and in Elasticsearch index I can only see 268 as in the images shows

any idea what could be the issue here?

Noa · Accepted Answer · 2021-06-18 19:42:14Z

0

This issue is Releated to reaching the max number for fields's mapping on Elastivsearch to Solve this issue increase this limit on Elasticsearch's index via

PUT my_data_index/_settings
{
  "index.mapping.total_fields.limit": 20000

}

answered Jun 18, 2021 at 19:42

Noa

3551 gold badge7 silver badges33 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Missing documents while connecting Mongodb and elasticsearch via mongo-connector

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related