76 questions
1
vote
1
answer
622
views
Is BulkIngester (replacement of 'Bulk Processor') in elasticsearch java api thread safe?
Use case: I have multiple kafka listeners for various topics. Each topic Listener will run in multiple threads (using spring's 'ConcurrentKafkaListenerContainer'). Listeners will be performing Update/...
- 17
0
votes
0
answers
151
views
Uploading a CSV file in sql using SP_OAMethod procedures to do a BULK sync with Salesforce using BULK API Version 2
My query is that I have a CSV file with data in the format as accepted by Salesforce and require to do a BULK sync using BULKApi Version 2. I now need to send this data from sql using SP_OAMethod ...
0
votes
1
answer
105
views
Unable to use _bulk API on Elasticsearch on Windows
I am new to windows . I was working my entire on linux. Can somebody help me here for _bulk api indexing
C:\ELK\master-1\elasticsearch\sample_data>curl -k -H 'Content-Type: application/x-ndjson' -X ...
0
votes
1
answer
198
views
remove an item using bulk update
I have an object with different values in Elasticsearch. When I want to update it, I use bulk api like below:
inventory_edition.append(
{
'_op_type': 'update',
'_index': value_index,
'_id': ...
1
vote
1
answer
932
views
Elasticsearch | Mapping exclude field with bulk API
I am using bulk api to create index and store data fields. Also I want to set mapping to exclude a field "field1" from the source. I know this can be done using "create Index API" ...
- 41
0
votes
1
answer
1k
views
how Elasticsearch bulk API works with threadpool
While trying to index around 20000 records (each record has around 100kb size) using python's bulk API, I saw that max write queue size consumed is 6. So are my following understanding about bulk ...
- 323
0
votes
1
answer
2k
views
Elasticsearch multiple JSON insert bulk
I am trying to insert multiple JSON documents in Elastic search. I have done with the single document as the following curl sample
curl --request POST \
--url 'http://localhost:9200/articles/_doc/?...
- 3,359
0
votes
1
answer
381
views
getting [ERROR] 'The [dims] property must be specified for field [vector].' despite being set in mapping
I am trying to upload dense vectors to Elasticsearch endpoint.
created index with mapping as below:
mapping = {
"mappings": {
"properties" : {
"vector": {
...
- 401
0
votes
1
answer
2k
views
Upload new line JSON to Elasticsearch bulk API
I'm trying to upload a new line JSON to Elasticsearch using the Bulk API. The bulk JSON I'm uploading looks like this, with each JSON on a new line:
{"ip": "x.x.x.x", "seen": true, "classification": "...
- 37
0
votes
0
answers
178
views
why index only one doc takes so long time
There is a scheduler task (every ten minutes) that synchronize changed data from Oracle to ElasticSearch, but occasionally there are SocketTimeOutException
java.net.SocketTimeoutException: 30,000 ...
- 8,545
1
vote
0
answers
228
views
couldn't find method readString(Path) while loading BulkApi
I'm going to load the bulk of data using BulkApi but while compiling the code it shows me the error.
private static final Logger log = LoggerFactory.getLogger(BulkApiService.class);
@Autowired
...
- 707
3
votes
1
answer
3k
views
Elasticsearch Java API 7.0 Bulk Insert trying to basic example, throwing errors
Java 1.8, Elasticsearch Low and High Level Rest Clients 7.0.0
I am trying the simple example from the docs found here: Bulk API
BulkRequest bulkRequest = new BulkRequest();
request.add(new ...
- 4,549
0
votes
1
answer
215
views
Index json files in bulk elasticsearch 5.6
I have a folder with around 590,035 json files. Each file is a document that has to be indexed. If I index each document using python then it is taking more than 30 hours. How do I index these ...
user10005853
1
vote
0
answers
1k
views
ElasticSearch illegal_argument_exception for client.bulk() Node.js
I try to send data to ES and I was using POST request to send data line by line and it worked fine with small files.
When I changed to use client.bulk(), I keep getting the following error message:
"...
- 11
0
votes
1
answer
2k
views
Post request for bulk API is giving status code of 406, How to resolve it?
I am using Elastic search 6.1 version
My data is appending correctly and I am adding '\n' at the end of the request.
My code is as follows:
def insert_in_bulk(self, filee, rtype):
U = urljoin(...
- 171
0
votes
1
answer
693
views
How to insert an element into already present list in elastic search
Say I have documents stored like below.
document 1
{
id : '1',
title : "This is a test document1",
list : ["value1" , "value2"],
...
}
document 2
{
...
- 87
1
vote
2
answers
2k
views
How to bulk change JSON format for Elastic Search
I have the below format and was hoping to pre-process in bulk using elasticsearch.
{"title":"April","url":"https://simple.wikipedia.org/wiki/April", "abstract":"April is the 4th month of the year, ...
- 87
1
vote
0
answers
738
views
How to get "related documents" in a query in Elasticearch?
I have a problem in elasticsearch i am not being able to solve:
I have two different types in an index, one called 'keywords' and another one called 'products'.
Keywords have the string property '...
32
votes
5
answers
37k
views
BULK API : Malformed action/metadata line [3], expected START_OBJECT but found [VALUE_STRING]
Using Elasticsearch 5.5,getting the following error while posting this bulk request, unable to figure out what is wrong with the request.
"type": "illegal_argument_exception",
"reason": "Malformed ...
0
votes
1
answer
144
views
how to check if ElasticSearch BulkProcessor Service exists
I am using elastic package for golang. I want to use its BulkProcessor for sending huge amount of documents in background. As per shown in wiki, I could create a processor. But I don't want to create ...
- 1,898
0
votes
1
answer
3k
views
usage of BulkIndexRequest from golang package 'elastic'
I want to use BulkIndexRequest from Go package elastic. I try to find examples but it seems that they don't exist. So if anybody has used it, could you please help me to use it for creating request ...
- 1,898
4
votes
2
answers
4k
views
Specify the _id field using Bulk.IndexMany in ElasticSearch
I'm facing a problem inserting document using bulk API (C# NEST v5.4). I've an array of documents and inside of the array I've my ID.
My code is:
documents = documents .ToArray();
Client.Bulk(bd =&...
- 121
2
votes
0
answers
6k
views
How to do Elasticsearch Bulk Insert in Python
I have a JSON array which contains a list of data that needs to be pushed to an ElasticSearch instance. Something like
[ {"Message": "2aTeFCKTYWwfrF"}, {"Message": "2aTeFCKTYWwfrD"}, {"Message": "...
- 1,700
0
votes
0
answers
740
views
update_by_query with upsert python elasticsearch
I wish to do an update_by_query request using my python elasticsearch client. I wish to query with index, type and a field which is NOT id. Based on that field I wish to update some of the other ...
- 873
0
votes
1
answer
1k
views
bulk insert for elasticsearch nest 5.x
I have been trying to use to the bulk insert function but every time I use it is showing some mapping error . Has the bulk insert function declaration have changed from nest 1.x to nest 5.x because in ...
- 131