How can I create Bulk CRUD Operations request in Elasticsearch version 8 using JAVA APIs?

Question

We wanted to create IndexRequest, DeleteRequest, UpdateRequest and BulkRequest in Elasticsearch version 8 using JAVA APIs. But I don't see any java documentation in elasticsearch v8 official website. Previously in elasticsearch version 7, we used below code in order to perform operations.

IndexRequest indexRequest = Requests.indexRequest(index).id(key).source(source);


BulkRequest bulkRequest = Requests.bulkRequest();
bulkRequest.add(indexRequest);

Also following Elasticsearch Java API Client [8.1] , but no luck.

Problem arises when we try to do Requests.indexRequest(), this Request class is not available in version 8.

So, Is it possible to create similar request in ES version 8 also?

Update 1:-

My point here is that I need to keep a list of request operation which maybe arbitrary ( maybe 1st five are inserts, next 2 are update and next 2 are delete requests and at the end 1 insert operation ). And that list needed to be flushed via Bulk maintaining the type of request received. I am using BulkRequest.Builder bulkRequestBuilder = new BulkRequest.Builder();

But my issue is with bulk update. I am unable to find any update API for bulkrequest for elasticsearch version 8.

For Insert:-

bulkRequestBuilder.operations(op -> op.index(idx -> idx.index(index).id(key).document(source)));

For Delete:-

bulkRequestBuilder.operations(op -> op.delete(d -> d.index(index).id(key)));

And flushing the bulk operation:-

BulkResponse bulkResponse = client.bulk(bulkRequestBuilder.build());

I am looking for something similar to above mentioned insert and delete operation.

Like, bulkRequestBuilder.operations(op->op.update(u->u.index(index).id(key)....))

Sagar Patel · Accepted Answer · 2022-05-04 07:17:22Z

2

You can use Fluent DSL like below as mentioned here:

List<Product> products = fetchProducts();

BulkRequest.Builder br = new BulkRequest.Builder();

for (Product product : products) {
    br.operations(op -> op           
        .index(idx -> idx            
            .index("products")       
            .id(product.getSku())
            .document(product)
        )
    );
}

BulkResponse result = esClient.bulk(br.build());

You can use Classic Builder like below (Not Recommndate):

IndexRequest.Builder<Product> indexReqBuilder = new IndexRequest.Builder<>();
indexReqBuilder.index("product");
indexReqBuilder.id("id");
indexReqBuilder.document(product);
        
List<BulkOperation> list = new ArrayList<BulkOperation>();
list.add(indexReqBuilder);
        
BulkRequest.Builder br = new BulkRequest.Builder();
br.index("");
br.operations(list);
        
client.bulk(br.build());

Update Request:

As mentioned here in document, UpdateRequest class support TDocument and TPartialDocument as parameter. when you want to index document as parial document (means only update) then you can use TPartialDocument and when you want to index document as upsert then you can use TDocument. you can pass Void class for other parameter.

You can check this discussion as well which give you some understanding.

client.update(b -> b
                .index("")
                .id("")
                .doc(product),
                Product.class
            );

client.update(new UpdateRequest.Builder<Void, Product>()
                .index("product")
                .id("")
                .doc(product)
                .build(),
                Void.class
            );

Bulk Update request:

BulkRequest.Builder br = new BulkRequest.Builder();
for (Product product : products) {
    ObjectMapper mapper = new ObjectMapper();
    String json = mapper.writeValueAsString(product);
    br.operations(op -> op
                    .update(idx -> idx.index("products").id("").withJson(new StringReader(json))));
        }

edited May 4, 2022 at 7:17

answered Apr 27, 2022 at 9:16

Sagar Patel

5,5161 gold badge11 silver badges23 bronze badges

Sign up to request clarification or add additional context in comments.

12 Comments

Dunggeon Over a year ago

Thanks for reply. Your solution as well as @Amit's solution helped me a lot. Can you help me in how to create UpdateRequest in elasticsearch version 8. Similar to below version 7 code. UpdateRequest updateRequest = new UpdateRequest().index(index).id(key).doc(source).docAsUpsert(isUpsert);

Sagar Patel Over a year ago

I have updated my answer. please check. there are no example available for UpdateRequest so difficult to implement.

Dunggeon Over a year ago

Thanks @Sagar Patel, Although your answer regarding UpdateRequest is suitable for older elasticsearch version. But can you help me in finding solution for updaterequest in bulk for elasticsearch version 8.

Sagar Patel Over a year ago

@PiyushKumar answer which i have posted it is for version 8 only. they dont have upsert method in new version. if you want to pass partial document then you need to pass <Void, Product> and if document as upsert then you need to pass <Product,Void>

Dunggeon Over a year ago

Thanks for reply. Your solution helped me a lot. But my point here is that I need to keep a list of request operation which maybe arbitrary ( maybe 1st five are inserts, next 2 are update and next 2 are delete requests and at the end 1 insert operation ). And that list needed to be flushed via Bulk maintaining the type of request received. I am using BulkRequest.Builder bulkRequestBuilder. continue.........

|

Amit · Accepted Answer · 2022-04-27 09:11:42Z

1

You can use the refer the following code which I used to bulk index.

String PRODUCT_INDEX="product"
final BulkResponse bulkResponse = esUtil.getESClient().bulk(builder -> {
                for (Product product : products) {
                    builder.index(PRODUCT_INDEX)
                            .operations(ob -> {
                                ob.index(ib -> ib.document(product).pipeline("score").id(product.getId())));
                                return ob;
                            });
                }
                return builder;
            });

answered Apr 27, 2022 at 9:11

Amit

32.5k7 gold badges68 silver badges101 bronze badges

Comments

Vitalizzare · Accepted Answer · 2024-06-19 05:32:42Z

0

This update works fine:

BulkRequest.Builder br = new BulkRequest.Builder().index(index_name);
            for (Group g : grou`enter code here`ps_to_be_updated) {
                br.operations(op -> op.update(o -> o
                        .id(g.getId())
                        .action(a ->
                                a.doc(g)
                        )
                ));

edited Jun 19, 2024 at 5:32

Vitalizzare

7,63611 gold badges23 silver badges49 bronze badges

answered Jun 13, 2024 at 7:08

Lorenzo Gardoni

313 bronze badges

Collectives™ on Stack Overflow

How can I create Bulk CRUD Operations request in Elasticsearch version 8 using JAVA APIs?

3 Answers 3

12 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

12 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related