ElasticSearch Bulk Indexing

Question

I am using version 2.1 and the 2.0 (alpha) client with the Nest C# client... I am attempting to bulk insert some records, however, the server is returning an error. I am utilizing the client on a windows client speaking to a Linux server (I don't believe this should matter however).

    public static void AddQuestionsToElasticSearch()
    {
        var es = new ElasticsearchClient(new ConnectionConfiguration(
            new Uri("http://elasticserver:9200"))
        );

        var foobars = new FooBar().Parse();
        var descriptor = new BulkDescriptor();

        descriptor.CreateMany<FooBar>(foobars, (bd, q) => bd.Id(q.Id.ToString()).Index("foobars"));

        Console.WriteLine($"Inserting foobars into ES...");
        var sw = new Stopwatch();
        sw.Start();

        var result = es.Bulk<FooBar>(descriptor);

        sw.Stop();
        Console.WriteLine($"Finished inserting foobars {GetTimeTaken(sw.Elapsed)}");
    }

Update - Error Info

The error I'm getting is in the response returned from the Bulk() method... the two properties on the BulkResponse returned are:

OriginalException: "The remote server returned an error: (400) Bad Request"

ServerError.Error: "Validation Failed: 1: no requests added"

Are you sure new FooBar().Parse(); returns actually any items? — Rob
– Rob, Commented Jan 12, 2016 at 9:08
Yes. It does return valid data.... to emulate the data source simply new []{new FooBar(), new FooBar(), .... }; — bbqchickenrobot
– bbqchickenrobot, Commented Jan 12, 2016 at 15:13

Frederik Struck-Schøning · Accepted Answer · 2018-09-12 23:37:58Z

5

You've made a simple mistake - you're using the low level ElasticsearchClient from Elasticsearch.Net to make the request, but sending it a strongly typed bulk request from NEST. To rectify is simple enough, you just need to use the ElasticClient from NEST

public static void AddQuestionsToElasticSearch()
{
    var es = new ElasticClient(new Uri("http://elasticserver:9200"));

    var foobars = new FooBar().Parse();
    var descriptor = new BulkDescriptor();

    descriptor.CreateMany<FooBar>(foobars, (bd, q) => bd.Id(q.Id.ToString()).Index("foobars"));

    Console.WriteLine($"Inserting foobars into ES...");
    var sw = new Stopwatch();
    sw.Start();

    var result = es.Bulk(descriptor);

    sw.Stop();
    Console.WriteLine($"Finished inserting foobars {GetTimeTaken(sw.Elapsed)}");
}

ElasticClient from NEST is the high level client and uses ElasticsearchClient from Elasticsearch.Net under the covers.

edited Sep 12, 2018 at 23:37

Frederik Struck-Schøning

13.3k9 gold badges63 silver badges70 bronze badges

answered Jan 12, 2016 at 21:56

Russ Cam

126k34 gold badges211 silver badges275 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

bbqchickenrobot Over a year ago

Thanks @Russ - Will try that now. I'm brandnew to ES so forgive me but I have another minor question - should I be using CreateMany<>() or IndexMany()? Couldn't tell the diff from the docs (probably missed somen).

Russ Cam Over a year ago

CreateMany<T>() will fail for any individual create call where a document already exists in the index with the same type and Id, whereas IndexMany<T> will insert the document or overwrite an existing document with the same type and Id (unless you've specified type, it will be inferred from T). Essentially, CreateMany<T> is useful in situations where you want to utilize Optimistic Concurrency Control. See elastic.co/guide/en/elasticsearch/reference/current/…

Collectives™ on Stack Overflow

ElasticSearch Bulk Indexing

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related