How to use elasticsearch.net bulk API to import json file by btye array to elasticsearch?

Question

I have some json file need to import to elasticsearch.

I use the curl api. The below is sample and it work fine for me.

curl -XPOST http://localhost:9200/index_local/_doc/_bulk -H "Content-Type: application/json" --data-binary @sample.json

I use the HttpWebRequest to simulate and it work fine for me too.

public void Post(string fileName)
{
    try
    {
        // get the byte array 
        var data = File.ReadAllBytes(fileName);

        // create HttpRequest
        var httpRequest = (HttpWebRequest)WebRequest.Create(@"http://localhost:9200/index_local/_doc/_bulk");
        httpRequest.Method = "POST";
        httpRequest.ContentType = "application/json";
        httpRequest.ContentLength = data.Length;

        // set the file byte array to the stream
        using (var requestStream = httpRequest.GetRequestStream())
        {
            requestStream.Write(data, 0, data.Length);
        }

        // get the response
        using (var response = httpRequest.GetResponse() as HttpWebResponse)
        {
            using (var responseStream = new StreamReader(response.GetResponseStream()))
            {
                // read the result
                Console.WriteLine(responseStream.ReadToEnd());
            }
        }
    }
    catch (Exception e)
    {
        Console.WriteLine(e.Message);
    }
}

But I can't find the bulk api with import json file with elasticsearch.net.

Is there have some function equal to HttpWebRequest that can post the json file to elasticsearch ?

Is elasticsearch.net library ElasticLowLevelClient or ElasticClient support to import json file with btye array ?

Russ Cam · Accepted Answer · 2019-11-24 22:46:51Z

1

Assuming sample.json is a JSON file with a valid structure for the bulk API, you can send the request with

var client = new ElasticClient();
var bytes = File.ReadAllBytes("sample.json");

var bulkResponse = client.LowLevel.Bulk<BulkResponse>(
    bytes,
    new BulkRequestParameters
    {
        RequestConfiguration = new RequestConfiguration
        {
            RequestTimeout = TimeSpan.FromMinutes(3)
        }
    });

if (!bulkResponse.IsValid)
{
    // handle failure
}

This sets a specific request timeout for this request, which you might set to something larger than normal if the bulk is larger than a normal request, which it typically is. If sample.json is larger than around 5MB, you might consider reading the file in line pairs (bulk action and document) in batches, and sending as multiple bulk requests.

answered Nov 24, 2019 at 22:46

Russ Cam

126k34 gold badges211 silver badges275 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to use elasticsearch.net bulk API to import json file by btye array to elasticsearch?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related